Bump version

Option for alternative docker images for manylinux1

.gitignore

@@ -85,10 +85,16 @@ celerybeat-schedule
 # virtualenv
 .venv
 venv/
+venv3/
 ENV/
+env/
+env3/
 
 # Spyder project settings
 .spyderproject
 
 # Rope project settings
 .ropeproject
+
+# VSCode project settings
+/.vscode

.travis.yml

@@ -1,17 +1,46 @@
+language: generic
+
 matrix:
   include:
+    # Linux Python 2
     - sudo: required
+      language: python
       services:
         - docker
+      env:
+        - "PYTHON=python2"
+
+    # Linux Python 3
+    - sudo: required
+      language: python
+      services:
+        - docker
+      env:
+        - "PYTHON=python3"
+      before_install:
+        - sudo apt-get -qq update
+        - sudo apt-get install -y python3-pip
+
+    # macOS Python 2
     - os: osx
+      env:
+        - "PYTHON=python2"
+
+    # macOS Python 3
+    - os: osx
+      env:
+        - "PYTHON=python3"
+      before_install:
+        - brew update
+        - brew install python3
 
 script:
   - |
     if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
-      pip install .
-      python ./run_tests.py
+      $PYTHON -m pip install -r requirements-dev.txt
+      $PYTHON ./bin/run_tests.py
     else
       # linux test requires root to clean up the wheelhouse (docker runs as root)
-      sudo pip install .
-      sudo python ./run_tests.py
+      sudo $PYTHON -m pip install -r requirements-dev.txt
+      sudo $PYTHON ./bin/run_tests.py
     fi

CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at joerick@mac.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

README.md

@@ -36,32 +36,37 @@ Usage
 - Create a `.travis.yml` file in your repo.
 
     ```
+    language: python
+    
     matrix:
       include:
         - sudo: required
           services:
             - docker
+          env: PIP=pip
         - os: osx
+          language: generic
+          env: PIP=pip2
 
     script:
-      - pip install cibuildwheel==0.2.0
+      - $PIP install cibuildwheel==0.7.0
       - cibuildwheel --output-dir wheelhouse
     ```
 
-  Then setup a deployment method by following the [Travis CI deployment docs](https://docs.travis-ci.com/user/deployment/).
+  Then setup a deployment method by following the [Travis CI deployment docs](https://docs.travis-ci.com/user/deployment/), or see [Delivering to PyPI](#delivering-to-pypi) below.
 
 - Create an `appveyor.yml` file in your repo.
 
     ```
     build_script:
-      - pip install cibuildwheel==0.2.0
+      - pip install cibuildwheel==0.7.0
       - cibuildwheel --output-dir wheelhouse
     artifacts:
       - path: "wheelhouse\\*.whl"
         name: Wheels
     ```
     
-  Appveyor will store the built wheels for you - you can access them from the project console. Alternatively, you may want to store them in the same place as the Travis CI build. See [Appveyor deployment docs](https://www.appveyor.com/docs/deployment/) for more info.
+  Appveyor will store the built wheels for you - you can access them from the project console. Alternatively, you may want to store them in the same place as the Travis CI build. See [Appveyor deployment docs](https://www.appveyor.com/docs/deployment/) for more info, or see [Delivering to PyPI](#delivering-to-pypi) below.
     
 - Commit those files, enable building of your repo on Travis CI and Appveyor, and push.
 
@@ -69,6 +74,30 @@ All being well, you should get wheels delivered to you in a few minutes.
 
 > ⚠️ Got an error? Check the [checklist](#it-didnt-work) below.
 
+### Configuration overview
+
+`cibuildwheel` allows for easy customization of the various phases of the build process demonstrated above:
+
+|   | Option |   |
+|---|---|---|
+| **Target wheels** | `CIBW_PLATFORM` | Override the auto-detected target platform |
+|   | `CIBW_SKIP` | Skip certain Python versions |
+| **Build environment** | `CIBW_ENVIRONMENT` | Set environment variables needed during the build |
+|   | `CIBW_BEFORE_BUILD` | Execute a shell command preparing each wheel's build |
+|   | `CIBW_MANYLINUX1_X86_64_IMAGE` | Specify an alternative manylinx1 x86_64 docker image |
+|   | `CIBW_MANYLINUX1_I686_IMAGE` | Specify an alternative manylinux1 i686 docker image |
+| **Tests** | `CIBW_TEST_COMMAND` | Execute a shell command to test all built wheels |
+|   | `CIBW_TEST_REQUIRES` | Install Python dependencies before running the tests |
+
+A more detailed description of the options, the allowed values, and some examples can be found in the [Options](#options) section.
+
+### Linux builds on Docker
+
+Linux wheels are built in the [`manylinux1` docker images](https://github.com/pypa/manylinux) to provide binary compatible wheels on Linux, according to [PEP 513](https://www.python.org/dev/peps/pep-0513/). Because of this, when building with `cibuildwheel` on Linux, a few things should be taken into account:
+- Progams and libraries cannot be installed on the Travis CI Ubuntu host with `apt-get`, but can be installed inside of the Docker image using `yum` or manually. The same goes for environment variables that are potentially needed to customize the wheel building. `cibuildwheel` supports this by providing the `CIBW_ENVIRONMENT` and `CIBW_BEFORE_BUILD` options to setup the build environment inside the running Docker image. See [below](#options) for details on these options.
+- The project directory is mounted in the running Docker instance as `/project`, the output directory for the wheels as `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system.
+- Alternative dockers images can be specified with the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options to allow for a custom, preconfigured build environment for the Linux builds. See [below](#options) for more details.
+
 
 Options
 -------
@@ -113,12 +142,84 @@ Default: `auto`
 
 For `linux` you need Docker running, on Mac or Linux. For `macos`, you need a Mac machine, and note that this script is going to automatically install MacPython on your system, so don't run on your development machine. For `windows`, you need to run in Windows, and it will build and test for all versions of Python at `C:\PythonXX[-x64]`.
 
+| Environment variable: `CIBW_SKIP`
+| ---
+
+Optional.
+
+Space-separated list of builds to skip. Each build has an identifier like `cp27-manylinux1_x86_64` or `cp34-macosx_10_6_intel` - you can list ones to skip here and `cibuildwheel` won't try to build them.
+
+The format is `python_tag-platform_tag`. The tags are as defined in [PEP 0425](https://www.python.org/dev/peps/pep-0425/#details).
+
+Python tags look like `cp27` `cp34` `cp35` `cp36`
+
+Platform tags look like `macosx_10_6_intel` `manylinux1_x86_64` `manylinux1_i686` `win32` `win_amd64`
+
+You can also use shell-style globbing syntax (as per `fnmatch`) 
+
+Examples:
+- Skip building on Python 2.7 on the Mac: `cp27-macosx_10_6_intel`  
+- Skip building on Python 2.7 on all platforms: `cp27-*`  
+- Skip Python 2.7 on Windows: `cp27-win*`  
+- Skip Python 2.7 on 32bit Windows: `cp27-win32`  
+- Skip Python 3.4 and Python 3.5: `cp34-* cp35-*`
+- Skip Python 3.6 on Linux: `cp36-manylinux*`
+- Only build on Python 3.6: `cp27-* cp34-* cp35-*`
+
+| Environment variable: `CIBW_ENVIRONMENT`
+| ---
+
+Optional.
+
+A space-separated list of environment variables to set during the build. Bash syntax should be used (even on Windows!).
+
+You must set this variable to pass variables to Linux builds (since they execute in a Docker container). It also works for the other platforms.
+
+You can use `$PATH` syntax to insert other variables, or the `$(pwd)` syntax to insert the output of other shell commands.
+
+Example: `CFLAGS="-g -Wall" CXXFLAGS="-Wall"`  
+Example: `PATH=$PATH:/usr/local/bin`  
+Example: `BUILD_TIME="$(date)"`  
+Example: `PIP_EXTRA_INDEX_URL="https://pypi.myorg.com/simple"`  
+
+Platform-specific variants also available:
+`CIBW_ENVIRONMENT_MACOS` | `CIBW_ENVIRONMENT_WINDOWS` | `CIBW_ENVIRONMENT_LINUX`
+
+In addition to the above, `cibuildwheel` always defines the environment variable `CIBUILDWHEEL=1`. This can be useful for [building wheels with optional extensions](https://github.com/joerick/cibuildwheel/wiki/Building-packages-with-optional-C-extensions).
+
+| Environment variable: `CIBW_BEFORE_BUILD`
+| ---
+
+Optional.
+
+A shell command to run before building the wheel. This option allows you to run a command in **each** Python environment before the `pip wheel` command. This is useful if you need to set up some dependency so it's available during the build.
+
+If dependencies are required to build your wheel (for example if you include a header from a Python module), set this to `{pip} install .`, and the dependencies will be installed automatically by pip. However, this means your package will be built twice - if your package takes a long time to build, you might wish to manually list the dependencies here instead.
+
+The active Python binary can be accessed using `{python}`, and pip with `{pip}`. These are useful when you need to write `python3` or `pip3` on a Python 3.x build.
+
+Example: `{pip} install .`  
+Example: `{pip} install pybind11`  
+Example: `yum install -y libffi-dev && {pip} install .`
+
+Platform-specific variants also available:  
+ `CIBW_BEFORE_BUILD_MACOS` | `CIBW_BEFORE_BUILD_WINDOWS` | `CIBW_BEFORE_BUILD_LINUX`
+
+| Environment variables: `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE`
+| ---
+
+Optional.
+
+An alternative docker image to be used for building [`manylinux1`](https://github.com/pypa/manylinux) wheels. `cibuildwheel` will then pull these instead of the official images, [`quay.io/pypa/manylinux1_x86_64`](https://quay.io/pypa/manylinux1_i686) and [`quay.io/pypa/manylinux1_i686`](https://quay.io/pypa/manylinux1_i686).
+
+Beware to specify a valid docker image that can be used the same as the official, default docker images: all necessary Python and pip versions need to be present in `/opt/python/`, and the `auditwheel` tool needs to be present for `cibuildwheel` to work. Apart from that, the architecture and relevant shared system libraries need to be manylinux1-compatible in order to produce valid `manylinux1` wheels (see https://github.com/pypa/manylinux and [PEP 513](https://www.python.org/dev/peps/pep-0513/) for more details).
+
 | Environment variable: `CIBW_TEST_COMMAND`
 | ---
 
 Optional.
 
-Shell command to run the tests. The project root should be included in the command as "{project}". The wheel will be installed automatically and available for import from the tests.
+Shell command to run tests after the build. The wheel will be installed automatically and available for import from the tests. The project root should be included in the command as "{project}".
 
 Example: `nosetests {project}/tests`
 
@@ -132,39 +233,6 @@ Space-separated list of dependencies required for running the tests.
 Example: `pytest`  
 Example: `nose==1.3.7 moto==0.4.31`
 
-| Environment variable: `CIBW_BEFORE_BUILD`
-| ---
-
-Optional.
-
-Shell command to run before building the wheel. This option allows you to run a command in **each** Python environment before the `pip wheel` command. This is useful if you need to set up some dependency so it's available during the build.
-
-The active Python binary can be accessed using `{python}`, and pip with `{pip}`. These are useful when you need to write `python3` or `pip3` on a Python 3.x build.
-
-Example: `{pip} install pybind11`
-
-Platform-specific variants also available:  
- `CIBW_BEFORE_BUILD_MACOS` | `CIBW_BEFORE_BUILD_WINDOWS` | `CIBW_BEFORE_BUILD_LINUX`
-
-| Environment variable: `CIBW_SKIP`
-| ---
-
-Optional.
-
-Space-separated list of builds to skip. Each build has an identifier like `cp27-manylinux1_x86_64` or `cp34-macosx_10_6_intel` - you can list ones to skip here and `cibuildwheel` won't try to build them.
-
-The format is `python_tag-platform_tag`. The tags are as defined in [PEP 0425](https://www.python.org/dev/peps/pep-0425/#details).
-
-Python tags look like `cp27` `cp34` `cp35` `cp36`
-
-Platform tags look like `macosx_10_6_intel` `manylinux1_x86_64` `manylinux1_i386` `win32` `win_amd64`
-
-You can also use shell-style globbing syntax (as per `fnmatch`) 
-
-Example: `cp27-macosx_10_6_intel `  (don't build on Python 2 on Mac)  
-Example: `cp27-win*`  (don't build on Python 2.7 on Windows)  
-Example: `cp34-* cp35-*`  (don't build on Python 3.4 or Python 3.5)  
-
 --
 
 #### Example YML syntax
@@ -211,6 +279,12 @@ Obviously, manual steps are for chumps, so we can automate this a little by usin
 
 > Quick note from me - using S3 as a storage didn't work due to a [bug](https://issues.apache.org/jira/browse/LIBCLOUD-792) in libcloud. Feel free to use my fork of that package that fixes the bug `pip install https://github.com/joerick/libcloud/archive/v1.5.0-s3fix.zip`
 
+### Automatic method
+
+If you don't need much control over the release of a package, you can set up cibuildwheel to deliver the wheels straight to PyPI. This doesn't require any cloud storage to work - you just need to bump the version and tag it.
+
+Check out [this example repo](https://github.com/joerick/cibuildwheel-autopypi-example) for instructions on how to set this up.
+
 It didn't work!
 ---------------
 
@@ -226,14 +300,67 @@ Working examples
 Here are some repos that use cibuildwheel. 
 
 - [pyinstrument_cext](https://github.com/joerick/pyinstrument_cext)
+- [websockets](https://github.com/aaugustin/websockets)
+- [Parselmouth](https://github.com/YannickJadoul/Parselmouth)
+- [python-admesh](https://github.com/admesh/python-admesh)
 
 > Add repo here! Send a PR.
 
 Legal note
 ----------
 
-Since `cibuildwheel` runs the wheel through delocate or auditwheel, it will automatically bundle library dependencies. This is similar to static linking - it might have some licence implications. Check the license for any code you're pulling in to make sure that's allowed.
+Since `cibuildwheel` runs the wheel through delocate or auditwheel, it will automatically bundle library dependencies. This is similar to static linking, so it might have some licence implications. Check the license for any code you're pulling in to make sure that's allowed.
 
+Changelog
+=========
+
+### 0.6.0
+
+- On the Linux build, the host filesystem is now accessible via `/host` (#36)
+- Fixed a bug where setup.py scripts would run the wrong version of Python when running subprocesses on Linux (#35)
+
+### 0.5.1
+
+- Fixed a couple of bugs on Python 3.
+- Added experimental support for Mac builds on [Bitrise.io](https://www.bitrise.io)
+
+### 0.5.0
+
+- `CIBW_ENVIRONMENT` added. You can now set environment variables for each build, even within the Docker container on Linux. This is a big one! (#21)
+- `CIBW_BEFORE_BUILD` now runs in a system shell on all platforms. You can now do things like `CIBW_BEFORE_BUILD="cmd1 && cmd2"`. (#32)
+
+### 0.4.1
+
+- Fixed a bug on Windows where subprocess' output was hidden (#23)
+- Fixed a bug on Appveyor where logs would appear in the wrong order due to output buffering (#24, thanks @YannickJadoul!)
+
+### 0.4.0
+
+- Fixed a bug that was increasing the build time by building the wheel twice. This was a problem for large projects that have a long build time. If you're upgrading and you need the old behaviour, use `CIBW_BEFORE_BUILD={pip} install .`, or install exactly the dependencies you need in `CIBW_BEFORE_BUILD`. See #18.
+
+### 0.3.0
+
+- Removed Python 2.6 support on Linux (#12)
+
+### 0.2.1
+
+11 June 2017
+
+- Changed the build process to install the package before building the wheel - this allows direct dependencies to be installed first (#9, thanks @tgarc!)
+- Added Python 3 support for the main process, for systems where Python 3 is the default (#8, thanks @tgarc).
+
+### 0.2.0
+
+13 April 2017
+
+- Added `CIBW_SKIP` option, letting users explicitly skip a build 
+- Added `CIBW_BEFORE_BUILD` option, letting users run a shell command before the build starts
+
+### 0.1.3
+
+31 March 2017
+
+- First public release!
 
 Contributing
 ============
@@ -257,6 +384,13 @@ Other notes:
 - The platforms are very similar, until they're not. I'd rather have straight-forward code than totally DRY code, so let's keep airy platfrom abstractions to a minimum.
 - I might want to break the options into a shared config file one day, so that config is more easily shared. That has motivated some of the design decisions.
 
+Maintainers
+-----------
+
+- Joe Rickerby [@joerick](https://github.com/joerick)
+- Tomas Garcia [@tgarc](https://github.com/tgarc)
+- Yannick Jadoul [@YannickJadoul](https://github.com/YannickJadoul)
+
 Credits
 -------
 

appveyor.yml

@@ -1,3 +1,10 @@
+environment:
+  matrix:
+    - PYTHON: "C:\\Python27\\python.exe"
+    - PYTHON: "C:\\Python35\\python.exe"
+
 build_script:
-  - pip install .
-  - python ./run_tests.py
+  - "%PYTHON% -m pip install -r requirements-dev.txt"
+  # the '-u' flag is required so the output is in the correct order. 
+  # See https://github.com/joerick/cibuildwheel/pull/24 for more info.
+  - "%PYTHON% -u ./bin/run_tests.py"

bin/dev_run_test

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+cd "$(dirname "$0")"
+cd ..
+
+CIBW_PLATFORM=linux ./bin/run_test.py $1

bin/run_test.py

@@ -0,0 +1,45 @@
+#!/usr/bin/python
+
+from __future__ import print_function
+import os, sys, subprocess, shutil, json
+from glob import glob
+
+def single_run(test_project):
+    # load project settings into environment
+    env_file = os.path.join(test_project, 'environment.json')
+    project_env = {}
+    if os.path.exists(env_file):
+        with open(env_file) as f:
+            project_env = json.load(f)
+
+    # run the build
+    env = os.environ.copy()
+    project_env = {str(k): str(v) for k, v in project_env.items()} # unicode not allowed in env
+    env.update(project_env)
+    print('Building %s with environment %s' % (test_project, project_env))
+    subprocess.check_call([sys.executable, '-m', 'cibuildwheel', test_project], env=env)
+    wheels = glob('wheelhouse/*.whl')
+    print('%s built successfully. %i wheels built.' % (test_project, len(wheels)))
+
+    # check some wheels were actually built
+    assert len(wheels) >= 4
+
+    # clean up
+    shutil.rmtree('wheelhouse')
+
+if __name__ == '__main__':
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("test_project_dir")
+    args = parser.parse_args()
+
+    project_path = os.path.abspath(args.test_project_dir)
+    
+    if not os.path.exists(project_path):
+        print('No test project not found.', file=sys.stderr)
+        exit(2)
+
+    single_run(project_path)
+
+    print('Project built successfully.')

bin/run_tests.py

@@ -0,0 +1,29 @@
+#!/usr/bin/python
+
+from __future__ import print_function
+import os, sys, subprocess, shutil, json
+from glob import glob
+
+if __name__ == '__main__':
+    # move cwd to the project root
+    os.chdir(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+    ### run the unit tests
+
+    subprocess.check_call([sys.executable, '-m', 'pytest', 'unit_test'])
+
+    ### run the integration tests
+
+    test_projects = glob('test/??_*')
+
+    if len(test_projects) == 0:
+        print('No test projects found. Aborting.', file=sys.stderr)
+        exit(2)
+
+    print('Testing projects:', test_projects)
+
+    run_test_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'run_test.py')
+    for project_path in test_projects:
+        subprocess.check_call([sys.executable, run_test_path, project_path])
+
+    print('%d projects built successfully.' % len(test_projects))

cibuildwheel/__init__.py

@@ -1 +1 @@
-__version__ = '0.2.0'
+__version__ = '0.7.0'

cibuildwheel/__main__.py

@@ -3,6 +3,7 @@ import argparse, os, subprocess, sys, textwrap
 
 import cibuildwheel
 import cibuildwheel.linux, cibuildwheel.windows, cibuildwheel.macos
+from cibuildwheel.environment import parse_environment, EnvironmentParseError
 from cibuildwheel.util import BuildSkipper
 
 def get_option_from_environment(option_name, platform=None):
@@ -58,8 +59,10 @@ def main():
             platform = 'macos'
         elif 'APPVEYOR' in os.environ:
             platform = 'windows'
+        elif 'BITRISE_BUILD_NUMBER' in os.environ:
+            platform = 'macos'
         else:
-            print('Unable to detect platform. cibuildwheel should run on your CI server, '
+            print('cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server, '
                   'Travis CI and Appveyor are supported. You can run on your development '
                   'machine using the --platform argument. Check --help output for more '
                   'information.',
@@ -72,12 +75,26 @@ def main():
     project_dir = args.project_dir
     before_build = get_option_from_environment('CIBW_BEFORE_BUILD', platform=platform)
     skip_config = os.environ.get('CIBW_SKIP', '')
+    environment_config = get_option_from_environment('CIBW_ENVIRONMENT', platform=platform) or ''
+
+    try:
+        environment = parse_environment(environment_config)
+    except (EnvironmentParseError, ValueError) as e:
+        print('cibuildwheel: Malformed environment option "%s"' % environment_config, file=sys.stderr)
+        import traceback
+        traceback.print_exc(None, sys.stderr)
+        exit(2)
 
     skip = BuildSkipper(skip_config)
 
+    # Add CIBUILDWHEEL environment variable
+    # This needs to be passed on to the docker container in linux.py
+    os.environ['CIBUILDWHEEL'] = '1'
+
     try:
         project_setup_py = os.path.join(project_dir, 'setup.py')
-        name_output = subprocess.check_output([sys.executable, project_setup_py, '--name'])
+        name_output = subprocess.check_output([sys.executable, project_setup_py, '--name'],
+                                              universal_newlines=True)
         # the last line of output is the name
         package_name = name_output.strip().splitlines()[-1]
     except subprocess.CalledProcessError as err:
@@ -102,10 +119,26 @@ def main():
         test_requires=test_requires,
         before_build=before_build,
         skip=skip,
+        environment=environment,
     )
 
+    if platform == 'linux':
+        manylinux1_x86_64_image = os.environ.get('CIBW_MANYLINUX1_X86_64_IMAGE', None)
+        manylinux1_i686_image = os.environ.get('CIBW_MANYLINUX1_I686_IMAGE', None)
+
+        build_options.update(
+            manylinux1_images={'x86_64': manylinux1_x86_64_image, 'i686': manylinux1_i686_image},
+        )
+    elif platform == 'macos':
+        pass
+    elif platform == 'windows':
+        pass
+
     print_preamble(platform, build_options)
 
+    if not os.path.exists(output_dir):
+        os.makedirs(output_dir)
+
     if platform == 'linux':
         cibuildwheel.linux.build(**build_options)
     elif platform == 'windows':

cibuildwheel/bashlex_eval.py

@@ -0,0 +1,71 @@
+import subprocess, shlex, sys
+from collections import namedtuple
+import bashlex
+
+NodeExecutionContext = namedtuple('NodeExecutionContext', ['environment', 'input'])
+
+def evaluate(value, environment):
+    if not value:
+        # empty string evaluates to empty string
+        # (but trips up bashlex)
+        return ''
+
+    command_node = bashlex.parsesingle(value)
+
+    if len(command_node.parts) != 1:
+        raise ValueError('"%s" has too many parts' % value)
+
+    value_word_node = command_node.parts[0]
+    
+    return evaluate_node(
+        value_word_node, 
+        context=NodeExecutionContext(environment=environment, input=value)
+    )
+
+
+def evaluate_node(node, context):
+    if node.kind == 'word':
+        return evaluate_word_node(node, context=context)
+    elif node.kind == 'commandsubstitution':
+        return evaluate_command_node(node.command, context=context)
+    elif node.kind == 'parameter':
+        return evaluate_parameter_node(node, context=context)
+    else:
+        raise ValueError('Unsupported bash construct: "%s"' % node.word)
+
+
+def evaluate_word_node(node, context):
+    word_start = node.pos[0]
+    word_end = node.pos[1]
+    word_string = context.input[word_start:word_end]
+    letters = list(word_string)
+
+    for part in node.parts:
+        part_start = part.pos[0] - word_start
+        part_end = part.pos[1] - word_start
+
+        # Set all the characters in the part to None
+        for i in range(part_start, part_end):
+            letters[i] = None
+
+        letters[part_start] = evaluate_node(part, context=context)
+
+    # remove the None letters and concat
+    value = ''.join(l for l in letters if l is not None)
+
+    # apply bash-like quotes/whitespace treatment
+    return ' '.join(word.strip() for word in shlex.split(value))
+
+
+def evaluate_command_node(node, context):
+    words = [evaluate_node(part, context=context) for part in node.parts]
+    command = ' '.join(words)
+    output = subprocess.check_output(shlex.split(command), env=context.environment)
+
+    if sys.version_info[0] >= 3:
+        return output.decode('utf8', 'replace')
+    else:
+        return output
+
+def evaluate_parameter_node(node, context):
+    return context.environment.get(node.value, '')

cibuildwheel/environment.py

@@ -0,0 +1,80 @@
+import bashlex
+from . import bashlex_eval
+
+
+class EnvironmentParseError(Exception):
+    pass
+
+
+def parse_environment(env_string):
+    env_items = split_env_items(env_string)
+    assignments = [EnvironmentAssignment(item) for item in env_items]
+    return ParsedEnvironment(assignments=assignments)
+
+
+def split_env_items(env_string):
+    '''Splits space-separated variable assignments into a list of individual assignments.
+
+    >>> split_env_items('VAR=abc')
+    ['VAR=abc']
+    >>> split_env_items('VAR="a string" THING=3')
+    ['VAR="a string"', 'THING=3']
+    >>> split_env_items('VAR="a string" THING=\\'single "quotes"\\'')
+    ['VAR="a string"', 'THING=\\'single "quotes"\\'']
+    >>> split_env_items('VAR="dont \\\\"forget\\\\" about backslashes"')
+    ['VAR="dont \\\\"forget\\\\" about backslashes"']
+    >>> split_env_items('PATH="$PATH;/opt/cibw_test_path"')
+    ['PATH="$PATH;/opt/cibw_test_path"']
+    >>> split_env_items('PATH2="something with spaces"')
+    ['PATH2="something with spaces"']
+    '''
+    if not env_string:
+        return []
+
+    command_node = bashlex.parsesingle(env_string)
+    result = []
+
+    for word_node in command_node.parts:
+        part_string = env_string[word_node.pos[0]:word_node.pos[1]]
+        result.append(part_string)
+
+    return result
+
+
+class EnvironmentAssignment(object):
+    def __init__(self, assignment):
+        name, equals, value = assignment.partition('=')
+        if not equals:
+            raise EnvironmentParseError(assignment)
+        self.name = name
+        self.value = value
+
+    def evaluated_value(self, environment):
+        '''Returns the value of this assignment, as evaluated in the environment'''
+        return bashlex_eval.evaluate(self.value, environment=environment)
+
+    def as_shell_assignment(self):
+        return 'export %s=%s' % (self.name, self.value)
+
+    def __repr__(self):
+        return '%s=%s' % (self.name, self.value)
+
+
+class ParsedEnvironment(object):
+    def __init__(self, assignments):
+        self.assignments = assignments
+
+    def as_dictionary(self, prev_environment):
+        environment = prev_environment.copy()
+
+        for assignment in self.assignments:
+            value = assignment.evaluated_value(environment=environment)
+            environment[assignment.name] = value
+
+        return environment
+
+    def as_shell_commands(self):
+        return [a.as_shell_assignment() for a in self.assignments]
+
+    def __repr__(self):
+        return 'ParsedEnvironment(%r)' % [repr(a) for a in self.assignments]

cibuildwheel/linux.py

@@ -1,5 +1,5 @@
 from __future__ import print_function
-import os, subprocess
+import os, subprocess, sys
 from collections import namedtuple
 from .util import prepare_command
 
@@ -9,19 +9,23 @@ except ImportError:
     from pipes import quote as shlex_quote
 
 
-def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip):
+def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip, environment, manylinux1_images):
+    try:
+        subprocess.check_call(['docker', '--version'])
+    except:
+        print('cibuildwheel: Docker not found. Docker is required to run Linux builds. '
+              'If you\'re building on Travis CI, add `services: [docker]` to your .travis.yml.',
+              file=sys.stderr)
+        exit(2)
+
     PythonConfiguration = namedtuple('PythonConfiguration', ['identifier', 'path'])
     python_configurations = [
-        PythonConfiguration(identifier='cp26-manylinux1_x86_64', path='/opt/python/cp26-cp26m'),
-        PythonConfiguration(identifier='cp26-manylinux1_x86_64', path='/opt/python/cp26-cp26mu'),
         PythonConfiguration(identifier='cp27-manylinux1_x86_64', path='/opt/python/cp27-cp27m'),
         PythonConfiguration(identifier='cp27-manylinux1_x86_64', path='/opt/python/cp27-cp27mu'),
         PythonConfiguration(identifier='cp33-manylinux1_x86_64', path='/opt/python/cp33-cp33m'),
         PythonConfiguration(identifier='cp34-manylinux1_x86_64', path='/opt/python/cp34-cp34m'),
         PythonConfiguration(identifier='cp35-manylinux1_x86_64', path='/opt/python/cp35-cp35m'),
         PythonConfiguration(identifier='cp36-manylinux1_x86_64', path='/opt/python/cp36-cp36m'),
-        PythonConfiguration(identifier='cp26-manylinux1_i686', path='/opt/python/cp26-cp26m'),
-        PythonConfiguration(identifier='cp26-manylinux1_i686', path='/opt/python/cp26-cp26mu'),
         PythonConfiguration(identifier='cp27-manylinux1_i686', path='/opt/python/cp27-cp27m'),
         PythonConfiguration(identifier='cp27-manylinux1_i686', path='/opt/python/cp27-cp27mu'),
         PythonConfiguration(identifier='cp33-manylinux1_i686', path='/opt/python/cp33-cp33m'),
@@ -34,39 +38,50 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
     python_configurations = [c for c in python_configurations if not skip(c.identifier)]
 
     platforms = [
-        ('manylinux1_x86_64', 'quay.io/pypa/manylinux1_x86_64'),
-        ('manylinux1_i686', 'quay.io/pypa/manylinux1_i686'),
+        ('manylinux1_x86_64', manylinux1_images.get('x86_64') or 'quay.io/pypa/manylinux1_x86_64'),
+        ('manylinux1_i686', manylinux1_images.get('i686') or 'quay.io/pypa/manylinux1_i686'),
     ]
 
     for platform_tag, docker_image in platforms:
         platform_configs = [c for c in python_configurations if c.identifier.endswith(platform_tag)]
+        if not platform_configs:
+            continue
 
         bash_script = '''
             set -o errexit
             set -o xtrace
             cd /project
 
+            {environment_exports}
+
             for PYBIN in {pybin_paths}; do
+                # Setup
+                rm -rf /tmp/built_wheel
+                rm -rf /tmp/delocated_wheel
+                mkdir /tmp/built_wheel
+                mkdir /tmp/delocated_wheel
+
                 if [ ! -z {before_build} ]; then
-                    PATH=$PYBIN:$PATH sh -c {before_build}
+                    PATH="$PYBIN:$PATH" sh -c {before_build}
                 fi
 
-                "$PYBIN/pip" wheel . -w /tmp/linux_wheels
-            done
+                # Build that wheel
+                PATH="$PYBIN:$PATH" "$PYBIN/pip" wheel . -w /tmp/built_wheel --no-deps
+                built_wheel=(/tmp/built_wheel/*.whl)
 
-            for whl in /tmp/linux_wheels/*.whl; do
-                if [[ "$whl" == *none-any.whl ]]; then
-                    # pure python wheel - just copy to the output
-                    cp "$whl" /output
+                # Delocate the wheel
+                # NOTE: 'built_wheel' here is a bash array of glob matches; "$built_wheel" returns
+                # the first element
+                if [[ "$built_wheel" == *none-any.whl ]]; then
+                    # pure python wheel - just copy
+                    mv "$built_wheel" /tmp/delocated_wheel
                 else
-                    auditwheel repair "$whl" -w /output
+                    auditwheel repair "$built_wheel" -w /tmp/delocated_wheel
                 fi
-            done
+                delocated_wheel=(/tmp/delocated_wheel/*.whl)
 
-            # Install packages and test
-            for PYBIN in {pybin_paths}; do
                 # Install the wheel we just built
-                "$PYBIN/pip" install {package_name} --no-index -f /output
+                "$PYBIN/pip" install "$delocated_wheel"
 
                 # Install any requirements to run the tests
                 if [ ! -z "{test_requires}" ]; then
@@ -76,9 +91,12 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
                 # Run the tests from a different directory
                 if [ ! -z {test_command} ]; then
                     pushd $HOME
-                    PATH=$PYBIN:$PATH sh -c {test_command}
+                    PATH="$PYBIN:$PATH" sh -c {test_command}
                     popd
                 fi
+
+                # we're all done here; move it to output
+                mv "$delocated_wheel" /output
             done
         '''.format(
             package_name=package_name,
@@ -90,20 +108,28 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             before_build=shlex_quote(
                 prepare_command(before_build, python='python', pip='pip') if before_build else ''
             ),
+            environment_exports='\n'.join(environment.as_shell_commands()),
         )
 
         docker_process = subprocess.Popen([
                 'docker',
                 'run',
+                '--env',
+                'CIBUILDWHEEL',
                 '--rm',
                 '-i',
                 '-v', '%s:/project' % os.path.abspath(project_dir),
                 '-v', '%s:/output' % os.path.abspath(output_dir),
+                '-v', '/:/host',
                 docker_image,
                 '/bin/bash'],
-            stdin=subprocess.PIPE)
+            stdin=subprocess.PIPE, universal_newlines=True)
 
-        docker_process.communicate(bash_script)
+        try:
+            docker_process.communicate(bash_script)
+        except KeyboardInterrupt:
+            docker_process.kill()
+            docker_process.wait()
 
         if docker_process.returncode != 0:
             exit(1)

cibuildwheel/macos.py

@@ -1,5 +1,5 @@
 from __future__ import print_function
-import os, subprocess, shlex, sys
+import os, subprocess, shlex, sys, shutil
 from collections import namedtuple
 from glob import glob
 try:
@@ -10,7 +10,7 @@ except ImportError:
 from .util import prepare_command
 
 
-def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip):
+def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip, environment):
     PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'identifier', 'url'])
     python_configurations = [
         PythonConfiguration(version='2.7', identifier='cp27-macosx_10_6_intel', url='https://www.python.org/ftp/python/2.7.13/python-2.7.13-macosx10.6.pkg'),
@@ -19,69 +19,94 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
         PythonConfiguration(version='3.6', identifier='cp36-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.6.0/python-3.6.0-macosx10.6.pkg'),
     ]
 
-    def shell(args, env=None, cwd=None):
+    pkgs_output = subprocess.check_output(['pkgutil',  '--pkgs'])
+    if sys.version_info[0] >= 3:
+        pkgs_output = pkgs_output.decode('utf8')
+    installed_system_packages = pkgs_output.splitlines()
+
+    def call(args, env=None, cwd=None, shell=False):
         # print the command executing for the logs
-        print('+ ' + ' '.join(shlex_quote(a) for a in args))
-        return subprocess.check_call(args, env=env, cwd=cwd)
+        if shell:
+            print('+ %s' % args)
+        else:
+            print('+ ' + ' '.join(shlex_quote(a) for a in args))
+
+        return subprocess.check_call(args, env=env, cwd=cwd, shell=shell)
+
+    abs_project_dir = os.path.abspath(project_dir)
 
     for config in python_configurations:
         if skip(config.identifier):
             print('cibuildwheel: Skipping build %s' % config.identifier, file=sys.stderr)
             continue
 
-        # download the pkg
-        shell(['curl', '-L', '-o', '/tmp/Python.pkg', config.url])
-        # install
-        shell(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
+        # if this version of python isn't installed, get it from python.org and install
+        python_package_identifier = 'org.python.Python.PythonFramework-%s' % config.version
+        if python_package_identifier not in installed_system_packages:
+            # download the pkg
+            call(['curl', '-L', '-o', '/tmp/Python.pkg', config.url])
+            # install
+            call(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
 
         env = os.environ.copy()
         env['PATH'] = os.pathsep.join([
             '/Library/Frameworks/Python.framework/Versions/%s/bin' % config.version,
             env['PATH'],
         ])
+        env = environment.as_dictionary(prev_environment=env)
 
         python = 'python3' if config.version[0] == '3' else 'python2'
         pip = 'pip3' if config.version[0] == '3' else 'pip2'
 
         # check what version we're on
-        shell(['which', python], env=env)
-        shell([python, '--version'], env=env)
+        call(['which', python], env=env)
+        call([python, '--version'], env=env)
 
         # install pip & wheel
-        shell([python, '-m', 'ensurepip', '--upgrade'], env=env)
-        shell([pip, '--version'], env=env)
-        shell([pip, 'install', 'wheel'], env=env)
-        shell([pip, 'install', 'delocate'], env=env)
+        call([python, '-m', 'ensurepip', '--upgrade'], env=env)
+        call([pip, '--version'], env=env)
+        call([pip, 'install', 'wheel'], env=env)
+        call([pip, 'install', 'delocate'], env=env)
+
+        # setup dirs
+        if os.path.exists('/tmp/built_wheel'):
+            shutil.rmtree('/tmp/built_wheel')
+        os.makedirs('/tmp/built_wheel')
+        if os.path.exists('/tmp/delocated_wheel'):
+            shutil.rmtree('/tmp/delocated_wheel')
+        os.makedirs('/tmp/delocated_wheel')
 
         # run the before_build command
         if before_build:
             before_build_prepared = prepare_command(before_build, python=python, pip=pip)
-            shell(shlex.split(before_build_prepared), env=env)
+            call(before_build_prepared, env=env, shell=True)
 
-        # build the wheel to temp dir
-        temp_wheel_dir = '/tmp/tmpwheel%s' % config.version
-        shell([pip, 'wheel', project_dir, '-w', temp_wheel_dir], env=env)
-        temp_wheel = glob(temp_wheel_dir+'/*.whl')[0]
+        # build the wheel
+        call([pip, 'wheel', abs_project_dir, '-w', '/tmp/built_wheel', '--no-deps'], env=env)
+        built_wheel = glob('/tmp/built_wheel/*.whl')[0]
 
-        if temp_wheel.endswith('none-any.whl'):
-            # pure python wheel - just copy to output_dir
-            shell(['cp', temp_wheel, output_dir], env=env)
+        if built_wheel.endswith('none-any.whl'):
+            # pure python wheel - just move
+            shutil.move(built_wheel, '/tmp/delocated_wheel')
         else:
             # list the dependencies
-            shell(['delocate-listdeps', temp_wheel], env=env)
+            call(['delocate-listdeps', built_wheel], env=env)
             # rebuild the wheel with shared libraries included and place in output dir
-            shell(['delocate-wheel', '-w', output_dir, temp_wheel], env=env)
+            call(['delocate-wheel', '-w', '/tmp/delocated_wheel', built_wheel], env=env)
+        delocated_wheel = glob('/tmp/delocated_wheel/*.whl')[0]
 
         # install the wheel
-        shell([pip, 'install', package_name, '--no-index', '--find-links', output_dir], env=env)
+        call([pip, 'install', delocated_wheel], env=env)
 
         # test the wheel
         if test_requires:
-            shell([pip, 'install'] + test_requires, env=env)
+            call([pip, 'install'] + test_requires, env=env)
         if test_command:
             # run the tests from $HOME, with an absolute path in the command
             # (this ensures that Python runs the tests against the installed wheel
             # and not the repo code)
-            abs_project_dir = os.path.abspath(project_dir)
             test_command_absolute = test_command.format(project=abs_project_dir)
-            shell(shlex.split(test_command_absolute), cwd=os.environ['HOME'], env=env)
+            call(shlex.split(test_command_absolute), cwd=os.environ['HOME'], env=env)
+
+        # we're all done here; move it to output
+        shutil.move(delocated_wheel, output_dir)

cibuildwheel/util.py

@@ -22,3 +22,20 @@ class BuildSkipper(object):
 
     def __repr__(self):
         return 'BuildSkipper(%r)' % ' '.join(self.patterns)
+
+
+# Taken from https://stackoverflow.com/a/107717
+class Unbuffered(object):
+    def __init__(self, stream):
+        self.stream = stream
+
+    def write(self, data):
+        self.stream.write(data)
+        self.stream.flush()
+
+    def writelines(self, datas):
+        self.stream.writelines(datas)
+        self.stream.flush()
+
+    def __getattr__(self, attr):
+        return getattr(self.stream, attr)

cibuildwheel/windows.py

@@ -1,16 +1,25 @@
 from __future__ import print_function
-import os, tempfile, subprocess, urllib2, sys
+import os, tempfile, subprocess, sys, shutil
+try:
+    from urllib2 import urlopen
+except ImportError:
+    from urllib.request import urlopen
 from collections import namedtuple
+from glob import glob
 
-from .util import prepare_command
+from .util import prepare_command, Unbuffered
 
 
-def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip):
+def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip, environment):
+    # Python under AppVeyor/Windows seems to be buffering by default, giving problems interleaving subprocess call output with unflushed calls to 'print'
+    sys.stdout.flush()
+    sys.stdout = Unbuffered(sys.stdout)
+
     # run_with_env is a cmd file that sets the right environment variables to
     run_with_env = os.path.join(tempfile.gettempdir(), 'appveyor_run_with_env.cmd')
     if not os.path.exists(run_with_env):
         with open(run_with_env, 'wb') as f:
-            request = urllib2.urlopen('https://github.com/ogrisel/python-appveyor-demo/raw/09a1c8672e5015a74d8f69d07add6ee803c176ec/appveyor/run_with_env.cmd')
+            request = urlopen('https://github.com/ogrisel/python-appveyor-demo/raw/09a1c8672e5015a74d8f69d07add6ee803c176ec/appveyor/run_with_env.cmd')
             f.write(request.read())
 
     def shell(args, env=None, cwd=None):
@@ -33,11 +42,20 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
         PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64'),
     ]
 
+    abs_project_dir = os.path.abspath(project_dir)
+    temp_dir = tempfile.mkdtemp(prefix='cibuildwheel')
+    built_wheel_dir = os.path.join(temp_dir, 'built_wheel')
+
     for config in python_configurations:
         if skip(config.identifier):
             print('cibuildwheel: Skipping build %s' % config.identifier, file=sys.stderr)
             continue
 
+        # setup dirs
+        if os.path.exists(built_wheel_dir):
+            shutil.rmtree(built_wheel_dir)
+        os.makedirs(built_wheel_dir)
+
         env = os.environ.copy()
         # set up environment variables for run_with_env
         env['PYTHON_VERSION'] = config.version
@@ -47,6 +65,7 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             os.path.join(config.path, 'Scripts'),
             env['PATH']
         ])
+        env = environment.as_dictionary(prev_environment=env)
 
         # for the logs - check we're running the right version of python
         shell(['python', '--version'], env=env)
@@ -63,10 +82,11 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             shell([before_build_prepared], env=env)
 
         # build the wheel
-        shell(['pip', 'wheel', project_dir, '-w', output_dir], env=env)
+        shell(['pip', 'wheel', abs_project_dir, '-w', built_wheel_dir, '--no-deps'], env=env)
+        built_wheel = glob(built_wheel_dir+'/*.whl')[0]
 
         # install the wheel
-        shell(['pip', 'install', package_name, '--no-index', '-f', output_dir], env=env)
+        shell(['pip', 'install', built_wheel], env=env)
 
         # test the wheel
         if test_requires:
@@ -75,6 +95,8 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             # run the tests from c:\, with an absolute path in the command
             # (this ensures that Python runs the tests against the installed wheel
             # and not the repo code)
-            abs_project_dir = os.path.abspath(project_dir)
             test_command_absolute = test_command.format(project=abs_project_dir)
             shell([test_command_absolute], cwd='c:\\', env=env)
+
+        # we're all done here; move it to output
+        shutil.move(built_wheel, output_dir)

requirements-dev.txt

@@ -0,0 +1,2 @@
+-e .
+pytest

run_tests.py

@@ -1,39 +0,0 @@
-from __future__ import print_function
-import os, sys, subprocess, shutil, json
-from glob import glob
-
-# move cwd to the project root
-os.chdir(os.path.dirname(os.path.abspath(__file__)))
-
-test_projects = glob('test/??_*')
-
-if len(test_projects) == 0:
-    print('No test projects found. Aborting.', file=sys.stderr)
-    exit(2)
-
-print('Testing projects:', test_projects)
-
-for project_path in test_projects:
-    # load project settings into environment
-    env_file = os.path.join(project_path, 'environment.json')
-    project_env = {}
-    if os.path.exists(env_file):
-        with open(env_file) as f:
-            project_env = json.load(f)
-
-    # run the build
-    env = os.environ.copy()
-    project_env = {str(k): str(v) for k, v in project_env.items()} # unicode not allowed in env
-    env.update(project_env)
-    print('Building %s with environment %s' % (project_path, project_env))
-    subprocess.check_call(['cibuildwheel', project_path], env=env)
-    wheels = glob('wheelhouse/*.whl')
-    print('%s built successfully. %i wheels built.' % (project_path, len(wheels)))
-
-    # check some wheels were actually built
-    assert len(wheels) >= 4
-
-    # clean up
-    shutil.rmtree('wheelhouse')
-
-print('%d projects built successfully.' % len(test_projects))

setup.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.2.0
+current_version = 0.7.0
 commit = True
 tag = True
 message = Bump version

setup.py

@@ -8,7 +8,8 @@ except ImportError:
 
 setup(
     name='cibuildwheel',
-    version='0.2.0',
+    version='0.7.0',
+    install_requires=['bashlex'],
     description="Build Python wheels on CI with minimal configuration.",
     long_description='For readme please see http://github.com/joerick/cibuildwheel',
     author="Joe Rickerby",

test/01_basic/setup.py

@@ -1,5 +1,10 @@
+import os
+
 from setuptools import setup, Extension
 
+if os.environ.get('CIBUILDWHEEL', '0') != '1':
+    raise Exception('CIBUILDWHEEL environment variable is not set to 1')
+
 setup(
     name="spam",
     ext_modules=[Extension('spam', sources=['spam.c'])],

test/03_before_build/environment.json

@@ -1,4 +1,4 @@
 {
-    "CIBW_BEFORE_BUILD": "{python} -c \"import sys; open('/tmp/pythonversion.txt', 'w').write(sys.version)\"",
-    "CIBW_BEFORE_BUILD_WINDOWS": "{python} -c \"import sys; open('c:\\pythonversion.txt', 'w').write(sys.version)\""
+    "CIBW_BEFORE_BUILD": "{python} -c \"import sys; open('/tmp/pythonversion.txt', 'w').write(sys.version)\" && {python} -c \"import sys; open('/tmp/pythonexecutable.txt', 'w').write(sys.executable)\"",
+    "CIBW_BEFORE_BUILD_WINDOWS": "{python} -c \"import sys; open('c:\\pythonversion.txt', 'w').write(sys.version)\" && {python} -c \"import sys; open('c:\\pythonexecutable.txt', 'w').write(sys.executable)\""
 }

test/03_before_build/setup.py

@@ -1,8 +1,8 @@
 from setuptools import setup, Extension
-import sys
+import sys, os
 
 if sys.argv[-1] != '--name':
-    # assert that the Python version as written to version.txt in the CIBW_BEFORE_BUILD step
+    # assert that the Python version as written to pythonversion.txt in the CIBW_BEFORE_BUILD step
     # is the same one as is currently running.
     version_file = 'c:\\pythonversion.txt' if sys.platform == 'win32' else '/tmp/pythonversion.txt'
     with open(version_file) as f:
@@ -11,6 +11,15 @@ if sys.argv[-1] != '--name':
     print('sys.version', sys.version)
     assert stored_version == sys.version
 
+    # check that the executable also was written
+    executable_file = 'c:\\pythonexecutable.txt' if sys.platform == 'win32' else '/tmp/pythonexecutable.txt'
+    with open(executable_file) as f:
+        stored_executable = f.read()
+    print('stored_executable', stored_executable)
+    print('sys.executable', sys.executable)
+    # windows/mac are case insensitive
+    assert os.path.realpath(stored_executable).lower() == os.path.realpath(sys.executable).lower()
+
 setup(
     name="spam",
     ext_modules=[Extension('spam', sources=['spam.c'])],

test/05_environment/environment.json

@@ -0,0 +1,4 @@
+{
+    "CIBW_ENVIRONMENT": "CIBW_TEST_VAR=\"a b c\" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3=\"$(echo 'test string 3')\" PATH=$PATH:/opt/cibw_test_path",
+    "CIBW_ENVIRONMENT_WINDOWS": "CIBW_TEST_VAR=\"a b c\" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3=\"$(echo 'test string 3')\" PATH=\"$PATH;/opt/cibw_test_path\""
+}

test/05_environment/setup.py

@@ -0,0 +1,24 @@
+from setuptools import setup, Extension
+import sys, os
+
+if sys.argv[-1] != '--name':
+    # explode if environment isn't correct, as set in CIBW_ENVIRONMENT
+    CIBW_TEST_VAR = os.environ.get('CIBW_TEST_VAR')
+    CIBW_TEST_VAR_2 = os.environ.get('CIBW_TEST_VAR_2')
+    PATH = os.environ.get('PATH')
+
+    if CIBW_TEST_VAR != 'a b c':
+        raise Exception('CIBW_TEST_VAR should equal "a b c". It was "%s"' % CIBW_TEST_VAR)
+    if CIBW_TEST_VAR_2 != '1':
+        raise Exception('CIBW_TEST_VAR_2 should equal "1". It was "%s"' % CIBW_TEST_VAR_2)
+    if '/opt/cibw_test_path' not in PATH:
+        raise Exception('PATH should contain "/opt/cibw_test_path". It was "%s"' % PATH)
+    if '$PATH' in PATH:
+        raise Exception('$PATH should be expanded in PATH. It was "%s"' % PATH)
+
+
+setup(
+    name="spam",
+    ext_modules=[Extension('spam', sources=['spam.c'])],
+    version="0.1.0",
+)

test/05_environment/spam.c

@@ -0,0 +1,48 @@
+#include <Python.h>
+
+static PyObject *
+spam_system(PyObject *self, PyObject *args)
+{
+    const char *command;
+    int sts;
+
+    if (!PyArg_ParseTuple(args, "s", &command))
+        return NULL;
+    sts = system(command);
+    return PyLong_FromLong(sts);
+}
+
+/* Module initialization */
+
+#if PY_MAJOR_VERSION >= 3
+    #define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
+    #define MOD_DEF(m, name, doc, methods, module_state_size) \
+        static struct PyModuleDef moduledef = { \
+            PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
+        m = PyModule_Create(&moduledef);
+    #define MOD_RETURN(m) return m;
+#else
+    #define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
+    #define MOD_DEF(m, name, doc, methods, module_state_size) \
+        m = Py_InitModule3(name, methods, doc);
+    #define MOD_RETURN(m) return;
+#endif
+
+static PyMethodDef module_methods[] = {
+    {"system", (PyCFunction)spam_system, METH_VARARGS, 
+     "Execute a shell command."},
+    {NULL}  /* Sentinel */
+};
+
+MOD_INIT(spam)
+{
+    PyObject* m;
+
+    MOD_DEF(m, 
+            "spam", 
+            "Example module", 
+            module_methods,
+            -1)
+
+    MOD_RETURN(m)
+}

test/05_environment/version.txt

@@ -0,0 +1,2 @@
+3.6.0 (default, Feb  7 2017, 23:55:32) 
+[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]

test/06_docker_images/environment.json

@@ -0,0 +1,4 @@
+{
+    "CIBW_MANYLINUX1_X86_64_IMAGE": "dockcross/manylinux-x64",
+    "CIBW_MANYLINUX1_I686_IMAGE": "dockcross/manylinux-x86"
+}

test/06_docker_images/setup.py

@@ -0,0 +1,15 @@
+import os, sys
+
+from setuptools import setup, Extension
+
+if sys.argv[-1] != '--name':
+    # check that we're running in the correct docker image as specified in the
+    # environment options CIBW_MANYLINUX1_*_IMAGE
+    if 'linux' in sys.platform and not os.path.exists('/dockcross'):
+        raise Exception('/dockcross directory not found. Is this test running in the correct docker image?')
+
+setup(
+    name="spam",
+    ext_modules=[Extension('spam', sources=['spam.c'])],
+    version="0.1.0",
+)

test/06_docker_images/spam.c

@@ -0,0 +1,48 @@
+#include <Python.h>
+
+static PyObject *
+spam_system(PyObject *self, PyObject *args)
+{
+    const char *command;
+    int sts;
+
+    if (!PyArg_ParseTuple(args, "s", &command))
+        return NULL;
+    sts = system(command);
+    return PyLong_FromLong(sts);
+}
+
+/* Module initialization */
+
+#if PY_MAJOR_VERSION >= 3
+    #define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
+    #define MOD_DEF(m, name, doc, methods, module_state_size) \
+        static struct PyModuleDef moduledef = { \
+            PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
+        m = PyModule_Create(&moduledef);
+    #define MOD_RETURN(m) return m;
+#else
+    #define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
+    #define MOD_DEF(m, name, doc, methods, module_state_size) \
+        m = Py_InitModule3(name, methods, doc);
+    #define MOD_RETURN(m) return;
+#endif
+
+static PyMethodDef module_methods[] = {
+    {"system", (PyCFunction)spam_system, METH_VARARGS, 
+     "Execute a shell command."},
+    {NULL}  /* Sentinel */
+};
+
+MOD_INIT(spam)
+{
+    PyObject* m;
+
+    MOD_DEF(m, 
+            "spam", 
+            "Example module", 
+            module_methods,
+            -1)
+
+    MOD_RETURN(m)
+}

unit_test/environment_test.py

@@ -0,0 +1,84 @@
+from cibuildwheel.environment import parse_environment
+
+
+def test_basic_parsing():
+    environment_recipe = parse_environment('VAR=1 VBR=2')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'VAR': '1', 'VBR': '2'}
+    assert environment_cmds == ['export VAR=1', 'export VBR=2']
+
+def test_quotes():
+    environment_recipe = parse_environment('A=1 VAR="1 NOT_A_VAR=2" VBR=\'vbr\'')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'A': '1', 'VAR': '1 NOT_A_VAR=2', 'VBR': 'vbr'}
+    assert environment_cmds == ['export A=1', 'export VAR="1 NOT_A_VAR=2"', 'export VBR=\'vbr\'']
+
+def test_inheritance():
+    environment_recipe = parse_environment('PATH=$PATH:/usr/local/bin')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={'PATH': '/usr/bin'}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'PATH': '/usr/bin:/usr/local/bin'}
+    assert environment_cmds == ['export PATH=$PATH:/usr/local/bin']
+
+def test_shell_eval():
+    environment_recipe = parse_environment('VAR="$(echo "a test" string)"')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'VAR': 'a test string'}
+    assert environment_cmds == ['export VAR="$(echo "a test" string)"']
+
+def test_shell_eval_and_env():
+    environment_recipe = parse_environment('VAR="$(echo "$PREV_VAR" string)"')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={'PREV_VAR': '1 2 3'}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'PREV_VAR': '1 2 3', 'VAR': '1 2 3 string'}
+    assert environment_cmds == ['export VAR="$(echo "$PREV_VAR" string)"']
+
+def test_empty_var():
+    environment_recipe = parse_environment('CFLAGS=')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={'CFLAGS': '-Wall'}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'CFLAGS': ''}
+    assert environment_cmds == ['export CFLAGS=']
+
+def test_no_vars():
+    environment_recipe = parse_environment('')
+
+    environment_dict = environment_recipe.as_dictionary(prev_environment={})
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {}
+    assert environment_cmds == []
+
+def test_no_vars_pass_through():
+    environment_recipe = parse_environment('')
+
+    environment_dict = environment_recipe.as_dictionary(prev_environment={'CIBUILDWHEEL': 'awesome'})
+
+    assert environment_dict == {'CIBUILDWHEEL': 'awesome'}