Bump version

macOS: update to python 2.7.15

.gitignore

@@ -85,10 +85,16 @@ celerybeat-schedule
 # virtualenv
 .venv
 venv/
+venv3/
 ENV/
+env/
+env3/
 
 # Spyder project settings
 .spyderproject
 
 # Rope project settings
 .ropeproject
+
+# VSCode project settings
+/.vscode

.travis.yml

@@ -1,17 +1,46 @@
+language: generic
+
 matrix:
   include:
+    # Linux Python 2
     - sudo: required
+      language: python
       services:
         - docker
+      env:
+        - "PYTHON=python2"
+
+    # Linux Python 3
+    - sudo: required
+      language: python
+      services:
+        - docker
+      env:
+        - "PYTHON=python3"
+      before_install:
+        - sudo apt-get -qq update
+        - sudo apt-get install -y python3-pip
+
+    # macOS Python 2
     - os: osx
+      env:
+        - "PYTHON=python2"
+
+    # macOS Python 3
+    - os: osx
+      env:
+        - "PYTHON=python3"
+      before_install:
+        - brew update
+        - brew outdated python || brew upgrade python
 
 script:
   - |
     if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
-      pip install .
-      python ./run_tests.py
+      $PYTHON -m pip install -r requirements-dev.txt
+      $PYTHON ./bin/run_tests.py
     else
       # linux test requires root to clean up the wheelhouse (docker runs as root)
-      sudo pip install .
-      sudo python ./run_tests.py
+      sudo $PYTHON -m pip install -r requirements-dev.txt
+      sudo $PYTHON ./bin/run_tests.py
     fi

README.md

@@ -15,7 +15,7 @@ What does it do?
 |   | macOS 10.6+ | manylinux i686 | manylinux x86_64 |  Windows 32bit | Windows 64bit |
 |---|---|---|---|---|---|
 | Python 2.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
-| Python 3.3 |    | ✅ | ✅ | ✅ | ✅ |
+| Python 3.3 |    |    |    | ✅ | ✅ |
 | Python 3.4 | ✅ | ✅ | ✅ | ✅ | ✅ |
 | Python 3.5 | ✅ | ✅ | ✅ | ✅ | ✅ |
 | Python 3.6 | ✅ | ✅ | ✅ | ✅ | ✅ |
@@ -36,15 +36,20 @@ Usage
 - Create a `.travis.yml` file in your repo.
 
     ```
+    language: python
+    
     matrix:
       include:
         - sudo: required
           services:
             - docker
+          env: PIP=pip
         - os: osx
+          language: generic
+          env: PIP=pip2
 
     script:
-      - pip install cibuildwheel==0.4.1
+      - $PIP install cibuildwheel==0.8.0
       - cibuildwheel --output-dir wheelhouse
     ```
 
@@ -54,7 +59,7 @@ Usage
 
     ```
     build_script:
-      - pip install cibuildwheel==0.4.1
+      - pip install cibuildwheel==0.8.0
       - cibuildwheel --output-dir wheelhouse
     artifacts:
       - path: "wheelhouse\\*.whl"
@@ -69,6 +74,31 @@ All being well, you should get wheels delivered to you in a few minutes.
 
 > ⚠️ Got an error? Check the [checklist](#it-didnt-work) below.
 
+### Configuration overview
+
+`cibuildwheel` allows for easy customization of the various phases of the build process demonstrated above:
+
+|   | Option |   |
+|---|---|---|
+| **Target wheels** | `CIBW_PLATFORM` | Override the auto-detected target platform |
+|   | `CIBW_SKIP` | Skip certain Python versions |
+| **Build parameters** | `CIBW_BUILD_VERBOSITY` | Increase or decrease the output of `pip wheel` |
+| **Build environment** | `CIBW_ENVIRONMENT` | Set environment variables needed during the build |
+|   | `CIBW_BEFORE_BUILD` | Execute a shell command preparing each wheel's build |
+|   | `CIBW_MANYLINUX1_X86_64_IMAGE` | Specify an alternative manylinx1 x86_64 docker image |
+|   | `CIBW_MANYLINUX1_I686_IMAGE` | Specify an alternative manylinux1 i686 docker image |
+| **Tests** | `CIBW_TEST_COMMAND` | Execute a shell command to test all built wheels |
+|   | `CIBW_TEST_REQUIRES` | Install Python dependencies before running the tests |
+
+A more detailed description of the options, the allowed values, and some examples can be found in the [Options](#options) section.
+
+### Linux builds on Docker
+
+Linux wheels are built in the [`manylinux1` docker images](https://github.com/pypa/manylinux) to provide binary compatible wheels on Linux, according to [PEP 513](https://www.python.org/dev/peps/pep-0513/). Because of this, when building with `cibuildwheel` on Linux, a few things should be taken into account:
+- Progams and libraries cannot be installed on the Travis CI Ubuntu host with `apt-get`, but can be installed inside of the Docker image using `yum` or manually. The same goes for environment variables that are potentially needed to customize the wheel building. `cibuildwheel` supports this by providing the `CIBW_ENVIRONMENT` and `CIBW_BEFORE_BUILD` options to setup the build environment inside the running Docker image. See [below](#options) for details on these options.
+- The project directory is mounted in the running Docker instance as `/project`, the output directory for the wheels as `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system.
+- Alternative dockers images can be specified with the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options to allow for a custom, preconfigured build environment for the Linux builds. See [below](#options) for more details.
+
 
 Options
 -------
@@ -113,12 +143,97 @@ Default: `auto`
 
 For `linux` you need Docker running, on Mac or Linux. For `macos`, you need a Mac machine, and note that this script is going to automatically install MacPython on your system, so don't run on your development machine. For `windows`, you need to run in Windows, and it will build and test for all versions of Python at `C:\PythonXX[-x64]`.
 
+| Environment variable: `CIBW_SKIP`
+| ---
+
+Optional.
+
+Space-separated list of builds to skip. Each build has an identifier like `cp27-manylinux1_x86_64` or `cp34-macosx_10_6_intel` - you can list ones to skip here and `cibuildwheel` won't try to build them.
+
+The format is `python_tag-platform_tag`. The tags are as defined in [PEP 0425](https://www.python.org/dev/peps/pep-0425/#details).
+
+Python tags look like `cp27` `cp34` `cp35` `cp36`
+
+Platform tags look like `macosx_10_6_intel` `manylinux1_x86_64` `manylinux1_i686` `win32` `win_amd64`
+
+You can also use shell-style globbing syntax (as per `fnmatch`) 
+
+Examples:
+- Skip building on Python 2.7 on the Mac: `cp27-macosx_10_6_intel`  
+- Skip building on Python 2.7 on all platforms: `cp27-*`  
+- Skip Python 2.7 on Windows: `cp27-win*`  
+- Skip Python 2.7 on 32bit Windows: `cp27-win32`  
+- Skip Python 3.4 and Python 3.5: `cp34-* cp35-*`
+- Skip Python 3.6 on Linux: `cp36-manylinux*`
+- Only build on Python 3.6: `cp27-* cp34-* cp35-*`
+
+| Environment variable: `CIBW_BUILD_VERBOSITY`
+| ---
+
+Optional.
+
+An number from 1 to 3 to increase the level of verbosity (corresponding to invoking pip with `-v`, `-vv`, and `-vvv`), between -1 and -3 (`-q`, `-qq`, and `-qqq`), or just 0 (default verbosity). These flags are useful while debugging a build when the output of the actual build invoked by `pip wheel` is required.
+
+Platform-specific variants also available:
+`CIBW_BUILD_VERBOSITY_MACOS` | `CIBW_BUILD_VERBOSITY_WINDOWS` | `CIBW_BUILD_VERBOSITY_LINUX`
+
+| Environment variable: `CIBW_ENVIRONMENT`
+| ---
+
+Optional.
+
+A space-separated list of environment variables to set during the build. Bash syntax should be used (even on Windows!).
+
+You must set this variable to pass variables to Linux builds (since they execute in a Docker container). It also works for the other platforms.
+
+You can use `$PATH` syntax to insert other variables, or the `$(pwd)` syntax to insert the output of other shell commands.
+
+Example: `CFLAGS="-g -Wall" CXXFLAGS="-Wall"`  
+Example: `PATH=$PATH:/usr/local/bin`  
+Example: `BUILD_TIME="$(date)"`  
+Example: `PIP_EXTRA_INDEX_URL="https://pypi.myorg.com/simple"`  
+
+Platform-specific variants also available:
+`CIBW_ENVIRONMENT_MACOS` | `CIBW_ENVIRONMENT_WINDOWS` | `CIBW_ENVIRONMENT_LINUX`
+
+In addition to the above, `cibuildwheel` always defines the environment variable `CIBUILDWHEEL=1`. This can be useful for [building wheels with optional extensions](https://github.com/joerick/cibuildwheel/wiki/Building-packages-with-optional-C-extensions).
+
+| Environment variable: `CIBW_BEFORE_BUILD`
+| ---
+
+Optional.
+
+A shell command to run before building the wheel. This option allows you to run a command in **each** Python environment before the `pip wheel` command. This is useful if you need to set up some dependency so it's available during the build.
+
+If dependencies are required to build your wheel (for example if you include a header from a Python module), set this to `{pip} install .`, and the dependencies will be installed automatically by pip. However, this means your package will be built twice - if your package takes a long time to build, you might wish to manually list the dependencies here instead.
+
+The active Python binary can be accessed using `{python}`, and pip with `{pip}`. These are useful when you need to write `python3` or `pip3` on a Python 3.x build. `{project}` can be used as a placeholder for the absolute path to the project's root.
+
+Example: `{pip} install .`  
+Example: `{pip} install pybind11`  
+Example: `yum install -y libffi-dev && {pip} install .`
+
+Platform-specific variants also available:  
+ `CIBW_BEFORE_BUILD_MACOS` | `CIBW_BEFORE_BUILD_WINDOWS` | `CIBW_BEFORE_BUILD_LINUX`
+
+| Environment variables: `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE`
+| ---
+
+Optional.
+
+An alternative docker image to be used for building [`manylinux1`](https://github.com/pypa/manylinux) wheels. `cibuildwheel` will then pull these instead of the official images, [`quay.io/pypa/manylinux1_x86_64`](https://quay.io/pypa/manylinux1_i686) and [`quay.io/pypa/manylinux1_i686`](https://quay.io/pypa/manylinux1_i686).
+
+Beware to specify a valid docker image that can be used the same as the official, default docker images: all necessary Python and pip versions need to be present in `/opt/python/`, and the `auditwheel` tool needs to be present for `cibuildwheel` to work. Apart from that, the architecture and relevant shared system libraries need to be manylinux1-compatible in order to produce valid `manylinux1` wheels (see https://github.com/pypa/manylinux and [PEP 513](https://www.python.org/dev/peps/pep-0513/) for more details).
+
+Example: `dockcross/manylinux-x64`  
+Example: `dockcross/manylinux-x86`
+
 | Environment variable: `CIBW_TEST_COMMAND`
 | ---
 
 Optional.
 
-Shell command to run the tests. The project root should be included in the command as "{project}". The wheel will be installed automatically and available for import from the tests.
+Shell command to run tests after the build. The wheel will be installed automatically and available for import from the tests. Just like in the `CIBW_BEFORE_BUILD` option, `{python}`, `{pip}`, and `{project}` will respectively be replaced by the Python executable, the pip executable, and the absolute project directory.
 
 Example: `nosetests {project}/tests`
 
@@ -132,42 +247,6 @@ Space-separated list of dependencies required for running the tests.
 Example: `pytest`  
 Example: `nose==1.3.7 moto==0.4.31`
 
-| Environment variable: `CIBW_BEFORE_BUILD`
-| ---
-
-Optional.
-
-Shell command to run before building the wheel. This option allows you to run a command in **each** Python environment before the `pip wheel` command. This is useful if you need to set up some dependency so it's available during the build.
-
-If dependencies are required to build your wheel (for example if you include a header from a Python module), set this to `{pip} install .`, and the dependencies will be installed automatically by pip. However, this means your package will be built twice - if your package takes a long time to build, you might wish to manually list the dependencies here instead.
-
-The active Python binary can be accessed using `{python}`, and pip with `{pip}`. These are useful when you need to write `python3` or `pip3` on a Python 3.x build.
-
-Example: `{pip} install .`  
-Example: `{pip} install pybind11`
-
-Platform-specific variants also available:  
- `CIBW_BEFORE_BUILD_MACOS` | `CIBW_BEFORE_BUILD_WINDOWS` | `CIBW_BEFORE_BUILD_LINUX`
-
-| Environment variable: `CIBW_SKIP`
-| ---
-
-Optional.
-
-Space-separated list of builds to skip. Each build has an identifier like `cp27-manylinux1_x86_64` or `cp34-macosx_10_6_intel` - you can list ones to skip here and `cibuildwheel` won't try to build them.
-
-The format is `python_tag-platform_tag`. The tags are as defined in [PEP 0425](https://www.python.org/dev/peps/pep-0425/#details).
-
-Python tags look like `cp27` `cp34` `cp35` `cp36`
-
-Platform tags look like `macosx_10_6_intel` `manylinux1_x86_64` `manylinux1_i386` `win32` `win_amd64`
-
-You can also use shell-style globbing syntax (as per `fnmatch`) 
-
-Example: `cp27-macosx_10_6_intel `  (don't build on Python 2 on Mac)  
-Example: `cp27-win*`  (don't build on Python 2.7 on Windows)  
-Example: `cp34-* cp35-*`  (don't build on Python 3.4 or Python 3.5)  
-
 --
 
 #### Example YML syntax
@@ -235,6 +314,9 @@ Working examples
 Here are some repos that use cibuildwheel. 
 
 - [pyinstrument_cext](https://github.com/joerick/pyinstrument_cext)
+- [websockets](https://github.com/aaugustin/websockets)
+- [Parselmouth](https://github.com/YannickJadoul/Parselmouth)
+- [python-admesh](https://github.com/admesh/python-admesh)
 
 > Add repo here! Send a PR.
 
@@ -246,6 +328,36 @@ Since `cibuildwheel` runs the wheel through delocate or auditwheel, it will auto
 Changelog
 =========
 
+### 0.7.1
+
+- macOS: Fix Pip bugs resulting from PyPI TLS 1.2 enforcement
+- macOS: Fix brew Python3 version problems in the CI
+
+### 0.7.0
+
+- You can now specify a custom docker image using the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options. (#46)
+- Fixed a bug where cibuildwheel would download and build a package from PyPI(!) instead of building the package on the local machine. (#51)
+
+### 0.6.0
+
+- On the Linux build, the host filesystem is now accessible via `/host` (#36)
+- Fixed a bug where setup.py scripts would run the wrong version of Python when running subprocesses on Linux (#35)
+
+### 0.5.1
+
+- Fixed a couple of bugs on Python 3.
+- Added experimental support for Mac builds on [Bitrise.io](https://www.bitrise.io)
+
+### 0.5.0
+
+- `CIBW_ENVIRONMENT` added. You can now set environment variables for each build, even within the Docker container on Linux. This is a big one! (#21)
+- `CIBW_BEFORE_BUILD` now runs in a system shell on all platforms. You can now do things like `CIBW_BEFORE_BUILD="cmd1 && cmd2"`. (#32)
+
+### 0.4.1
+
+- Fixed a bug on Windows where subprocess' output was hidden (#23)
+- Fixed a bug on Appveyor where logs would appear in the wrong order due to output buffering (#24, thanks @YannickJadoul!)
+
 ### 0.4.0
 
 - Fixed a bug that was increasing the build time by building the wheel twice. This was a problem for large projects that have a long build time. If you're upgrading and you need the old behaviour, use `CIBW_BEFORE_BUILD={pip} install .`, or install exactly the dependencies you need in `CIBW_BEFORE_BUILD`. See #18.
@@ -301,6 +413,7 @@ Maintainers
 
 - Joe Rickerby [@joerick](https://github.com/joerick)
 - Tomas Garcia [@tgarc](https://github.com/tgarc)
+- Yannick Jadoul [@YannickJadoul](https://github.com/YannickJadoul)
 
 Credits
 -------

appveyor.yml

@@ -1,5 +1,10 @@
+environment:
+  matrix:
+    - PYTHON: "C:\\Python27\\python.exe"
+    - PYTHON: "C:\\Python35\\python.exe"
+
 build_script:
-  - pip install .
+  - "%PYTHON% -m pip install -r requirements-dev.txt"
   # the '-u' flag is required so the output is in the correct order. 
   # See https://github.com/joerick/cibuildwheel/pull/24 for more info.
-  - python -u ./run_tests.py
+  - "%PYTHON% -u ./bin/run_tests.py"

bin/dev_run_test

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+cd "$(dirname "$0")"
+cd ..
+
+CIBW_PLATFORM=linux ./bin/run_test.py $1

bin/run_test.py

@@ -0,0 +1,45 @@
+#!/usr/bin/python
+
+from __future__ import print_function
+import os, sys, subprocess, shutil, json
+from glob import glob
+
+def single_run(test_project):
+    # load project settings into environment
+    env_file = os.path.join(test_project, 'environment.json')
+    project_env = {}
+    if os.path.exists(env_file):
+        with open(env_file) as f:
+            project_env = json.load(f)
+
+    # run the build
+    env = os.environ.copy()
+    project_env = {str(k): str(v) for k, v in project_env.items()} # unicode not allowed in env
+    env.update(project_env)
+    print('Building %s with environment %s' % (test_project, project_env))
+    subprocess.check_call([sys.executable, '-m', 'cibuildwheel', test_project], env=env)
+    wheels = glob('wheelhouse/*.whl')
+    print('%s built successfully. %i wheels built.' % (test_project, len(wheels)))
+
+    # check some wheels were actually built
+    assert len(wheels) >= 4
+
+    # clean up
+    shutil.rmtree('wheelhouse')
+
+if __name__ == '__main__':
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("test_project_dir")
+    args = parser.parse_args()
+
+    project_path = os.path.abspath(args.test_project_dir)
+    
+    if not os.path.exists(project_path):
+        print('No test project not found.', file=sys.stderr)
+        exit(2)
+
+    single_run(project_path)
+
+    print('Project built successfully.')

bin/run_tests.py

@@ -0,0 +1,29 @@
+#!/usr/bin/python
+
+from __future__ import print_function
+import os, sys, subprocess, shutil, json
+from glob import glob
+
+if __name__ == '__main__':
+    # move cwd to the project root
+    os.chdir(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+    ### run the unit tests
+
+    subprocess.check_call([sys.executable, '-m', 'pytest', 'unit_test'])
+
+    ### run the integration tests
+
+    test_projects = glob('test/??_*')
+
+    if len(test_projects) == 0:
+        print('No test projects found. Aborting.', file=sys.stderr)
+        exit(2)
+
+    print('Testing projects:', test_projects)
+
+    run_test_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'run_test.py')
+    for project_path in test_projects:
+        subprocess.check_call([sys.executable, run_test_path, project_path])
+
+    print('%d projects built successfully.' % len(test_projects))

cibuildwheel/__init__.py

@@ -1 +1 @@
-__version__ = '0.4.1'
+__version__ = '0.8.0'

cibuildwheel/__main__.py

@@ -3,6 +3,7 @@ import argparse, os, subprocess, sys, textwrap
 
 import cibuildwheel
 import cibuildwheel.linux, cibuildwheel.windows, cibuildwheel.macos
+from cibuildwheel.environment import parse_environment, EnvironmentParseError
 from cibuildwheel.util import BuildSkipper
 
 def get_option_from_environment(option_name, platform=None):
@@ -58,6 +59,8 @@ def main():
             platform = 'macos'
         elif 'APPVEYOR' in os.environ:
             platform = 'windows'
+        elif 'BITRISE_BUILD_NUMBER' in os.environ:
+            platform = 'macos'
         else:
             print('cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server, '
                   'Travis CI and Appveyor are supported. You can run on your development '
@@ -71,10 +74,29 @@ def main():
     test_requires = os.environ.get('CIBW_TEST_REQUIRES', '').split()
     project_dir = args.project_dir
     before_build = get_option_from_environment('CIBW_BEFORE_BUILD', platform=platform)
+    build_verbosity = get_option_from_environment('CIBW_BUILD_VERBOSITY', platform=platform) or ''
     skip_config = os.environ.get('CIBW_SKIP', '')
+    environment_config = get_option_from_environment('CIBW_ENVIRONMENT', platform=platform) or ''
+
+    try:
+        build_verbosity = min(3, max(-3, int(build_verbosity)))
+    except ValueError:
+        build_verbosity = 0
+
+    try:
+        environment = parse_environment(environment_config)
+    except (EnvironmentParseError, ValueError) as e:
+        print('cibuildwheel: Malformed environment option "%s"' % environment_config, file=sys.stderr)
+        import traceback
+        traceback.print_exc(None, sys.stderr)
+        exit(2)
 
     skip = BuildSkipper(skip_config)
 
+    # Add CIBUILDWHEEL environment variable
+    # This needs to be passed on to the docker container in linux.py
+    os.environ['CIBUILDWHEEL'] = '1'
+
     try:
         project_setup_py = os.path.join(project_dir, 'setup.py')
         name_output = subprocess.check_output([sys.executable, project_setup_py, '--name'],
@@ -102,13 +124,27 @@ def main():
         test_command=test_command,
         test_requires=test_requires,
         before_build=before_build,
+        build_verbosity=build_verbosity,
         skip=skip,
+        environment=environment,
     )
 
+    if platform == 'linux':
+        manylinux1_x86_64_image = os.environ.get('CIBW_MANYLINUX1_X86_64_IMAGE', None)
+        manylinux1_i686_image = os.environ.get('CIBW_MANYLINUX1_I686_IMAGE', None)
+
+        build_options.update(
+            manylinux1_images={'x86_64': manylinux1_x86_64_image, 'i686': manylinux1_i686_image},
+        )
+    elif platform == 'macos':
+        pass
+    elif platform == 'windows':
+        pass
+
     print_preamble(platform, build_options)
 
     if not os.path.exists(output_dir):
-        os.mkdir(output_dir)
+        os.makedirs(output_dir)
 
     if platform == 'linux':
         cibuildwheel.linux.build(**build_options)

cibuildwheel/bashlex_eval.py

@@ -0,0 +1,71 @@
+import subprocess, shlex, sys
+from collections import namedtuple
+import bashlex
+
+NodeExecutionContext = namedtuple('NodeExecutionContext', ['environment', 'input'])
+
+def evaluate(value, environment):
+    if not value:
+        # empty string evaluates to empty string
+        # (but trips up bashlex)
+        return ''
+
+    command_node = bashlex.parsesingle(value)
+
+    if len(command_node.parts) != 1:
+        raise ValueError('"%s" has too many parts' % value)
+
+    value_word_node = command_node.parts[0]
+    
+    return evaluate_node(
+        value_word_node, 
+        context=NodeExecutionContext(environment=environment, input=value)
+    )
+
+
+def evaluate_node(node, context):
+    if node.kind == 'word':
+        return evaluate_word_node(node, context=context)
+    elif node.kind == 'commandsubstitution':
+        return evaluate_command_node(node.command, context=context)
+    elif node.kind == 'parameter':
+        return evaluate_parameter_node(node, context=context)
+    else:
+        raise ValueError('Unsupported bash construct: "%s"' % node.word)
+
+
+def evaluate_word_node(node, context):
+    word_start = node.pos[0]
+    word_end = node.pos[1]
+    word_string = context.input[word_start:word_end]
+    letters = list(word_string)
+
+    for part in node.parts:
+        part_start = part.pos[0] - word_start
+        part_end = part.pos[1] - word_start
+
+        # Set all the characters in the part to None
+        for i in range(part_start, part_end):
+            letters[i] = None
+
+        letters[part_start] = evaluate_node(part, context=context)
+
+    # remove the None letters and concat
+    value = ''.join(l for l in letters if l is not None)
+
+    # apply bash-like quotes/whitespace treatment
+    return ' '.join(word.strip() for word in shlex.split(value))
+
+
+def evaluate_command_node(node, context):
+    words = [evaluate_node(part, context=context) for part in node.parts]
+    command = ' '.join(words)
+    output = subprocess.check_output(shlex.split(command), env=context.environment)
+
+    if sys.version_info[0] >= 3:
+        return output.decode('utf8', 'replace')
+    else:
+        return output
+
+def evaluate_parameter_node(node, context):
+    return context.environment.get(node.value, '')

cibuildwheel/environment.py

@@ -0,0 +1,80 @@
+import bashlex
+from . import bashlex_eval
+
+
+class EnvironmentParseError(Exception):
+    pass
+
+
+def parse_environment(env_string):
+    env_items = split_env_items(env_string)
+    assignments = [EnvironmentAssignment(item) for item in env_items]
+    return ParsedEnvironment(assignments=assignments)
+
+
+def split_env_items(env_string):
+    '''Splits space-separated variable assignments into a list of individual assignments.
+
+    >>> split_env_items('VAR=abc')
+    ['VAR=abc']
+    >>> split_env_items('VAR="a string" THING=3')
+    ['VAR="a string"', 'THING=3']
+    >>> split_env_items('VAR="a string" THING=\\'single "quotes"\\'')
+    ['VAR="a string"', 'THING=\\'single "quotes"\\'']
+    >>> split_env_items('VAR="dont \\\\"forget\\\\" about backslashes"')
+    ['VAR="dont \\\\"forget\\\\" about backslashes"']
+    >>> split_env_items('PATH="$PATH;/opt/cibw_test_path"')
+    ['PATH="$PATH;/opt/cibw_test_path"']
+    >>> split_env_items('PATH2="something with spaces"')
+    ['PATH2="something with spaces"']
+    '''
+    if not env_string:
+        return []
+
+    command_node = bashlex.parsesingle(env_string)
+    result = []
+
+    for word_node in command_node.parts:
+        part_string = env_string[word_node.pos[0]:word_node.pos[1]]
+        result.append(part_string)
+
+    return result
+
+
+class EnvironmentAssignment(object):
+    def __init__(self, assignment):
+        name, equals, value = assignment.partition('=')
+        if not equals:
+            raise EnvironmentParseError(assignment)
+        self.name = name
+        self.value = value
+
+    def evaluated_value(self, environment):
+        '''Returns the value of this assignment, as evaluated in the environment'''
+        return bashlex_eval.evaluate(self.value, environment=environment)
+
+    def as_shell_assignment(self):
+        return 'export %s=%s' % (self.name, self.value)
+
+    def __repr__(self):
+        return '%s=%s' % (self.name, self.value)
+
+
+class ParsedEnvironment(object):
+    def __init__(self, assignments):
+        self.assignments = assignments
+
+    def as_dictionary(self, prev_environment):
+        environment = prev_environment.copy()
+
+        for assignment in self.assignments:
+            value = assignment.evaluated_value(environment=environment)
+            environment[assignment.name] = value
+
+        return environment
+
+    def as_shell_commands(self):
+        return [a.as_shell_assignment() for a in self.assignments]
+
+    def __repr__(self):
+        return 'ParsedEnvironment(%r)' % [repr(a) for a in self.assignments]

cibuildwheel/linux.py

@@ -1,7 +1,7 @@
 from __future__ import print_function
 import os, subprocess, sys
 from collections import namedtuple
-from .util import prepare_command
+from .util import prepare_command, get_build_verbosity_extra_flags
 
 try:
     from shlex import quote as shlex_quote
@@ -9,7 +9,7 @@ except ImportError:
     from pipes import quote as shlex_quote
 
 
-def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip):
+def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, build_verbosity, skip, environment, manylinux1_images):
     try:
         subprocess.check_call(['docker', '--version'])
     except:
@@ -22,13 +22,11 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
     python_configurations = [
         PythonConfiguration(identifier='cp27-manylinux1_x86_64', path='/opt/python/cp27-cp27m'),
         PythonConfiguration(identifier='cp27-manylinux1_x86_64', path='/opt/python/cp27-cp27mu'),
-        PythonConfiguration(identifier='cp33-manylinux1_x86_64', path='/opt/python/cp33-cp33m'),
         PythonConfiguration(identifier='cp34-manylinux1_x86_64', path='/opt/python/cp34-cp34m'),
         PythonConfiguration(identifier='cp35-manylinux1_x86_64', path='/opt/python/cp35-cp35m'),
         PythonConfiguration(identifier='cp36-manylinux1_x86_64', path='/opt/python/cp36-cp36m'),
         PythonConfiguration(identifier='cp27-manylinux1_i686', path='/opt/python/cp27-cp27m'),
         PythonConfiguration(identifier='cp27-manylinux1_i686', path='/opt/python/cp27-cp27mu'),
-        PythonConfiguration(identifier='cp33-manylinux1_i686', path='/opt/python/cp33-cp33m'),
         PythonConfiguration(identifier='cp34-manylinux1_i686', path='/opt/python/cp34-cp34m'),
         PythonConfiguration(identifier='cp35-manylinux1_i686', path='/opt/python/cp35-cp35m'),
         PythonConfiguration(identifier='cp36-manylinux1_i686', path='/opt/python/cp36-cp36m'),
@@ -38,8 +36,8 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
     python_configurations = [c for c in python_configurations if not skip(c.identifier)]
 
     platforms = [
-        ('manylinux1_x86_64', 'quay.io/pypa/manylinux1_x86_64'),
-        ('manylinux1_i686', 'quay.io/pypa/manylinux1_i686'),
+        ('manylinux1_x86_64', manylinux1_images.get('x86_64') or 'quay.io/pypa/manylinux1_x86_64'),
+        ('manylinux1_i686', manylinux1_images.get('i686') or 'quay.io/pypa/manylinux1_i686'),
     ]
 
     for platform_tag, docker_image in platforms:
@@ -52,6 +50,8 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             set -o xtrace
             cd /project
 
+            {environment_exports}
+
             for PYBIN in {pybin_paths}; do
                 # Setup
                 rm -rf /tmp/built_wheel
@@ -60,11 +60,11 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
                 mkdir /tmp/delocated_wheel
 
                 if [ ! -z {before_build} ]; then
-                    PATH=$PYBIN:$PATH sh -c {before_build}
+                    PATH="$PYBIN:$PATH" sh -c {before_build}
                 fi
 
                 # Build that wheel
-                "$PYBIN/pip" wheel . -w /tmp/built_wheel --no-deps
+                PATH="$PYBIN:$PATH" "$PYBIN/pip" wheel . -w /tmp/built_wheel --no-deps {build_verbosity_flag}
                 built_wheel=(/tmp/built_wheel/*.whl)
 
                 # Delocate the wheel
@@ -89,7 +89,7 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
                 # Run the tests from a different directory
                 if [ ! -z {test_command} ]; then
                     pushd $HOME
-                    PATH=$PYBIN:$PATH sh -c {test_command}
+                    PATH="$PYBIN:$PATH" sh -c {test_command}
                     popd
                 fi
 
@@ -101,20 +101,25 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             pybin_paths=' '.join(c.path+'/bin' for c in platform_configs),
             test_requires=' '.join(test_requires),
             test_command=shlex_quote(
-                test_command.format(project='/project') if test_command else ''
+                prepare_command(test_command, python='python', pip='pip', project='/project') if test_command else ''
             ),
             before_build=shlex_quote(
-                prepare_command(before_build, python='python', pip='pip') if before_build else ''
+                prepare_command(before_build, python='python', pip='pip', project='/project') if before_build else ''
             ),
+            build_verbosity_flag=' '.join(get_build_verbosity_extra_flags(build_verbosity)),
+            environment_exports='\n'.join(environment.as_shell_commands()),
         )
 
         docker_process = subprocess.Popen([
                 'docker',
                 'run',
+                '--env',
+                'CIBUILDWHEEL',
                 '--rm',
                 '-i',
                 '-v', '%s:/project' % os.path.abspath(project_dir),
                 '-v', '%s:/output' % os.path.abspath(output_dir),
+                '-v', '/:/host',
                 docker_image,
                 '/bin/bash'],
             stdin=subprocess.PIPE, universal_newlines=True)

cibuildwheel/macos.py

@@ -7,51 +7,72 @@ try:
 except ImportError:
     from pipes import quote as shlex_quote
 
-from .util import prepare_command
+from .util import prepare_command, get_build_verbosity_extra_flags
 
 
-def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip):
+def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, build_verbosity, skip, environment):
     PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'identifier', 'url'])
     python_configurations = [
-        PythonConfiguration(version='2.7', identifier='cp27-macosx_10_6_intel', url='https://www.python.org/ftp/python/2.7.13/python-2.7.13-macosx10.6.pkg'),
+        PythonConfiguration(version='2.7', identifier='cp27-macosx_10_6_intel', url='https://www.python.org/ftp/python/2.7.15/python-2.7.15-macosx10.6.pkg'),
         PythonConfiguration(version='3.4', identifier='cp34-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.4.4/python-3.4.4-macosx10.6.pkg'),
-        PythonConfiguration(version='3.5', identifier='cp35-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.5.3/python-3.5.3-macosx10.6.pkg'),
-        PythonConfiguration(version='3.6', identifier='cp36-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.6.0/python-3.6.0-macosx10.6.pkg'),
+        PythonConfiguration(version='3.5', identifier='cp35-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.5.4/python-3.5.4-macosx10.6.pkg'),
+        PythonConfiguration(version='3.6', identifier='cp36-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.6.5/python-3.6.5-macosx10.6.pkg'),
     ]
+    get_pip_url = 'https://bootstrap.pypa.io/get-pip.py'
+    get_pip_script = '/tmp/get-pip.py'
 
-    def shell(args, env=None, cwd=None):
+    pkgs_output = subprocess.check_output(['pkgutil',  '--pkgs'])
+    if sys.version_info[0] >= 3:
+        pkgs_output = pkgs_output.decode('utf8')
+    installed_system_packages = pkgs_output.splitlines()
+
+    def call(args, env=None, cwd=None, shell=False):
         # print the command executing for the logs
-        print('+ ' + ' '.join(shlex_quote(a) for a in args))
-        return subprocess.check_call(args, env=env, cwd=cwd)
+        if shell:
+            print('+ %s' % args)
+        else:
+            print('+ ' + ' '.join(shlex_quote(a) for a in args))
+
+        return subprocess.check_call(args, env=env, cwd=cwd, shell=shell)
+
+    abs_project_dir = os.path.abspath(project_dir)
+
+    # get latest pip once and for all
+    call(['curl', '-L', '-o', get_pip_script, get_pip_url])
 
     for config in python_configurations:
         if skip(config.identifier):
             print('cibuildwheel: Skipping build %s' % config.identifier, file=sys.stderr)
             continue
 
-        # download the pkg
-        shell(['curl', '-L', '-o', '/tmp/Python.pkg', config.url])
-        # install
-        shell(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
+        # if this version of python isn't installed, get it from python.org and install
+        python_package_identifier = 'org.python.Python.PythonFramework-%s' % config.version
+        if python_package_identifier not in installed_system_packages:
+            # download the pkg
+            call(['curl', '-L', '-o', '/tmp/Python.pkg', config.url])
+            # install
+            call(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
 
         env = os.environ.copy()
         env['PATH'] = os.pathsep.join([
             '/Library/Frameworks/Python.framework/Versions/%s/bin' % config.version,
             env['PATH'],
         ])
+        env = environment.as_dictionary(prev_environment=env)
 
         python = 'python3' if config.version[0] == '3' else 'python2'
         pip = 'pip3' if config.version[0] == '3' else 'pip2'
 
         # check what version we're on
-        shell(['which', python], env=env)
-        shell([python, '--version'], env=env)
+        call(['which', python], env=env)
+        call([python, '--version'], env=env)
 
         # install pip & wheel
-        shell([python, '-m', 'ensurepip', '--upgrade'], env=env)
-        shell([pip, '--version'], env=env)
-        shell([pip, 'install', 'wheel'], env=env)
-        shell([pip, 'install', 'delocate'], env=env)
+        call([python, get_pip_script, '--no-setuptools', '--no-wheel'], env=env)
+        call([pip, '--version'], env=env)
+        call([pip, 'install', '--upgrade', 'setuptools'], env=env)
+        call([pip, 'install', 'wheel'], env=env)
+        call([pip, 'install', 'delocate'], env=env)
 
         # setup dirs
         if os.path.exists('/tmp/built_wheel'):
@@ -63,11 +84,11 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
 
         # run the before_build command
         if before_build:
-            before_build_prepared = prepare_command(before_build, python=python, pip=pip)
-            shell(shlex.split(before_build_prepared), env=env)
+            before_build_prepared = prepare_command(before_build, python=python, pip=pip, project=abs_project_dir)
+            call(before_build_prepared, env=env, shell=True)
 
         # build the wheel
-        shell([pip, 'wheel', project_dir, '-w', '/tmp/built_wheel', '--no-deps'], env=env)
+        call([pip, 'wheel', abs_project_dir, '-w', '/tmp/built_wheel', '--no-deps'] + get_build_verbosity_extra_flags(build_verbosity), env=env)
         built_wheel = glob('/tmp/built_wheel/*.whl')[0]
 
         if built_wheel.endswith('none-any.whl'):
@@ -75,24 +96,23 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             shutil.move(built_wheel, '/tmp/delocated_wheel')
         else:
             # list the dependencies
-            shell(['delocate-listdeps', built_wheel], env=env)
+            call(['delocate-listdeps', built_wheel], env=env)
             # rebuild the wheel with shared libraries included and place in output dir
-            shell(['delocate-wheel', '-w', '/tmp/delocated_wheel', built_wheel], env=env)
+            call(['delocate-wheel', '-w', '/tmp/delocated_wheel', built_wheel], env=env)
         delocated_wheel = glob('/tmp/delocated_wheel/*.whl')[0]
 
         # install the wheel
-        shell([pip, 'install', delocated_wheel], env=env)
+        call([pip, 'install', delocated_wheel], env=env)
 
         # test the wheel
         if test_requires:
-            shell([pip, 'install'] + test_requires, env=env)
+            call([pip, 'install'] + test_requires, env=env)
         if test_command:
             # run the tests from $HOME, with an absolute path in the command
             # (this ensures that Python runs the tests against the installed wheel
             # and not the repo code)
-            abs_project_dir = os.path.abspath(project_dir)
-            test_command_absolute = test_command.format(project=abs_project_dir)
-            shell(shlex.split(test_command_absolute), cwd=os.environ['HOME'], env=env)
+            test_command_prepared = prepare_command(test_command, python=python, pip=pip, project=abs_project_dir)
+            call(shlex.split(test_command_prepared), cwd=os.environ['HOME'], env=env)
 
         # we're all done here; move it to output
         shutil.move(delocated_wheel, output_dir)

cibuildwheel/util.py

@@ -1,7 +1,7 @@
 from fnmatch import fnmatch
 
 
-def prepare_command(command, python, pip):
+def prepare_command(command, python, pip, project):
     '''
     Preprocesses a command by expanding variables like {python} or {pip}.
 
@@ -10,7 +10,16 @@ def prepare_command(command, python, pip):
     Python 3, the user could write `{python} setup.py test`. This command would expand
     it out to python2 or python3 as appropriate.
     '''
-    return command.format(python=python, pip=pip)
+    return command.format(python=python, pip=pip, project=project)
+
+
+def get_build_verbosity_extra_flags(level):
+    if level > 0:
+        return ['-' + level * 'v']
+    elif level < 0:
+        return ['-' + -level * 'q']
+    else:
+        return []
 
 
 class BuildSkipper(object):
@@ -22,3 +31,20 @@ class BuildSkipper(object):
 
     def __repr__(self):
         return 'BuildSkipper(%r)' % ' '.join(self.patterns)
+
+
+# Taken from https://stackoverflow.com/a/107717
+class Unbuffered(object):
+    def __init__(self, stream):
+        self.stream = stream
+
+    def write(self, data):
+        self.stream.write(data)
+        self.stream.flush()
+
+    def writelines(self, datas):
+        self.stream.writelines(datas)
+        self.stream.flush()
+
+    def __getattr__(self, attr):
+        return getattr(self.stream, attr)

cibuildwheel/windows.py

@@ -7,13 +7,13 @@ except ImportError:
 from collections import namedtuple
 from glob import glob
 
-from .util import prepare_command
+from .util import prepare_command, get_build_verbosity_extra_flags, Unbuffered
 
 
-def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip):
+def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, build_verbosity, skip, environment):
     # Python under AppVeyor/Windows seems to be buffering by default, giving problems interleaving subprocess call output with unflushed calls to 'print'
     sys.stdout.flush()
-    sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
+    sys.stdout = Unbuffered(sys.stdout)
 
     # run_with_env is a cmd file that sets the right environment variables to
     run_with_env = os.path.join(tempfile.gettempdir(), 'appveyor_run_with_env.cmd')
@@ -42,6 +42,7 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
         PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64'),
     ]
 
+    abs_project_dir = os.path.abspath(project_dir)
     temp_dir = tempfile.mkdtemp(prefix='cibuildwheel')
     built_wheel_dir = os.path.join(temp_dir, 'built_wheel')
 
@@ -64,23 +65,25 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             os.path.join(config.path, 'Scripts'),
             env['PATH']
         ])
+        env = environment.as_dictionary(prev_environment=env)
 
         # for the logs - check we're running the right version of python
         shell(['python', '--version'], env=env)
         shell(['python', '-c', '"import struct; print(struct.calcsize(\'P\') * 8)\"'], env=env)
 
         # prepare the Python environment
-        shell(['pip', 'install', '--disable-pip-version-check', '--user', '--upgrade', 'pip'],
+        shell(['python', '-m', 'pip', 'install', '--upgrade', 'pip'],
               env=env)
+        shell(['pip', 'install', '--upgrade', 'setuptools'], env=env)
         shell(['pip', 'install', 'wheel'], env=env)
 
         # run the before_build command
         if before_build:
-            before_build_prepared = prepare_command(before_build, python='python', pip='pip')
+            before_build_prepared = prepare_command(before_build, python='python', pip='pip', project=abs_project_dir)
             shell([before_build_prepared], env=env)
 
         # build the wheel
-        shell(['pip', 'wheel', project_dir, '-w', built_wheel_dir, '--no-deps'], env=env)
+        shell(['pip', 'wheel', abs_project_dir, '-w', built_wheel_dir, '--no-deps'] + get_build_verbosity_extra_flags(build_verbosity), env=env)
         built_wheel = glob(built_wheel_dir+'/*.whl')[0]
 
         # install the wheel
@@ -93,9 +96,8 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             # run the tests from c:\, with an absolute path in the command
             # (this ensures that Python runs the tests against the installed wheel
             # and not the repo code)
-            abs_project_dir = os.path.abspath(project_dir)
-            test_command_absolute = test_command.format(project=abs_project_dir)
-            shell([test_command_absolute], cwd='c:\\', env=env)
+            test_command_prepared = prepare_command(test_command, python='python', pip='pip', project=abs_project_dir)
+            shell([test_command_prepared], cwd='c:\\', env=env)
 
         # we're all done here; move it to output
         shutil.move(built_wheel, output_dir)

requirements-dev.txt

@@ -0,0 +1,2 @@
+-e .
+pytest

run_tests.py

@@ -1,39 +0,0 @@
-from __future__ import print_function
-import os, sys, subprocess, shutil, json
-from glob import glob
-
-# move cwd to the project root
-os.chdir(os.path.dirname(os.path.abspath(__file__)))
-
-test_projects = glob('test/??_*')
-
-if len(test_projects) == 0:
-    print('No test projects found. Aborting.', file=sys.stderr)
-    exit(2)
-
-print('Testing projects:', test_projects)
-
-for project_path in test_projects:
-    # load project settings into environment
-    env_file = os.path.join(project_path, 'environment.json')
-    project_env = {}
-    if os.path.exists(env_file):
-        with open(env_file) as f:
-            project_env = json.load(f)
-
-    # run the build
-    env = os.environ.copy()
-    project_env = {str(k): str(v) for k, v in project_env.items()} # unicode not allowed in env
-    env.update(project_env)
-    print('Building %s with environment %s' % (project_path, project_env))
-    subprocess.check_call(['cibuildwheel', project_path], env=env)
-    wheels = glob('wheelhouse/*.whl')
-    print('%s built successfully. %i wheels built.' % (project_path, len(wheels)))
-
-    # check some wheels were actually built
-    assert len(wheels) >= 4
-
-    # clean up
-    shutil.rmtree('wheelhouse')
-
-print('%d projects built successfully.' % len(test_projects))

setup.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.4.1
+current_version = 0.8.0
 commit = True
 tag = True
 message = Bump version

setup.py

@@ -8,7 +8,8 @@ except ImportError:
 
 setup(
     name='cibuildwheel',
-    version='0.4.1',
+    version='0.8.0',
+    install_requires=['bashlex'],
     description="Build Python wheels on CI with minimal configuration.",
     long_description='For readme please see http://github.com/joerick/cibuildwheel',
     author="Joe Rickerby",

test/01_basic/setup.py

@@ -1,5 +1,10 @@
+import os
+
 from setuptools import setup, Extension
 
+if os.environ.get('CIBUILDWHEEL', '0') != '1':
+    raise Exception('CIBUILDWHEEL environment variable is not set to 1')
+
 setup(
     name="spam",
     ext_modules=[Extension('spam', sources=['spam.c'])],

test/03_before_build/environment.json

@@ -1,4 +1,4 @@
 {
-    "CIBW_BEFORE_BUILD": "{python} -c \"import sys; open('/tmp/pythonversion.txt', 'w').write(sys.version)\"",
-    "CIBW_BEFORE_BUILD_WINDOWS": "{python} -c \"import sys; open('c:\\pythonversion.txt', 'w').write(sys.version)\""
+    "CIBW_BEFORE_BUILD": "{python} -c \"import sys; open('/tmp/pythonversion.txt', 'w').write(sys.version)\" && {python} -c \"import sys; open('/tmp/pythonexecutable.txt', 'w').write(sys.executable)\"",
+    "CIBW_BEFORE_BUILD_WINDOWS": "{python} -c \"import sys; open('c:\\pythonversion.txt', 'w').write(sys.version)\" && {python} -c \"import sys; open('c:\\pythonexecutable.txt', 'w').write(sys.executable)\""
 }

test/03_before_build/setup.py

@@ -1,8 +1,8 @@
 from setuptools import setup, Extension
-import sys
+import sys, os
 
 if sys.argv[-1] != '--name':
-    # assert that the Python version as written to version.txt in the CIBW_BEFORE_BUILD step
+    # assert that the Python version as written to pythonversion.txt in the CIBW_BEFORE_BUILD step
     # is the same one as is currently running.
     version_file = 'c:\\pythonversion.txt' if sys.platform == 'win32' else '/tmp/pythonversion.txt'
     with open(version_file) as f:
@@ -11,6 +11,15 @@ if sys.argv[-1] != '--name':
     print('sys.version', sys.version)
     assert stored_version == sys.version
 
+    # check that the executable also was written
+    executable_file = 'c:\\pythonexecutable.txt' if sys.platform == 'win32' else '/tmp/pythonexecutable.txt'
+    with open(executable_file) as f:
+        stored_executable = f.read()
+    print('stored_executable', stored_executable)
+    print('sys.executable', sys.executable)
+    # windows/mac are case insensitive
+    assert os.path.realpath(stored_executable).lower() == os.path.realpath(sys.executable).lower()
+
 setup(
     name="spam",
     ext_modules=[Extension('spam', sources=['spam.c'])],

test/05_environment/environment.json

@@ -0,0 +1,4 @@
+{
+    "CIBW_ENVIRONMENT": "CIBW_TEST_VAR=\"a b c\" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3=\"$(echo 'test string 3')\" PATH=$PATH:/opt/cibw_test_path",
+    "CIBW_ENVIRONMENT_WINDOWS": "CIBW_TEST_VAR=\"a b c\" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3=\"$(echo 'test string 3')\" PATH=\"$PATH;/opt/cibw_test_path\""
+}

test/05_environment/setup.py

@@ -0,0 +1,24 @@
+from setuptools import setup, Extension
+import sys, os
+
+if sys.argv[-1] != '--name':
+    # explode if environment isn't correct, as set in CIBW_ENVIRONMENT
+    CIBW_TEST_VAR = os.environ.get('CIBW_TEST_VAR')
+    CIBW_TEST_VAR_2 = os.environ.get('CIBW_TEST_VAR_2')
+    PATH = os.environ.get('PATH')
+
+    if CIBW_TEST_VAR != 'a b c':
+        raise Exception('CIBW_TEST_VAR should equal "a b c". It was "%s"' % CIBW_TEST_VAR)
+    if CIBW_TEST_VAR_2 != '1':
+        raise Exception('CIBW_TEST_VAR_2 should equal "1". It was "%s"' % CIBW_TEST_VAR_2)
+    if '/opt/cibw_test_path' not in PATH:
+        raise Exception('PATH should contain "/opt/cibw_test_path". It was "%s"' % PATH)
+    if '$PATH' in PATH:
+        raise Exception('$PATH should be expanded in PATH. It was "%s"' % PATH)
+
+
+setup(
+    name="spam",
+    ext_modules=[Extension('spam', sources=['spam.c'])],
+    version="0.1.0",
+)

test/05_environment/spam.c

@@ -0,0 +1,48 @@
+#include <Python.h>
+
+static PyObject *
+spam_system(PyObject *self, PyObject *args)
+{
+    const char *command;
+    int sts;
+
+    if (!PyArg_ParseTuple(args, "s", &command))
+        return NULL;
+    sts = system(command);
+    return PyLong_FromLong(sts);
+}
+
+/* Module initialization */
+
+#if PY_MAJOR_VERSION >= 3
+    #define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
+    #define MOD_DEF(m, name, doc, methods, module_state_size) \
+        static struct PyModuleDef moduledef = { \
+            PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
+        m = PyModule_Create(&moduledef);
+    #define MOD_RETURN(m) return m;
+#else
+    #define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
+    #define MOD_DEF(m, name, doc, methods, module_state_size) \
+        m = Py_InitModule3(name, methods, doc);
+    #define MOD_RETURN(m) return;
+#endif
+
+static PyMethodDef module_methods[] = {
+    {"system", (PyCFunction)spam_system, METH_VARARGS, 
+     "Execute a shell command."},
+    {NULL}  /* Sentinel */
+};
+
+MOD_INIT(spam)
+{
+    PyObject* m;
+
+    MOD_DEF(m, 
+            "spam", 
+            "Example module", 
+            module_methods,
+            -1)
+
+    MOD_RETURN(m)
+}

test/05_environment/version.txt

@@ -0,0 +1,2 @@
+3.6.0 (default, Feb  7 2017, 23:55:32) 
+[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]

test/06_docker_images/environment.json

@@ -0,0 +1,4 @@
+{
+    "CIBW_MANYLINUX1_X86_64_IMAGE": "dockcross/manylinux-x64",
+    "CIBW_MANYLINUX1_I686_IMAGE": "dockcross/manylinux-x86"
+}

test/06_docker_images/setup.py

@@ -0,0 +1,15 @@
+import os, sys
+
+from setuptools import setup, Extension
+
+if sys.argv[-1] != '--name':
+    # check that we're running in the correct docker image as specified in the
+    # environment options CIBW_MANYLINUX1_*_IMAGE
+    if 'linux' in sys.platform and not os.path.exists('/dockcross'):
+        raise Exception('/dockcross directory not found. Is this test running in the correct docker image?')
+
+setup(
+    name="spam",
+    ext_modules=[Extension('spam', sources=['spam.c'])],
+    version="0.1.0",
+)

test/06_docker_images/spam.c

@@ -0,0 +1,48 @@
+#include <Python.h>
+
+static PyObject *
+spam_system(PyObject *self, PyObject *args)
+{
+    const char *command;
+    int sts;
+
+    if (!PyArg_ParseTuple(args, "s", &command))
+        return NULL;
+    sts = system(command);
+    return PyLong_FromLong(sts);
+}
+
+/* Module initialization */
+
+#if PY_MAJOR_VERSION >= 3
+    #define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
+    #define MOD_DEF(m, name, doc, methods, module_state_size) \
+        static struct PyModuleDef moduledef = { \
+            PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
+        m = PyModule_Create(&moduledef);
+    #define MOD_RETURN(m) return m;
+#else
+    #define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
+    #define MOD_DEF(m, name, doc, methods, module_state_size) \
+        m = Py_InitModule3(name, methods, doc);
+    #define MOD_RETURN(m) return;
+#endif
+
+static PyMethodDef module_methods[] = {
+    {"system", (PyCFunction)spam_system, METH_VARARGS, 
+     "Execute a shell command."},
+    {NULL}  /* Sentinel */
+};
+
+MOD_INIT(spam)
+{
+    PyObject* m;
+
+    MOD_DEF(m, 
+            "spam", 
+            "Example module", 
+            module_methods,
+            -1)
+
+    MOD_RETURN(m)
+}

unit_test/environment_test.py

@@ -0,0 +1,84 @@
+from cibuildwheel.environment import parse_environment
+
+
+def test_basic_parsing():
+    environment_recipe = parse_environment('VAR=1 VBR=2')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'VAR': '1', 'VBR': '2'}
+    assert environment_cmds == ['export VAR=1', 'export VBR=2']
+
+def test_quotes():
+    environment_recipe = parse_environment('A=1 VAR="1 NOT_A_VAR=2" VBR=\'vbr\'')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'A': '1', 'VAR': '1 NOT_A_VAR=2', 'VBR': 'vbr'}
+    assert environment_cmds == ['export A=1', 'export VAR="1 NOT_A_VAR=2"', 'export VBR=\'vbr\'']
+
+def test_inheritance():
+    environment_recipe = parse_environment('PATH=$PATH:/usr/local/bin')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={'PATH': '/usr/bin'}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'PATH': '/usr/bin:/usr/local/bin'}
+    assert environment_cmds == ['export PATH=$PATH:/usr/local/bin']
+
+def test_shell_eval():
+    environment_recipe = parse_environment('VAR="$(echo "a test" string)"')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'VAR': 'a test string'}
+    assert environment_cmds == ['export VAR="$(echo "a test" string)"']
+
+def test_shell_eval_and_env():
+    environment_recipe = parse_environment('VAR="$(echo "$PREV_VAR" string)"')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={'PREV_VAR': '1 2 3'}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'PREV_VAR': '1 2 3', 'VAR': '1 2 3 string'}
+    assert environment_cmds == ['export VAR="$(echo "$PREV_VAR" string)"']
+
+def test_empty_var():
+    environment_recipe = parse_environment('CFLAGS=')
+
+    environment_dict = environment_recipe.as_dictionary(
+        prev_environment={'CFLAGS': '-Wall'}
+    )
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {'CFLAGS': ''}
+    assert environment_cmds == ['export CFLAGS=']
+
+def test_no_vars():
+    environment_recipe = parse_environment('')
+
+    environment_dict = environment_recipe.as_dictionary(prev_environment={})
+    environment_cmds = environment_recipe.as_shell_commands()
+
+    assert environment_dict == {}
+    assert environment_cmds == []
+
+def test_no_vars_pass_through():
+    environment_recipe = parse_environment('')
+
+    environment_dict = environment_recipe.as_dictionary(prev_environment={'CIBUILDWHEEL': 'awesome'})
+
+    assert environment_dict == {'CIBUILDWHEEL': 'awesome'}