Bump version

Add python-admesh to the list of working examples

.gitignore

@@ -85,7 +85,10 @@ celerybeat-schedule
 # virtualenv
 .venv
 venv/
+venv3/
 ENV/
+env/
+env3/
 
 # Spyder project settings
 .spyderproject

.travis.yml

@@ -1,17 +1,46 @@
+language: generic
+
 matrix:
   include:
+    # Linux Python 2
     - sudo: required
+      language: python
       services:
         - docker
+      env:
+        - "PYTHON=python2"
+
+    # Linux Python 3
+    - sudo: required
+      language: python
+      services:
+        - docker
+      env:
+        - "PYTHON=python3"
+      before_install:
+        - sudo apt-get -qq update
+        - sudo apt-get install -y python3-pip
+
+    # macOS Python 2
     - os: osx
+      env:
+        - "PYTHON=python2"
+
+    # macOS Python 3
+    - os: osx
+      env:
+        - "PYTHON=python3"
+      before_install:
+        - brew update
+        - brew install python3
 
 script:
   - |
     if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
-      pip install -r requirements-dev.txt
-      python ./bin/run_tests.py
+      $PYTHON -m pip install -r requirements-dev.txt
+      $PYTHON ./bin/run_tests.py
     else
       # linux test requires root to clean up the wheelhouse (docker runs as root)
-      sudo pip install -r requirements-dev.txt
-      sudo python ./bin/run_tests.py
+      sudo $PYTHON -m pip install -r requirements-dev.txt
+      sudo $PYTHON ./bin/run_tests.py
     fi

README.md

@@ -44,7 +44,7 @@ Usage
         - os: osx
 
     script:
-      - pip install cibuildwheel==0.5.0
+      - pip install cibuildwheel==0.6.0
       - cibuildwheel --output-dir wheelhouse
     ```
 
@@ -54,7 +54,7 @@ Usage
 
     ```
     build_script:
-      - pip install cibuildwheel==0.5.0
+      - pip install cibuildwheel==0.6.0
       - cibuildwheel --output-dir wheelhouse
     artifacts:
       - path: "wheelhouse\\*.whl"
@@ -69,6 +69,27 @@ All being well, you should get wheels delivered to you in a few minutes.
 
 > ⚠️ Got an error? Check the [checklist](#it-didnt-work) below.
 
+### Configuration overview
+
+`cibuildwheel` allows for easy customization of the various phases of the build process demonstrated above:
+
+|   | Option |   |
+|---|---|---|
+| **Target wheels** | `CIBW_PLATFORM` | Override the auto-detected target platform |
+|   | `CIBW_SKIP` | Skip certain Python versions |
+| **Build environment** | `CIBW_ENVIRONMENT` | Set environment variables needed during the build |
+|   | `CIBW_BEFORE_BUILD` | Execute a shell command preparing each wheel's build |
+| **Tests** | `CIBW_TEST_COMMAND` | Execute a shell command to test all built wheels |
+|   | `CIBW_TEST_REQUIRES` | Install Python dependencies before running the tests |
+
+A more detailed description of the options, the allowed values, and some examples can be found in the [Options](#options) section.
+
+### Linux builds on Docker
+
+Linux wheels are built in the [`manylinux1` docker images](https://github.com/pypa/manylinux) to provide binary compatible wheels on Linux, according to [PEP 513](https://www.python.org/dev/peps/pep-0513/). Because of this, when building with `cibuildwheel` on Linux, a few things should be taken into account:
+- Progams and libraries cannot be installed on the Travis CI Ubuntu host with `apt-get`, but can be installed inside of the Docker image using `yum` or manually. The same goes for environment variables that are potentially needed to customize the wheel building. `cibuildwheel` supports this by providing the `CIBW_ENVIRONMENT` and `CIBW_BEFORE_BUILD` options to setup the build environment inside the running Docker image. See [below](#options) for details on these options.
+- The project directory is mounted in the running Docker instance as `/project`, the output directory for the wheels as `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system.
+
 
 Options
 -------
@@ -124,7 +145,7 @@ The format is `python_tag-platform_tag`. The tags are as defined in [PEP 0425](h
 
 Python tags look like `cp27` `cp34` `cp35` `cp36`
 
-Platform tags look like `macosx_10_6_intel` `manylinux1_x86_64` `manylinux1_i386` `win32` `win_amd64`
+Platform tags look like `macosx_10_6_intel` `manylinux1_x86_64` `manylinux1_i686` `win32` `win_amd64`
 
 You can also use shell-style globbing syntax (as per `fnmatch`) 
 
@@ -258,6 +279,8 @@ Here are some repos that use cibuildwheel.
 
 - [pyinstrument_cext](https://github.com/joerick/pyinstrument_cext)
 - [websockets](https://github.com/aaugustin/websockets)
+- [Parselmouth](https://github.com/YannickJadoul/Parselmouth)
+- [python-admesh](https://github.com/admesh/python-admesh)
 
 > Add repo here! Send a PR.
 
@@ -269,6 +292,16 @@ Since `cibuildwheel` runs the wheel through delocate or auditwheel, it will auto
 Changelog
 =========
 
+### 0.5.1
+
+- Fixed a couple of bugs on Python 3.
+- Added experimental support for Mac builds on [Bitrise.io](https://www.bitrise.io)
+
+### 0.5.0
+
+- `CIBW_ENVIRONMENT` added. You can now set environment variables for each build, even within the Docker container on Linux. This is a big one! (#21)
+- `CIBW_BEFORE_BUILD` now runs in a system shell on all platforms. You can now do things like `CIBW_BEFORE_BUILD="cmd1 && cmd2"`. (#32)
+
 ### 0.4.1
 
 - Fixed a bug on Windows where subprocess' output was hidden (#23)

appveyor.yml

@@ -1,5 +1,10 @@
+environment:
+  matrix:
+    - PYTHON: "C:\\Python27\\python.exe"
+    - PYTHON: "C:\\Python35\\python.exe"
+
 build_script:
-  - pip install -r requirements-dev.txt
+  - "%PYTHON% -m pip install -r requirements-dev.txt"
   # the '-u' flag is required so the output is in the correct order. 
   # See https://github.com/joerick/cibuildwheel/pull/24 for more info.
-  - python -u ./bin/run_tests.py
+  - "%PYTHON% -u ./bin/run_tests.py"

bin/run_test.py

@@ -17,7 +17,7 @@ def single_run(test_project):
     project_env = {str(k): str(v) for k, v in project_env.items()} # unicode not allowed in env
     env.update(project_env)
     print('Building %s with environment %s' % (test_project, project_env))
-    subprocess.check_call(['cibuildwheel', test_project], env=env)
+    subprocess.check_call([sys.executable, '-m', 'cibuildwheel', test_project], env=env)
     wheels = glob('wheelhouse/*.whl')
     print('%s built successfully. %i wheels built.' % (test_project, len(wheels)))
 

bin/run_tests.py

@@ -10,7 +10,7 @@ if __name__ == '__main__':
 
     ### run the unit tests
 
-    subprocess.check_call(['python', '-m', 'pytest', 'unit_test'])
+    subprocess.check_call([sys.executable, '-m', 'pytest', 'unit_test'])
 
     ### run the integration tests
 

cibuildwheel/__init__.py

@@ -1 +1 @@
-__version__ = '0.5.0'
+__version__ = '0.6.0'

cibuildwheel/__main__.py

@@ -59,6 +59,8 @@ def main():
             platform = 'macos'
         elif 'APPVEYOR' in os.environ:
             platform = 'windows'
+        elif 'BITRISE_BUILD_NUMBER' in os.environ:
+            platform = 'macos'
         else:
             print('cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server, '
                   'Travis CI and Appveyor are supported. You can run on your development '

cibuildwheel/bashlex_eval.py

@@ -1,4 +1,4 @@
-import subprocess, shlex
+import subprocess, shlex, sys
 from collections import namedtuple
 import bashlex
 
@@ -60,8 +60,12 @@ def evaluate_word_node(node, context):
 def evaluate_command_node(node, context):
     words = [evaluate_node(part, context=context) for part in node.parts]
     command = ' '.join(words)
-    return subprocess.check_output(shlex.split(command), env=context.environment)
+    output = subprocess.check_output(shlex.split(command), env=context.environment)
 
+    if sys.version_info[0] >= 3:
+        return output.decode('utf8', 'replace')
+    else:
+        return output
 
 def evaluate_parameter_node(node, context):
     return context.environment.get(node.value, '')

cibuildwheel/linux.py

@@ -62,11 +62,11 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
                 mkdir /tmp/delocated_wheel
 
                 if [ ! -z {before_build} ]; then
-                    PATH=$PYBIN:$PATH sh -c {before_build}
+                    PATH="$PYBIN:$PATH" sh -c {before_build}
                 fi
 
                 # Build that wheel
-                "$PYBIN/pip" wheel . -w /tmp/built_wheel --no-deps
+                PATH="$PYBIN:$PATH" "$PYBIN/pip" wheel . -w /tmp/built_wheel --no-deps
                 built_wheel=(/tmp/built_wheel/*.whl)
 
                 # Delocate the wheel
@@ -91,7 +91,7 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
                 # Run the tests from a different directory
                 if [ ! -z {test_command} ]; then
                     pushd $HOME
-                    PATH=$PYBIN:$PATH sh -c {test_command}
+                    PATH="$PYBIN:$PATH" sh -c {test_command}
                     popd
                 fi
 
@@ -120,6 +120,7 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
                 '-i',
                 '-v', '%s:/project' % os.path.abspath(project_dir),
                 '-v', '%s:/output' % os.path.abspath(output_dir),
+                '-v', '/:/host',
                 docker_image,
                 '/bin/bash'],
             stdin=subprocess.PIPE, universal_newlines=True)

cibuildwheel/macos.py

@@ -19,6 +19,11 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
         PythonConfiguration(version='3.6', identifier='cp36-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.6.0/python-3.6.0-macosx10.6.pkg'),
     ]
 
+    pkgs_output = subprocess.check_output(['pkgutil',  '--pkgs'])
+    if sys.version_info[0] >= 3:
+        pkgs_output = pkgs_output.decode('utf8')
+    installed_system_packages = pkgs_output.splitlines()
+
     def call(args, env=None, cwd=None, shell=False):
         # print the command executing for the logs
         if shell:
@@ -33,10 +38,13 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             print('cibuildwheel: Skipping build %s' % config.identifier, file=sys.stderr)
             continue
 
-        # download the pkg
-        call(['curl', '-L', '-o', '/tmp/Python.pkg', config.url])
-        # install
-        call(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
+        # if this version of python isn't installed, get it from python.org and install
+        python_package_identifier = 'org.python.Python.PythonFramework-%s' % config.version
+        if python_package_identifier not in installed_system_packages:
+            # download the pkg
+            call(['curl', '-L', '-o', '/tmp/Python.pkg', config.url])
+            # install
+            call(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
 
         env = os.environ.copy()
         env['PATH'] = os.pathsep.join([

cibuildwheel/util.py

@@ -22,3 +22,20 @@ class BuildSkipper(object):
 
     def __repr__(self):
         return 'BuildSkipper(%r)' % ' '.join(self.patterns)
+
+
+# Taken from https://stackoverflow.com/a/107717
+class Unbuffered(object):
+    def __init__(self, stream):
+        self.stream = stream
+
+    def write(self, data):
+        self.stream.write(data)
+        self.stream.flush()
+
+    def writelines(self, datas):
+        self.stream.writelines(datas)
+        self.stream.flush()
+
+    def __getattr__(self, attr):
+        return getattr(self.stream, attr)

cibuildwheel/windows.py

@@ -7,13 +7,13 @@ except ImportError:
 from collections import namedtuple
 from glob import glob
 
-from .util import prepare_command
+from .util import prepare_command, Unbuffered
 
 
 def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip, environment):
     # Python under AppVeyor/Windows seems to be buffering by default, giving problems interleaving subprocess call output with unflushed calls to 'print'
     sys.stdout.flush()
-    sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
+    sys.stdout = Unbuffered(sys.stdout)
 
     # run_with_env is a cmd file that sets the right environment variables to
     run_with_env = os.path.join(tempfile.gettempdir(), 'appveyor_run_with_env.cmd')

setup.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.5.0
+current_version = 0.6.0
 commit = True
 tag = True
 message = Bump version

setup.py

@@ -8,7 +8,7 @@ except ImportError:
 
 setup(
     name='cibuildwheel',
-    version='0.5.0',
+    version='0.6.0',
     install_requires=['bashlex'],
     description="Build Python wheels on CI with minimal configuration.",
     long_description='For readme please see http://github.com/joerick/cibuildwheel',