Bump version

Unbuffered stdout for Windows and displaying the output of subprocess calls

Fix #23

CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at joerick@mac.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

README.md

@@ -44,24 +44,24 @@ Usage
         - os: osx
 
     script:
-      - pip install cibuildwheel==0.2.1
+      - pip install cibuildwheel==0.4.1
       - cibuildwheel --output-dir wheelhouse
     ```
 
-  Then setup a deployment method by following the [Travis CI deployment docs](https://docs.travis-ci.com/user/deployment/).
+  Then setup a deployment method by following the [Travis CI deployment docs](https://docs.travis-ci.com/user/deployment/), or see [Delivering to PyPI](#delivering-to-pypi) below.
 
 - Create an `appveyor.yml` file in your repo.
 
     ```
     build_script:
-      - pip install cibuildwheel==0.2.1
+      - pip install cibuildwheel==0.4.1
       - cibuildwheel --output-dir wheelhouse
     artifacts:
       - path: "wheelhouse\\*.whl"
         name: Wheels
     ```
     
-  Appveyor will store the built wheels for you - you can access them from the project console. Alternatively, you may want to store them in the same place as the Travis CI build. See [Appveyor deployment docs](https://www.appveyor.com/docs/deployment/) for more info.
+  Appveyor will store the built wheels for you - you can access them from the project console. Alternatively, you may want to store them in the same place as the Travis CI build. See [Appveyor deployment docs](https://www.appveyor.com/docs/deployment/) for more info, or see [Delivering to PyPI](#delivering-to-pypi) below.
     
 - Commit those files, enable building of your repo on Travis CI and Appveyor, and push.
 
@@ -139,8 +139,11 @@ Optional.
 
 Shell command to run before building the wheel. This option allows you to run a command in **each** Python environment before the `pip wheel` command. This is useful if you need to set up some dependency so it's available during the build.
 
+If dependencies are required to build your wheel (for example if you include a header from a Python module), set this to `{pip} install .`, and the dependencies will be installed automatically by pip. However, this means your package will be built twice - if your package takes a long time to build, you might wish to manually list the dependencies here instead.
+
 The active Python binary can be accessed using `{python}`, and pip with `{pip}`. These are useful when you need to write `python3` or `pip3` on a Python 3.x build.
 
+Example: `{pip} install .`  
 Example: `{pip} install pybind11`
 
 Platform-specific variants also available:  
@@ -238,8 +241,38 @@ Here are some repos that use cibuildwheel.
 Legal note
 ----------
 
-Since `cibuildwheel` runs the wheel through delocate or auditwheel, it will automatically bundle library dependencies. This is similar to static linking - it might have some licence implications. Check the license for any code you're pulling in to make sure that's allowed.
+Since `cibuildwheel` runs the wheel through delocate or auditwheel, it will automatically bundle library dependencies. This is similar to static linking, so it might have some licence implications. Check the license for any code you're pulling in to make sure that's allowed.
 
+Changelog
+=========
+
+### 0.4.0
+
+- Fixed a bug that was increasing the build time by building the wheel twice. This was a problem for large projects that have a long build time. If you're upgrading and you need the old behaviour, use `CIBW_BEFORE_BUILD={pip} install .`, or install exactly the dependencies you need in `CIBW_BEFORE_BUILD`. See #18.
+
+### 0.3.0
+
+- Removed Python 2.6 support on Linux (#12)
+
+### 0.2.1
+
+11 June 2017
+
+- Changed the build process to install the package before building the wheel - this allows direct dependencies to be installed first (#9, thanks @tgarc!)
+- Added Python 3 support for the main process, for systems where Python 3 is the default (#8, thanks @tgarc).
+
+### 0.2.0
+
+13 April 2017
+
+- Added `CIBW_SKIP` option, letting users explicitly skip a build 
+- Added `CIBW_BEFORE_BUILD` option, letting users run a shell command before the build starts
+
+### 0.1.3
+
+31 March 2017
+
+- First public release!
 
 Contributing
 ============
@@ -263,6 +296,12 @@ Other notes:
 - The platforms are very similar, until they're not. I'd rather have straight-forward code than totally DRY code, so let's keep airy platfrom abstractions to a minimum.
 - I might want to break the options into a shared config file one day, so that config is more easily shared. That has motivated some of the design decisions.
 
+Maintainers
+-----------
+
+- Joe Rickerby [@joerick](https://github.com/joerick)
+- Tomas Garcia [@tgarc](https://github.com/tgarc)
+
 Credits
 -------
 

appveyor.yml

@@ -1,3 +1,5 @@
 build_script:
   - pip install .
-  - python ./run_tests.py
+  # the '-u' flag is required so the output is in the correct order. 
+  # See https://github.com/joerick/cibuildwheel/pull/24 for more info.
+  - python -u ./run_tests.py

cibuildwheel/__init__.py

@@ -1 +1 @@
-__version__ = '0.2.1'
+__version__ = '0.4.1'

cibuildwheel/__main__.py

@@ -59,7 +59,7 @@ def main():
         elif 'APPVEYOR' in os.environ:
             platform = 'windows'
         else:
-            print('Unable to detect platform. cibuildwheel should run on your CI server, '
+            print('cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server, '
                   'Travis CI and Appveyor are supported. You can run on your development '
                   'machine using the --platform argument. Check --help output for more '
                   'information.',
@@ -77,7 +77,8 @@ def main():
 
     try:
         project_setup_py = os.path.join(project_dir, 'setup.py')
-        name_output = subprocess.check_output([sys.executable, project_setup_py, '--name'])
+        name_output = subprocess.check_output([sys.executable, project_setup_py, '--name'],
+                                              universal_newlines=True)
         # the last line of output is the name
         package_name = name_output.strip().splitlines()[-1]
     except subprocess.CalledProcessError as err:
@@ -106,6 +107,9 @@ def main():
 
     print_preamble(platform, build_options)
 
+    if not os.path.exists(output_dir):
+        os.mkdir(output_dir)
+
     if platform == 'linux':
         cibuildwheel.linux.build(**build_options)
     elif platform == 'windows':

cibuildwheel/linux.py

@@ -1,5 +1,5 @@
 from __future__ import print_function
-import os, subprocess
+import os, subprocess, sys
 from collections import namedtuple
 from .util import prepare_command
 
@@ -10,18 +10,22 @@ except ImportError:
 
 
 def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip):
+    try:
+        subprocess.check_call(['docker', '--version'])
+    except:
+        print('cibuildwheel: Docker not found. Docker is required to run Linux builds. '
+              'If you\'re building on Travis CI, add `services: [docker]` to your .travis.yml.',
+              file=sys.stderr)
+        exit(2)
+
     PythonConfiguration = namedtuple('PythonConfiguration', ['identifier', 'path'])
     python_configurations = [
-        PythonConfiguration(identifier='cp26-manylinux1_x86_64', path='/opt/python/cp26-cp26m'),
-        PythonConfiguration(identifier='cp26-manylinux1_x86_64', path='/opt/python/cp26-cp26mu'),
         PythonConfiguration(identifier='cp27-manylinux1_x86_64', path='/opt/python/cp27-cp27m'),
         PythonConfiguration(identifier='cp27-manylinux1_x86_64', path='/opt/python/cp27-cp27mu'),
         PythonConfiguration(identifier='cp33-manylinux1_x86_64', path='/opt/python/cp33-cp33m'),
         PythonConfiguration(identifier='cp34-manylinux1_x86_64', path='/opt/python/cp34-cp34m'),
         PythonConfiguration(identifier='cp35-manylinux1_x86_64', path='/opt/python/cp35-cp35m'),
         PythonConfiguration(identifier='cp36-manylinux1_x86_64', path='/opt/python/cp36-cp36m'),
-        PythonConfiguration(identifier='cp26-manylinux1_i686', path='/opt/python/cp26-cp26m'),
-        PythonConfiguration(identifier='cp26-manylinux1_i686', path='/opt/python/cp26-cp26mu'),
         PythonConfiguration(identifier='cp27-manylinux1_i686', path='/opt/python/cp27-cp27m'),
         PythonConfiguration(identifier='cp27-manylinux1_i686', path='/opt/python/cp27-cp27mu'),
         PythonConfiguration(identifier='cp33-manylinux1_i686', path='/opt/python/cp33-cp33m'),
@@ -40,6 +44,8 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
 
     for platform_tag, docker_image in platforms:
         platform_configs = [c for c in python_configurations if c.identifier.endswith(platform_tag)]
+        if not platform_configs:
+            continue
 
         bash_script = '''
             set -o errexit
@@ -47,30 +53,33 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             cd /project
 
             for PYBIN in {pybin_paths}; do
+                # Setup
+                rm -rf /tmp/built_wheel
+                rm -rf /tmp/delocated_wheel
+                mkdir /tmp/built_wheel
+                mkdir /tmp/delocated_wheel
+
                 if [ ! -z {before_build} ]; then
                     PATH=$PYBIN:$PATH sh -c {before_build}
                 fi
 
-                # install the package first to take care of dependencies
-                "$PYBIN/pip" install .
+                # Build that wheel
+                "$PYBIN/pip" wheel . -w /tmp/built_wheel --no-deps
+                built_wheel=(/tmp/built_wheel/*.whl)
 
-                "$PYBIN/pip" wheel --no-deps . -w /tmp/linux_wheels
-            done
-
-            for whl in /tmp/linux_wheels/*.whl; do
-                if [[ "$whl" == *none-any.whl ]]; then
-                    # pure python wheel - just copy to the output
-                    cp "$whl" /output
+                # Delocate the wheel
+                # NOTE: 'built_wheel' here is a bash array of glob matches; "$built_wheel" returns
+                # the first element
+                if [[ "$built_wheel" == *none-any.whl ]]; then
+                    # pure python wheel - just copy
+                    mv "$built_wheel" /tmp/delocated_wheel
                 else
-                    auditwheel repair "$whl" -w /output
+                    auditwheel repair "$built_wheel" -w /tmp/delocated_wheel
                 fi
-            done
+                delocated_wheel=(/tmp/delocated_wheel/*.whl)
 
-            # Install packages and test
-            for PYBIN in {pybin_paths}; do
                 # Install the wheel we just built
-                "$PYBIN/pip" install {package_name} \
-                    --upgrade --force-reinstall --no-deps --no-index -f /output
+                "$PYBIN/pip" install "$delocated_wheel"
 
                 # Install any requirements to run the tests
                 if [ ! -z "{test_requires}" ]; then
@@ -83,6 +92,9 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
                     PATH=$PYBIN:$PATH sh -c {test_command}
                     popd
                 fi
+
+                # we're all done here; move it to output
+                mv "$delocated_wheel" /output
             done
         '''.format(
             package_name=package_name,
@@ -107,7 +119,11 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
                 '/bin/bash'],
             stdin=subprocess.PIPE, universal_newlines=True)
 
-        docker_process.communicate(bash_script)
+        try:
+            docker_process.communicate(bash_script)
+        except KeyboardInterrupt:
+            docker_process.kill()
+            docker_process.wait()
 
         if docker_process.returncode != 0:
             exit(1)

cibuildwheel/macos.py

@@ -1,5 +1,5 @@
 from __future__ import print_function
-import os, subprocess, shlex, sys
+import os, subprocess, shlex, sys, shutil
 from collections import namedtuple
 from glob import glob
 try:
@@ -53,31 +53,35 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
         shell([pip, 'install', 'wheel'], env=env)
         shell([pip, 'install', 'delocate'], env=env)
 
+        # setup dirs
+        if os.path.exists('/tmp/built_wheel'):
+            shutil.rmtree('/tmp/built_wheel')
+        os.makedirs('/tmp/built_wheel')
+        if os.path.exists('/tmp/delocated_wheel'):
+            shutil.rmtree('/tmp/delocated_wheel')
+        os.makedirs('/tmp/delocated_wheel')
+
         # run the before_build command
         if before_build:
             before_build_prepared = prepare_command(before_build, python=python, pip=pip)
             shell(shlex.split(before_build_prepared), env=env)
 
-        # install the package first to take care of dependencies
-        shell([pip, 'install', project_dir], env=env)
+        # build the wheel
+        shell([pip, 'wheel', project_dir, '-w', '/tmp/built_wheel', '--no-deps'], env=env)
+        built_wheel = glob('/tmp/built_wheel/*.whl')[0]
 
-        # build the wheel to temp dir
-        temp_wheel_dir = '/tmp/tmpwheel%s' % config.version
-        shell([pip, 'wheel', project_dir, '-w', temp_wheel_dir, '--no-deps'], env=env)
-        temp_wheel = glob(temp_wheel_dir+'/*.whl')[0]
-
-        if temp_wheel.endswith('none-any.whl'):
-            # pure python wheel - just copy to output_dir
-            shell(['cp', temp_wheel, output_dir], env=env)
+        if built_wheel.endswith('none-any.whl'):
+            # pure python wheel - just move
+            shutil.move(built_wheel, '/tmp/delocated_wheel')
         else:
             # list the dependencies
-            shell(['delocate-listdeps', temp_wheel], env=env)
+            shell(['delocate-listdeps', built_wheel], env=env)
             # rebuild the wheel with shared libraries included and place in output dir
-            shell(['delocate-wheel', '-w', output_dir, temp_wheel], env=env)
+            shell(['delocate-wheel', '-w', '/tmp/delocated_wheel', built_wheel], env=env)
+        delocated_wheel = glob('/tmp/delocated_wheel/*.whl')[0]
 
-        # now install the package from the generated wheel
-        shell([pip, 'install', package_name, '--upgrade', '--force-reinstall',
-               '--no-deps', '--no-index', '--find-links', output_dir], env=env)
+        # install the wheel
+        shell([pip, 'install', delocated_wheel], env=env)
 
         # test the wheel
         if test_requires:
@@ -89,3 +93,6 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             abs_project_dir = os.path.abspath(project_dir)
             test_command_absolute = test_command.format(project=abs_project_dir)
             shell(shlex.split(test_command_absolute), cwd=os.environ['HOME'], env=env)
+
+        # we're all done here; move it to output
+        shutil.move(delocated_wheel, output_dir)

cibuildwheel/windows.py

@@ -1,15 +1,20 @@
 from __future__ import print_function
-import os, tempfile, subprocess, sys
+import os, tempfile, subprocess, sys, shutil
 try:
     from urllib2 import urlopen
 except ImportError:
     from urllib.request import urlopen
 from collections import namedtuple
+from glob import glob
 
 from .util import prepare_command
 
 
 def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, skip):
+    # Python under AppVeyor/Windows seems to be buffering by default, giving problems interleaving subprocess call output with unflushed calls to 'print'
+    sys.stdout.flush()
+    sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
+
     # run_with_env is a cmd file that sets the right environment variables to
     run_with_env = os.path.join(tempfile.gettempdir(), 'appveyor_run_with_env.cmd')
     if not os.path.exists(run_with_env):
@@ -37,11 +42,19 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
         PythonConfiguration(version='3.6.x', arch="64", identifier='cp36-win_amd64', path='C:\Python36-x64'),
     ]
 
+    temp_dir = tempfile.mkdtemp(prefix='cibuildwheel')
+    built_wheel_dir = os.path.join(temp_dir, 'built_wheel')
+
     for config in python_configurations:
         if skip(config.identifier):
             print('cibuildwheel: Skipping build %s' % config.identifier, file=sys.stderr)
             continue
 
+        # setup dirs
+        if os.path.exists(built_wheel_dir):
+            shutil.rmtree(built_wheel_dir)
+        os.makedirs(built_wheel_dir)
+
         env = os.environ.copy()
         # set up environment variables for run_with_env
         env['PYTHON_VERSION'] = config.version
@@ -66,16 +79,12 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             before_build_prepared = prepare_command(before_build, python='python', pip='pip')
             shell([before_build_prepared], env=env)
 
-        # install the package first to take care of dependencies
-        shell(['pip', 'install', project_dir], env=env)
-
         # build the wheel
-        shell(['pip', 'wheel', project_dir, '-w', output_dir, '--no-deps'], env=env)
+        shell(['pip', 'wheel', project_dir, '-w', built_wheel_dir, '--no-deps'], env=env)
+        built_wheel = glob(built_wheel_dir+'/*.whl')[0]
 
         # install the wheel
-        shell(['pip', 'install', package_name, '--upgrade',
-               '--force-reinstall', '--no-deps', '--no-index', '-f',
-               output_dir], env=env)
+        shell(['pip', 'install', built_wheel], env=env)
 
         # test the wheel
         if test_requires:
@@ -87,3 +96,6 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
             abs_project_dir = os.path.abspath(project_dir)
             test_command_absolute = test_command.format(project=abs_project_dir)
             shell([test_command_absolute], cwd='c:\\', env=env)
+
+        # we're all done here; move it to output
+        shutil.move(built_wheel, output_dir)

setup.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.2.1
+current_version = 0.4.1
 commit = True
 tag = True
 message = Bump version

setup.py

@@ -8,7 +8,7 @@ except ImportError:
 
 setup(
     name='cibuildwheel',
-    version='0.2.1',
+    version='0.4.1',
     description="Build Python wheels on CI with minimal configuration.",
     long_description='For readme please see http://github.com/joerick/cibuildwheel',
     author="Joe Rickerby",