Develop Python C Extension in Windows

Python programs with one simple file can be divided into two parts. One is the runtime provided by python and its modules. The other is the logic part provided by the python script called from console.

So developing python c extensions involves a python environment and a compiling environment. We compile the python c extension in compiling environment and run python script to import the compiled c extension.

There're 2 ways to get a python environment. You can download official distributions from python.org or git clone the source code of cpython from github if you like.

As for the compiling environment, there're multiple ways to get a compiling environment in windows. You can use msvc in visual studio. You can use wsl, which basically is Linux. Or you can use msys, cygwin etc.

In this tutorial, we go with following environment.

Compile cpython

TLDR;

  • run %CPYTHON_ROOT%\PCBuild\get_externals.bat
  • open %CPYTHON_ROOT%\PCBuild/pcbuild.sln with visual studio
  • build with both Release and Debug configuration

For more details, you can follow the official guide.

After compiling, you will find the pyds and libs in %CPYTHON_ROOT%\PCBuild\win32\.

Please pay attention the output of Debug build has a "_d" suffix. If you search "_DEBUG" in cpython, you will find multiple results.

Some of them choose which executable files to run. Some of them choose which module to load at run-time.

So be aware if you run python in Debug mode, it will try to load module_d.pyd. If you run python in Release mode it will try to load module.pyd.

If python raise an import exception that it cannot find the module, please make sure the directory containing the pyd file is in sys.path and you are running python in correct mode.

Compile the c extension

Just follow this guide, you can get your code running easily.

Since we are using a self built cpython, some differences must be noticed.

  1. Use %CPYTHON_ROOT%/include instead of the include folder of official python distribution.
  2. USE %CPYTHON_ROOT%/PCBuild/win32 instead of the libs folder of official python distirbution.
  3. Remember to add %CPYTHON_ROOT%/PCBuild/win32 to the search path of your python project, since all pyd and symbols are there.
  4. Choose your self built python environment instead of some official python environment.
  5. If python raises some import exception, you can run the python with -vv argument. It will provide more information about search path when loading a pyd module.
4. use self built python environment
3. add search path

Debug your extension

It is pretty straightforward if you have run your code in Debug mode. Add breakpoint, call that method or function in python script and debugger will break.

One Example

Here's one example extension for demenstration.