Python programs with one simple file can be divided into two parts. One is the runtime provided by python and its modules. The other is the logic part provided by the python script called from console.
So developing python c extensions involves a python environment and a compiling environment. We compile the python c extension in compiling environment and run python script to import the compiled c extension.
As for the compiling environment, there're multiple ways to get a compiling environment in windows. You can use msvc in visual studio. You can use wsl, which basically is Linux. Or you can use msys, cygwin etc.
In this tutorial, we go with following environment.
- cpython v3.7.6 compiled from source code
- msvc in visual studio 2019
- run %CPYTHON_ROOT%\PCBuild\get_externals.bat
- open %CPYTHON_ROOT%\PCBuild/pcbuild.sln with visual studio
- build with both Release and Debug configuration
For more details, you can follow the official guide.
After compiling, you will find the pyds and libs in %CPYTHON_ROOT%\PCBuild\win32\.
Please pay attention the output of Debug build has a "_d" suffix. If you search "_DEBUG" in cpython, you will find multiple results.
So be aware if you run python in Debug mode, it will try to load module_d.pyd. If you run python in Release mode it will try to load module.pyd.
If python raise an import exception that it cannot find the module, please make sure the directory containing the pyd file is in sys.path and you are running python in correct mode.
Compile the c extension
Just follow this guide, you can get your code running easily.
Since we are using a self built cpython, some differences must be noticed.
- Use %CPYTHON_ROOT%/include instead of the include folder of official python distribution.
- USE %CPYTHON_ROOT%/PCBuild/win32 instead of the libs folder of official python distirbution.
- Remember to add %CPYTHON_ROOT%/PCBuild/win32 to the search path of your python project, since all pyd and symbols are there.
- Choose your self built python environment instead of some official python environment.
- If python raises some import exception, you can run the python with -vv argument. It will provide more information about search path when loading a pyd module.
Debug your extension
It is pretty straightforward if you have run your code in Debug mode. Add breakpoint, call that method or function in python script and debugger will break.
Here's one example extension for demenstration.