| Feb | MAR | Apr |
| 12 | ||
| 2020 | 2021 | 2022 |
COLLECTED BY
Collection: Corentin Barreau's Web Archives
●Python »
Python Developer's Guide »
git¶
git; this is also used to refer to git
itself. git is easily available for all common operating systems.
Install
As the CPython repo is hosted on GitHub, please refer to either the
GitHub setup instructions
or the git project instructions for step-by-step
installation directions. You may also want to consider a graphical client
such as TortoiseGitorGitHub Desktop.
Configure
Configure your name and email and create
an SSH key
as this will allow you to interact with GitHub without typing a username
and password each time you execute a command, such as git pull,
git push, or git fetch. On Windows, you should also
enable autocrlf.
https://github.com/<username>/cpython.
Clone your GitHub fork (replace <username> with your username):
$ git clone git@github.com:<username>/cpython.git
(You can use both SSH-based or HTTPS-based URLs.)
Configure an upstream remote:
$ cd cpython $ git remote add upstream git@github.com:python/cpython.gitVerify that your setup is correct:
$ git remote -v origin git@github.com:<your-username>/cpython.git (fetch) origin git@github.com:<your-username>/cpython.git (push) upstream git@github.com:python/cpython.git (fetch) upstream git@github.com:python/cpython.git (push)If you did everything correctly, you should now have a copy of the code in the
cpython directory and two remotes that refer to your own GitHub fork
(origin) and the official CPython repository (upstream).
If you want a working copy of an already-released version of Python,
i.e., a version in maintenance mode, you can checkout
a release branch. For instance, to checkout a working copy of Python 3.8,
do git checkout 3.8.
You will need to re-compile CPython when you do such an update.
Do note that CPython will notice that it is being run from a working copy.
This means that if you edit CPython’s source code in your working copy,
changes to Python code will be picked up by the interpreter for immediate
use and testing. (If you change C code, you will need to recompile the
affected files as described below.)
Patches for the documentation can be made from the same repository; see
Documenting Python.
Misc/SpecialBuilds.txt file, the most critical one is the Py_DEBUG flag
which creates what is known as a “pydebug” build. This flag turns on various
extra sanity checks which help catch common issues. The use of the flag is so
common that turning on the flag is a basic compile option.
You should always develop under a pydebug build of CPython (the only instance of
when you shouldn’t is if you are taking performance measurements). Even when
working only on pure Python code the pydebug build provides several useful
checks that one should not skip.
zlib library for compression).
Depending on what you intend to work on, you might need to install these
additional requirements so that the compiled interpreter supports the
desired features.
If you want to install these optional dependencies, consult the
Install dependencies section below.
If you don’t need to install them, the basic steps for building Python
for development is to configure it and then compile it.
Configuration is typically:
./configure --with-pydebugMore flags are available to
configure, but this is the minimum you should
do to get a pydebug build of CPython.
Note
You might need to run make clean before or after re-running configure
in a particular build directory.
Once configure is done, you can then compile CPython with:
make -s -j2This will build CPython with only warnings and errors being printed to stderr and utilize up to 2 CPU cores. If you are using a multi-core machine with more than 2 cores (or a single-core machine), you can adjust the number passed into the
-j flag to match the number of cores you have (or if your
version of Make supports it, you can use -j without a number and Make
will not limit the number of steps that can run simultaneously.).
At the end of the build you should see a success message, possibly followed
by a list of extension modules that haven’t been built because their
dependencies were missing:
Python build finished successfully! The necessary bits to build these optional modules were not found: _bz2 _dbm _gdbm _lzma _sqlite3 _ssl _tkinter _uuid readline zlib To find the necessary bits, look in setup.py in detect_modules() for the module's name.If the build failed and you are using a C89 or C99-compliant compiler, please open a bug report. If you decide to Install dependencies, you will need to re-run both
configure and make.
Once CPython is done building you will then have a working build
that can be run in-place; ./python on most machines (and what is used in
all examples), ./python.exe wherever a case-insensitive filesystem is used
(e.g. on OS X by default), in order to avoid conflicts with the Python
directory. There is normally no need to install your built copy
of Python! The interpreter will realize where it is being run from
and thus use the files found in the working copy. If you are worried
you might accidentally install your working copy build, you can add
--prefix=/tmp/python to the configuration step. When running from your
working directory, it is best to avoid using the --enable-shared flag
to configure; unless you are very careful, you may accidentally run
with code from an older, installed shared Python library rather than from
the interpreter you just built.
-Wno-unused-value -Wno-empty-body -Qunused-arguments. You can set your
CFLAGS environment variable to these flags when running configure.
If you are using clang with ccache, turn off the noisy
parentheses-equality warnings with the -Wno-parentheses-equality flag.
These warnings are caused by clang not having enough information to detect
that extraneous parentheses in expanded macros are valid, because the
preprocessing is done separately by ccache.
If you are using LLVM 2.8, also use the -no-integrated-as flag in order to
build the ctypes module (without the flag the rest of CPython will
still build properly).
PCBuild\build.batAfter this build succeeds, you can open the
PCBuild\pcbuild.sln solution in
Visual Studio to continue development.
See the readme for more details on what other software is necessary and how
to build.
Note
Python 2.7 uses Microsoft Visual Studio 2008, which is most easily
obtained through an MSDN subscription. To use the build files in the
PCbuild directory you will also need Visual Studio 2010, see the 2.7
readme for more details. If you have VS 2008 but not 2010 you can use the
build files in the PC/VS9.0 directory, see the VS9 readme for details.
Note
If you are using the Windows Subsystem for Linux (WSL), clone the
repository from a native Windows terminal program like cmd.exe command prompt
or PowerShell as well as use a build of git targeted for Windows, e.g., the
official one from https://git-scm.com. Otherwise, Visual Studio will
not be able to find all the project’s files and will fail the build.
zlib)
on Linux and macOs/OS X. On Windows,
extensions are already included and built automatically.
yum based systems:
$ sudo yum install yum-utils $ sudo yum-builddep python3OnFedora and other
DNF based systems:
$ sudo dnf install dnf-plugins-core # install this to use 'dnf builddep' $ sudo dnf builddep python3OnDebian, Ubuntu, and other
apt based systems, try to get the
dependencies for the Python you’re working on by using the apt command.
First, make sure you have enabled the source packages in the sources list.
You can do this by adding the location of the source packages, including
URL, distribution name and component name, to /etc/apt/sources.list.
Take Ubuntu Bionic for example:
deb-src http://archive.ubuntu.com/ubuntu/ bionic main
For other distributions, like Debian, change the URL and names to correspond
with the specific distribution.
Then you should update the packages index:
$ sudo apt-get update
Now you can install the build dependencies via apt:
$ sudo apt-get build-dep python3.6
If that package is not available for your system, try reducing the minor
version until you find a package that is available.
$ xcode-select --install
This will also ensure that the system header files are installed into
/usr/include.
OnMac OS X systems (versions 10.0 - 10.7) and OS X 10.8, use the C
compiler and other development utilities provided by Apple’s Xcode Developer
Tools. The Developer Tools are not shipped with Mac OS X.
For these older releases (versions 10.0 - 10.8), you will need to download either the
correct version of the Command Line Tools, if available, or install them from the
full Xcode app or package for that OS X release. Older versions may be
available either as a no-cost download through Apple’s App Store or from
the Apple Developer web site.
Also note that OS X does not include several libraries used by the Python
standard library, including libzma, so expect to see some extension module
build failures unless you install local copies of them. As of OS X 10.11,
Apple no longer provides header files for the deprecated system version of
OpenSSL which means that you will not be able to build the _ssl extension.
One solution is to install these libraries from a third-party package
manager, like HomebreworMacPorts, and then add the appropriate paths
for the header and library files to your configure command. For example,
with Homebrew:
$ brew install openssl xz gdbm
and configure Python versions >= 3.7:
./configure --with-pydebug --with-openssl=$(brew --prefix openssl)
orconfigure Python versions < 3.7:
$ CPPFLAGS="-I$(brew --prefix openssl)/include" \ LDFLAGS="-L$(brew --prefix openssl)/lib" \ ./configure --with-pydebugand
make:
$ make -s -j2
orMacPorts:
$ sudo port install pkgconfig openssl xz gdbm
and configure:
$ CPPFLAGS="-I/opt/local/include" \ LDFLAGS="-L/opt/local/lib" \ ./configure --with-pydebugand
make:
$ make -s -j2
There will sometimes be optional modules added for a new release which
won’t yet be identified in the OS level build dependencies. In those cases,
just ask for assistance on the core-mentorship list. If working on bug
fixes for Python 2.7, use python in place of python3 in the above
commands.
Explaining how to build optional dependencies on a UNIX based system without
root access is beyond the scope of this guide.
Note
While you need a C compiler to build CPython, you don’t need any
knowledge of the C language to contribute! Vast areas of CPython are
written completely in Python: as of this writing, CPython contains slightly
more Python code than C.
configure¶
configure script to test for availability of the functionality.
Python’s configure script is generated from configure.ac using Autoconf.
Instead of editing configure, edit configure.ac and then run
autoreconf to regenerate configure and a number of other files (such as
pyconfig.h).
When submitting a patch with changes made to configure.ac, you should also
include the generated files.
Note that running autoreconf is not the same as running autoconf. For
example, autoconf by itself will not regenerate pyconfig.h.in.
autoreconf runs autoconf and a number of other tools repeatedly as is
appropriate.
Python’s configure.ac script typically requires a specific version of
Autoconf. At the moment, this reads: AC_PREREQ(2.69).
If the system copy of Autoconf does not match this version, you will need to
install your own copy of Autoconf.
Parser/asdl_c.pyorPython/makeopcodetargets.py while running make.
Python auto-generates some of its own code, and a full build from scratch needs
to run the auto-generation scripts. However, this makes the Python build require
an already installed Python interpreter; this can also cause version mismatches
when trying to build an old (2.x) Python with a new (3.x) Python installed, or
vice versa.
To overcome this problem, auto-generated files are also checked into the
Git repository. So if you don’t touch the auto-generation scripts, there’s
no real need to auto-generate anything.
Doc
The official documentation. This is what https://docs.python.org/ uses.
See also Building the documentation.
Grammar
Contains the EBNF grammar file for
Python.
Include
Contains all interpreter-wide header files.
Lib
The part of the standard library implemented in pure Python.
Mac
Mac-specific code (e.g., using IDLE as an OS X application).
Misc
Things that do not belong elsewhere. Typically this is varying kinds of
developer-specific documentation.
Modules
The part of the standard library (plus some other code) that is implemented
in C.
Objects
Code for all built-in types.
PC
Windows-specific code.
PCbuild
Build files for the version of MSVC currently used for the Windows
installers provided on python.org.
Parser
Code related to the parser. The definition of the AST nodes is also kept
here.
Programs
Source code for C executables, including the main function for the
CPython interpreter (in versions prior to Python 3.5, these files are
in the Modules directory).
Python
The code that makes up the core CPython runtime. This includes the
compiler, eval loop and various built-in modules.
Tools
Various tools that are (or have been) used to maintain Python.
git
●1.2. Get the source code
●1.3. Compile and build
●1.3.1. UNIX
●1.3.1.1. Clang
●1.3.2. Windows
●1.4. Install dependencies
●1.4.1. Linux
●1.4.2. macOS and OS X
●1.5. Regenerate configure
●1.6. Troubleshoot the build
●1.6.1. Avoid recreating auto-generated files
●1.7. Editors and Tools
●1.8. Directory structure
git
●1.2. Get the source code
●1.3. Compile and build
●1.3.1. UNIX
●1.3.2. Windows
●1.4. Install dependencies
●1.4.1. Linux
●1.4.2. macOS and OS X
●1.5. Regenerate configure
●1.6. Troubleshoot the build
●1.6.1. Avoid recreating auto-generated files
●1.7. Editors and Tools
●1.8. Directory structure
●2. Where to Get Help
●3. Lifecycle of a Pull Request
●4. Running & Writing Tests
●5. Increase Test Coverage
●6. Helping with Documentation
●7. Documenting Python
●8. Silence Warnings From the Test Suite
●9. Fixing “easy” Issues (and Beyond)
●10. Issue Tracking
●11. Triaging an Issue
●12. Following Python’s Development
●13. Porting Python to a new platform
●14. How to Become a Core Developer
●15. Developer Log
●16. Accepting Pull Requests
●17. Development Cycle
●18. Continuous Integration
●19. Adding to the Stdlib
●20. Changing the Python Language
●21. Experts Index
●22. gdb Support
●23. Exploring CPython’s Internals
●24. Changing CPython’s Grammar
●25. Design of CPython’s Compiler
●26. Design of CPython’s Garbage Collector
●27. Updating standard library extension modules
●28. Coverity Scan
●29. Dynamic Analysis with Clang
●30. Running a buildbot worker
●31. Core Developer Motivations and Affiliations
●32. Git Bootcamp and Cheat Sheet
●33. Appendix: Topics
●Python »
Python Developer's Guide »