Python初始化,终结和线程

python lib

python-initialization">

在Python初始化之前¶

在一个植入了 Python 的应用程序中,Py_Initialize() 函数必须在任何其他 Python/C API 函数之前被调用;例外的只有个别函数和 全局配置变量

在初始化Python之前,可以安全地调用以下函数:

  • 配置函数:

    • PyImport_AppendInittab()

    • PyImport_ExtendInittab()

    • PyInitFrozenExtensions()

    • PyMem_SetAllocator()

    • PyMem_SetupDebugHooks()

    • PyObject_SetArenaAllocator()

    • Py_SetPath()

    • Py_SetProgramName()

    • Py_SetPythonHome()

    • Py_SetStandardStreamEncoding()

    • PySys_AddWarnOption()

    • PySys_AddXOption()

    • PySys_ResetWarnOptions()

  • 信息函数:

    • Py_IsInitialized()

    • PyMem_GetAllocator()

    • PyObject_GetArenaAllocator()

    • Py_GetBuildInfo()

    • Py_GetCompiler()

    • Py_GetCopyright()

    • Py_GetPlatform()

    • Py_GetVersion()

  • 公用

    • Py_DecodeLocale()

  • 内存分配器:

    • PyMem_RawMalloc()

    • PyMem_RawRealloc()

    • PyMem_RawCalloc()

    • PyMem_RawFree()

注解

以下函数 不应该 在 Py_Initialize(): Py_EncodeLocale(), Py_GetPath(), Py_GetPrefix(), Py_GetExecPrefix(), Py_GetProgramFullPath(), Py_GetPythonHome(), Py_GetProgramName()PyEval_InitThreads() 前调用。

全局配置变量¶

Python 有负责控制全局配置中不同特性和选项的变量。这些标志默认被 命令行选项

当一个选项设置一个旗标时,该旗标的值将是设置选项的次数。 例如,-b 会将 Py_BytesWarningFlag 设为 1 而 -bb 会将 Py_BytesWarningFlag 设为 2.

Py_BytesWarningFlag

Issue a warning when comparing bytes or bytearray with

str or bytes with int. Issue an error if greater

or equal to 2.

-b 选项设置。

Py_DebugFlag

Turn on parser debugging output (for expert only, depending on compilation

options).

Set by the -d option and the PYTHONDEBUG environment

variable.

Py_DontWriteBytecodeFlag

If set to non-zero, Python won't try to write .pyc files on the

import of source modules.

Set by the -B option and the PYTHONDONTWRITEBYTECODE

environment variable.

Py_FrozenFlag

Suppress error messages when calculating the module search path in

Py_GetPath().

Private flag used by _freeze_importlib and frozenmain programs.

Py_HashRandomizationFlag

Set to 1 if the PYTHONHASHSEED environment variable is set to

a non-empty string.

If the flag is non-zero, read the PYTHONHASHSEED environment

variable to initialize the secret hash seed.

Py_IgnoreEnvironmentFlag

忽略所有 PYTHON* 环境变量,例如可能已设置的 PYTHONPATHPYTHONHOME

-E-I 选项设置。

Py_InspectFlag

When a script is passed as first argument or the -c option is used,

enter interactive mode after executing the script or the command, even when

sys.stdin does not appear to be a terminal.

Set by the -i option and the PYTHONINSPECT environment

variable.

Py_InteractiveFlag

-i 选项设置。

Py_IsolatedFlag

Run Python in isolated mode. In isolated mode sys.path contains

neither the script's directory nor the user's site-packages directory.

-I 选项设置。

3.4 新版功能.

Py_LegacyWindowsFSEncodingFlag

If the flag is non-zero, use the mbcs encoding instead of the UTF-8

encoding for the filesystem encoding.

Set to 1 if the PYTHONLEGACYWINDOWSFSENCODING environment

variable is set to a non-empty string.

有关更多详细信息,请参阅 PEP 529。

可用性: Windows。

Py_LegacyWindowsStdioFlag

If the flag is non-zero, use io.FileIO instead of

WindowsConsoleIO for sys standard streams.

Set to 1 if the PYTHONLEGACYWINDOWSSTDIO environment

variable is set to a non-empty string.

有关更多详细信息,请参阅 PEP 528。

可用性: Windows。

Py_NoSiteFlag

禁用 site 的导入及其所附带的基于站点对 sys.path 的操作。 如果 site 会在稍后被显式地导入也会禁用这些操作 (如果你希望触发它们则应调用 site.main())。

-S 选项设置。

Py_NoUserSiteDirectory

不要将 用户site-packages目录 添加到 sys.path

Set by the -s and -I options, and the

PYTHONNOUSERSITE environment variable.

Py_OptimizeFlag

Set by the -O option and the PYTHONOPTIMIZE environment

variable.

Py_QuietFlag

即使在交互模式下也不显示版权和版本信息。

-q 选项设置。

3.2 新版功能.

Py_UnbufferedStdioFlag

Force the stdout and stderr streams to be unbuffered.

Set by the -u option and the PYTHONUNBUFFERED

environment variable.

Py_VerboseFlag

Print a message each time a module is initialized, showing the place

(filename or built-in module) from which it is loaded. If greater or equal

to 2, print a message for each file that is checked for when

searching for a module. Also provides information on module cleanup at exit.

Set by the -v option and the PYTHONVERBOSE environment

variable.

Initializing and finalizing the interpreter¶

void Py_Initialize()

Initialize the Python interpreter. In an application embedding Python,

this should be called before using any other Python/C API functions; see

Before Python Initialization for the few exceptions.

This initializes

the table of loaded modules (sys.modules), and creates the fundamental

modules builtins, __main__ and sys. It also initializes

the module search path (sys.path). It does not set sys.argv; use

PySys_SetArgvEx() for that. This is a no-op when called for a second time

(without calling Py_FinalizeEx() first). There is no return value; it is a

fatal error if the initialization fails.

注解

On Windows, changes the console mode from O_TEXT to O_BINARY, which will

also affect non-Python uses of the console using the C Runtime.

void Py_InitializeEx(int initsigs)

This function works like Py_Initialize() if initsigs is 1. If

initsigs is 0, it skips initialization registration of signal handlers, which

might be useful when Python is embedded.

int Py_IsInitialized()

Return true (nonzero) when the Python interpreter has been initialized, false

(zero) if not. After Py_FinalizeEx() is called, this returns false until

Py_Initialize() is called again.

int Py_FinalizeEx()

Undo all initializations made by Py_Initialize() and subsequent use of

Python/C API functions, and destroy all sub-interpreters (see

Py_NewInterpreter() below) that were created and not yet destroyed since

the last call to Py_Initialize(). Ideally, this frees all memory

allocated by the Python interpreter. This is a no-op when called for a second

time (without calling Py_Initialize() again first). Normally the

return value is 0. If there were errors during finalization

(flushing buffered data), -1 is returned.

This function is provided for a number of reasons. An embedding application

might want to restart Python without having to restart the application itself.

An application that has loaded the Python interpreter from a dynamically

loadable library (or DLL) might want to free all memory allocated by Python

before unloading the DLL. During a hunt for memory leaks in an application a

developer might want to free all memory allocated by Python before exiting from

the application.

Bugs and caveats: The destruction of modules and objects in modules is done

in random order; this may cause destructors (__del__() methods) to fail

when they depend on other objects (even functions) or modules. Dynamically

loaded extension modules loaded by Python are not unloaded. Small amounts of

memory allocated by the Python interpreter may not be freed (if you find a leak,

please report it). Memory tied up in circular references between objects is not

freed. Some memory allocated by extension modules may not be freed. Some

extensions may not work properly if their initialization routine is called more

than once; this can happen if an application calls Py_Initialize() and

Py_FinalizeEx() more than once.

3.6 新版功能.

void Py_Finalize()

This is a backwards-compatible version of Py_FinalizeEx() that

disregards the return value.

Process-wide parameters¶

int Py_SetStandardStreamEncoding(const char *encoding, const char *errors)

This function should be called before Py_Initialize(), if it is

called at all. It specifies which encoding and error handling to use

with standard IO, with the same meanings as in str.encode().

It overrides PYTHONIOENCODING values, and allows embedding code

to control IO encoding when the environment variable does not work.

encoding and/or errors may be NULL to use

PYTHONIOENCODING and/or default values (depending on other

settings).

Note that sys.stderr always uses the "backslashreplace" error

handler, regardless of this (or any other) setting.

If Py_FinalizeEx() is called, this function will need to be called

again in order to affect subsequent calls to Py_Initialize().

Returns 0 if successful, a nonzero value on error (e.g. calling after the

interpreter has already been initialized).

3.4 新版功能.

void Py_SetProgramName(const wchar_t *name)

This function should be called before Py_Initialize() is called for

the first time, if it is called at all. It tells the interpreter the value

of the argv[0] argument to the main() function of the program

(converted to wide characters).

This is used by Py_GetPath() and some other functions below to find

the Python run-time libraries relative to the interpreter executable. The

default value is 'python'. The argument should point to a

zero-terminated wide character string in static storage whose contents will not

change for the duration of the program's execution. No code in the Python

interpreter will change the contents of this storage.

Use Py_DecodeLocale() to decode a bytes string to get a

wchar_* string.

wchar* Py_GetProgramName()

Return the program name set with Py_SetProgramName(), or the default.

The returned string points into static storage; the caller should not modify its

value.

wchar_t* Py_GetPrefix()

Return the prefix for installed platform-independent files. This is derived

through a number of complicated rules from the program name set with

Py_SetProgramName() and some environment variables; for example, if the

program name is '/usr/local/bin/python', the prefix is '/usr/local'. The

returned string points into static storage; the caller should not modify its

value. This corresponds to the prefix variable in the top-level

Makefile and the --prefix argument to the configure

script at build time. The value is available to Python code as sys.prefix.

It is only useful on Unix. See also the next function.

wchar_t* Py_GetExecPrefix()

Return the exec-prefix for installed platform-dependent files. This is

derived through a number of complicated rules from the program name set with

Py_SetProgramName() and some environment variables; for example, if the

program name is '/usr/local/bin/python', the exec-prefix is

'/usr/local'. The returned string points into static storage; the caller

should not modify its value. This corresponds to the exec_prefix

variable in the top-level Makefile and the --exec-prefix

argument to the configure script at build time. The value is

available to Python code as sys.exec_prefix. It is only useful on Unix.

Background: The exec-prefix differs from the prefix when platform dependent

files (such as executables and shared libraries) are installed in a different

directory tree. In a typical installation, platform dependent files may be

installed in the /usr/local/plat subtree while platform independent may

be installed in /usr/local.

Generally speaking, a platform is a combination of hardware and software

families, e.g. Sparc machines running the Solaris 2.x operating system are

considered the same platform, but Intel machines running Solaris 2.x are another

platform, and Intel machines running Linux are yet another platform. Different

major revisions of the same operating system generally also form different

platforms. Non-Unix operating systems are a different story; the installation

strategies on those systems are so different that the prefix and exec-prefix are

meaningless, and set to the empty string. Note that compiled Python bytecode

files are platform independent (but not independent from the Python version by

which they were compiled!).

System administrators will know how to configure the mount or

automount programs to share /usr/local between platforms

while having /usr/local/plat be a different filesystem for each

platform.

wchar_t* Py_GetProgramFullPath()

Return the full program name of the Python executable; this is computed as a

side-effect of deriving the default module search path from the program name

(set by Py_SetProgramName() above). The returned string points into

static storage; the caller should not modify its value. The value is available

to Python code as sys.executable.

wchar_t* Py_GetPath()

Return the default module search path; this is computed from the program name

(set by Py_SetProgramName() above) and some environment variables.

The returned string consists of a series of directory names separated by a

platform dependent delimiter character. The delimiter character is ':'

on Unix and Mac OS X, ';' on Windows. The returned string points into

static storage; the caller should not modify its value. The list

sys.path is initialized with this value on interpreter startup; it

can be (and usually is) modified later to change the search path for loading

modules.

void Py_SetPath(const wchar_t *)

Set the default module search path. If this function is called before

Py_Initialize(), then Py_GetPath() won't attempt to compute a

default search path but uses the one provided instead. This is useful if

Python is embedded by an application that has full knowledge of the location

of all modules. The path components should be separated by the platform

dependent delimiter character, which is ':' on Unix and Mac OS X, ';'

on Windows.

This also causes sys.executable to be set only to the raw program

name (see Py_SetProgramName()) and for sys.prefix and

sys.exec_prefix to be empty. It is up to the caller to modify these

if required after calling Py_Initialize().

Use Py_DecodeLocale() to decode a bytes string to get a

wchar_* string.

The path argument is copied internally, so the caller may free it after the

call completes.

const char* Py_GetVersion()

Return the version of this Python interpreter. This is a string that looks

something like

"3.0a5+ (py3k:63103M, May 12 2008, 00:53:55) \n[GCC 4.2.3]"

The first word (up to the first space character) is the current Python version;

the first three characters are the major and minor version separated by a

period. The returned string points into static storage; the caller should not

modify its value. The value is available to Python code as sys.version.

const char* Py_GetPlatform()

Return the platform identifier for the current platform. On Unix, this is

formed from the "official" name of the operating system, converted to lower

case, followed by the major revision number; e.g., for Solaris 2.x, which is

also known as SunOS 5.x, the value is 'sunos5'. On Mac OS X, it is

'darwin'. On Windows, it is 'win'. The returned string points into

static storage; the caller should not modify its value. The value is available

to Python code as sys.platform.

const char* Py_GetCopyright()

Return the official copyright string for the current Python version, for example

'Copyright1991-1995StichtingMathematischCentrum,Amsterdam'

The returned string points into static storage; the caller should not modify its

value. The value is available to Python code as sys.copyright.

const char* Py_GetCompiler()

Return an indication of the compiler used to build the current Python version,

in square brackets, for example:

"[GCC 2.7.2.2]"

The returned string points into static storage; the caller should not modify its

value. The value is available to Python code as part of the variable

sys.version.

const char* Py_GetBuildInfo()

Return information about the sequence number and build date and time of the

current Python interpreter instance, for example

"#67, Aug  1 1997, 22:34:28"

The returned string points into static storage; the caller should not modify its

value. The value is available to Python code as part of the variable

sys.version.

void PySys_SetArgvEx(int argc, wchar_t **argv, int updatepath)

Set sys.argv based on argc and argv. These parameters are

similar to those passed to the program's main() function with the

difference that the first entry should refer to the script file to be

executed rather than the executable hosting the Python interpreter. If there

isn't a script that will be run, the first entry in argv can be an empty

string. If this function fails to initialize sys.argv, a fatal

condition is signalled using Py_FatalError().

If updatepath is zero, this is all the function does. If updatepath

is non-zero, the function also modifies sys.path according to the

following algorithm:

  • If the name of an existing script is passed in argv[0], the absolute

    path of the directory where the script is located is prepended to

    sys.path.

  • Otherwise (that is, if argc is 0 or argv[0] doesn't point

    to an existing file name), an empty string is prepended to

    sys.path, which is the same as prepending the current working

    directory (".").

Use Py_DecodeLocale() to decode a bytes string to get a

wchar_* string.

注解

It is recommended that applications embedding the Python interpreter

for purposes other than executing a single script pass 0 as updatepath,

and update sys.path themselves if desired.

See CVE-2008-5983.

On versions before 3.1.3, you can achieve the same effect by manually

popping the first sys.path element after having called

PySys_SetArgv(), for example using:

PyRun_SimpleString("import sys; sys.path.pop(0)\n");

3.1.3 新版功能.

void PySys_SetArgv(int argc, wchar_t **argv)

This function works like PySys_SetArgvEx() with updatepath set

to 1 unless the python interpreter was started with the

-I.

Use Py_DecodeLocale() to decode a bytes string to get a

wchar_* string.

在 3.4 版更改: The updatepath value depends on -I.

void Py_SetPythonHome(const wchar_t *home)

Set the default "home" directory, that is, the location of the standard

Python libraries. See PYTHONHOME for the meaning of the

argument string.

The argument should point to a zero-terminated character string in static

storage whose contents will not change for the duration of the program's

execution. No code in the Python interpreter will change the contents of

this storage.

Use Py_DecodeLocale() to decode a bytes string to get a

wchar_* string.

w_char* Py_GetPythonHome()

Return the default "home", that is, the value set by a previous call to

Py_SetPythonHome(), or the value of the PYTHONHOME

environment variable if it is set.

Thread State and the Global Interpreter Lock¶

The Python interpreter is not fully thread-safe. In order to support

multi-threaded Python programs, there's a global lock, called the global

interpreter lock or GIL, that must be held by the current thread before

it can safely access Python objects. Without the lock, even the simplest

operations could cause problems in a multi-threaded program: for example, when

two threads simultaneously increment the reference count of the same object, the

reference count could end up being incremented only once instead of twice.

Therefore, the rule exists that only the thread that has acquired the

GIL may operate on Python objects or call Python/C API functions.

In order to emulate concurrency of execution, the interpreter regularly

tries to switch threads (see sys.setswitchinterval()). The lock is also

released around potentially blocking I/O operations like reading or writing

a file, so that other Python threads can run in the meantime.

The Python interpreter keeps some thread-specific bookkeeping information

inside a data structure called PyThreadState. There's also one

global variable pointing to the current PyThreadState: it can

be retrieved using PyThreadState_Get().

Releasing the GIL from extension code¶

Most extension code manipulating the GIL has the following simple

structure:

Savethethreadstateinalocalvariable.

Releasetheglobalinterpreterlock.

...DosomeblockingI/Ooperation...

Reacquiretheglobalinterpreterlock.

Restorethethreadstatefromthelocalvariable.

This is so common that a pair of macros exists to simplify it:

Py_BEGIN_ALLOW_THREADS

...DosomeblockingI/Ooperation...

Py_END_ALLOW_THREADS

The Py_BEGIN_ALLOW_THREADS macro opens a new block and declares a

hidden local variable; the Py_END_ALLOW_THREADS macro closes the

block.

The block above expands to the following code:

PyThreadState*_save;

_save=PyEval_SaveThread();

...DosomeblockingI/Ooperation...

PyEval_RestoreThread(_save);

Here is how these functions work: the global interpreter lock is used to protect the pointer to the

current thread state. When releasing the lock and saving the thread state,

the current thread state pointer must be retrieved before the lock is released

(since another thread could immediately acquire the lock and store its own thread

state in the global variable). Conversely, when acquiring the lock and restoring

the thread state, the lock must be acquired before storing the thread state

pointer.

注解

Calling system I/O functions is the most common use case for releasing

the GIL, but it can also be useful before calling long-running computations

which don't need access to Python objects, such as compression or

cryptographic functions operating over memory buffers. For example, the

standard zlib and hashlib modules release the GIL when

compressing or hashing data.

非Python创建的线程¶

When threads are created using the dedicated Python APIs (such as the

threading module), a thread state is automatically associated to them

and the code showed above is therefore correct. However, when threads are

created from C (for example by a third-party library with its own thread

management), they don't hold the GIL, nor is there a thread state structure

for them.

If you need to call Python code from these threads (often this will be part

of a callback API provided by the aforementioned third-party library),

you must first register these threads with the interpreter by

creating a thread state data structure, then acquiring the GIL, and finally

storing their thread state pointer, before you can start using the Python/C

API. When you are done, you should reset the thread state pointer, release

the GIL, and finally free the thread state data structure.

The PyGILState_Ensure() and PyGILState_Release() functions do

all of the above automatically. The typical idiom for calling into Python

from a C thread is:

PyGILState_STATEgstate;

gstate=PyGILState_Ensure();

/* Perform Python actions here. */

result=CallSomeFunction();

/* evaluate result or handle exception */

/* Release the thread. No Python API allowed beyond this point. */

PyGILState_Release(gstate);

Note that the PyGILState_*() functions assume there is only one global

interpreter (created automatically by Py_Initialize()). Python

supports the creation of additional interpreters (using

Py_NewInterpreter()), but mixing multiple interpreters and the

PyGILState_*() API is unsupported.

Another important thing to note about threads is their behaviour in the face

of the C fork() call. On most systems with fork(), after a

process forks only the thread that issued the fork will exist. That also

means any locks held by other threads will never be released. Python solves

this for os.fork() by acquiring the locks it uses internally before

the fork, and releasing them afterwards. In addition, it resets any

锁对象 in the child. When extending or embedding Python, there

is no way to inform Python of additional (non-Python) locks that need to be

acquired before or reset after a fork. OS facilities such as

pthread_atfork() would need to be used to accomplish the same thing.

Additionally, when extending or embedding Python, calling fork()

directly rather than through os.fork() (and returning to or calling

into Python) may result in a deadlock by one of Python's internal locks

being held by a thread that is defunct after the fork.

PyOS_AfterFork_Child() tries to reset the necessary locks, but is not

always able to.

高阶 API¶

These are the most commonly used types and functions when writing C extension

code, or when embedding the Python interpreter:

PyInterpreterState

This data structure represents the state shared by a number of cooperating

threads. Threads belonging to the same interpreter share their module

administration and a few other internal items. There are no public members in

this structure.

Threads belonging to different interpreters initially share nothing, except

process state like available memory, open file descriptors and such. The global

interpreter lock is also shared by all threads, regardless of to which

interpreter they belong.

PyThreadState

This data structure represents the state of a single thread. The only public

data member is PyInterpreterState*interp, which points to

this thread's interpreter state.

void PyEval_InitThreads()

Initialize and acquire the global interpreter lock. It should be called in the

main thread before creating a second thread or engaging in any other thread

operations such as PyEval_ReleaseThread(tstate). It is not needed before

calling PyEval_SaveThread() or PyEval_RestoreThread().

This is a no-op when called for a second time.

在 3.7 版更改: This function is now called by Py_Initialize(), so you don't

have to call it yourself anymore.

在 3.2 版更改: This function cannot be called before Py_Initialize() anymore.

int PyEval_ThreadsInitialized()

Returns a non-zero value if PyEval_InitThreads() has been called. This

function can be called without holding the GIL, and therefore can be used to

avoid calls to the locking API when running single-threaded.

在 3.7 版更改: The GIL is now initialized by Py_Initialize().

PyThreadState* PyEval_SaveThread()

Release the global interpreter lock (if it has been created and thread

support is enabled) and reset the thread state to NULL, returning the

previous thread state (which is not NULL). If the lock has been created,

the current thread must have acquired it.

void PyEval_RestoreThread(PyThreadState *tstate)

Acquire the global interpreter lock (if it has been created and thread

support is enabled) and set the thread state to tstate, which must not be

NULL. If the lock has been created, the current thread must not have

acquired it, otherwise deadlock ensues.

注解

Calling this function from a thread when the runtime is finalizing

will terminate the thread, even if the thread was not created by Python.

You can use _Py_IsFinalizing() or sys.is_finalizing() to

check if the interpreter is in process of being finalized before calling

this function to avoid unwanted termination.

PyThreadState* PyThreadState_Get()

Return the current thread state. The global interpreter lock must be held.

When the current thread state is NULL, this issues a fatal error (so that

the caller needn't check for NULL).

PyThreadState* PyThreadState_Swap(PyThreadState *tstate)

Swap the current thread state with the thread state given by the argument

tstate, which may be NULL. The global interpreter lock must be held

and is not released.

void PyEval_ReInitThreads()

This function is called from PyOS_AfterFork_Child() to ensure

that newly created child processes don't hold locks referring to threads

which are not running in the child process.

The following functions use thread-local storage, and are not compatible

with sub-interpreters:

PyGILState_STATE PyGILState_Ensure()

Ensure that the current thread is ready to call the Python C API regardless

of the current state of Python, or of the global interpreter lock. This may

be called as many times as desired by a thread as long as each call is

matched with a call to PyGILState_Release(). In general, other

thread-related APIs may be used between PyGILState_Ensure() and

PyGILState_Release() calls as long as the thread state is restored to

its previous state before the Release(). For example, normal usage of the

Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros is

acceptable.

The return value is an opaque "handle" to the thread state when

PyGILState_Ensure() was called, and must be passed to

PyGILState_Release() to ensure Python is left in the same state. Even

though recursive calls are allowed, these handles cannot be shared - each

unique call to PyGILState_Ensure() must save the handle for its call

to PyGILState_Release().

When the function returns, the current thread will hold the GIL and be able

to call arbitrary Python code. Failure is a fatal error.

注解

Calling this function from a thread when the runtime is finalizing

will terminate the thread, even if the thread was not created by Python.

You can use _Py_IsFinalizing() or sys.is_finalizing() to

check if the interpreter is in process of being finalized before calling

this function to avoid unwanted termination.

void PyGILState_Release(PyGILState_STATE)

Release any resources previously acquired. After this call, Python's state will

be the same as it was prior to the corresponding PyGILState_Ensure() call

(but generally this state will be unknown to the caller, hence the use of the

GILState API).

Every call to PyGILState_Ensure() must be matched by a call to

PyGILState_Release() on the same thread.

PyThreadState* PyGILState_GetThisThreadState()

Get the current thread state for this thread. May return NULL if no

GILState API has been used on the current thread. Note that the main thread

always has such a thread-state, even if no auto-thread-state call has been

made on the main thread. This is mainly a helper/diagnostic function.

int PyGILState_Check()

Return 1 if the current thread is holding the GIL and 0 otherwise.

This function can be called from any thread at any time.

Only if it has had its Python thread state initialized and currently is

holding the GIL will it return 1.

This is mainly a helper/diagnostic function. It can be useful

for example in callback contexts or memory allocation functions when

knowing that the GIL is locked can allow the caller to perform sensitive

actions or otherwise behave differently.

3.4 新版功能.

The following macros are normally used without a trailing semicolon; look for

example usage in the Python source distribution.

Py_BEGIN_ALLOW_THREADS

This macro expands to {PyThreadState*_save;_save=PyEval_SaveThread();.

Note that it contains an opening brace; it must be matched with a following

Py_END_ALLOW_THREADS macro. See above for further discussion of this

macro.

Py_END_ALLOW_THREADS

此宏扩展为 PyEval_RestoreThread(_save);}。 注意它包含一个右花括号;它必须与之前的 Py_BEGIN_ALLOW_THREADS 宏匹配。 请参阅上文以进一步讨论此宏。

Py_BLOCK_THREADS

This macro expands to PyEval_RestoreThread(_save);: it is equivalent to

Py_END_ALLOW_THREADS without the closing brace.

Py_UNBLOCK_THREADS

This macro expands to _save=PyEval_SaveThread();: it is equivalent to

Py_BEGIN_ALLOW_THREADS without the opening brace and variable

declaration.

Low-level API¶

All of the following functions must be called after Py_Initialize().

在 3.7 版更改: Py_Initialize() now initializes the GIL.

PyInterpreterState* PyInterpreterState_New()

Create a new interpreter state object. The global interpreter lock need not

be held, but may be held if it is necessary to serialize calls to this

function.

void PyInterpreterState_Clear(PyInterpreterState *interp)

Reset all information in an interpreter state object. The global interpreter

lock must be held.

void PyInterpreterState_Delete(PyInterpreterState *interp)

Destroy an interpreter state object. The global interpreter lock need not be

held. The interpreter state must have been reset with a previous call to

PyInterpreterState_Clear().

PyThreadState* PyThreadState_New(PyInterpreterState *interp)

Create a new thread state object belonging to the given interpreter object.

The global interpreter lock need not be held, but may be held if it is

necessary to serialize calls to this function.

void PyThreadState_Clear(PyThreadState *tstate)

Reset all information in a thread state object. The global interpreter lock

must be held.

void PyThreadState_Delete(PyThreadState *tstate)

Destroy a thread state object. The global interpreter lock need not be held.

The thread state must have been reset with a previous call to

PyThreadState_Clear().

PY_INT64_T PyInterpreterState_GetID(PyInterpreterState *interp)

Return the interpreter's unique ID. If there was any error in doing

so then -1 is returned and an error is set.

3.7 新版功能.

PyObject* PyThreadState_GetDict()

Return value: Borrowed reference.

Return a dictionary in which extensions can store thread-specific state

information. Each extension should use a unique key to use to store state in

the dictionary. It is okay to call this function when no current thread state

is available. If this function returns NULL, no exception has been raised and

the caller should assume no current thread state is available.

int PyThreadState_SetAsyncExc(unsigned long id, PyObject *exc)

Asynchronously raise an exception in a thread. The id argument is the thread

id of the target thread; exc is the exception object to be raised. This

function does not steal any references to exc. To prevent naive misuse, you

must write your own C extension to call this. Must be called with the GIL held.

Returns the number of thread states modified; this is normally one, but will be

zero if the thread id isn't found. If exc is NULL, the pending

exception (if any) for the thread is cleared. This raises no exceptions.

在 3.7 版更改: The type of the id parameter changed from long to

unsignedlong.

void PyEval_AcquireThread(PyThreadState *tstate)

Acquire the global interpreter lock and set the current thread state to

tstate, which should not be NULL. The lock must have been created earlier.

If this thread already has the lock, deadlock ensues.

PyEval_RestoreThread() is a higher-level function which is always

available (even when threads have not been initialized).

void PyEval_ReleaseThread(PyThreadState *tstate)

Reset the current thread state to NULL and release the global interpreter

lock. The lock must have been created earlier and must be held by the current

thread. The tstate argument, which must not be NULL, is only used to check

that it represents the current thread state --- if it isn't, a fatal error is

reported.

PyEval_SaveThread() is a higher-level function which is always

available (even when threads have not been initialized).

void PyEval_AcquireLock()

Acquire the global interpreter lock. The lock must have been created earlier.

If this thread already has the lock, a deadlock ensues.

3.2 版后已移除: This function does not update the current thread state. Please use

PyEval_RestoreThread() or PyEval_AcquireThread()

instead.

void PyEval_ReleaseLock()

Release the global interpreter lock. The lock must have been created earlier.

3.2 版后已移除: This function does not update the current thread state. Please use

PyEval_SaveThread() or PyEval_ReleaseThread()

instead.

Sub-interpreter support¶

While in most uses, you will only embed a single Python interpreter, there

are cases where you need to create several independent interpreters in the

same process and perhaps even in the same thread. Sub-interpreters allow

you to do that. You can switch between sub-interpreters using the

PyThreadState_Swap() function. You can create and destroy them

using the following functions:

PyThreadState* Py_NewInterpreter()

Create a new sub-interpreter. This is an (almost) totally separate environment

for the execution of Python code. In particular, the new interpreter has

separate, independent versions of all imported modules, including the

fundamental modules builtins, __main__ and sys. The

table of loaded modules (sys.modules) and the module search path

(sys.path) are also separate. The new environment has no sys.argv

variable. It has new standard I/O stream file objects sys.stdin,

sys.stdout and sys.stderr (however these refer to the same underlying

file descriptors).

The return value points to the first thread state created in the new

sub-interpreter. This thread state is made in the current thread state.

Note that no actual thread is created; see the discussion of thread states

below. If creation of the new interpreter is unsuccessful, NULL is

returned; no exception is set since the exception state is stored in the

current thread state and there may not be a current thread state. (Like all

other Python/C API functions, the global interpreter lock must be held before

calling this function and is still held when it returns; however, unlike most

other Python/C API functions, there needn't be a current thread state on

entry.)

Extension modules are shared between (sub-)interpreters as follows: the first

time a particular extension is imported, it is initialized normally, and a

(shallow) copy of its module's dictionary is squirreled away. When the same

extension is imported by another (sub-)interpreter, a new module is initialized

and filled with the contents of this copy; the extension's init function is

not called. Note that this is different from what happens when an extension is

imported after the interpreter has been completely re-initialized by calling

Py_FinalizeEx() and Py_Initialize(); in that case, the extension's

initmodule function is called again.

void Py_EndInterpreter(PyThreadState *tstate)

Destroy the (sub-)interpreter represented by the given thread state. The given

thread state must be the current thread state. See the discussion of thread

states below. When the call returns, the current thread state is NULL. All

thread states associated with this interpreter are destroyed. (The global

interpreter lock must be held before calling this function and is still held

when it returns.) Py_FinalizeEx() will destroy all sub-interpreters that

haven't been explicitly destroyed at that point.

错误和警告¶

Because sub-interpreters (and the main interpreter) are part of the same

process, the insulation between them isn't perfect --- for example, using

low-level file operations like os.close() they can

(accidentally or maliciously) affect each other's open files. Because of the

way extensions are shared between (sub-)interpreters, some extensions may not

work properly; this is especially likely when the extension makes use of

(static) global variables, or when the extension manipulates its module's

dictionary after its initialization. It is possible to insert objects created

in one sub-interpreter into a namespace of another sub-interpreter; this should

be done with great care to avoid sharing user-defined functions, methods,

instances or classes between sub-interpreters, since import operations executed

by such objects may affect the wrong (sub-)interpreter's dictionary of loaded

modules.

Also note that combining this functionality with PyGILState_*() APIs

is delicate, because these APIs assume a bijection between Python thread states

and OS-level threads, an assumption broken by the presence of sub-interpreters.

It is highly recommended that you don't switch sub-interpreters between a pair

of matching PyGILState_Ensure() and PyGILState_Release() calls.

Furthermore, extensions (such as ctypes) using these APIs to allow calling

of Python code from non-Python created threads will probably be broken when using

sub-interpreters.

异步通知¶

A mechanism is provided to make asynchronous notifications to the main

interpreter thread. These notifications take the form of a function

pointer and a void pointer argument.

int Py_AddPendingCall(int (*func)(void *), void *arg)

Schedule a function to be called from the main interpreter thread. On

success, 0 is returned and func is queued for being called in the

main thread. On failure, -1 is returned without setting any exception.

When successfully queued, func will be eventually called from the

main interpreter thread with the argument arg. It will be called

asynchronously with respect to normally running Python code, but with

both these conditions met:

  • on a bytecode boundary;

  • with the main thread holding the global interpreter lock

    (func can therefore use the full C API).

func must return 0 on success, or -1 on failure with an exception

set. func won't be interrupted to perform another asynchronous

notification recursively, but it can still be interrupted to switch

threads if the global interpreter lock is released.

This function doesn't need a current thread state to run, and it doesn't

need the global interpreter lock.

警告

This is a low-level function, only useful for very special cases.

There is no guarantee that func will be called as quick as

possible. If the main thread is busy executing a system call,

func won't be called before the system call returns. This

function is generally not suitable for calling Python code from

arbitrary C threads. Instead, use the PyGILState API.

3.1 新版功能.

分析和跟踪¶

The Python interpreter provides some low-level support for attaching profiling

and execution tracing facilities. These are used for profiling, debugging, and

coverage analysis tools.

This C interface allows the profiling or tracing code to avoid the overhead of

calling through Python-level callable objects, making a direct C function call

instead. The essential attributes of the facility have not changed; the

interface allows trace functions to be installed per-thread, and the basic

events reported to the trace function are the same as had been reported to the

Python-level trace functions in previous versions.

int (*Py_tracefunc)(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg)

The type of the trace function registered using PyEval_SetProfile() and

PyEval_SetTrace(). The first parameter is the object passed to the

registration function as obj, frame is the frame object to which the event

pertains, what is one of the constants PyTrace_CALL,

PyTrace_EXCEPTION, PyTrace_LINE, PyTrace_RETURN,

PyTrace_C_CALL, PyTrace_C_EXCEPTION, PyTrace_C_RETURN,

or PyTrace_OPCODE, and arg depends on the value of what:

what 的值

arg 的含义

PyTrace_CALL

总是 Py_None.

PyTrace_EXCEPTION

sys.exc_info() 返回的异常信息。

PyTrace_LINE

总是 Py_None.

PyTrace_RETURN

Value being returned to the caller,

or NULL if caused by an exception.

PyTrace_C_CALL

正在调用函数对象。

PyTrace_C_EXCEPTION

正在调用函数对象。

PyTrace_C_RETURN

正在调用函数对象。

PyTrace_OPCODE

总是 Py_None.

int PyTrace_CALL

The value of the what parameter to a Py_tracefunc function when a new

call to a function or method is being reported, or a new entry into a generator.

Note that the creation of the iterator for a generator function is not reported

as there is no control transfer to the Python bytecode in the corresponding

frame.

int PyTrace_EXCEPTION

The value of the what parameter to a Py_tracefunc function when an

exception has been raised. The callback function is called with this value for

what when after any bytecode is processed after which the exception becomes

set within the frame being executed. The effect of this is that as exception

propagation causes the Python stack to unwind, the callback is called upon

return to each frame as the exception propagates. Only trace functions receives

these events; they are not needed by the profiler.

int PyTrace_LINE

The value passed as the what parameter to a Py_tracefunc function

(but not a profiling function) when a line-number event is being reported.

It may be disabled for a frame by setting f_trace_lines to 0 on that frame.

int PyTrace_RETURN

The value for the what parameter to Py_tracefunc functions when a

call is about to return.

int PyTrace_C_CALL

The value for the what parameter to Py_tracefunc functions when a C

function is about to be called.

int PyTrace_C_EXCEPTION

The value for the what parameter to Py_tracefunc functions when a C

function has raised an exception.

int PyTrace_C_RETURN

The value for the what parameter to Py_tracefunc functions when a C

function has returned.

int PyTrace_OPCODE

The value for the what parameter to Py_tracefunc functions (but not

profiling functions) when a new opcode is about to be executed. This event is

not emitted by default: it must be explicitly requested by setting

f_trace_opcodes to 1 on the frame.

void PyEval_SetProfile(Py_tracefunc func, PyObject *obj)

Set the profiler function to func. The obj parameter is passed to the

function as its first parameter, and may be any Python object, or NULL. If

the profile function needs to maintain state, using a different value for obj

for each thread provides a convenient and thread-safe place to store it. The

profile function is called for all monitored events except PyTrace_LINE

PyTrace_OPCODE and PyTrace_EXCEPTION.

void PyEval_SetTrace(Py_tracefunc func, PyObject *obj)

Set the tracing function to func. This is similar to

PyEval_SetProfile(), except the tracing function does receive line-number

events and per-opcode events, but does not receive any event related to C function

objects being called. Any trace function registered using PyEval_SetTrace()

will not receive PyTrace_C_CALL, PyTrace_C_EXCEPTION or

PyTrace_C_RETURN as a value for the what parameter.

高级调试器支持¶

These functions are only intended to be used by advanced debugging tools.

PyInterpreterState* PyInterpreterState_Head()

Return the interpreter state object at the head of the list of all such objects.

PyInterpreterState* PyInterpreterState_Main()

Return the main interpreter state object.

PyInterpreterState* PyInterpreterState_Next(PyInterpreterState *interp)

Return the next interpreter state object after interp from the list of all

such objects.

PyThreadState * PyInterpreterState_ThreadHead(PyInterpreterState *interp)

Return the pointer to the first PyThreadState object in the list of

threads associated with the interpreter interp.

PyThreadState* PyThreadState_Next(PyThreadState *tstate)

Return the next thread state object after tstate from the list of all such

objects belonging to the same PyInterpreterState object.

Thread Local Storage Support¶

The Python interpreter provides low-level support for thread-local storage

(TLS) which wraps the underlying native TLS implementation to support the

Python-level thread local storage API (threading.local). The

CPython C level APIs are similar to those offered by pthreads and Windows:

use a thread key and functions to associate a void* value per

thread.

The GIL does not need to be held when calling these functions; they supply

their own locking.

Note that Python.h does not include the declaration of the TLS APIs,

you need to include pythread.h to use thread-local storage.

注解

None of these API functions handle memory management on behalf of the

void* values. You need to allocate and deallocate them yourself.

If the void* values happen to be PyObject*, these

functions don't do refcount operations on them either.

Thread Specific Storage (TSS) API¶

TSS API is introduced to supersede the use of the existing TLS API within the

CPython interpreter. This API uses a new type Py_tss_t instead of

int to represent thread keys.

3.7 新版功能.

参见

"A New C-API for Thread-Local Storage in CPython" (PEP 539)

Py_tss_t

This data structure represents the state of a thread key, the definition of

which may depend on the underlying TLS implementation, and it has an

internal field representing the key's initialization state. There are no

public members in this structure.

When Py_LIMITED_API is not defined, static allocation of

this type by Py_tss_NEEDS_INIT is allowed.

Py_tss_NEEDS_INIT

This macro expands to the initializer for Py_tss_t variables.

Note that this macro won't be defined with Py_LIMITED_API.

Dynamic Allocation¶

Dynamic allocation of the Py_tss_t, required in extension modules

built with Py_LIMITED_API, where static allocation of this type

is not possible due to its implementation being opaque at build time.

Py_tss_t* PyThread_tss_alloc()

Return a value which is the same state as a value initialized with

Py_tss_NEEDS_INIT, or NULL in the case of dynamic allocation

failure.

void PyThread_tss_free(Py_tss_t *key)

Free the given key allocated by PyThread_tss_alloc(), after

first calling PyThread_tss_delete() to ensure any associated

thread locals have been unassigned. This is a no-op if the key

argument is NULL.

注解

A freed key becomes a dangling pointer, you should reset the key to

NULL.

方法¶

The parameter key of these functions must not be NULL. Moreover, the

behaviors of PyThread_tss_set() and PyThread_tss_get() are

undefined if the given Py_tss_t has not been initialized by

PyThread_tss_create().

int PyThread_tss_is_created(Py_tss_t *key)

Return a non-zero value if the given Py_tss_t has been initialized

by PyThread_tss_create().

int PyThread_tss_create(Py_tss_t *key)

Return a zero value on successful initialization of a TSS key. The behavior

is undefined if the value pointed to by the key argument is not

initialized by Py_tss_NEEDS_INIT. This function can be called

repeatedly on the same key -- calling it on an already initialized key is a

no-op and immediately returns success.

void PyThread_tss_delete(Py_tss_t *key)

Destroy a TSS key to forget the values associated with the key across all

threads, and change the key's initialization state to uninitialized. A

destroyed key is able to be initialized again by

PyThread_tss_create(). This function can be called repeatedly on

the same key -- calling it on an already destroyed key is a no-op.

int PyThread_tss_set(Py_tss_t *key, void *value)

Return a zero value to indicate successfully associating a void*

value with a TSS key in the current thread. Each thread has a distinct

mapping of the key to a void* value.

void* PyThread_tss_get(Py_tss_t *key)

Return the void* value associated with a TSS key in the current

thread. This returns NULL if no value is associated with the key in the

current thread.

Thread Local Storage (TLS) API¶

3.7 版后已移除: This API is superseded by

Thread Specific Storage (TSS) API.

注解

This version of the API does not support platforms where the native TLS key

is defined in a way that cannot be safely cast to int. On such platforms,

PyThread_create_key() will return immediately with a failure status,

and the other TLS functions will all be no-ops on such platforms.

由于上面提到的兼容性问题,不应在新代码中使用此版本的API。

int PyThread_create_key()

void PyThread_delete_key(int key)

int PyThread_set_key_value(int key, void *value)

void* PyThread_get_key_value(int key)

void PyThread_delete_key_value(int key)

void PyThread_ReInitTLS()

以上是 Python初始化,终结和线程 的全部内容, 来源链接: utcz.com/z/508093.html

回到顶部