PythonArgumentClinicHowTo

Symbolic default values¶

The default value you provide for a parameter can't be any arbitrary

expression. Currently the following are explicitly supported:

• Numeric constants (integer and float)

• 字符串常量

• True, False, and None

• Simple symbolic constants like sys.maxsize, which must

  start with the name of the module

In case you're curious, this is implemented in from_builtin()

in Lib/inspect.py.

(In the future, this may need to get even more elaborate,

to allow full expressions like CONSTANT-1.)

Renaming the C functions and variables generated by Argument Clinic¶

Argument Clinic automatically names the functions it generates for you.

Occasionally this may cause a problem, if the generated name collides with

the name of an existing C function. There's an easy solution: override the names

used for the C functions. Just add the keyword "as"

to your function declaration line, followed by the function name you wish to use.

Argument Clinic will use that function name for the base (generated) function,

then add "_impl" to the end and use that for the name of the impl function.

For example, if we wanted to rename the C function names generated for

pickle.Pickler.dump, it'd look like this:

/*[clinic input]
pickle.Pickler.dump as pickler_dumper
...

The base function would now be named pickler_dumper(),

and the impl function would now be named pickler_dumper_impl().

Similarly, you may have a problem where you want to give a parameter

a specific Python name, but that name may be inconvenient in C. Argument

Clinic allows you to give a parameter different names in Python and in C,

using the same "as" syntax:

/*[clinic input]
pickle.Pickler.dump
    obj: object
    file as file_obj: object
    protocol: object = NULL
    *
    fix_imports: bool = True

Here, the name used in Python (in the signature and the keywords

array) would be file, but the C variable would be named file_obj.

You can use this to rename the self parameter too!

Converting functions using PyArg_UnpackTuple¶

To convert a function parsing its arguments with PyArg_UnpackTuple(),

simply write out all the arguments, specifying each as an object. You

may specify the type argument to cast the type as appropriate. All

arguments should be marked positional-only (add a / on a line by itself

after the last argument).

Currently the generated code will use PyArg_ParseTuple(), but this

will change soon.

Optional Groups¶

Some legacy functions have a tricky approach to parsing their arguments:

they count the number of positional arguments, then use a switch statement

to call one of several different PyArg_ParseTuple() calls depending on

how many positional arguments there are. (These functions cannot accept

keyword-only arguments.) This approach was used to simulate optional

arguments back before PyArg_ParseTupleAndKeywords() was created.

While functions using this approach can often be converted to

use PyArg_ParseTupleAndKeywords(), optional arguments, and default values,

it's not always possible. Some of these legacy functions have

behaviors PyArg_ParseTupleAndKeywords() doesn't directly support.

The most obvious example is the builtin function range(), which has

an optional argument on the left side of its required argument!

Another example is curses.window.addch(), which has a group of two

arguments that must always be specified together. (The arguments are

called x and y; if you call the function passing in x,

you must also pass in y—and if you don't pass in x you may not

pass in y either.)

In any case, the goal of Argument Clinic is to support argument parsing

for all existing CPython builtins without changing their semantics.

Therefore Argument Clinic supports

this alternate approach to parsing, using what are called optional groups.

Optional groups are groups of arguments that must all be passed in together.

They can be to the left or the right of the required arguments. They

can only be used with positional-only parameters.

To specify an optional group, add a [ on a line by itself before

the parameters you wish to group together, and a ] on a line by itself

after these parameters. As an example, here's how curses.window.addch

uses optional groups to make the first two parameters and the last

parameter optional:

/*[clinic input]
curses.window.addch
    [
    x: int
      X-coordinate.
    y: int
      Y-coordinate.
    ]
    ch: object
      Character to add.
    [
    attr: long
      Attributes for the character.
    ]
    /
...

注释:

• For every optional group, one additional parameter will be passed into the

  impl function representing the group. The parameter will be an int named

  group_{direction}_{number},

  where {direction} is either right or left depending on whether the group

  is before or after the required parameters, and {number} is a monotonically

  increasing number (starting at 1) indicating how far away the group is from

  the required parameters. When the impl is called, this parameter will be set

  to zero if this group was unused, and set to non-zero if this group was used.

  (By used or unused, I mean whether or not the parameters received arguments

  in this invocation.)

• If there are no required arguments, the optional groups will behave

  as if they're to the right of the required arguments.

• In the case of ambiguity, the argument parsing code

  favors parameters on the left (before the required parameters).

• Optional groups can only contain positional-only parameters.

• Optional groups are only intended for legacy code. Please do not

  use optional groups for new code.

Using real Argument Clinic converters, instead of "legacy converters"¶

To save time, and to minimize how much you need to learn

to achieve your first port to Argument Clinic, the walkthrough above tells

you to use "legacy converters". "Legacy converters" are a convenience,

designed explicitly to make porting existing code to Argument Clinic

easier. And to be clear, their use is acceptable when porting code for

Python 3.4.

However, in the long term we probably want all our blocks to

use Argument Clinic's real syntax for converters. Why? A couple

reasons:

• The proper converters are far easier to read and clearer in their intent.

• There are some format units that are unsupported as "legacy converters",

  because they require arguments, and the legacy converter syntax doesn't

  support specifying arguments.

• In the future we may have a new argument parsing library that isn't

  restricted to what PyArg_ParseTuple() supports; this flexibility

  won't be available to parameters using legacy converters.

Therefore, if you don't mind a little extra effort, please use the normal

converters instead of legacy converters.

In a nutshell, the syntax for Argument Clinic (non-legacy) converters

looks like a Python function call. However, if there are no explicit

arguments to the function (all functions take their default values),

you may omit the parentheses. Thus bool and bool() are exactly

the same converters.

All arguments to Argument Clinic converters are keyword-only.

All Argument Clinic converters accept the following arguments:

c_default

The default value for this parameter when defined in C.

Specifically, this will be the initializer for the variable declared

in the "parse function". See the section on default values

for how to use this.

Specified as a string.

annotation

The annotation value for this parameter. Not currently supported,

because PEP 8 mandates that the Python library may not use

annotations.

In addition, some converters accept additional arguments. Here is a list

of these arguments, along with their meanings:

accept

A set of Python types (and possibly pseudo-types);

this restricts the allowable Python argument to values of these types.

(This is not a general-purpose facility; as a rule it only supports

specific lists of types as shown in the legacy converter table.)

To accept None, add NoneType to this set.

bitwise

Only supported for unsigned integers. The native integer value of this

Python argument will be written to the parameter without any range checking,

even for negative values.

converter

Only supported by the object converter. Specifies the name of a

C "converter function"

to use to convert this object to a native type.

encoding

Only supported for strings. Specifies the encoding to use when converting

this string from a Python str (Unicode) value into a C char* value.

subclass_of

Only supported for the object converter. Requires that the Python

value be a subclass of a Python type, as expressed in C.

type

Only supported for the object and self converters. Specifies

the C type that will be used to declare the variable. Default value is

"PyObject*".

zeroes

Only supported for strings. If true, embedded NUL bytes ('\\0') are

permitted inside the value. The length of the string will be passed in

to the impl function, just after the string parameter, as a parameter named

<parameter_name>_length.

Please note, not every possible combination of arguments will work.

Usually these arguments are implemented by specific PyArg_ParseTuple

format units, with specific behavior. For example, currently you cannot

call unsigned_short without also specifying bitwise=True.

Although it's perfectly reasonable to think this would work, these semantics don't

map to any existing format unit. So Argument Clinic doesn't support it. (Or, at

least, not yet.)

下表显示了传统转换器映射到实参转换器的情况。左边是传统转换器，右边是要替换它的文本。

As an example, here's our sample pickle.Pickler.dump using the proper

converter:

/*[clinic input]
pickle.Pickler.dump
    obj: object
        The object to be pickled.
    /
Write a pickled representation of obj to the open file.
[clinic start generated code]*/

Argument Clinic will show you all the converters it has

available. For each converter it'll show you all the parameters

it accepts, along with the default value for each parameter.

Just run Tools/clinic/clinic.py--converters to see the full list.

Py_buffer¶

When using the Py_buffer converter

(or the 's*', 'w*', '*y', or 'z*' legacy converters),

you must not call PyBuffer_Release() on the provided buffer.

Argument Clinic generates code that does it for you (in the parsing function).

Advanced converters¶

Remember those format units you skipped for your first

time because they were advanced? Here's how to handle those too.

The trick is, all those format units take arguments—either

conversion functions, or types, or strings specifying an encoding.

(But "legacy converters" don't support arguments. That's why we

skipped them for your first function.) The argument you specified

to the format unit is now an argument to the converter; this

argument is either converter (for O&), subclass_of (for O!),

or encoding (for all the format units that start with e).

When using subclass_of, you may also want to use the other

custom argument for object(): type, which lets you set the type

actually used for the parameter. For example, if you want to ensure

that the object is a subclass of PyUnicode_Type, you probably want

to use the converter object(type='PyUnicodeObject*',subclass_of='&PyUnicode_Type').

One possible problem with using Argument Clinic: it takes away some possible

flexibility for the format units starting with e. When writing a

PyArg_Parse call by hand, you could theoretically decide at runtime what

encoding string to pass in to PyArg_ParseTuple(). But now this string must

be hard-coded at Argument-Clinic-preprocessing-time. This limitation is deliberate;

it made supporting this format unit much easier, and may allow for future optimizations.

This restriction doesn't seem unreasonable; CPython itself always passes in static

hard-coded encoding strings for parameters whose format units start with e.

Parameter default values¶

Default values for parameters can be any of a number of values.

At their simplest, they can be string, int, or float literals:

foo: str = "abc"
bar: int = 123
bat: float = 45.6

They can also use any of Python's built-in constants:

yep:  bool = True
nope: bool = False
nada: object = None

There's also special support for a default value of NULL, and

for simple expressions, documented in the following sections.

The NULL default value¶

For string and object parameters, you can set them to None to indicate

that there's no default. However, that means the C variable will be

initialized to Py_None. For convenience's sakes, there's a special

value called NULL for just this reason: from Python's perspective it

behaves like a default value of None, but the C variable is initialized

with NULL.

Expressions specified as default values¶

The default value for a parameter can be more than just a literal value.

It can be an entire expression, using math operators and looking up attributes

on objects. However, this support isn't exactly simple, because of some

non-obvious semantics.

Consider the following example:

foo: Py_ssize_t = sys.maxsize - 1

sys.maxsize can have different values on different platforms. Therefore

Argument Clinic can't simply evaluate that expression locally and hard-code it

in C. So it stores the default in such a way that it will get evaluated at

runtime, when the user asks for the function's signature.

What namespace is available when the expression is evaluated? It's evaluated

in the context of the module the builtin came from. So, if your module has an

attribute called "max_widgets", you may simply use it:

foo: Py_ssize_t = max_widgets

If the symbol isn't found in the current module, it fails over to looking in

sys.modules. That's how it can find sys.maxsize for example. (Since you

don't know in advance what modules the user will load into their interpreter,

it's best to restrict yourself to modules that are preloaded by Python itself.)

Evaluating default values only at runtime means Argument Clinic can't compute

the correct equivalent C default value. So you need to tell it explicitly.

When you use an expression, you must also specify the equivalent expression

in C, using the c_default parameter to the converter:

foo: Py_ssize_t(c_default="PY_SSIZE_T_MAX - 1") = sys.maxsize - 1

Another complication: Argument Clinic can't know in advance whether or not the

expression you supply is valid. It parses it to make sure it looks legal, but

it can't actually know. You must be very careful when using expressions to

specify values that are guaranteed to be valid at runtime!

Finally, because expressions must be representable as static C values, there

are many restrictions on legal expressions. Here's a list of Python features

you're not permitted to use:

• Function calls.

• Inline if statements (3iffooelse5).

• Automatic sequence unpacking (*[1,2,3]).

• List/set/dict comprehensions and generator expressions.

• Tuple/list/set/dict literals.

Using a return converter¶

By default the impl function Argument Clinic generates for you returns PyObject*.

But your C function often computes some C type, then converts it into the PyObject*

at the last moment. Argument Clinic handles converting your inputs from Python types

into native C types—why not have it convert your return value from a native C type

into a Python type too?

That's what a "return converter" does. It changes your impl function to return

some C type, then adds code to the generated (non-impl) function to handle converting

that value into the appropriate PyObject*.

The syntax for return converters is similar to that of parameter converters.

You specify the return converter like it was a return annotation on the

function itself. Return converters behave much the same as parameter converters;

they take arguments, the arguments are all keyword-only, and if you're not changing

any of the default arguments you can omit the parentheses.

(If you use both "as"and a return converter for your function,

the "as" should come before the return converter.)

There's one additional complication when using return converters: how do you

indicate an error has occurred? Normally, a function returns a valid (non-NULL)

pointer for success, and NULL for failure. But if you use an integer return converter,

all integers are valid. How can Argument Clinic detect an error? Its solution: each return

converter implicitly looks for a special value that indicates an error. If you return

that value, and an error has been set (PyErr_Occurred() returns a true

value), then the generated code will propagate the error. Otherwise it will

encode the value you return like normal.

Currently Argument Clinic supports only a few return converters:

bool
int
unsigned int
long
unsigned int
size_t
Py_ssize_t
float
double
DecodeFSDefault

None of these take parameters. For the first three, return -1 to indicate

error. For DecodeFSDefault, the return type is constchar*; return a NULL

pointer to indicate an error.

(There's also an experimental NoneType converter, which lets you

return Py_None on success or NULL on failure, without having

to increment the reference count on Py_None. I'm not sure it adds

enough clarity to be worth using.)

To see all the return converters Argument Clinic supports, along with

their parameters (if any),

just run Tools/clinic/clinic.py--converters for the full list.

Cloning existing functions¶

If you have a number of functions that look similar, you may be able to

use Clinic's "clone" feature. When you clone an existing function,

you reuse:

• its parameters, including

  • their names,

  • their converters, with all parameters,

  • their default values,

  • their per-parameter docstrings,

  • their kind (whether they're positional only,

    positional or keyword, or keyword only), and

• its return converter.

The only thing not copied from the original function is its docstring;

the syntax allows you to specify a new docstring.

Here's the syntax for cloning a function:

/*[clinic input]
module.class.new_function [as c_basename] = module.class.existing_function
Docstring for new_function goes here.
[clinic start generated code]*/

(The functions can be in different modules or classes. I wrote

module.class in the sample just to illustrate that you must

use the full path to both functions.)

Sorry, there's no syntax for partially-cloning a function, or cloning a function

then modifying it. Cloning is an all-or nothing proposition.

Also, the function you are cloning from must have been previously defined

in the current file.

Calling Python code¶

The rest of the advanced topics require you to write Python code

which lives inside your C file and modifies Argument Clinic's

runtime state. This is simple: you simply define a Python block.

A Python block uses different delimiter lines than an Argument

Clinic function block. It looks like this:

/*[python input]
# python code goes here
[python start generated code]*/

All the code inside the Python block is executed at the

time it's parsed. All text written to stdout inside the block

is redirected into the "output" after the block.

As an example, here's a Python block that adds a static integer

variable to the C code:

/*[python input]
print('static int __ignored_unused_variable__ = 0;')
[python start generated code]*/
staticint__ignored_unused_variable__=0;
/*[python checksum:...]*/

/*[clinic input]
_pickle.Pickler.dump
  self: self(type="PicklerObject *")
  obj: object
  /
Write a pickled representation of the given object to the open file.
[clinic start generated code]*/

/*[python input]
class PicklerObject_converter(self_converter):
    type = "PicklerObject *"
[python start generated code]*/
/*[clinic input]
_pickle.Pickler.dump
  self: PicklerObject
  obj: object
  /
Write a pickled representation of the given object to the open file.
[clinic start generated code]*/

/*[python input]
class ssize_t_converter(CConverter):
    type = 'Py_ssize_t'
    converter = 'ssize_t_converter'
[python start generated code]*/
/*[python end generated code: output=da39a3ee5e6b4b0d input=35521e4e733823c7]*/

/*[clinic input]
meth_o_sample
     argument: object
     /
[clinic start generated code]*/

dump <destination>

output <field> <destination>

output push
output pop
output preset <preset>

destination <name> <command> [...]

destination <name> new <type>

destination <name> clear

set line_prefix "string"
set line_suffix "string"

preserve

#ifdef HAVE_FUNCTIONNAME
staticmodule_functionname(...)
{
...
}
#endif /* HAVE_FUNCTIONNAME */

#ifdef HAVE_FUNCTIONNAME
{'functionname', ... },
#endif /* HAVE_FUNCTIONNAME */

#ifdef HAVE_FUNCTIONNAME
/*[clinic input]
module.functionname
...
[clinic start generated code]*/
staticmodule_functionname(...)
{
...
}
#endif /* HAVE_FUNCTIONNAME */

MODULE_FUNCTIONNAME_METHODDEF

#ifndef MODULE_FUNCTIONNAME_METHODDEF
#define MODULE_FUNCTIONNAME_METHODDEF
#endif /* !defined(MODULE_FUNCTIONNAME_METHODDEF) */

Warning in file "Modules/posixmodule.c" on line 12357:
Destination buffer 'buffer' not empty at end of file, emptying.

#/*[python input]
#print("def foo(): pass")
#[python start generated code]*/
deffoo():pass
#/*[python checksum:...]*/