pygenstub

PyPI version. Project license. Azure Pipelines build status.

pygenstub is a utility for generating stub files from docstrings in source files. It takes a source file as input and creates a stub file with the same base name and the .pyi extension.

If the docstring of a function includes a sig field, the value of that field will be used to generate a prototype by matching the types to the parameters in the same order. For example, for the code given below:

def foo(a, b):
    """Do foo.

    :sig: (int, str) -> None
    """

pygenstub will generate the following stub:

def foo(a: int, b: str) -> None: ...

pygenstub consists of a single module which itself contains signatures. You can see the source code and the autogenerated stub file as an example.

Getting started

pygenstub has been tested with Python 2.7, Python 3.4+, and compatible versions of PyPy. You can install the latest version using pip:

pip install pygenstub

Installation creates a script named pygenstub which can be used as follows:

pygenstub foo.py

This command will generate the file foo.pyi in the same directory as the input file. If the output file already exists, it will be overwritten.

If pygenstub is activated as a Sphinx extension (after autodoc), it will insert type comments for parameters and return values in the docstring. It will also remove the signature fields to exclude them from the output:

extensions = [
    'sphinx.ext.autodoc',
    'pygenstub'
]

As an example of the output, you can see the API documentation for pygenstub itself.

Getting help

The documentation is available on: https://pygenstub.tekir.org/

The source code can be obtained from: https://github.com/uyar/pygenstub

License

Copyright (C) 2016-2019 H. Turgut Uyar <uyar@tekir.org>

pygenstub is released under the GPL license, version 3 or later. Read the included LICENSE.txt for details.

Tip

pygenstub can be used with PyCharm file watchers to update stub files automatically when source files are modified.

Features

This document describes how pygenstub generates the stub files from the source.

Methods

Methods are handled the same way as functions except that there is no type hint for the self parameter (assuming it’s the first parameter):

code:
class Foo:
    def foo(self, a):
        """Do foo.

        :sig: (int) -> None
        """
stub:
class Foo:
    def foo(self, a: int) -> None: ...

Imported names

Imported type names in the source will be used in the stub file if needed:

code:
from x import A, B, C

def foo(a, b):
    """Do foo.

    :sig: (A, B) -> A
    """
stub:
from x import A, B

def foo(a: A, b: B) -> A: ...

Note that the name C is not imported in the stub file.

Qualified names

Qualified (dotted) type names will generate import lines in the stub file if they are not already imported:

code:
from z import x

def foo(a, b):
    """Do foo.

    :sig: (x.A, y.B) -> m.n.C
    """
stub:
from z import x
import y
import m.n

def foo(a: x.A, b: y.B) -> m.n.C: ...

Names from the typing module

Unresolved names will be looked up in the typing module.

code:
def foo(a, b):
    """Do foo.

    :sig: (List[int], Mapping[str, int]) -> Iterable[str]
    """
stub:
from typing import Iterable, List, Mapping

def foo(a: List[int], b: Mapping[str, int]) -> Iterable[str]: ...

Default values

If a parameter has a default value, the prototype will contain the triple dots placeholder for it:

code:
def foo(a, b=''):
    """Do foo.

    :sig: (int, str) -> None
    """
stub:
def foo(a: int, b: str = ...) -> None: ...

Base classes

The imports needed for base classes will be included or generated using the same rules as described above (imported, dotted, etc.):

code:
from x import A

class Foo(A, y.B):
    def foo(self, a):
        """Do foo.

        :sig: (int) -> None
        """
stub:
from x import A
import y

class Foo(A, y.B):
    def foo(self, a: int) -> None: ...

Class signatures

If the docstring of a class has a signature field, it will be used as the signature field of its __init__ method unless that method already has a signature.

code:
class Foo:
    """A foo.

    :sig: (int) -> None
    """

    def __init__(self, a):
        self.a = a
stub:
class Foo:
    def __init__(self, a: int) -> None: ...

Signature comments

Type hints for assignments can be written using # sig: comments.

code:
n = 42  # sig: int
stub:
n = ...  # type: int

The rules for importing names as described above also apply here.

Note

The reason for using # sig comment instead of a # type comment would be to avoid having to import the types.

Instance variables

Within classes, assignments to attributes of self will generate assignments with type comments under the class:

code:
class Foo:
    def foo(self):
        self.y = 'spam'  # sig: str
stub:
class Foo:
    y = ...  # type: str

Long lines

If the prototype line gets too long, it will be divided into multiple lines:

code:
def some_long_func_name(some_long_param_name_1, some_long_param_name_2):
    """Do foo.

    :sig: (some_long_type_1, some_long_type_2) -> some_long_type_3
    """
stub:
def some_long_func_name(
    some_long_param_name_1: some_long_type_1,
    some_long_param_name_2: some_long_type_2,
) -> some_long_type_3: ...

Type aliases

Type aliases can be defined using a # sigalias comment:

code:
# sigalias: A = int

def foo(a):
    """Do foo.

    :sig: (A) -> None
    """
stub:
A = int

def foo(a: A) -> None: ...

Command-line interface

In the simplest form, pygenstub takes a Python source file as parameter and generates a stub in the same directory:

$ pygenstub foo.py

New in version 1.4.0: It can also generate stubs for multiple source files:

$ pygenstub foo.py bar.py baz.py

New in version 1.4.0: If the input is a directory, it will recursively generate stubs for all Python source files in the hierarchy:

$ pygenstub foodir

New in version 1.4.0: By default, stub files will be placed in the same directory as their corresponding source files. The -o option can be used to change the directory where the stub files will be stored:

$ pygenstub foo.py -o out

New in version 1.4.0: By default, pygenstub will ignore any function, method, or variable that doesn’t have a type signature. If used with the --generic option, it will generate output and use the type Any in such cases:

$ pygenstub --generic foo.py

New in version 1.4.0: pygenstub can also generate stubs for modules using the -m option. In this case, an output directory must be given:

$ pygenstub -m foo -o out

Multiple modules are also supported:

$ pygenstub -m foo -m bar -o out

When used together with the --generic option, it can generate stub templates for existing modules, even without any signatures in the expected format:

$ pygenstub --generic -m xml -o out

Note that this is not a main feature for pygenstub; the stubgen utility in mypy is probably better suited for this job.

API

pygenstub is a utility for generating stub files from docstrings in source files.

It takes a source file as input and creates a stub file with the same base name and the .pyi extension.

For more information, please refer to the documentation: https://pygenstub.tekir.org/

class pygenstub.ClassNode(name, bases, signature=None)

Bases: pygenstub.StubNode

A node representing a class in a stub tree.

Parameters:
  • name (str) – Name of class.
  • bases (Sequence[str]) – Base classes of class.
  • signature (Optional[str]) – Signature of class, to be used in __init__ method.
get_code()

Get the stub code for this class.

Return type:List[str]
Returns:Lines of stub code for this class.
class pygenstub.FunctionNode(name, parameters, rtype, decorators=None)

Bases: pygenstub.StubNode

A node representing a function in a stub tree.

Parameters:
  • name (str) – Name of function.
  • parameters (Sequence[Tuple[str, str, bool]]) – List of parameter triples (name, type, has_default).
  • rtype (str) – Type of return value.
  • decorators (Optional[Sequence[str]]) – Decorators of function.
get_code()

Get the stub code for this function.

Return type:List[str]
Returns:Lines of stub code for this function.
class pygenstub.StubGenerator(source, generic=False)

Bases: ast.NodeVisitor

A transformer that generates stub declarations from a source code.

Parameters:
  • source (str) – Source code to generate the stub for.
  • generic (bool) – Whether to produce generic stubs.
collect_aliases()

Collect the type aliases in the source.

static generate_import_from(module_, names)

Generate an import line.

Parameters:
  • module (str) – Name of module to import the names from.
  • names (Set[str]) – Names to import.
Return type:

str
Returns:

Import line in stub code.
generate_stub()

Generate the stub code for this source.

Return type:str
Returns:Generated stub code.
get_function_node(node)

Process a function node.

Parameters:node – Node to process.
Return type:FunctionNode
Returns:Generated function node in stub tree.
visit_Assign(node)

Visit an assignment node.

visit_AsyncFunctionDef(node)

Visit an async function node.

visit_ClassDef(node)

Visit a class node.

visit_FunctionDef(node)

Visit a function node.

visit_Import(node)

Visit an import node.

visit_ImportFrom(node)

Visit an from-import node.

class pygenstub.StubNode

Bases: object

A node in a stub tree.

add_child(node)

Add a function/method or class node to this node.

Parameters:node – Function or class node to add.
add_variable(node)

Add a variable node to this node.

Parameters:node – Variable node to add.
get_code()

Get the stub code for this node.

The stub code for a node consists of the type annotations of its variables, followed by the prototypes of its functions/methods and classes.

Return type:List[str]
Returns:Lines of stub code for this node.
class pygenstub.VariableNode(name, type_)

Bases: pygenstub.StubNode

A node representing an assignment in a stub tree.

Parameters:
  • name (str) – Name of variable that is being assigned to.
  • type (str) – Type of variable.
get_code()

Get the type annotation for this variable.

Return type:List[str]
Returns:Lines of stub code for this variable.
pygenstub.extract_signature(docstring)

Extract the signature from a docstring.

Parameters:docstring (str) – Docstring to extract the signature from.
Return type:Optional[str]
Returns:Extracted signature, or None if there’s no signature.
pygenstub.get_aliases(lines)

Get the type aliases in the source.

Parameters:lines (Sequence[str]) – Lines of the source code.
Return type:Dict[str, str]
Returns:Aliases and their their definitions.
pygenstub.get_fields(node, fields_tag='field_list')

Get the field names and their values from a node.

Parameters:
  • node (Document docutils.nodes.document) – Node to get the fields from.
  • fields_tag (str) – Tag of child node that contains the fields.
Return type:

Dict[str, str]
Returns:

Names and values of fields.
pygenstub.get_mod_paths(mod_name, out_dir)

Get source and stub paths for a module.

pygenstub.get_pkg_paths(pkg_name, out_dir)

Recursively get all source and stub paths for a package.

pygenstub.get_signature(node)

Get the signature of a function or a class.

Parameters:node (Union[ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef]) – Node to get the signature from.
Return type:Optional[str]
Returns:Value of signature field in node docstring, or None if there’s no signature.
pygenstub.get_stub(source, generic=False)

Get the stub code for a source code.

Parameters:
  • source (str) – Source code to generate the stub for.
  • generic (bool) – Whether to produce generic stubs.
Return type:

str
Returns:

Generated stub code.
pygenstub.main(argv=None)

Start the command line interface.

pygenstub.parse_signature(signature)

Parse a signature into its input and return parameter types.

This will also collect the types that are required by any of the input and return types.

Parameters:signature (str) – Signature to parse.
Return type:Tuple[List[str], str, Set[str]]
Returns:Input parameter types, return type, and all required types.
pygenstub.process_docstring(app, what, name, obj, options, lines)

Modify the docstring before generating documentation.

This will insert type declarations for parameters and return type into the docstring, and remove the signature field so that it will be excluded from the generated document.

pygenstub.setup(app)

Register to Sphinx.

pygenstub.split_parameter_types(parameters)

Split a parameter types declaration into individual types.

The input is the left hand side of a signature (the part before the arrow), excluding the parentheses.

Parameters:parameters (str) – Comma separated parameter types.
Return type:List[str]
Returns:Parameter types.

History

1.4.0 (2019-03-27)

  • Generate stubs for multiple input files.
  • Generate stubs for directories.
  • Generate stubs for modules.
  • Add option for storing stub files in an output directory.
  • Add option to generate generic stubs.

1.3.0 (2019-03-07)

  • Support names renamed with as during import.
  • Support class variables.

1.2.4 (2019-02-01)

  • More spacing fixes for consecutive class stubs.

1.2.3 (2019-01-31)

  • Improve layout for stub generation with one-line class stubs.

1.2.2 (2019-01-08)

  • Fix stub generation for superclass declarations with multiple dots.

1.2.1 (2018-12-27)

  • Fix problem with function decorators that have parameters.
  • Switch to poetry for project management.

1.2 (2018-10-19)

  • Generate stub files that follow the same layout as Black.
  • Fixed stub generation for async functions with decorators.

1.1 (2018-05-25)

  • Added support for generating async function stubs.

1.0 (2018-05-25)

  • Include type alias expansion in parameter type docstring.

1.0b9 (2018-03-23)

  • Fixed bug about missing newlines after type aliases.

1.0b8 (2018-03-23)

  • Added simplistic support for defining type aliases.

1.0b7 (2018-01-18)

  • Added support for getting class signature from init method in Sphinx.

1.0b6 (2017-07-26)

  • Fixed handling of * separator for keyword-only arguments.
  • Added support for keyword-only arguments with default values.
  • Added –version option to command line arguments.

1.0b5 (2017-07-26)

  • Added support for property, staticmethod, and classmethod decorators.
  • Added support for keyword-only arguments.

1.0b4 (2017-06-16)

  • Collect builtin types from the builtins module.

1.0b3 (2017-06-16)

  • Fixes for *args and **kwargs on Python 2 code.

1.0b2 (2017-05-26)

  • Added support for Python 2 again.

1.0b1 (2017-05-09)

  • Added support for using type hints in Sphinx autodoc.

1.0a6 (2017-03-06)

  • Improvements on imported names.

1.0a5 (2017-02-07)

  • Support for methods.
  • Support for instance variables.
  • Support for base classes.
  • Shortened the field name from “signature” to “sig”.
  • Use three dots instead of actual value for parameter defaults.
  • Dropped support for Python 2.

1.0a4 (2017-01-06)

  • Long stubs are now spread over multiple lines.
  • Better handling of parameter defaults that are tuples.
  • Bugfix: handling of parameter defaults that have the value None.

1.0a3 (2017-01-06)

  • Proper support for names from the typing module in input parameters.
  • Added parameter default values to stubs.

1.0a2 (2017-01-03)

  • Support for Python 2.7.

1.0a1 (2017-01-03)

  • First release on PyPI.