Skip to content

[Python] Enhance object API __init__ with typed keyword arguments #8615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

crackcomm
Copy link

This commit significantly improves the developer experience for the Python Object-Based API by overhauling the generated __init__ method for T-suffixed classes.

Previously, T objects had to be instantiated with an empty constructor, and their fields had to be populated manually one by one. This was verbose and not idiomatic Python.

This change modifies the Python code generator (GenInitialize) to produce __init__ methods that are:

  1. Keyword-Argument-Friendly: The constructor now accepts all table/struct fields as keyword arguments, allowing for concise, single-line object creation.

  2. Fully Typed: The signature of the __init__ method is now annotated with Python type hints. This provides immediate benefits for static analysis tools (like Mypy) and IDEs, enabling better autocompletion and type checking.

  3. Correctly Optional: The generator now correctly wraps types in Optional[...] if their default value is None. This applies to strings, vectors, and other nullable fields, ensuring strict type safety.

The new approach remains fully backward-compatible, as all arguments have default values. Existing code that uses the empty constructor will continue to work without modification.

Example of a Generated __init__

Before:

class KeyValueT(object):
    def __init__(self):
        self.key = None  # type: str
        self.value = None  # type: str

After:

class KeyValueT(object):
    def __init__(self, key: Optional[str] = None, value: Optional[str] = None):
        self.key = key
        self.value = value

Example of User Code

Before:

# Old, verbose way
kv = KeyValueT()
kv.key = "instrument"
kv.value = "EUR/USD"

After:

# New, Pythonic way
kv = KeyValueT(key="instrument", value="EUR/USD")

@github-actions github-actions bot added c++ codegen Involving generating code from schema python labels Jun 16, 2025
This commit significantly improves the developer experience for the Python Object-Based API by overhauling the generated `__init__` method for `T`-suffixed classes.

Previously, `T` objects had to be instantiated with an empty constructor, and their fields had to be populated manually one by one. This was verbose and not idiomatic Python.

This change modifies the Python code generator (`GenInitialize`) to produce `__init__` methods that are:

1.  **Keyword-Argument-Friendly**: The constructor now accepts all table/struct fields as keyword arguments, allowing for concise, single-line object creation.

2.  **Fully Typed**: The signature of the `__init__` method is now annotated with Python type hints. This provides immediate benefits for static analysis tools (like Mypy) and IDEs, enabling better autocompletion and type checking.

3.  **Correctly Optional**: The generator now correctly wraps types in `Optional[...]` if their default value is `None`. This applies to strings, vectors, and other nullable fields, ensuring strict type safety.

The new approach remains **fully backward-compatible**, as all arguments have default values. Existing code that uses the empty constructor will continue to work without modification.

#### Example of a Generated `__init__`

**Before:**

```python
class KeyValueT(object):
    def __init__(self):
        self.key = None  # type: str
        self.value = None  # type: str
```

**After:**

```python
class KeyValueT(object):
    def __init__(self, key: Optional[str] = None, value: Optional[str] = None):
        self.key = key
        self.value = value
```

#### Example of User Code

**Before:**

```python
# Old, verbose way
kv = KeyValueT()
kv.key = "instrument"
kv.value = "EUR/USD"
```

**After:**

```python
# New, Pythonic way
kv = KeyValueT(key="instrument", value="EUR/USD")
```
@crackcomm crackcomm force-pushed the push-uvkkkyywuuzy branch from 3df6969 to 2043352 Compare June 16, 2025 04:43
@fliiiix
Copy link
Contributor

fliiiix commented Jun 23, 2025

Nice, but im almost certain this will break Python 2 support

File "script.py", line 2
    def __init__(self, key: Optional[str] = None, value: Optional[str] = None):
                          ^
SyntaxError: invalid syntax

Some guarding against that would be probably useful version_.major == 3


if (IsScalar(type.base_type)) {
field_type = TypeOf(type, imports);
if (field->IsOptional()) { field_type += " | None"; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the | syntax was introduced in Python 3.10, and since flatbuffers supports Python all the way back to 2.7 should be avoided.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a part of .pyi generator. It's already used there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double checking the documentation for stubs, it just states "all tools support the | syntax", so I expect the syntax is more "python-like" rather than fully python and should be fine.


// Build signature with keyword arguments, type hints, and default values.
signature_params +=
", " + field_field + ": " + field_type + " = " + default_value;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typing is opt-in with --python-typing, no typing imports or annotations should be output if not explicitly requested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also the type info should end up in the .pyi files rather than the py code directly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best I can tell when looking at the generator code, when --python-typing is specified it will output at least some typing annotations in the .py file.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is your suggestion? Should we keep the old behavior (self.a = a # type: float) and no typed __init__, and defer all typing to .pyi, or old typing behavior (comments) if no --python-typing and new if enabled?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated code seems to be really inconsistent when type info is output, so it's hard to make a good suggestion.

When looking at the MonsterExtra.py test output for an example, it seems like the primary API has at least some (but not complete) type info, while the newer object API at the end created with --gen-object-api relegates all typing to the .pyi stub file.

Based on that, as this adds to the newer object API, I think that means that the typing should be removed from the .py file and existing comments for the types should probably stay. Realistically this should be made more consistent, such as moving the original non-object API typing info to the stub files, but as far as this incremental change that makes the most sense to me.

BTW it may be worth putting each parameter on a separate line for the generated __init__ functions, otherwise larger tables can horizontally scroll to infinity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ codegen Involving generating code from schema python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants