Shipping types for type-checking

[G-Rath]

I'm looking to use this library in a project that uses mypy, and it looks like currently there are no types available - is this something you'd be open to adding as part of the package?

My understanding is there are three primary ways this can be addressed:

1. adding types to the actual code
2. shipping stubs next to the code, in .pyi files
3. publishing stubs as a dedicated package, ideally via typeshed

If I understand correctly, provided the package author is onboard with shipping types in their package, doing 2. should be very straightforward, requiring creating a markdownify/__init__.pyi with some initial type information (which mypy can generate for you), and maybe including a py.typed file in the published package.

This means that the types don't need to be perfect from the get-go, and can be easily improved over time - I would be happy to do such a pull request using the basic types I've stubbed out locally.

I don't mind looking at doing 1. or 3. either, but I consider myself something of a guest in the Python ecosystem as I mainly use it for scripts and have not published any packages, so those options feel like they'd required more time and complexity which is why I'm suggesting option 2. to start with

[AlexVonB]

Hey @G-Rath , thanks for proposing this! The types needed for this package would be minimal, right? Providing types for markdownify would be enough for most usecases. Maintaining a large list of types calls for errors down the road (missed updates in the types, stale types etc), but keeping types just for markdownify would be feasable. Could this be done right in the code, only in this function, to avoid having another file? Best! Alex

[G-Rath]

@AlexVonB yes for the consumer the types can be minimal - this is what we've got so far in our stub, though we are just calling markdownify with no options:

from _typeshed import Incomplete

def markdownify(
  html: str,
  strip: list[str] = ...,
  convert: list[str] = ...,
  autolinks: bool = ...,
  bullets: str = ...,  # An iterable of bullet types.
  code_language: str = ...,
  code_language_callback: Incomplete = ...,
  default_title: bool = ...,
  escape_asterisks: bool = ...,
  escape_underscores: bool = ...,
  escape_misc: bool = ...,
  heading_style: str = ...,
  keep_inline_images_in: list[str] = ...,
  newline_style: str = ...,
  strip_document: str = ...,
  strong_em_symbol: str = ...,
  sub_symbol: str = ...,
  sup_symbol: str = ...,
  table_infer_header: bool = ...,
  wrap: bool = ...,
  wrap_width: int = ...,
) -> str: ...

My understanding is that you could ship supports for types right now by publishing a version with the above in a markdownify/__init__.pyi.

I'm happy to have a crack at trying to inline the types if you'd prefer, though I would say if you go that route you should aim to eventually use types throughout the code with mypy running in CI to ensure that they're correct (which I'm also happy to help with)

[AlexVonB]

Having this as a base looks great. I once tried adding types to all of the code (which would be the best solution) but remember hitting a dealbreaking issue and stopping.

mypy is just the static type checker, right? All inline types would be https://peps.python.org/pep-0484/ and not require any new dependencies? Then having the types would be cool. @chrispy-snps whats your take on having types?

[G-Rath]

Yup having types should not require any additional runtime dependencies - you'd just want mypy as a development dependency for checking that the types are valid

[chrispy-snps]

We use typing annotations in our own code and find them very useful. At my day job, I am "that guy" who remind everything to include typing annotations in their code. I would love to see this!

I've only been a Python developer for two years so I don't have a lot of knowledge about older Python versions. Would this affect our Python version compatibility?

For example, the current code uses the six library to help with Python 2/3 compatibility. Would typing annotations even parse in a Python 2 version? If not, is it time to raise our minimum version requirement so we can modernize our code with best practices?

[G-Rath]

Personally I'm not sure how Python 2/3 compatibility works in regards to typings, which is an example of why I think it could be good to start with shipping types in a dedicated .pyi file rather than in the source as that should just be a minor version bump and that'd let downstream folks start consuming (and contributing back to!) the types while in parallel work could be done on the next major that'd drop support for the older versions of Python and move the types into the source files themselves

[AlexVonB]

Re: Python 2.7: Python 3.0 came out 2008, thats 17 years ago and 2.7 was sunset 2020 (5 years ago). I would be fine with removing 2.7 support from markdown (if absolutely needed by some, they could use an old markdownify version). Removing six from the package would simplify some minor things.

That being said, starting with the pyi stub sounds best to me; adding inline types later on can be done anyways.

@G-Rath you suggested adding the pyi and mypy via PR, would that be something you are still up to? We could contribute by removing 2.7 support and six from the project. That PR would be kind of large, but would start some good change.

Anyways, thanks for your input so far! Sometimes it needs some outside nudges :D

Edit: Oh, 3.8 is EOL, as well :D 3.9 gets security patches until 25-10; the latest supported version is 3.13. Maybe we could push to 3.13? Or at least 3.9?