implement pydantic model data type

[cosmicBboy]

fixes #764

[codecov]

Codecov Report

Merging #779 (37acb98) into dev (ebfecc1) will decrease coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev     #779      +/-   ##
==========================================
- Coverage   97.71%   97.63%   -0.08%     
==========================================
  Files          45       44       -1     
  Lines        4026     4059      +33     
==========================================
+ Hits         3934     3963      +29     
- Misses         92       96       +4     

Impacted Files	Coverage Δ
pandera/checks.py	98.51% <ø> (ø)
pandera/error_formatters.py	92.59% <ø> (-0.27%)	⬇️
pandera/model.py	95.07% <ø> (ø)
pandera/engines/pandas_engine.py	97.45% <100.00%> (+0.20%)	⬆️
pandera/schemas.py	99.24% <100.00%> (+<0.01%)	⬆️
pandera/typing/config.py	100.00% <100.00%> (ø)
pandera/check_utils.py	90.00% <0.00%> (-6.67%)	⬇️
pandera/constants.py

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ebfecc1...37acb98. Read the comment docs.

[jeffzi]

Really cool use of DataType !

I have 2 remarks:

1. If coerce is set to False, validation passes since the schema think there are no columns to validate. Pandera should warn if the global dtype is a PydanticModel and coerce is False.

2. PydanticModel(Record).check(df) always return False. It's not a big deal since most users won't call it. We can override check and raise a NotImplementedError for now.

To expand on 2., one problem with the current DataType.check() method is that it validates the data type without access to the data itself. Therefore you cannot re-use the coerce logic to validate with check. I've ran into this problem when implementing decimal and date dtypes, which is why I've been holding back my PR. Those dtypes can be coerced by not validated without looking at the data. I will open a separate issue because it deserves its own discussion.

Minimal example:

import pandas as pd
from pydantic import BaseModel
import pandera as pa
from pandera.engines.pandas_engine import PydanticModel


class Record(BaseModel):
    """Pydantic record model."""

    age: int


class PydanticSchema(pa.SchemaModel):
    """Pandera schema using the pydantic model."""

    class Config:
        """Config with dataframe-level data type."""

        dtype = PydanticModel(Record)
        coerce = False # <------ 

df = pd.DataFrame({"age": [21, "foo"]})
PydanticSchema.validate(df) # passes
#>    age
0   21
1  foo

PydanticModel(Record).check(df) # unsupported, expects a data type
#> False

[cosmicBboy]

@jeffzi I ended up adding back the mypy nox session back, will tackle the pre-commit mypy issue in #786

also, please ✅ this PR if all looks good

[jeffzi]

Did you see my 2 remarks above?

If coerce is set to False, validation passes since the schema think there are no columns to validate. Pandera should warn if the global dtype is a PydanticModel and coerce is False.

^ I think this one is important to fix before merging.

[cosmicBboy]

Ah, right, those slipped my mind. I think for (1) I'm gonna play around with adding a auto_coerce: bool class attribute to DataType, by default this will be false be it'll be True for PydanticModel. Warnings are helpful but I think it's better UX to make it easy to do the right thing, in this case, during the coerce step the schema will coerce the dtype if auto_coerce is True.

[cosmicBboy]

@jeffzi I decided to special-case the PydanticModel dtype in the DataFrameSchema constructor, raising a SchemaInitError if coerce=False and dtype is a PydanticModel... in this case I thought it best to have pandera barf in this case instead of a warning.