add polars LazyFrame generic type, element-wise checks, add docs (#1521)

* add polars LazyFrame generic type, add docs

Signed-off-by: cosmicBboy <[email protected]>

* readme, makefile fixes

Signed-off-by: cosmicBboy <[email protected]>

* add support for pl.DataFrame

Signed-off-by: cosmicBboy <[email protected]>

* fix failure case collection logic

Signed-off-by: cosmicBboy <[email protected]>

* update docs

Signed-off-by: cosmicBboy <[email protected]>

* add custom check docs

Signed-off-by: cosmicBboy <[email protected]>

* add polars elementwise check support

Signed-off-by: cosmicBboy <[email protected]>

* update docs

Signed-off-by: cosmicBboy <[email protected]>

* add NotImplementedError for polars strategies

Signed-off-by: cosmicBboy <[email protected]>

* update docs

Signed-off-by: cosmicBboy <[email protected]>

---------

Signed-off-by: cosmicBboy <[email protected]>

Author: cosmicBboy

Makefile

@@ -21,10 +21,10 @@ requirements:
 	pip install -r requirements-dev.txt
 
 docs-clean:
-	rm -rf docs/**/generated docs/**/methods docs/_build docs/source/_contents
+	rm -rf docs/source/reference/generated docs/**/generated docs/**/methods docs/_build docs/source/_contents
 
 docs: docs-clean
-	python -m sphinx -E "docs/source" "docs/_build" && make -C docs doctest
+	python -m sphinx -W -E "docs/source" "docs/_build" && make -C docs doctest
 
 quick-docs:
 	python -m sphinx -E "docs/source" "docs/_build" -W && \

README.md

@@ -42,8 +42,9 @@ This is useful in production-critical or reproducible research settings. With
 
 1. Define a schema once and use it to validate
    [different dataframe types](https://pandera.readthedocs.io/en/stable/supported_libraries.html)
-   including [pandas](http://pandas.pydata.org), [dask](https://dask.org),
-   [modin](https://modin.readthedocs.io/), and [pyspark](https://spark.apache.org/docs/3.2.0/api/python/user_guide/pandas_on_spark/index.html).
+   including [pandas](http://pandas.pydata.org), [polars](https://docs.pola.rs/),
+   [dask](https://dask.org), [modin](https://modin.readthedocs.io/),
+   and [pyspark](https://spark.apache.org/docs/3.2.0/api/python/user_guide/pandas_on_spark/index.html).
 1. [Check](https://pandera.readthedocs.io/en/stable/checks.html) the types and
    properties of columns in a `DataFrame` or values in a `Series`.
 1. Perform more complex statistical validation like
@@ -100,6 +101,7 @@ pip install pandera[modin]       # validate modin dataframes
 pip install pandera[modin-ray]   # validate modin dataframes with ray
 pip install pandera[modin-dask]  # validate modin dataframes with dask
 pip install pandera[geopandas]   # validate geopandas geodataframes
+pip install pandera[polars]      # validate polars dataframes
 ```
 
 </details>
@@ -120,6 +122,7 @@ conda install -c conda-forge pandera-modin       # validate modin dataframes
 conda install -c conda-forge pandera-modin-ray   # validate modin dataframes with ray
 conda install -c conda-forge pandera-modin-dask  # validate modin dataframes with dask
 conda install -c conda-forge pandera-geopandas   # validate geopandas geodataframes
+conda install -c conda-forge pandera-polars      # validate polars dataframes
 ```
 
 </details>

docs/source/conf.py

@@ -132,7 +132,7 @@
 # documentation.
 
 announcement = """
-📢 Pandera 0.16.0 now supports <a href="pyspark_sql.html">Pyspark SQL</a> 🎉.
+📢 Pandera 0.19.0 now supports <a href="/polars.html">Polars</a> 🎉.
 If you like this project, <a href='https://github.com/unionai-oss/pandera' target='_blank'>give us a star ⭐️! </a>
 """
 

docs/source/dataframe_models.rst

@@ -94,7 +94,7 @@ As you can see in the examples above, you can define a schema by sub-classing
 The :func:`~pandera.decorators.check_types` decorator is required to perform validation of the dataframe at
 run-time.
 
-Note that :class:`~pandera.api.pandas.model_components.Field` s apply to both
+Note that :class:`~pandera.api.dataframe.model_components.Field` s apply to both
 :class:`~pandera.api.pandas.components.Column` and :class:`~pandera.api.pandas.components.Index`
 objects, exposing the built-in :class:`Check` s via key-word arguments.
 
@@ -714,7 +714,7 @@ Column/Index checks
     * Similarly to ``pydantic``, :func:`classmethod` decorator is added behind the scenes
       if omitted.
     * You still may need to add the ``@classmethod`` decorator *after* the
-      :func:`~pandera.api.pandas.model_components.check` decorator if your static-type checker or
+      :func:`~pandera.api.dataframe.model_components.check` decorator if your static-type checker or
       linter complains.
     * Since ``checks`` are class methods, the first argument value they receive is a
       DataFrameModel subclass, not an instance of a model.
@@ -830,7 +830,7 @@ Aliases
 -------
 
 :class:`~pandera.api.pandas.model.DataFrameModel` supports columns which are not valid python variable names via the argument
-`alias` of :class:`~pandera.api.pandas.model_components.Field`.
+`alias` of :class:`~pandera.api.dataframe.model_components.Field`.
 
 Checks must reference the aliased names.
 
@@ -878,7 +878,7 @@ the class scope, and it will respect the alias.
 .. note::
 
     To access a variable from the class scope, you need to make it a class attribute,
-    and therefore assign it a default :class:`~pandera.api.pandas.model_components.Field`.
+    and therefore assign it a default :class:`~pandera.api.dataframe.model_components.Field`.
 
 .. testcode:: dataframe_schema_model
 

docs/source/dtype_validation.rst

@@ -123,7 +123,7 @@ express this same type with the class-based API, we need to use an
         dt: Series[Annotated[pd.DatetimeTZDtype, "ns", "UTC"]]
 
 Or alternatively, you can pass in the ``dtype_kwargs`` into
-:py:func:`~pandera.api.pandas.model_components.Field`:
+:py:func:`~pandera.api.dataframe.model_components.Field`:
 
 .. testcode:: dtype_validation
 

docs/source/index.rst

@@ -78,7 +78,7 @@ This is useful in production-critical data pipelines or reproducible research
 settings. With ``pandera``, you can:
 
 #. Define a schema once and use it to validate :ref:`different dataframe types <supported-dataframe-libraries>`
-   including `pandas <http://pandas.pydata.org>`_, `dask <https://dask.org/>`_,
+   including `pandas <http://pandas.pydata.org>`_, `polars <https://docs.pola.rs/>`, `dask <https://dask.org/>`_,
    `modin <https://modin.readthedocs.io/>`_, and
    `pyspark.pandas <https://spark.apache.org/docs/3.2.0/api/python/user_guide/pandas_on_spark/index.html>`_.
 #. :ref:`Check<checks>` the types and properties of columns in a
@@ -137,6 +137,7 @@ Installing additional functionality:
       pip install pandera[modin-ray]   # validate modin dataframes with ray
       pip install pandera[modin-dask]  # validate modin dataframes with dask
       pip install pandera[geopandas]   # validate geopandas geodataframes
+      pip install pandera[polars]      # validate polars dataframes
 
 .. tabbed:: conda
 
@@ -153,6 +154,7 @@ Installing additional functionality:
       conda install -c conda-forge pandera-modin-ray   # validate modin dataframes with ray
       conda install -c conda-forge pandera-modin-dask  # validate modin dataframes with dask
       conda install -c conda-forge pandera-geopandas   # validate geopandas geodataframes
+      conda install -c conda-forge pandera-polars      # validate polars dataframes
 
 Quick Start
 -----------