Hacker News new | ask | show | jobs
by tomtom1337 126 days ago
Definitely do not parse each row into eg pydantic models. You lose the entire performance benefit of pandas / polars by doing this.

If you need it, use a dataframe validation library to ensure that values are within certain ranges.

There are not yet good, fast implementations of proper types in Python dataframes (or databases for that matter) that I am aware of.