Hacker News new | ask | show | jobs
by joobus 1919 days ago
> With Mypy, you get all the benefits of high-level dynamic typing for rapid experimentation, and all the benefits of rigorous type checking...

Except that most typed languages are far faster than python. Retrofitting existing python projects with types makes sense for reliability, but starting a new project with python and types makes less sense, imo.

5 comments

The post isn't about performance, and is aimed at people using Python for whatever reason, so, sure, retrofit away.

That said, if you care about the performance improvements that typing can give you with Mypy, you might want to look here:

https://github.com/mypyc/mypyc-benchmark-results/blob/master...

It won't be going toe to toe with Rust any time soon, but a 4x to 17x speedup is nothing to sneeze at.

That's cool if there's any indication that this could actually be used in production. Namely, what is the compatibility with the ecosystem. The history of Python could be aptly characterized as a series of big problems -> tools promising miraculous solutions -> tools utterly failing to deliver on those solutions because they failed to consider some important part of the ecosystem. We need more than a benchmark graph to suggest that this is a panacea to Python's longstanding performance problems.
Mypy itself uses this extensively; it was developed because typechecking itself was getting too slow. So it’s seen some practical real-world use. It’s not a panacea (heck, it’s barely documented) but it’s an appealing option if you have some clear hotspot in Python that you need to optimize.
There is a very similar tool called Cython that is used by scikitlearn. Coincidentally Cython also solves a lot of python 2/3 problems since it compiles for both.
Also, Mypy has a thousand paper cuts compared to other type systems. Pretty sure we still can't model JSON in Mypy (e.g., JSON = Union[List['JSON'], Dict[str, 'JSON'], None, bool, int, float, str] or whatever) because Mypy doesn't support recursive types. Moreover, since it's largely shoehorned into existing Python syntax, it's very awkward. E.g., I'm still not sure what the scoping rules are for TypeVar, describing a callback that takes kwargs involves defining a protocol class with a `__call__()` method that takes your function's kwargs, etc. Further, getting the type definitions to load properly is incredibly painful, the error message directs you to a page with a few things you might try to debug further, but never has my issue been among them. Also, like everything in the Python world, Mypy is slow, and you really want your type checks to be instant.

Most of these things can be improved upon, but progress feels slow and there are so many other serious issues in the Python ecosystem (performance, package management, etc) that I've given up (after 15 years of daily use). It makes me sad, but Go and Rust seem to offer better tradeoffs for the things that I care about.

Using that logic, it never made sense to start any project with Python, correct?

People use Python for its ease and speed of development, and extensive ecosystem. Things that need to be fast can have the appropriate effort expended to execute as fast as necessary.

> People use Python for its ease and speed of development, and extensive ecosystem.

Of these, I think only the ecosystem stands out anymore. For work purposes I wouldn't be able to move away from Python anytime soon, but if I could pick any language for new code it wouldn't be Python. If I had to pick the closest competitor, Julia seems like a great Python-like language in terms of speed and ease of development with much better typing and performance, but with a less extensive ecosystem. Nim might also fit that bill. If ecosystem was important, Go probably fills a similar space without being quite as steep of a learning curve as some of the other languages.

> it never made sense to start any project with Python, correct?

Well as a Perl fan I certainly don't disagree with that conclusion ;)

But yeah, you're definitely right that there's immense value to being able to quickly prototype.

I start new projects with python and mypy. If you're convinced that there are some problems that better be solved in python, you can easily see that once you have a project in python, mypy just adds more safety to it. So there is nothing wrong with python+mypy for new projects.
Cython is an option when a (potential) 100x speedup is required. Choosing the right algorithm up front, thru the rapid experimentation Python gives, can sometimes make it faster than a real world implementation in a static language, that didn't get enough time to experiment.