Hacker News new | ask | show | jobs
by narush 1371 days ago
This is a cool overview, and I certainly learned new things about the Python language from it. Thanks for posting!

We do lots of Python metaprogramming at Mito [1], but generally avoid all of this fancy Python fluff to get it done. Specifically, we avoid metaclasses, invisible decorators, etc. Instead, we take a much simpler approach of having Python code that literally populates as template .py file, and then writes it to the correct location in our codebase.

As a concrete example: we’re a spreadsheet, so we let our users transform data in a bunch of different ways - adding a column, writing a formula, deleting some rows. Anytime I want to add a new transform (say, encoding a column), I tell the metaprogramming package “python -m metaprogramming step —name “Encoding A Column”. It will ask me some questions about the parameters and types of those parameters, and then write most of the 4-6 boilerplate Python and Typescript files I need automatically! You can see it here [2].

This is still metaprogramming (it’s certainly code that writes code). But the code you end up with at the end of the day is very simple Python code that is extremely easy to understand / maintain long-term.

I’ll pass on the fancy stuff for now. Thanks though!

[1] https://trymito.io

[2] https://github.com/mito-ds/monorepo/blob/dev/mitosheet/dev/c...

2 comments

I personally consider code generation an anti-pattern in Python. With its dynamic nature and the two-step execution model, Python is essentially its own macro language, and any generation can be done at runtime.
unfortunately pep-484 changed all that. now you have to use static generation for basically anything that used to be at runtime previously, if you want it to have any kind of compatibility with mypy, pylance, etc.
MyPy doesn't require everything to be type-hinted. It's fine to sprinkle in hints where you want them.
when people using your library need pylance type hinting to show up for all your public methods, and those public methods are in fact generated as proxies for some other object. This was trivial as a dynamic runtime thing before, now must be statically generated. I can share examples if you'd like.
So the only problem is type hinting in IDEs. That's a small price to pay.
How do you deal with changes over time?

Like let's say you want to add a new required property to every type of something... like you want every encoding to have an accepts_tainted_value property. You don't want a default because you want to be forced to think through every case.

Can you regenerate the code? If so, are you keeping the "real" object specification in some other structure? Or do you just change a superclass to make the property required then fix the generated source based on the errors you get?