Hacker News new | ask | show | jobs
by haberman 1594 days ago
A notable downside that is not mentioned: the Python interpreter is far from Valgrind-clean. Valgrind is generally a powerful tool for debugging memory errors, but if you wrap your C code in Python, Valgrind will be so noisy as to be ineffective. Python startup is also very heavyweight; combined with the ~60x slowdown from Valgrind this is something you are going to notice.

The Python interpreter does not even use malloc()/free() directly by default; it layers its own memory allocator on top called PyMalloc (https://docs.python.org/3/c-api/memory.html#pymalloc). You can disable this by setting an environment variable (PYTHONMALLOC=malloc), but even then you will see many Valgrind warnings inside the Python interpreter itself.

2 comments

That's a good argument for keeping these kind of tests separate. Indeed Austin has dedicated Valgrind integration tests just for this reason.
I’ve had success running valgrind on python code with a very small suppression list to cover all of the python interpreter issues (at least for safety, not leaks) and let me focus on the code.
Is such a suppression list publicly available somewhere?
https://github.com/python-pillow/docker-images/blob/main/ubu...

This docker image is what we’re using in CI to do memory safety checks against the pillow c extension.

That is indeed very short! Thanks!