My experience has been that to efficiently write computationally expensive operations I need to know the low-level libraries that Python interfaces with. Other languages provide me this functionality natively and I only go to the low-level stuff only after a few iterations.