[82716f12]

> I agree, which is why you should design your modules correctly and import only the stuff you need.

How do you achieve this without making gazillions of modules, where each module has just a few stuff?

Are you saying just use local import everywhere?

[ActorNightly]

You can have one module with all the capability, but the capability is separated into submodules. And the __init__.py of the module doesn't auto import the submodules

So when you do this

    import bigmodule

It doesn't do anything functionality, and you may only have some small top level things available for you, like bigmodule.config, or bigmodule.logging.

Then, you have your big initializer code in bigmodule.financedata. But the stuff you need for running scripts is in bigmodule.scripts.

So when you write

   from bigmodule import financedata

This code will take a while.

But if you write

   from bigmodule import scripts

This will load fast.

You don't need to have gazillion modules, just good organization. Also, in general, its a good practice to gate intensive compute/network operations behind an explicit function you need to call.

Also thank you for focusing the convo on the tech stuff instead of repeating finance bro myths

[82716f12]

That sounds good in theory, but I haven’t come across a codebase that’s really optimized and free of unnecessary imports, especially in large companies.

And when you have to depend on external libraries beyond your control, how do you typically handle those situations?

[ActorNightly]

As codebases get larger, inefficiencies are bound to happen. Having explicit imports that I can go look up and see where they are is better for resolving this, because you can trace how every module gets imported and time them. Having interpreter code that runs with lazy loading that is all hidden is not the way to solve this.

As for external libraries, you import them in places where you need to use them only to avoid the same pitfalls. Its also pretty easy to analyze the import process within those libraries, and then again import specific submodules only that limit what actually gets loaded.