I read somewhere on HackerNews that Python is kinda like a default language and rest such as Go, C, C++, Java are just for Optimization, once you figure out your solution.
Python has problems scaling to medium-size code bases because programs above a certain size tend to become difficult to reason about. This is not a performance issue, and it’s the #1 issue that I choose to rewrite Python programs in other languages. Python type annotations help.
IMO modern languages like Go and Java are pretty easy to get into and you can use them for a first implementation without really sacrificing development time relative to Python, as long as you have invested the time to learn those languages and the associated tools. (C++ is not like that unless you have made a very serious investment in setting it all up.)
I’m not trying to say that any of these languages are better/worse, just that they are differently suited for particular situations (program size, team experience, etc)
> Python has problems scaling to medium-size code bases because programs above a certain size tend to become difficult to reason about.
In mye experience Python programs are not more difficult to reason about than equivalent Java programs. To the contrary an over-reliance on certain design patterns and ubiquitous, inescapable OOP complicates Java code bases, while the static typing is so weak it affords little safety compared to e.g. Python.
Worth keeping in mind a that a Python program will be about half the LOC of a Java program doing the same thing. (See the reference section here [1].) In other words you can get further with Python before passing the complexity threshold.
Bugs are also proportional to lines of code [2], which is another element that favours Python over more verbose languages like Java.
> Python is not maintanable above 50k to 100k lines of code and because of that people consider this code bases very large
Note that this is a speculation, not a conclusion, the article is not very thorough, and LOC is mostly used because it is convenient to measure, not because it is what we are trying to measure. I'd personally consider 50k a "medium" code base, and 10k is "small".
> Worth keeping in mind a that a Python program will be about half the LOC of a Java program doing the same thing.
LOC is a confounding variable.
> Bugs are also proportional to lines of code...
Not supported by the citation. The citation measures bugs per line of code, and finds that for 500 kloc of code, the average number of bugs will be somewhere between zero and 25,000. That's a very wide range.
To be clear, I'm not really trying to fight against Python or for Java here. I'm just giving my reasoning for why I might personally choose one or the other. I think that the idea that you would switch languages because Python is slow is actually far more situational. You might have ten reasons to choose Python or Java, runtime performance may only be one of those factors, it may not be heavily weighted, and in some cases, Python runtime performance can be extremely fast (I do a lot of NumPy stuff... it's great).
The relationship between programming language and code quality is, at best, a difficult relationship to study. It's hard to make any kind of direct statement like "using language X results in more bugs than using language Y" and back it up by evidence, even though we believe it to be true. Individual statistics which relate some variable to LOC is not useful in isolation.
Both of these are straight up false. Java can be written with static functions and single-depth inheritance trees just fine. Hell, the direction the language took for quite a few years now are pretty much this with records, pattern matching, etc.
And while Java is not Haskell, it has a moderately strong type system with quite a good generic implementation making even some more advanced functional patterns expressible.
Your claim on LOCs is also false (as well as most of this kind of claim) Java has at most a tiny bit of constant overhead, it won’t make it anywhere close to 2x.
While this is something I very much appreciate about Python I think it's also true that structure needs to increase with the size of your codebase and the size of your team to avoid being driven mad. Python takes the "just apply discipline" approach which can absolutely work for lots of code but falls apart a bit with large heterogeneous teams.
> Python has problems scaling to medium-size code bases because programs above a certain size tend to become difficult to reason about. This is not a performance issue, and it’s the #1 issue that I choose to rewrite Python programs in other languages. Python type annotations help.
I think you're spot on. Is some ways, it is a tooling problem, as type annotations require mypy (or some other type checker), and enforcement via e.g. CI.
Crucially, retrofitting them to a codebase is a difficult and tedious problem. On the other hand, it's relatively easy to do for new projects, but so is choosing a different language.
> Python has problems scaling to medium-size code bases because programs above a certain size tend to become difficult to reason about.
In my experience, this is because a lot of Python programmers are trying to write Java code in Python, carrying Java's awful and unreadable paradigms into a language that has no reason for them.
> Python has problems scaling to medium-size code bases because programs above a certain size tend to become difficult to reason about.
The other issue with Python is that it uses indentation for scoping. Combine that with the fact that it is super easy to mess up indentation when you are moving code around via copy/paste and it is super easy to change the meaning (ie accidentally move a statement out of an if block).
Using indentation for scoping is great for small projects and for beginners. However, once you get to medium or large projects, having the extra redundancy of curly braces is reassuring.
No. The lack of curly braces doesn't have a big impact on python being hard to reason about as the code base grows, it's the lack of tooling that comes with a strong statically typed language. It makes refactoring hard across a large code base.
Python is a great scripting language, and it's a great language for hackers that work on their own personal projects. It's not a great language for enterprise microservices on teams of like 20 people. The growing pains are real.
Whenever someone complains about Python's indentation-as-syntax, my mind translates it as "this programmer writes terribly formatted code". If your code is properly formatted, then indentation will never be a problem.
Copy/pasting large blocks, yeah, you can mess it up. But any editor worth using will let you select several lines of text and hit Tab or Shift-Tab to add/remove an indentation level, so fixing it only takes a couple seconds.
Indentation has never been a problem with any of the code bases I've ever used. Ok, maybe there was that one script with tabs in it from ~2003?
Not using a programming editor? I select the text and hit Tab or Shift+Tab, not exactly rocket surgery. The lack of redundant noise characters pays off every single day, where as odd indentation problems are a once-a-decade issue.
Python is a pretty terrible language to work with. Tooling sucks. Dependency conflicts are common. There's no test framework/runner worth a damn. Web frameworks are inferior to those in most other languages.
If you're doing data science things, it's hard to beat pandas/numpy. I get that those are popular in that community because the barrier to entry with Python is low. People who are just looking for a tool to solve immediate problems would do well to reach for Python.
The problems start when you try to write more complex things. Then you run into the weaknesses in tooling, testing, and performance. People would do well to skip Python and go right to another solution.
I say this having spent half of the last decade working on Python projects. If not for data science and academia keeping it alive, it'd be disappearing along with Perl.
> Web frameworks are inferior to those in most other languages.
That's quite a statement. Go on, for most of the other languages, show me a web framework that's better than Django.
Python absolutely has deployment challenges, but the performance is Good Enough™ and the speed of modelling and maintaining and accessing databases in Django's ORM is so much better than anything else I've found. Its Admin interface too is superb for same-day turnarounds on little CRUD projects. Nothing comes close.
And I've written quite complex systems around Django. Ones that are multi-headed websites, APIs to physical hardware IO, ANPR, card readers, ID printers. Python hasn't let us down.
> That's quite a statement. Go on, for most of the other languages, show me a web framework that's better than Django.
I think that’s subjective, so take this with a grain of salt. I do have a different opinion though. Also, my point isn’t to argue, but to encourage consideration.
I’ve used Django and Python quite extensively in the past and I would say that, for me, Phoenix/Elixir and Ruby on Rails are both better web frameworks than Django.
The reason I feel this way is that Phoenix/Elixir, for example, has much better dependency management and tooling than Python/Django (e.g., Mix is vastly superior to Pip, in my opinion).
Also, when I was using Django in the past, even the core team suggested using a different directory layout than is generated by default. This meant that every Django application I worked on had a completely different project layout. Both Phoenix and Rails are more opinionated regarding their project layouts. Some may consider this bad, but the benefit is I can go to any Phoenix or Rails project and instantly be productive because I know where all my controllers, models, views, templates, and contexts are going to be. I also like that Phoenix is much faster, more scalable, and makes better use of hardware resources than either Python or Ruby.
I think Python and Django are excellent, but I’d place them at #3 in my top-three list of web frameworks. That’s just my opinion though.
What I took issue with was the suggestion that "most other languages" have a better web framework. I'd wager most other languages don't even have a web framework. And even amongst those that do, there's some real dirt out there, and much worse examples of packaging, tooling, etc. Two Python frameworks, Django and FastAPI would be in my as-objective-as-possible top 10.
I am a self admitted python- and ORM-hater and I agree. Django is pretty fantastic, and I have successfully used it to spin up several complete CRUD projects in a day or two
These are the kind of comments that really made me feel insecure when I started out to program. Every programming environment/language has its advantages and drawbacks and python is obviously not a terrible language. People build amazing stuff with python.
> Python is a pretty terrible language to work with.
That's just like, your opinion, man.
> Tooling sucks.
Not in my experience. Lot of tools could be better, but I would not say there is a lack of un-sucky tooling at this point. Poetry and pytest in particular are largely excellent to work with.
> Dependency conflicts are common.
This has been a problem historically but it's leagues better today, especially with lockfiles.
> There's no test framework/runner worth a damn.
Um, what? Unittest (stdlib), Pytest, tox, nose, hypothesis, schemathesis, and a few other lesser known ones.
> Web frameworks are inferior to those in most other languages.
FastAPI is pretty amazing. Flask is well regarded as a good intro framework. Django I'm not a fan of personally, but many love it.
> If not for data science and academia keeping it alive, it'd be disappearing along with Perl.
And machine learning. And web dev. And sysadmin scripting.
For the little data sciences like statistics and mildly complex ML, R is really good. At least for data exploration. The REPL/emacs combo I use is much better than python/jupyter. By better I mean it's far more productive to test ideas, compute statistics, charts, etc. Unfortunately R programs I write are not as structured as python programs. I wouldn't like to maintain that kind of code for a long time, nor would I build big applications in R.
Sounds like inexperience or confusion, to be honest. None of those are problems particular to Python. Meanwhile you did not mention any of Python's significant issues, such as performance or deploying end-user apps. But it is best-in-class for what it was originally designed for.
The built-in unittest module is essentially a clone of JUnit, so it can't be much worse, and most of the remaining limitations are fixed by pytest which people seem pretty happy with. What test frameworks are you comparing them to?
The thing I want is a good monorepo solution for Python. There's at least 3 for JavaScript.
As much hate as JS gets, its package managers are also much better than Pip.
In order to manage complexity of Python as you do your big projects, you really need to do "enterprisey" patterns like DI. Even though it's rare for Python programmers to write like that.
This is possible with existing general monorepo tools. We use Bazel for Python and other languages. With Python, we have a single requirements.txt at the root of the monorepo, but each target (a Python package, a module, an app, or some combination thereof) depends only on the packages it needs and the transitive packages of its dependencies.
Then if you need to do something like build a distributable archive for your app (say, to deploy it to lambda), you can use Bazel's dependency graph to get only the packages that the app transitively depends on.
What about the opposite case? I have several python packages with one requirements.txt each. But for development I want to have a single “root” virtual env?
Of things you mention I feel the testing part the most. I started python recently enough that I've only ever used pytest, but something about it just feels too...magical? I'm holding out hope for a new test framework that I'd feel more at home with, but I'm not aware of any such projects; do they exist?
You could give the built-in unittest module [1] a try. It's less magical (although also less flexible) and many of its old limitations that people used to cite as advantages of pytest (e.g. built-in test discovery) have since been fixed.
I'm relatively new to Python, coming from Ruby and JS, and the built-in unittest module is just fine. I haven't had any issues with it, besides the documentation for setting it up following the conventions is really hard to find. But once it works, I like how simple it is. If you know OOP you'll feel right at home. You can simply create your own test case classes if you want to re-use stuff. No magical bloated configuration.
> People would do well to skip Python and go right to another solution.
Okay, so what solution? What do you recommend? I'm guessing if you're qualified to make statements like your first paragraph, you can bring your wealth of skill and experience to the table and show us what we should be using.
Python is the Visual Basic of 2022. Easy to understand, easy to learn and very forgiving. Much nicer than the main alternative (Javascript).
Unskilled developers can be very productive very quickly. Skilled developers can write some really beautiful code with it.
Has issues with the ecosystem - it's a headache with M1 and far too many Python projects are just broken. The latter is because it's used by scientists / students so is more a function of the community than the language.
I use it as a scripting language, which it excels in as long as you don't have many dependencies.
Go: fast and cgo is really powerful. The language itself is a little verbose for my tastes though - it makes the simple verbose. The package management story is weird (import the source code from github). Better than having the dependency hell of npm or Python though.
I don't know who coined it but I love the term "second-best language for everything" when describing python. The ecosystem is so broad you can almost always find tools to do whatever you need, and it lets folks jump between domains simply by knowing the language.
> I read somewhere on HackerNews that Python is kinda like a default language and rest such as Go, C, C++, Java are just for Optimization, once you figure out your solution.
Python is extremely convenient when you are trying to be productive. It is a good general purpose tool, especially if you're doing anything web related. My project starts in seconds and requires no compilation. Stack traces and tooling for the language is far beyond most other popular languages IMO. The standard library is excellent, but also the quality of popular python libraries is unmatched by other languages in my experience.
Python is not super fast at doing some stuff. If you're doing that stuff all the time it may not be a good choice. But for most of what I do, python is really good enough. A user will not notice a few 10s of Ms for that web request, and usually that isn't even the scale anyway for this type of stuff.
Python is an excellent choice for that. What I tend to do is build with Python and, if (and that's a big if) the thing needs to be orders of magnitude faster, then we profile the app to find the hot spots and optimize those, either in Python, with some better approach, or in another language still being called from Python. A rewrite of a whole service from scratch is relatively rare in my experience (unless the original is really terrible and impossible to maintain).
I take a similar approach. Python allows me to rapidly prototype and ... gets the job done — fast (in terms of developer productivity). Once performance becomes a main criteria, it's not so difficult to fork/exec some Go/Rust/C/C++ executable.
I'm thinking about using MPI for that - it should minimize the fork/exec time. Another approach is the extensive use of queues (which is a good thing anyway) and process the queue messages using a more optimized backend.
There's an element of truth to that in some cases, but it's mostly just something Python fans say to excuse the fact that Python is so slow.
It's not true for two reasons:
1. In practice writing a prototype in one language, then completely scrapping it and rewriting it in another language almost never happens for fairly obvious reasons.
2. The "slow bits" of a Python program are rarely neatly concentrated in a few parts of the program that you can rewrite in a fast language. The whole program is slow, so there's no small part that you can rewrite to make it all faster.
3. There are other languages that are just as easy to use as Python (easier even!) but are also fast enough that you don't need to rewrite anything in most cases. Typescript is probably the best option at the moment but there's also Dart, Lua and probably others I've forgotten.
Python is popular because it's seen as very beginner friendly so there's a mountain of content out there teaching people Python. Also the REPL makes it good for beginners and for science.
The point is you're very productive up front, and have a number of choices with the rest of your time. Few rewrite from the ground up because they don't need to.
On first startup we used Tcl, and C code for performance, similar to what is fashionable with Python nowadays.
I had enough of the rewrite in C experience that since then, if the language doesn't come with a compiler on the box, either JIT or AOT, I only see it as scripting language for sysadmin work or learning purposes.
AOLServer had a lot of defaults that would magnify the performance issues of TCL. The documentation did not cover all of the necessary configuration options to get a system to scale and you had to dig into driver.c to understand how to tune the system. I think the two big undocumented bottlenecks were maxsok and backlog. I bet if you were writing the same applications today, using modern hardware and Naviserver you would not have to write C modules anymore.
Our product was a competitor to AOLServer with CMS tooling for call centers.
Also Vignette, remember it?
When we dediced to move away from TCL/C, after exhausting the performance options, J2EE was going to be the next version.
Due to a twist of fate of being Microsoft partners, we got invited to take part into .NET launch product line, and that Java rewrite became a .NET one instead.
The original founders, after going through these experiences, went out and created OutSystems based on the learnings, naturally in .NET and later a Java backend was added as well.
Yes, it has from the beginning been a fantastic prototyping tool, with a growing number of exit strategies for when you outgrow it. Or make do when you don't.
It's had its deficiencies as well, exacerbated due to lack of investment, but that is changing. These are being knocked out one by one, every release.
IMO modern languages like Go and Java are pretty easy to get into and you can use them for a first implementation without really sacrificing development time relative to Python, as long as you have invested the time to learn those languages and the associated tools. (C++ is not like that unless you have made a very serious investment in setting it all up.)
I’m not trying to say that any of these languages are better/worse, just that they are differently suited for particular situations (program size, team experience, etc)