[afandian]

Ditto. I used it for something that I would have written in C (and had prior). A glorious language. Gives safety to places where pointer arithmetic and function pointers adds danger while keeping structs and arrays of primitives. Plus great libraries and memory management.

And then I thought "why not use it for something web-like". My experience was dreadful. After coming from Python+Django, it felt like writing Java 1.4 (not a good thing). Especially for returning JSON objects with lots of mixed types.

So I switched to Clojure and never looked back (Python would have been an equally valid choice).

Go's a great language, but it's being applied in some very odd (and potentially unsuitable) places.

[pramodliv1]

I'm running Python + Django with uWSGI. Building the app is a pleasure. With type hints in Python 3.5, the code becomes almost as maintainable as statically typed languages too.

One problem I have is that the application is extremely CPU intensive. I can't get past 35-40 requests per second with 500 concurrent users (at 4 CPU cores, 14 GB RAM), which seemed too expensive economically. (I cached as much data as possible, both at the Nginx tier and with Redis) and tuned the number of uWSGI worker processes.

Do I have to try other languages or do you think I have more room for optimization with Python?

[true_religion]

Presuming you've actually profiled it to know its CPU intensive, or you're in a field where this 'goes without saying' (e.g. you're doing a bunch of math calculations within Python)....

Then I'd suggest you use Cython for a speed up or try using PyPy which can be 200% faster without any changes.

When I used Python and worked with financial data, every live algorithm would be recoded as a Cython extension.

[pramodliv1]

Thanks for the suggestions!

[milesvp]

Without a doubt you can serve more traffic than that with python, more than likely, the bottleneck isn't your language. There's a good chance if all you did was port your codebase to another language, you'd have the same basic usage profile, maybe +/-50%.

Having done web development for over a decade. My experience is that algorithm is far more important than language. After that data structures trumps language. After that doing work you don't have to do trumps language.

First look for really ugly nested loops in your code. I can't tell you the number of times this type of things ends up in your codebase, especially since dev environments often have a small subset of data. Even an O(n!) algorithm is fast for small values of n.

Next look to see what kinds of datastructures you're using. For instance, dicts are a seductive way to store data, they have a theoretical O(1) lookup, but they have a drawback of randomizing memory lookups, which means that for subsequent looks for a large dict, you'll end up with lots of cache misses, can make each lookup 1000 times slower. Meanwhile, a tuple has O(n) lookups, but if you're finding that you need to iterate over the data, you're going to benefit from memory locality. So know your tradeoffs. Also code that had one usage profile at launch, can often have a very different usage profile a year later, so it doesn't hurt to revisit this occasionally.

Last look for code that isn't doing anything. Are you calling the database twice in a row, asking for the same data, and not using it? It sounds obvious, but over time these kinds of things have a way of showing up in a codebase, and they can really add up.

I also should mention caching at this point. It can be incredibly hard to get caching right, but the performance savings are pretty big. It goes under the bucket of not doing things that you don't need to. Do whatever you can to do page caching. Putting an nginx cache in front of the webserver serving python for the majority of your pages could easily get you to hundreds of requests per second.

[StavrosK]

I had the same experience as you. Django (and RoR, I guess?) blows everything else I've tried out of the water in terms of ecosystem, and Python is a joy to program in. Go is fantastic when I want either a small service or a script that I want to deploy as a single executable and have it be reasonably fast, but I find Python much easier to write in. However, that may be because I'm more familiar with Python.

How did you like Clojure? Does the ecosystem it compare well against Django? I tried it once during an on-site interview but was put off by the multi-minute startup times.

[afandian]

It's unfair to compare a framework like Django to a language like Clojure.

So, comparing the languages, I tried Clojure because a colleague was using it and to me it feels like the best language I have ever used professionally (comparing to C#, C, Obj-C, Python, Java, JavaScript, Ruby, Go) and I'm very happy using it. Great functional programming features, great libraries, I really like the LISP syntax and the stress I'd been having with Python kind of melted away (things like state in objects and the type system (which kind of doesn't really exist most of the time with Clojure)).

In terms of doing web type things, the Liberator and Ring libraries feel a lot like a big chunk of Django's routing and middleware functionality. Selmer does a good job of templating.

For most user interface work I'm using React.js (which I also really like) so my Clojure code is only serving up JSON on an HTTP API. But for simple templated HTML, Selmer works well.

One question is "Django is an entire framework. How does it compare to lots of little libraries?". I have to say that Django is probably the most well constructed pieces of software I have worked with and it hangs together very well. Clojure with a few libraries is very good, but beating Django is a very high bar to reach for any framework or group of libraries.

Startup time is an over-done argument. Once you're running a REPL you don't need to restart it, so development isn't slowed down. Once you're running a server you don't need to restart it, so production isn't slowed down.

[StavrosK]

I see, thanks. The startup time problem was because of tests, they had to re-run every time and the suite took multiple minutes just to start running.

Since you work with React, how do you like JS? I'd imagine that, coming from Clojure, you'd be frustrated with its inconsistencies, as I am, coming from Python.

[afandian]

I don't like JS at all. But React provides a very nice container to write as little as possible of it! And that's fine with me.

I've tried ClojureScript (without React) and it didn't quite click -- some of JavaScript's weird behaviour still shines through.

[StavrosK]

Ah, I'll have to give React a go, thanks!