> It means that scalability would become an issue if and when Binary Duel is used by more than a few thousands of people an hour.
In this fortunate case, I will be very happy to host the infrastructure on a K8 cluster with autoscaling, self-healing, a distributed database, a Redis server and so on.
Yeah, it is kind of provocatory. Of course you can't build a business out of an in-memory game, but it is fun to ship something as it is. Without thinking too much about scalability.
I don’t disagree with the premise for very small applications, but
> The quiz state machine is full in memory
> The queue system for the matchmaking is in memory
Means those will be nuked as soon as a new version gets uploaded. it really wouldn’t have been much extra work to put this into SQLite as well, and would lead to a much better user experience during updates
You are right, but i can roll updates during the night without causing any issue.
Since after you finished playing you don't have a way to access your old games anymore
That works for the specific case presented here. As soon as you have users from other timezones, ”during the night“ becomes a really complicated concept quickly.
> i can roll updates during the night without causing any issue
This isn't universally true.
I work on a telephony platform, some components of which need to be upgraded very carefully, because if you just restart the daemon without first transferring load elsewhere then you drop people's calls.
OK, there are less calls at night time, but there are calls all of the time and people don't expect their calls to randomly drop just because it's night time.
I wanted to learn some Go and I thought "I will just reimplement my music library with it". So basically the task was to write a backend API that maintains a database of music files with associated ID3 tags, and so on.
The original backend is Python over SQLite, which works fine enough.
As I started to reimplement the functionalities, I though "why should I use a database anyway ?" and started just pushing the ID3 tags a list of structs. It turns out that having an in-memory list of structs for 50k music files is not a problem at all for a raspberry pie. It's not that musch memory and it's fast enough to create so that I don't have to persist it and it's fast enough to scan for a search.
In the end, I may use SQLite anyway because FTS5 is just so good, but yes... sometimes you don't need a database.
Missing from the title is "enough" for what? For any trivial personal project? Python, Ruby, PHP are each enough. There's not a need for literature about personal low-risk development or deployment.
That said, Scalability? But at what COST? [http://www.frankmcsherry.org/assets/COST.pdf] was a good note that orienting around scaleable technologies can be very inefficient- scalable does not mean either fast or economical.
I like the idea, think about what you really need, keep it simple.
I also try to move centralized computing to decentralized computing more and more.
And that can also be done to let the client do more computing.
And also storage, is it needed to store in central, or can the user store it.
Many times people thinking that we should have a central system, with all the truth.
But let it be a part of the users.
It makes also the central part simple, and because the central part is simple, it is easier to scale.
I am a fan of "distributed computing" - but I do not make it complicated.
I have created a generic broker daemon, which reads messages (in) and places them in the appropriate service (queue) which, at this time, is using sqlite.
Then, a bunch of daemons (workers) connect to the broker service, and processes anything in the queue (process) and tells the broker when the message is OK or errored (output)
I also have commands to check a service queue, active workers, etc.
The broker is generic, simple, and fast. The workers do the actual, specific job.
The communication between the client<-->broker and broker<-->worker is ZeroMQ.
There is also a "publish" if you choose to join, which some client programs do.. so they can see any errors on-the-fly.
Installation/deployment is on a linux server using SystemD. Nothing fancy or complicated.
Purpose is that this broker could serve useful for many projects I do in the future.
If the data in your app doesn't matter at all, and you have no availability requirement, sure keep everything in memory. I don't see what Go has to do with it however, any language should be fine.
The author states that scalability doesn't matter here. Any language would have been fine. The title should be "do everything in memory in a single process until you can't ".
We developers take proud for being able to get our head around a complex solution. And unfortunately sometimes that becomes a driver for building complex solutions for problems, that otherwise could have been solved much simpler. But in that case we loose interest.
Me, personally, I often find myself in situations like this. Where if I clearly understand the solution in my head, I might be a bit reluctant to actually code it
and get the job done. Kinda like, you know the solution exists, and then you move on to something more interesting.
You can get pretty far with simple in memory solutions when your problem space fits entirely within the constraints. We have a Go microservice for linking accounts to their proper federated logins with some associated metadata. There’s a refresh process that runs every fifteen minutes.
The entire set fits happily in memory and takes milliseconds to respond.
> We need to choose a database: so, let’s start with that.
...Wait, ten years ago the advice was that this should be one of your last decisions: your application should be able to work with minimal changes with any DBMS, and tying yourself to a specific vendor from the start was considered a bad idea. Did it change while I was not looking?
Can attest - for similar use cases. I did much the same (shameless plug, just like the original): https://friendsyeights.com/
Game state is kept in memory, the server is single process (Node.js as it happens), connections use websockets, it just works.
Does it scale? No. Could it scale with a similar approach? Sure, with some additional work: separate games could be distributed over many instances (a sharding strategy). There's no redundancy there, but it's a game, not a credit card processing system.
However, I do snapshot game state in JSON files on disk so that games are not terminated by every new deployment. In a single-process model, that's safe when using synchronous filesystem calls. Just don't try it in a multithreaded system. Though it may take many moons, you will eventually lose your "poor man's database" to a race condition.
Still, would I use this technique to host people's taxes? Hard no. I wouldn't even use it to host their blog posts. Durability matters for those use cases.
Worth shipping, but not worth investing massive amounts of time in.
On the other hand, suppose you actually hoped to grow and make enough money on something to pay the salary of a few dozen people. Let's say you start with golang and sqlite3, ready to handle 1000s of people an hour, everything is going great, and then someone posts your thing to Hacker News (or it goes viral elsewhere). Suddenly you have millions of eyeballs an hour, of whom your non-scalable infrastructure turns away 99.9%. What's the chance you get another shot after you've re-architected everything to be more scalable?
I don't have a good idea how likely that is; at some point I seem to recall hearing that Stack Overflow was run on just two servers, made possible because it was just efficient. But it's certainly something to take into consideration if you're hoping to grow large; and only something to ignore if you don't really care.
Half way down the article: The questions are stored in a sqlite database