| As far as system design is concerned, I feel like I have an answer as to why they're like this. Typically, FAANG (and wannabe-FAANG) companies have overindexed on algorithm questions; which has been meant to draw people from a research/mathematics-heavy background which was well aligned with their initial needs by which they committed a sort of "original sin." Since then they've forgotten to balance these requirements against the business applications they build, which require CRUD-heavy work and usually requires knowledge of how systems (databases, queues, performance tuning, load testing, etc.) work. Because said companies continue to overindex on algorithm skills, this means that aforementioned knowledge is deprioritized. When systems fail to deliver the expected performance, these software engineers seeking new solutions with wild tradeoffs that may not have been required if they were able to come up with the right system model for the software. As an example, at two FAANGs that I've closely studied (and one of which I worked at), people always seem to be selecting serverless functions for scalability, and then adding queues and another set of serverless function to work the queues and slowly write to a database, only to work around the fact that it may have been easier to just have a background thread in a server-based model that processes results and commits them. (This is just one example; I'm sure there are cases where a queue may have been necessary to facilitate an async process. This doesn't discount the fact that serverless is usually the wrong model for most stuff out there.) It also has a secondary effect on the software engineering market, where said engineers are now able to market their overly complicated solutions, selling it as THE way to build software and dismissing everything else as a toy. This also helps said FAANGs to market their cloud services; with sales people excitedly speaking about some supposed "innovation". I remember a video from a FAANG organized event where they talked about how their database supports PITR upto the second. At the time, I was young, impressionable and lacked said knowledge, and came away mesmerized about how they could do this and thinking about how I could never write code to do that. Many years later, having read some introductory material about distributed systems and MySQL internals, I happened to remember "why wouldn't any database worth it's salt support PITR; after all, it's just replicating the WAL entries to another host!" However, said practices have already taken hold, and it has lead to a generation of engineers who would continue to seek cloud services and design overly complicated solutions. |