Ask HN:"Engineering” a sequence a projects to become a more confident generalist

[gaandmaraa]

Is there a particular path that can truly accelerate one's learning to become a true generalist (ability to stitch layers together with confidence). I've been lurking around HN for a while now, and when questions of this vein are asked I've often noticed that people usually say something to the effect of "dude. just build stuff" which frankly is really a good way to get started (definitely beats analysis-paralysis). But I'm trying to understand if there's a particularly clever way to sequence the "building" of said projects in a particular way that leads to incremental insights that in turn converge to a less frustrating and more fulfilling that could give me a big-picture understanding of how things are connected end to end.

I'm looking for something along the lines of:

Step1 => Build a ridiculously simple MVC app like a todo list that lets people add things to an online list (like http://todomvc.com/)

Step2 => Back the app with some RESTful backend service that includes persistence (ex: mySQL/redis)

Step3 => Add simplified monitoring to the app to get a sense of how your app is performing (ex: How many 4xx errors yesterday)

Step4 => Hook your monitoring to some simple analytics processing (ex: How many ppl from SF have added things to the list)

Step5 => Deploy it onto AWS and configure your machines to figure out 2 flavors of the page (A/B testing + DevOps)

Step6 => Make your analytics processing "offline" by queueing events from the RESTful service into an MQ

Step7 => Make your analytics interesting by storing this data (warehousing) by using things like elastic search to answer more interesting questions (ex: What are the 'general' things ppl from SF tend to add) etc.

The idea here is to tackle things that progress naturally and minimize any new dev frustration by incrementally adding more bells and whistles. Does the HN community have any opinion on such examples that they've found worked for them and things they'd recommend to relatively new developers (1-2 years of experience)?

[jtchang]

I like angersock's list since it shows a lot of areas that I would expect a generalist to know. Obviously not exhaustive but it's a pretty good list.

That said you are going about it wrong if you are looking for a linear path to become a generalist. It is more like a giant graph of areas and you start from one you like and branch. The "right" node to start at is what you enjoy the most.

[angersock]

That is a very tame definition of "generalist" you have there.

Here, let's propose some things I'd expect a generalist to be able to do:

  IT
  ===========
  
  - Provision wired networks for small businesses/homes/labs
  - Do basic troubleshooting of printers and networked printing (and know when to throw in towel)
  - Setup and debug towers and rack servers from parts
  - Provision wireless networks using access points for small businesses/homes/labs
  - Setup and debug mail clients
  - Setup and debug file servers
  - Setup and provision virtual machines with network access
  - Setup and debug scripts for common administration tasks
  - Setup and secure user accounts on remote systems with centralized management
  - Setup disaster recovery systems and policies for above
  - Write project plans and deliverables for doing above
  - Write cost analyses and bills of materials for doing above
  - Explain existing service providers and when they'd be preferable to doing things in-house
  
  Web
  ============
  
  - Write front-end JS, without frameworks, to do basic tasks
  - Write single-page app with proper history integration
  - Write front-end JS using AJAX endpoints
  - Write functional JS where appropriate
  - Write CSS
  - Write CSS with less pain using LESS/SASS/Stylus/whatever
  - Write CSS with rough idea of compatibility concerns
  - Write HTML from scratch
  - Write responsive web pages (and know when this is a fool's errand)
  - Explain how HTTP works, what it gets wrong, and what alternatives exist
  - Write JS using websockets
  - Write JS using webworkers
  - Write JS using canvas (2D or 3D)
  - Provision a basic website using turnkey hosting ( Heroku, etc. )
  - Provision a basic website using virtual hosting ( Linode, Digital Ocean, etc. )
  
  
  Software engineering
  =======================
  
  - Write useful and succinct comments in code
  - Write useful and succinct documentation for projects
  - Provide useful root-cause analysis for production failures
  - Provide reasonable estimates for project completion
  - Provision and use a centralized version control system (CVS, SVN, etc.) and explain its uses
  - Provision and use a decentralized version control system (git, mercurial, etc.) and explain its uses
  - Write useful and approriate tests for code (TDD, BDD, library unit tests, whatever)
  - Provision continuous integration services for an application
  
  Systems
  ==============
  
  - Explain how different data types can be represented in memory
  - Explain how filesystems work
  - Explain functions of a kernel
  - Explain at least 2 approaches to operating systems design
  - Explain basic architecture for at least 1 processor family, and be able to provide design considerations (MIPS, x86, x64, Itanium, ARM, SPARC, etc.)
  - Write organized project in a systems language (assembly, C, C++, D, Rust, Go)
  - Create build environment for native code projects
  - Create makefiles (or platform's equivalent)
  - Write in systems language: hash table, linked list, resizable array, binary tree
  - Write networking code in a systems language supporting IPv4, IPv6, and which can support lookups by hostname
  - Write UDP code with reliability mechanisms
  - Write TCP client code, explain common mistakes
  - Write multithreaded code, explain concurrency approaches
  - Write asynchronous code in a systems language
  - Write cross-language binding for systems language library ( Ruby over C, Python over C, etc.)
  - Create and document binary format for storing structure data to a file
  - Create and document binary format for streaming and message-based communication over a network
  - Explain why we prefer JSON for above purposes
  - Explain why JSON is slow and dumb, and we'd rather use <CapnProto, protocol buffers, STOMP, whatever > instead
  - Write code handling file IO using normal methods, using asynchronous methods, using memory mapping in systems language
  - Write code performing IPC in systems language
  
  Databases
  ===============
  
  - Provision an RDBMS (MariaDB, Postgres, whatever)
  - Write basic queries for fetching data
  - Write useful and succient data model documentation for database
  - Explain performance considerations for hardware running that RDBMS
  - Perform maintenance (backups, benchmarking) on RDBMS
  - Provision at least one NoSQL solution (neo4j, MongoDB, Redis, Cassandra, etc.)
  - Perform maintenance (backups, benchmarking) on NoSQL databases)
  - Explain when to use that NoSQL solution in preference of an RDBMS, and why.
  - Perform ETL using above systems
  - Perform map-reduce using above systems
  - Collect real-time time-series data using above systems
  - Connect to above systems using web framework of choice, using manual and secure queries
  - Connect to above systems using web framework of choice, using an ORM
  - Scale database of choice across multiple machines
  - Perform failover of database of choice across multiple machines
  
  Maths
  ==============
  
  - Understand basic linear algebra and rigid transforms (i.e., 2D graphics)
  - Understand basic statistcal properties
  - Understand basic themes of intergration and differentation
  - Understand IEEE 754 floating-point and basic numerical issues in computation
  - Understand trigonometric functions and their relations
  - Estimate computational complexity of implementing above

This is not an exhaustive list, and I don't expect everyone to be 100% on it. That said, if you call yourself a "generalist" and can't do like at least half of everything in each of those categories, you're either a novice generalist, or you're full of shit.

EDIT:

I left out so things applying to programming languages, applying to machine learning, applying to graphics, applying to electronics, and several other things. Being a generalist is hard.

This is also why a good one is expensive.

[gaandmaraa]

That's a massive compendium! I wish I had been more crisp in my question; the thought over here was to figure whether there's a simpler step by step process to minimize frustration and maximize your confidence on your path to becoming an enlightened generalist (which would naturally takes years to accomplish). I'm mostly interested in the "sequencing" of these steps which is why I provided a rather contrived example of the web app that grows in features with each iteration. For junior devs it's usually pretty hard to figure this out on their own so they end up either doing a deep dive in a very limited number of technologies (ex: Building a webservice in Spring + Hibernate ecosystem comes to mind) without or just float up top at very superficial layers.

You've done an incredible job at detailing the expectations of a generalist but might you have some recommendations on forging that path for people interested?