[adamgordonbell]

Chet's book on the early days of Android is super interesting. Lots of crazy OS building stories. He must have interviewed 50 people who worked on the first couple releases of Android. I had him share parts of that story on the podcast and result is one of my favorite episodes.

I always thought of Android as, of course it succeeded, it was a big google project. But the story is far more interesting. They were somewhat of a small group, part of an acquisition and didn't actually merge with the prevailing google culture. Kind of kept to themselves and worked insane hours and were really into coffee.

Also, most of them had previous worked on similar projects. They had done the T-Mobile Sidekick and some of them came from WebTV, I think and BeOS.

https://www.goodreads.com/en/book/show/61431659

https://corecursive.com/android-with-chet-haase/

[raphlinus]

I joined Android in April 2012. Looking back, at that time Android was incredibly lean for what it was accomplishing. The entire Frameworks team reported to one manager, and we would fit in a medium size conference room. That included all of UI toolkit and the bulk of the Java APIs an app would call into, including Binder. (Networking, phone, graphics, Bluetooth, etc were other teams, but we worked closely together. And I think graphics was like 5 people). At the time I was slightly worried that I had joined a mature project and that it had already seen most of its success. Little did I know. (Disclosure: Chet was my manager for a couple years, until I left in late 2016 to join Fuchsia, largely because I was impatient to develop in Rust).

Chet is a great comedian, I think he'll do well in his new interest. I'll bend the norms here and tell a joke. It'll be funny to Android insiders, apologies otherwise.

At one point, he hosted a get-together in his house, which was one of many things that led to incredible team cohesion. It was somewhat chilly, and we were sitting around in our jackets shivering a bit. One person said, "we should have an Android app for turning on the heat to warm us up." My response: "we already do, it's called GMSCore."

[chaboud]

That must have been an amazing ride. Note: in my opinion, a small, lean, focused team is the right way to build an OS. I haven't seen the "throw an army at it" approach succeed for building the core of an OS or platform, as it becomes almost impossible to keep coherence in the system.

And, yeah, that GMSCore joke is too real.

[manmal]

Indeed, it was interesting when VW announced they‘d hire 10k engineers to build an automotive OS. Of course it hasn’t worked out and they needed to restart the effort (with 5k devs it seems?).

[monero-xmr]

Small, lean, focused team is the correct way to build any massive undertaking in software. Maybe not in sending people to the moon, but in building consumer software.

[jethro_tell]

Well, small, lean, and focused also implies that the time to deliver is still a bit longer. If you're willing to wait 5 years for something, then a small lean focused team is the way to go. The issue with hiring 10k devs to build the core of an os is that just by definition, you'll have people starting the user land at the same time as the core and now you have UI project managers dictating ABI interfaces at the same time as you're trying to make your core OS.

A large dev team for something that complex it the classic case of the mythical man month. Some things just need to happen in series, and some things just need to happen first.

[p_l]

Various individual components involved in sending people to the moon were also composed of small focused engineering teams.

[agentgumshoe]

Much like societies, there is no 'one way' to do it all.

Best is to really understand what you are trying to do in the first place.

[ghaff]

Maybe?

How many people worked on the core of Windows NT? On a number of the big Unixes? Various minicomputer OSs?

What you probably do need is a chief architect--like Cutler in the NT case--to keep everyone lined up.

[wrs]

Depending on your definition of “core”, I think the cores of those were indeed implemented by a handful of people. Read “Showstopper” for the Windows NT story, for example, with a small gang of ex-DEC engineers putting the basic kernel abstractions in place.

Of course, the full functionality of the system comes after lots more people build functionality on top of that core. Which is exactly why it’s important for that core to be coherent, powerful, and well engineered.

You can have the best chief architect in the world, but if everyone’s building on sand you’re going to get a crappy system. Whereas if the core of the system is solid, it will guide people into doing things better even without an architect.

[ghaff]

I've read Showstopper. Also Soul of a New Machine. Don't really disagree. A lot depends on what you consider core and what you consider a system in the words of Fred Brooks.

[bitwize]

The Mythical Man Month was literally written because Fred Brooks threw an army at the problem of writing an OS -- IBM's OS/360 -- then found it didn't work and decided to write down why it didn't work. His idea of an ideal software team he called the "surgical team" and your chief architect is the "surgeon" -- the one responsible for the major design decisions who calls all the shots. The surgeon and his team shouldn't exceed about ten people.

The Mythical Man Month is, like the Bible, one of those tomes that everyone cites with reverence, yet no one seems to read or follow the principles of. The ideal team from corporate's perspective is what I call the RAMP -- Redundant Array of Mid Programmers. The idea being that if you get a bunch of mid programmers together and have them constantly communicating, you can get the output of one good programmer without the risk of one good programmer, since you can always replace any of the mid programmers that fail or falter. But this approach has a number of drawbacks: you don't actually get the output of one good programmer this way, and you don't get the speed of one good programmer either. Furthermore, you run into the same problems you do with actual RAID: similar components tend to fail on similar timelines, so you end up having to replace all the components at around the same time anyway. Programmers tend to burn out, or quit and look for greener pa$ture$, after a few years, so you may end up losing a significant chunk, if not all, of your team at around the same time.

But if you're an organization with billions in the bank, you can remain idiotic for much longer periods of time than any of your people are willing to stick around for and attempt to positively change things.

[agentgumshoe]

The other key factor is probably the person hiring/hiring being able to identify a 'good' programmer. Whether that's through their own competence or having some knowledge of the candidate outside their resume.

Speed to market is also a factor. If you pushing to release a new product, good engineers are very important. For big corporates with established marketshare and profits, it seems to be the thousand monkeys with typewriters approach

[bitwize]

Indeed. I've noticed that the more off the critical path is the corpo's software team, the more waste they are willing to tolerate when it comes to software development. That's how you come to things like super-scalable, cloud-based, kooberneteez-orchestrated microservice architectures for internal applications which serve a subset of a division of a company. And you need a team 200 strong to service that.

[chaboud]

The real core of NT, the kernel team, could have been fed by two pizzas (if they could agree on the toppings). I didn’t work at Microsoft, but I had some occasions to meet with them (I was working on high performance media tools that squeezed a lot out of a little at the time). They kept a tight ship and designed systems that they genuinely had understanding of. Linux has had similar guidance, and my contact with the Fuchsia team in the early days showed signs of the core design tenets (e.g., object capability permissions model) having been well considered before broadening the effort and resourcing.

Different ecosystems have gone through phases of mass and focus, but times of concise clarity of vision from small groups (e.g., hardware rendering in Android 4.x, early days of CoreAudio, NTFS) move mountains that large teams never could.

[eclipxe]

As someone currently working on GMSCore…I couldn’t stop laughing.

[chairhairair]

Is there any animosity towards the later efforts to “professionalize” some of the early simpler Android APIs? I’m thinking specifically of some of the drama surrounding the banning of SharedPreferences.

[jreck]

Most of that is being driven by the same people who worked on the original APIs. Being on a successful platform means all your rushed, good-enough APIs from years ago tend to now be load bearing regret.

[FpUser]

>"until I left in late 2016 to join Fuchsia, largely because I was impatient to develop in Rust"

Shows how different people are. I only care what I am developing. I have my preferences but if generally language is the last thing I care about bar some pathological cases.

[chihuahua]

I used to think that way until I ran into one of the pathological cases (Ruby) and now I'll think very carefully about which job offers to accept based on the language(s) involved.

[shermantanktop]

Me, before Ruby: “languages are languages, I can usually pick up what I need to in order to contribute.”

Me, after Ruby: “fools rush in where wise men fear to tread.”

[chihuahua]

exactly - last year I had to learn Go and Scala for a job, and it was a nice experience learning something new. 2 languages with advantages and disadvantages, and good support from VSCode and IntelliJ. This year, I had to learn Ruby, and found it to be a mess, with poor support from VSCode and RubyMine. Apparently you can't know what methods exist on a class until runtime, so the IDEs don't can't tell you much about your code.

Just like you shouldn't be so open-minded that your brain falls out, a language shouldn't be so dynamic that you can't tell anything about a line of code until you're in the middle of executing it. Maybe it's great for cranking out a CRUD web app in a weekend, but not so great for a product that's had 300 developers writing code for 10 years.

[dmux]

I've just inherited a Rails project that's been in development for the last 10 years. I know one of Ruby's mantras is something along the lines of "Optimized for Programmer Happiness," but after working with it for a couple of months, I have to ask "who is the programmer there?," certainly not the maintenance programmer. If code is read X times more than its written, then aiding comprehension is of the utmost importance. I'm not a huge fan of Go, but I do have to say that its readability greatly improves the developer-experience.

[boredtofears]

> Apparently you can't know what methods exist on a class until runtime, so the IDEs don't can't tell you much about your code.

If you didn't know that going in you did zero research about the job you took.

[ndesaulniers]

I didn't work on Android until 2016. As someone who was very much not old guard, Chet's book helped fill in a lot of details and historical context for me. I enjoyed the read and recommend it. So long Chet, and thanks for all the fish.

[jeffrallen]

I worked with some of them at WebTV and they were extraordinary hackers there, and at General Magic before that.

I miss Bowser the Rabbit most, though.

[jimmywetnips]

I was way too daft to understand the joke, here's the very unfunny but afaik accurate explanation from gpt

https://chat.openai.com/share/a5a62c3f-aaf7-4adb-a3e1-b13780...