Hacker News new | ask | show | jobs
by seregine 3481 days ago
Having worked at both places for ~4 years each, I would say Amazon is much more of a product company, and a platform is really a collection of compelling products.

Amazon really puts customers first. Their platform and organization are made up of small teams that own services with well-defined interfaces, accountable for customer metrics. All profits are reinvested, so resources and perks are scarce, efficiency matters, and management is tight. The platform emerged because internal teams thought of their infrastructure services as products with customers.

Google really puts ideas (or technology) first; it aims to hire the smartest people and rewards them for launching new things and solving complex problems rather than optimizing UX or making customers happy. Resources are ample and management is loose, so individual contributors can try new things with greater leisure. It's been compared to grad school. But simplifying customer experience is less of a priority, so the internal infrastructure was notoriously complex and hard to use. They're now learning to prioritize customers, but it's hard to change culture.

Of course, both companies are huge and diverse and evolving, so you'll find plenty of variance.

App Engine wasn't evidence of Google being a product company, nor does it exemplify the company's strategy. It was a grassroots project that for years didn't receive much leadership support, but was still allowed to launch and grow.

8 comments

App engine is the biggest enigma. On paper it is a near perfect cloud experience. It has a huge range of services covering the needs of most web applications: task queues, caching, RDBMS, storage, monitoring, logging, a fantastic dashboard; the list goes on and the scope of GCP and GAE in general is truly massive. At first glance the documentation is exhaustive, support quick, and you get the feeling that the product has the full backing of Google with many hundreds of engineers continually improving and iterating.

Yet then you get into the trenches of it and (IMO) you realize the sum of its parts is much less then the value of the individual pieces. You feel the pain of the documentation writers who had to transcribe examples and helper libraries to ten languages, "beta" features that have been out for years, "examples coming soon" in README's that are two years old.

Want to use python3? That's cool, use the flexible environment. But it doesn't support taskqueues or many other features.

Need websockets? Thats cool, we kinda have this socket API and similar for some languages and environments. It doesn't really work in the flexible environment though sorry :X.

All our python examples are in framework X, that's sufficient for everyone using framework Y, right?

Don't get me wrong, my company uses GAE and its benefits outweigh the costs for us. But there is a very real "Googliness" to the failings of the platform. The shear breadth and requirements of "fixing" and iterating on GAE must not be a very fun project to work on.

> All our python examples are in framework X, that's sufficient for everyone using framework Y, right?

I work on GCP on the Python samples.

We generally pick Flask, since our thinking is that for many API calls, it is pretty much identical code in other frameworks, and Flask has minimal boilerplate.

We have quick starts for Django for all our platforms. I think Flask + Django covers a huge chunk of Python frameworks people are using.

If you think we are missing important Python samples, you can file an issue here:

https://github.com/googlecloudplatform/python-docs-samples

There are a lot of potential samples to write, so any guidance on which ones people want is always helpful to us.

Thanks for your comment and I want to say, though my comment was rather pointy, I respect the work all of you are doing and I do see a lot of improvement in the platform.

I keep a local branch of the python-docs-sample repo and just took a gander to refresh my memory. Specifically I see a quite a few examples using class based views based from the webapp2 package. I don't think it's unreasonable to have this as a major reference point, but it does require some extra documentation reading when converting to say, function based views in Flask.

Our personal use case is python3 in the flexible environment and I'd like to point out two notes while I have you here (if it's appropriate):

1) Are task queues coming to GAE Flexible environment - python? (and more over is feature parity coming between the google.appengine and google.cloud packages)

2) It's undocumented that the flexible environment of python requires a specific configuration variable to be set in order to make a connection to cloudSQL. I raised a support ticket for it a few weeks ago and the documentation hasn't been changed. It took me a few hours to debug it personally and I would like to save others the effort, can users make a pull request on the docs directly? For reference the variable is "beta_settings: cloud_sql_instances:" in app.yaml (it's present in the python-docs-samples but has no comment explaining its significance/requirement).

EDIT: I can no longer edit my original comment, but it seems GAE flexibly environment for python does support web sockets, though I would question the effectiveness of stateful servers in GAE. Of course thats an implementation problem and not one with GAE.

Thanks for the feedback, not pointy at all.

You're right my original comment was misleading, most of the GAE Standard samples are webapp2, but that's because it comes built-in to the platform and can be specified in the app.yaml, so webapp2 doesn't require people to `pip -t` to vendor Flask into the project. It might be worth revisiting if some of those samples should be in Flask or in both.

1) Guessing you already know you can use Pub/Sub for background tasks, examples here in our Bookshelf app:

https://cloud.google.com/python/getting-started/using-pub-su...

I think Product knows that the developer experience for tasks could be better and closer to Standard, but we haven't announced any public roadmap for task queues on Flexible.

2) I see references to that variable in our docs so I'm not sure where you're saying it's missing. Unfortunately you can't submit PRs for our docs, I wish you could.

Thanks again for the feedback, getting pretty off-topic so maybe good to take any further conversation to #python in GCP Slack?

https://googlecloud-community.slack.com https://gcp-slack.appspot.com/

No thoughts on the Python 3 issue?
(Not GP but also work on Python on GCP)

What was the specific Python 3 issue? I didn't see anything in your post specifically about Python 3.

We are indeed working on creating a standalone Task Queues service that will work across all hosting platforms. You can sign up for the alpha here: https://goo.gl/Ya0AZd

(I work on the Python developer experience for Google Cloud Platform)

This is why I come to HN. Where else can you read an article about a high profile company, then find comments from people who work on what the article is about? Unless you guys are a brigade, it's pretty cool you happen to be browsing where the rest of us browse.
I am in Developer Relations so this is indeed part of my job. I'm glad we can be helpful.
"Their platform and organization are made up of small teams that own services with well-defined interfaces, accountable for customer metrics."

You are using the terminology of products and platforms with the exact opposite meaning of the article's author.

Those "services with well-defined interfaces" become a platform others can use to build their own products. Similarly with the e-commerce and fulfillment infrastructure third party sellers can use. And maybe the transportation infrastructure in the future.

I also think you vastly underestimate the quality of Google's UX. Type any thing you can think of into one simple box, and you get surprisingly useful auto-completions, relevant spelling corrections, and almost always find the answer to the question you had when you started typing. There are any number of complex back end services working in parallel to resolve your query, with results gathered, ranked, merged, and rendered in a fraction of a second. Pretty hard to beat that user experience.

This is how the author defines product focus, emphasis on combining many components internally to provide an outstanding end user experience, without necessarily making the components available to be used by others to create their own user experience for their customers.

Products and platforms overlap quite a bit, e.g. many enterprise products can be thought of as platforms, or AdSense and Android. That's why I think it's less useful to contrast the two companies on a product vs. platform axis, and why I find the cultural differences more interesting.

Google does make great products, especially when it's a matter of presenting a simple elegant interface to a complex internal system. I'm a big fan of Google Search and Maps UX, and Google Now.

But the UX of a product isn't just its immediate interface, it's also all the interactions you have with support, documentation, and change over time, and trust. The cultural differences are more evident there, though some teams at Google are getting pretty good at these things as well.

I would state the tradeoff very differently.

Amazon gives each team independence. Therefore it is virtually impossible to insist on consistency between what different teams do. Each team makes sense on its own, but the whole can be very, very confusing.

Google has a process that results in much greater internal consistency. It may not be a great UX, but it is consistent. Inside and out.

For small systems, Amazon is going to give a better UX. But for a complex system, I prefer what Google will produce.

as a long time Google user, I find it pretty ironic to use consistency and Google in the same sentence. If consistency exists internally at Google, none of it made in to their products unfortunately.
It's improving, but you still run into some of the interfaces that were designed 8 years ago.

But Microsoft is far worse in that department.

> But for a complex system, I prefer what Google will produce

Having worked there in teams near to their tablets I really think Amazon would have a hard time producing software of the complexity of Android or Chrome.

I have a question. Do you think this Amazon culture is the cause or the result of service-oriented architecture at Amazon? Or maybe am I completely off the mark here.

I found this quote from SEC filings. Jeff Bezos says:

> Service-oriented architecture -- or SOA -- is the fundamental building abstraction for Amazon technologies.

This was in 2010. https://www.sec.gov/Archives/edgar/data/1018724/000119312511...

It sounds like they have been committed to it at least since 2005? https://s3.amazonaws.com/aws001/trailhead/MigratingAmazonCom... I'd imagine internally it'd have to be a lot sooner than that because I think they were ready to release AWS by 2006.

Sorry, I don't mean to start a holy war. I just wonder if there's any connection... What do you think?

If we use Steve Yegge's famous rant for reference, it goes back to 2002.

https://plus.google.com/+RipRowan/posts/eVeouesvaVX

Great read, thanks.
you mean like the Amazon retail site?
I'm not so sure, if Amazon develops it's own logistics solutions I would say that those are a bit more complicated than Chrome.
To be fair, Amazon has developed a Silk browser for their Kindle Fire tablets.
Silk is a fork of Chromium, Amazon just piggy-backed on Google's hard work. Just like Fire OS itself (a fork of Android).
You mean the way Google piggy-packed on Apples work which in turn piggy-backed on KDEs work? Amazon did an extension to optimise rendering on small devices, which is complexity wise not too far off to what Apple & Google contributed to the rendering engines, which at the end is the tricky bit an a browser, not the Chrome.
No. Google contributed massive amounts to the webkit code and built a full browser around it (including a state of the art javascript engine).

Silk is a different story.

Really don't understand the downvotes. Whether they gave back to the community or not is another story, but they did piggy-back on previous code, even though adding a lot themselves and giving back a lot, but it still built on massive existing work.
Apparently Amazon also puts developers into the customer support rotation. If you want to improve customer experience, this is a great idea. I've worked small-medium business support before, and you're generally treated like chaff by the devs, who don't get to wear their own cut corners and bad decisions.
> App Engine wasn't evidence of Google being a product company, nor does it exemplify the company's strategy. It was a grassroots project that for years didn't receive much leadership support, but was still allowed to launch and grow.

App Engine was a precursor that came 5 years too early.

The hype about serverless is only starting now.

We jumped on Appengine very early, and have never regretted it. It's remarkably stable, and not having to worry about the security issue du jour, scaling, or any other sysadmin stuff means we can concentrate on what we do best - building apps.

I've been surprised how few people understand the value proposition, and how little competition there is. When I first heard about Azure I expected it to be PaaS, but it turned out to be Windows-first AWS.

I'm pretty sure Azure began as Paas as well, and added Iaas later.
Isn't Azure Cloud Services pretty close to Appengine?
Not as I understand it. Both AWS and Azure force you to use machines - you may put them behind load balancers, and they may start up automatically when required, but you still need to design the system and do provisioning and plumbing to make your application working.

With Appengine, you don't know anything about the hardware of software your application is running on - you simply upload your app.

I haven't used Azure, so please correct me if I'm wrong. I believe AWS Lambda is the beginnings of a "serverless" environment - maybe Azure has an equivalent.

Azure has a ton of serverless components. Functions is their lambda equivelant. Azure has many services you can create apps and capabilities without standing up a sever. Like anything else it has some pros and cons.
Was it really just "too early"? I always thought App Engine was a fantastic idea, and I wondered why it never seemed to catch on.

Choice of languages -- initially just Python and Java? Fiddly APIs, different from competing platforms but not actually super-simple for simple tasks? Lack of a straightforward way of running background tasks (still a bit of a mess)? Lack of management support? Maybe just underpowered at first for large sites, and insufficiently compelling for small sites to build a loyal fanbase?

I just built a new, very small project in App Engine (Python, standard environment). It works fine but the tools are quite fiddly. There are plenty of docs but they're a bit of a trainwreck, in the classic Google "the old way is deprecated, but the new way is still in beta" way (e.g. standard versus flexible environments).

App Engine seems to me like the classic Google product: some cool ideas but the initial experience was clunky and the pricing model scared people, especially since you were locking yourself into a proprietary architecture. Google engineers could point to various things it did to help with future scaling needs and took the advantages of things like the NoSQL data model for granted but everyone I knew who didn't work for Google was generally asking questions like “How long would it take to migrate if they cancel the service?” or “What's my coping strategy if they have another major outage?”.

I think a little time invested on customer service and user experience would have gone a long way.

I just realised there was another problem: no internal customers. If there had been an important team inside Google keen on using App Engine, that might have helped them figure out the right feature set. But instead they were just guessing at what users outside Google might want. (Compare to Gmail, which was and is very heavily used inside Google.)
Thousands of internal Google apps (most of the important ones we use day-to-day) run on App Engine. Many external apps do too - https://plus.google.com/107043289104010976501/posts/VYybt1BC... talks about developers.google.com, for example.

That doesn't help though, as Googlers learn to write apps the Google way (massively horizontally scalable, managed NoSQL data service), which looks very much like App Engine. Outside, people still wanted to run their relational databases and large VMs, and App Engine didn't let them do that. That's why we came out with Compute Engine.

I'm sure there are other reasons why App Engine didn't catch on, but the deal breaker for me was support.

They had an issue where outgoing emails just disappeared, without errors, and no way to debug what was happening. Support denied the issue for quite some time. Wasn't fixed for 2 or 3 weeks.

Great point on simplifying the UI being a priority. This takes a lot of coordination and willingness to say "We will drop features A, B and C to make the overall experience easier." This is ultimately both an organizational and cultural issue. Some companies optimize on centralization (historically Apple) while others have a more decentralized "empower the individual" ethos. This article from the same source highlights how Apple's centralized org is core to it's view of product integration. https://stratechery.com/2016/apples-organizational-crossroad...

When dealing with a platform, this user experience coordination becomes much more important. (Think the old Apple ecosystem versus all the crap that came pre-installed on the Wintel systems)

"rather than optimizing UX or making customers happy"

as someone who's dabbled a bit w both, I found GCP UI/UX to be far simpler than dealing w AWS

AWS has fallen flat in the past with cross-service initiatives. Tagging and IAM keys come to mind. I haven't personally used Google much but from what I hear their stuff is a bit more cohesive..

This new Org stuff might be a recent exception though; I haven't read through it all and tested it to know. Hopefully.

Reading GCP documentation is a delight. Reading AWS documentation feels like reading Salesforce's documentation.
Haha, I have never read Salesforce docs.. I must say though that I'm pretty happy with their docs. Sure, they are dense. However, they have most of what you need if you spend the time to read them. I MUCH prefer comprehensive docs to those that are lacking..
This is changing - I got a call from Google Cloud sales/support today. I am just using the GCP for prototyping so I have spent maybe $50 on it so far. Looks like they're stepping up their gane and really trying to be helpful and willing to listen to customers.