Hacker News new | ask | show | jobs
by jordanb 5532 days ago
Judging from things Steve Yegge has written about Amazon, and from friends of mine who have worked there, I don't get the impression that Amazon is a company dedicated to technical excellence.

In particular I seem to recall something about 40 million lines of C++ code in their core product, and about mediocre engineering talent and high levels of burnout due to the technical debt they're required to deal with...

In fact, ever-increasing engineering costs would be just as indicative of mounting technical debt as it would of an increasing commitment to R&D.

9 comments

I worked at Amazon from 2004-2007. On one hand, Amazon does some really excellent and novel work like Dynamo and AWS (S3, EC2, EBS, etc). They organize developer conferences and bring in really good people to lecture. They hire really solid distributed systems people like Pat Helland and give them reign to work with engineering teams to improve their designs.

On the other hand, when you're in the trenches of a product team at Amazon, good engineering takes a back seat to features, deadlines, and keeping everything running. There was an incredible amount of pressure from the top to maintain high levels of service, but not a lot of support for taking the time to refactor or rework those systems to address the underlying causes of service disruption. Everything was always an emergency, but we always had to make the shallow fixes.

Overall, I'm much happier at Google where there is a lot of support for good engineering at all levels of the organization, not just as a means to an end, but as an end in itself.

Too bad that Google doesn't touch the customer service standards at Amazon.
I bought an xbox and a game for it at the same time from amazon. The xbox was fine the game disk was defective. I contacted amazon via my account, got a call from a human in less than a half hour (mainly I'd say to confirm the xbox itself was OK), received the new game a couple of days later and a refund of my return costs for the game itself. The only (very) slight quibble I'd have was that the postage on the return exceeded what they refunded. They made the return process itself simple by emailing me barcoded return labels and frequent reminders. I feel very confident buying off amazon a lot more so than ebay for example
Good engineering can't take a backseat to features, deadlines, and keeping things running. Good engineering IS features, deadlines, and keeping things running.

If you aren't making features and meeting deadlines you aren't engineering anything. You're just messing around. Features and deadlines are the difference between science and engineering. There's never enough time, that's what makes it hard.

Of course, Amazon might be pushing time tables too far. I have no idea, and it seems like you do.

I really didn't get the impression that the parent was disappointed he wasn't allowed to 'mess around'. If management doesn't give a flip about technical debt that's a problem.
There is a balance between shipping a good product with good technology. If you are constantly pushed by deadlines and make things just work, it breaks stuff in the future. Any engineer that is continuously working under pressure of tight deadlines cant ship anything creative, solid, simple and open to change.

Good engineers also have to be good analysts. What often makes a programmer better is his/her opinion to say yes and no to products people/designers. But yet again there is a balance. haberman mentioned Google but I personally don't like the over-engineered Google culture. It's too left-wing, if tight deadlines are too right.

I would say the opposite is true. Amazon is a collection of largely independent product teams that make their own decisions and share very little code. Infrastructure services are used based on its merit. Stuff that works stays, things that don't die out. Unlike most tech. companies that depend on large, common infrastructure, Amazon is carefully designed to evolve.

Main problem is that a new family of species called AWS is so well-adapted and successful that its population is rapidly outgrowing some of the older, less dynamic species on which it depends. You can call that technical debt. I'd call it progress.

In my experience, their tech job listings list tech degrees as a must-have. That seems to be a common attribute of companies that care more about HR box ticking than technical excellence, alas.
I thought about going to Amazon when I decided to get out of the startup world about 6 months ago. I went through the standard application process, and promptly closed the tab when I found that they would only accept my r�sum� in MS Word format -- rejected PDFs, HTML, and even plaintext. It may seem like a silly reason not to work somewhere, but I think that the hiring process says more about a company than just about anything else.
Of course, that goes both way. Any engineer so fixated on not using Microsoft Word, and unwilling to submit a resume in the format requested by their potential employer, might not work out well in a number of environments where they might gasp be required to use operating systems, applications, and methodologies that they were unfamiliar with, or downright opposed to.

Works out well for everyone - so, to some degree, a valid filter.

I don't have anything that can (reliably) create .doc files that consistently view the same in MS Word as they do in the application I've created them (I create and update my resume in Pages on the Mac). Creating a PDF means I know what it'll look like when they see it.

I once sent a .doc file to a potential employer, and when I arrived for my interview I saw that the font it had fallen back to (since it didn't have the exact one I'd used) was an aliased version of Courier for some ridiculous reason. It was ugly and difficult to read, the spacing was entirely incorrect, and it was generally a mess. If I were an employer and received a resume that looked like that, I'd surely count it as points against them (though it's possible that accepting Word document resumes means that this happens frequently and you get used to it).

I don't mind using MS Word at work, but I don't use it at home and have no intention of paying for it, so being able to provide someone a reliable, working document is not a guarantee unless I'm using PDF.

Google Docs? LibreOffice? Just make it simple, if they are about box ticking then it does not matter at all how well your resume looks (as e.g. a PDF version would).

I mean I feel your pain, Word Docs give me the cold shivers, but there are some tools.

None of those things will produce a word document that looks exactly in Word like it did when I created it. This is the problem.
I am not sure it works as a filter - I personally wouldn't spend my time making a .doc resume since it would enable HR drones to "enhance" it.

Yes that can also be done with a PDF, but it is difficult enough that they don't.

" personally wouldn't spend my time making a .doc resume since it would enable HR drones to "enhance" it."

You don't think that filtering out people who have issues with the HR organization processing resume's isn't a "Filter?"

it's a filter. just not a filter for qualities I want to see in a potential employer, or, in potential co-workers. i want to see intelligence, flexibility, efficiency, substance, creativity, modernity, human-friendly systems and an emphasis on ROWE, not warm bodies in seats like clockwork, or conformity. Word? I mean, in a world where PDF and plain text exists? And non-MS/agnostic systems, and HTML? Seriously?
Not everyone owns a copy of Word these days
That's ridiculous. There are plenty of programs that output MS Word format, or even RTF.
Sure, but most of them aren't 100%, and tend to lose formatting nuances. My resume looks awful when exported to MS Word, so I always have to load it up in a pirated copy or on someone else's computer, tweak it there, and then send it on. I hate the idea of using commercial software that I haven't paid for, so it just irks me that this is required.
Yeah, but the MS Word format is vague enough, that the only way to ensure that someone else sees the same document you're seeing is to ensure that they're opening the document on the same version of Word, with the same OS and the same fonts that you have.
True, but Open/Libre Office is freely available and can save to Word formats.
In my experience, for anything but the simplest document, OpenOffice will absolutely mangle all formatting.
True, but I can create a word doc using Google Docs for free.
I never went to college. I submitted a PDF résumé. Amazon hired me.
I've never applied for a job at Amazon but I have done the same thing with other companies. When I was last looking for a job, I found a couple of interesting positions whose HR systems would only take resumes in MS Word. I closed the tab and kept searching.

For me, it's a practical and ideological thing. My resume is written in LaTeX (with a plain text version) and unless I'm missing something, there is no good or easy way to convert from LaTeX to MS Word. I also have doubts about a company that insists on word documents - it smells like inflexibility and a heavy-handed top down management approach.

To convert LaTeX to word, use something like http://hyperlatex.sourceforge.net/ to convert it to HTML, and then rename it .doc.

Word should be perfectly happy reading that. And a lot of places that say they only accept Word won't notice the difference. (I don't know about Amazon.)

Actually, a lot of the places that require Word format use a tool which extracts relevant information from the resume to automatically populate their database. For some reason those custom tools don't work with text documents (go figure).
I didn't say it worked with all. But it works with a lot.
I think the source of this is that they use third party tools that automatically store/index/rip apart your resume for keywords. Theres obviously no reason why the third party tools don't support plaintext files, but it seems that they don't.

Considering Amazon didn't develop the system, I don't think the choice of software by the HR department is really something that is safe to make generalizations from.

<quote> All the effort we put into technology might not matter that much if we kept technology off to the side in some sort of R&D department, but we don’t take that approach. Technology infuses all of our teams, all of our processes, our decision-making, and our approach to innovation in each of our businesses. It is deeply integrated into everything we do. </quote>

According to Bezos you're wrong. If technology infuses all of their processes that also includes hiring.

And let's not forget Conway's law. http://en.wikipedia.org/wiki/Conway%27s_Law

Everything matters. If only because good people will ignore companies that have stupid hiring processes.

I honestly think its a bit of a stretch to think that Bezos was talking about the HR department when he said that. Do you think he was talking about the people that vacuum the floors and sell sandwiches in the cafeteria too?
You're talking about HR here; they have no idea how to use anything other than Word. The idea of opening a PDF is foreign to their way of doing business, not just because a lack of know how, but because these are people who are very rigid in their processes. If I were to disregard companies that require Word I'd probably be unable to get a job where I live. I just disregard it, and what their qualifications supposedly are.
Word-only resumes indicate that the HR department is in control of the hiring process. This means that the company is passing up on excellent people for stupid reasons ("We're looking for people with "Unix" experience but this candidate only lists "Linux" and "BSD") and hiring incompetent people who happened to have won the Bingo game.

So the HR department is in the driver's seat on hiring is a good proxy for determining how good their engineering is.

Personally I think a better indicator than if a company is they Word-only or not is the quality of their job postings. HR is not competent to write job descriptions for engineers, and the idiotic descriptions that get passed around the internet are almost always an indicator that hiring is "HR's responsibility."

Those job descriptions are stuff they find on the internet. It's divorced from the reality of the position; you only find that out when you do an actual interview. I'm not saying there aren't places where HR is in control of the hiring processes, but I also know from person experience that it can be the case where HR is in charge of posting positions and taking applications to simply hand off to Programming.
This is... odd. I started at Amazon in January, and as of November, plain text was the preferred format for resumes. Then again, mine got into the system through the employee referral program, so perhaps it takes different inputs.
For what it's worth, their job site now says you paste your resume into a box: http://www.amazon.com/How-Apply-Help-Careers-Homepage/b/ref=...

I don't know about 6 months ago, perhaps their policy has changed.

I pretty much do the same - mostly since I decided to typeset my resume a while ago, and converting it to anything, particularly MS Word, just feels like a travesty.
> closed the tab when I found that they would only accept my r�sum� in MS Word format

Why not just send it in a decent format and see what happens? Just because they say they'll reject it doesn't mean they will.

I did -- I initially just dropped it in there, then saw the ".doc only" notice after it errored out.
I am truly surpised by the rigidness displayed regarding Word docs. Limiting opportunities in your life because an HR department uses the most common document app in the world seems self-defeating. Why not rule out cmapanies that use a certain type of printer paper?
I have no problems with whatever document app the HR team is using. If they send me a doc file, I will read the information contained therein. In creating a document the author puts in more work than the reader; so as long as the format is not too onerous the author should be able to choose whatever tool is convenient to him/her.

I hope that you agree that txt and pdf files are just as convenient formats to read as doc ones. IMHO it is irritating when someone insists that any document you send them should be in the format of the application they write documents with.

Furthermore, these are no absolutes. If the State asks me to send a document in doc format, I will. Despite pretenses, these companies (and HR depts) are not Republic of Greater Timbuktu really - so I will skip, thank you. :)

It's a "bad smell". Too many bad smells and you don't apply for a job, it's like Bayesian filtering.
Bayesian filtering that works both ways. In fact we currently take PDF/Word/txt/RTF at my job. I'd actual considering narrowing it down to just Word. The type of person who decides that they don't want to submit in Word is probably a type of person that's not a good cultural fit. And I guess we've already ruled out those that prefer to send their resume via smoke signals only.

As an employer it is more important to find good cultural fits than it is as an employee, since it is typically easier for employees to leave than it is to fire them.

My CV is a PDF because I made it in LaTeX. I did that because .doc is not a standardised format, does not make the promise to look right on any system, and doesn't support some of the nicer typesetting I have going on. In my opinion, forcing people to use a proprietary locked-down format is the kind of thing that should be fought. What really gets my goat is that most of these places still don't accept .docx, even though it is standardised, well-supported, and absolutely TRIVIAL to pull plaintext out of for indexing purposes.

However, because I took the initiative to teach myself an industry standard text markup language in order to make a CV that looks really nice and will always display and print properly (and is distributed in the most widely-agreed upon document format for that purpose), you want to exclude me? I'm not submitting in PDF to be difficult or to take a stance, it's a convenient, standardised and well supported format which I am using for its intended purpose.

Comparing smoke signals to PDF is completely disingenuous, as is bundling together "people who happen to submit their CV to you as a PDF" and "people who would flat-out refuse to submit their CV as a .doc".

I don't want to send my resume as a Word document because I wrote it in LaTeX, and it looks correct as a PDF. I don't understand why you wouldn't do me the courtesy of accepting the PDF.

This seems like a good cultural filter to me (on both sides).

I wouldn't rule out an employer for this, but it's annoying.

I use Ubuntu at home. As someone who will need to work with Ubuntu Server at work, I think that's a good thing for my employability. Using Microsoft Word in Ubuntu is not an option for me.

So it's a Giant Pain in the Butt to produce a Word-formatted resume (no, OpenOfice and Google Docs don't make it reliably look good), and there's no reason they need that because they're not going to edit it.

On the other hand, it's easy for me to produce a PDF, which will look better anyway, and it can be done using the OS I will be using at the job I'm applying for.

Being told, effectively, "we require you to install Windows so that you can submit a nice-looking resume for this job working with Unix" is annoying. It's a small factor in how interested I am in the job, but a factor nonetheless, because it shows (to me) that technical people are not running the company.

I was hired with no college degree, when Microsoft and Google would not even consider me on that basis.

Also, for Amazon and other companies, there is a world of difference going through their public-facing recruiting process and getting to know someone on an interesting team through your network, and getting hired through them.

I'm sure the same rules of networking apply anywhere, though, including Microsoft and Google. Degrees are an HR filter, but networking can bypass HR. So it goes.
More than that, it is very easy for only one programmer or researcher doing a bad job to create technical debt that will take disproportional time to pay off.
I've seen some negative stories about working at Amazon. I don't recall seeing anybody write that it's a great place to program. Anybody have any stories or links about the upside?
The upside is that the company has excellent developer tools and infrastructure (I mean aside from AWS), very accessible principal engineers, a huge amount of paying customers so your work has real impact, and it seems like most teams have autonomy to work and deliver in their own way. Within a week or two of joining I was given ownership of an important internal product while also getting up to speed on the main event (the retail website). The technical challenges are there for the taking.

I also mentioned some of this at http://news.ycombinator.com/item?id=2488980

The opportunity to have an impact cannot be overstated. I worked at A9, the search subsidiary of Amazon. Given that a substantial portion of purchases go through search, we were always keenly aware that even improvements that appeared to be modest could have very substantial impacts to the bottom line.
I know a lot of people who work/have worked there, and it seems to be a very love it or hate it company. They work people hard, but the people who like that kind of environment thrive. They seem to be pretty ruthless about improving in certain areas and are very metric-focused. I'm not sure it's a great place to program, but it's certainly a great place to learn about programming.
I've heard it is very dependent upon the team.
I get the feeling that you're both right and wrong. Right in that a lot of the legacy Amazon code is probably burdened with huge technical debt. I also get the feeling they're pushing the boundaries in other areas. A bit of the "chase the shiny coin" syndrome.

One of the issues is that none of us really know what goes on inside Amazon. They are notoriously secretive about most everything.

"Secrecy is the beginning of stupidity." It both reduces feedback, often dramatically, and wastes your time and energy in maintaining the secrecy.
unless its APPL....
Apple's ticker symbol is AAPL.
Yes, because users always know exactly what they want and provide valuable feedback..
> 40 million lines of C++ code in their core product

That was a long time ago. As Jeff Bezos confirms in the article, Amazon is a constellation of services now.

Yegge left a long time ago.
I don't see technical excellence having a lot to do with technical debt in terms of if you are excellent, you have little or no debt.

Business needs change too quickly for a system as large as Amazon; they simply cannot maintain net zero technical debt and not cripple their reaction time in the market.

The key is to know what level of debt is necessary and acceptable to take on and deal with going forward.

Maybe not working at Amazon, but their suite of companies are still technically innovating : their http://www.a2z.com subsidiary is doing some amazing stuff, with top-tier talent.