Hacker News new | ask | show | jobs
by nodesocket 3871 days ago
Google compute has taken some awesome strides in the last year. I now prefer GCE over AWS, because of pricing (per minute) and simplicity. AWS quickly can get overwhelming and overly complex.
4 comments

Not just Compute, Google Network, Cloud Storage, Container Engine (Kubernetese is OpenSource, so you can run it internally too), Cloud Shell, Pub/Sub, their new Cloud Console, are awesome.
Pub/Sub is pretty meh. The Google Cloud team still don't seem to understand the need for a Kafka-like unified log service, unlike AWS (Kinesis is 2 years old this week) or even IBM Bluemix (who just launched Message Hub, which is true hosted Kafka).
Depending on the sort of logging you're looking for, there's a logging API (https://cloud.google.com/logging/docs/api/) and you can also stream to BigQuery.

(Disclaimer: I work on GCP)

This is the best, but I keep thinking that I will no longer need ELK, but still do. Can you create a dashboard system that runs off of bigquery?
One thing you gotta remember is, PubSub charges per volume, regardless of speed (in other words, scaling is free). AWS will charge you varying orders of magnitudes for varying scales, in addition to volume.
What's meh about it versus Kinesis?
Cloud Pub/Sub is really a competitor to Amazon SQS, not Kinesis. It's more helpful to think of Kafka and Kinesis as databases containing first-class, immutable streams; writing to the streams and reading from the streams are completely decoupled, unlike in a traditional pub-sub system. Jay Kreps' blog post explains it better than I can:

https://engineering.linkedin.com/distributed-systems/log-wha...

In my limited Pub/Sub experience, this seems to be how it works. You publish to a topic (an immutable stream), and then create a decoupled subscription that reads messages from the topic. Am I missing something?
I think this sentence [1] helps to explain the difference:

> When you create a subscription, the system establishes a sync point. That is, your subscriber is guaranteed to receive any message published after this point.

[1] https://cloud.google.com/pubsub/subscriber

With Kafka or Kinesis, I can write events to a stream/topic completely independently of any consumer. I can then bring as many consumers online as I want, and they can start processing from the beginning of my stream if they want. If one of my consumers has a bug in it, I can ask it to go back and start again. That's what I mean by an immutable stream in Kafka or Kinesis.

Take a look at PubSub + Google Cloud Dataflow combo.
There are some great aspects to Google Compute, but I hope this change results in catch up on features needed for corporate adoption compared to AWS.

The lack of a billing API and the lack of centralized management are really painful if you're trying to adopt it across an organization.

There is a billing API: https://cloud.google.com/billing/

There have been some recent introductions of account and key management that help too.

Excellent, thanks. Hopefully we'll see tools like Cloudability support it then.

I'll check it out again, really the frustrating management aspect was the lack of org oversight over multiple projects started within your domain.

Pricing is fine. Tech support is a joke. I wish I could pay more for Google to actually offer good support. We tried GCE for a while but it just didn't cut it in terms of mean response time to questions. AWS really leads on support.
We do have paid support tiers (https://cloud.google.com/support/) with explicit time-to-response targets. Which were you using?

Disclaimer: I work on GCE.

Well tbh we got answers but they usually were; nope not offered, cannot be fixed. 3 weeks later I met a Google engineer working with GCE at a party who fixed it for us the day after hearing about our issue, which means it probably was able to be fixed from the first time.
I will admit that the party support plan is a bit pricey.
Sounds like we should throw more parties ;).
I think this is a top priority customer ask; I'm running it up to our new brass.
I haven't used GCE but I've had one of the worst support experiences ever with Google Cloud Billing Support while using Google App Engine.

I had a very simple question about billing (why was my bill higher than it seemed like it should be). Each reply would take a week and they'd often consist of copy and paste messages asking me to enter information I had already supplied or requests that I take screenshots of my console (all information they should readily have available). Then right at the end they swapped out who I was talking to for someone else and asked me to look up more information they already had and just ignored my question in the last email which would mean another week before I got an answer.

I had, luckily, experimented with the configuration and figured out what was wrong. The default instance class in reality is one higher (F2) than the documentation says it is ("If you do not specify a class, F1 is assigned by default."). Nowhere on the Console does it list what instance class is being used (which would have made the problem obvious) so there was really no way of knowing this without just guessing what the problem was. They never did answer my question "What is the default instance class?" (instead just abruptly ended the support ticket after proposed my theory about what was wrong).

Then I started getting emails about a billing account being past due. It was an old billing account from before I moved to High Replication (I have no idea how I ended up with two billing accounts...it was during the dark time when the console was even worse than it is now). That billing account was assigned to no projects and had no oustanding balance. I jumped in and just deleted the unused billing account. Then a few days later they sent a scary email saying that the billing account had been terminated (even though I had deleted it) which made my scramble to make sure they didn't close my in-use billing account out of nowhere (they hadn't thankfully).

None of this has left me with any confidence in Google's Cloud offerings.

I plan to migrate off GAE as soon as I can rewrite the app (luckily it's not very big).

Can you say more about your experience with AWS support? My experience has been consistently quite bad. I've had to resort to AWS technical support perhaps a dozen times over the last few years. They're always slow to respond (even if you have an SLA promising otherwise), and it always takes half a dozen go-rounds to get to a resolution. They will stick with you until the issue is resolved, but they don't add much value along the way. It's a slow process of "did you do obvious thing X" (even if you've already provided facts indicating that couldn't be the problem), "please provide six different pieces of information" (that don't seem to bear on the actual question), and "I've never seen this before, let me go research it".
For the lowest support plan (Silver), high priority issues were addressed pretty quickly. Low priority ones are a bit slow, but hey, they're low priority. Plus the Silver support plan is very cheap.
Has anyone tried Cloud BigTable? Performance numbers are compelling but I'm not always sure where it fits in with the rest of the GCP storage options.
Bigtable is best thought of as an "event database". High reads, high writes, single index, accessible through the Hbase API. Cassandra and Hbase are similar technologies that are inspired by the original Bigtable paper.

One big benefit of Bigtable is its scalability. To scale up, you turn the 'scale' knob. By contrast, Cassandra and Hbase are headaches to scale (Apple has acquired Cassandra companies to aid in operation and scale).

Here's a couple of guys from Sungard, who scaled to about 3,000,000 writes per second with a couple weekends' worth of effort (something only few beyond the likes of Facebook, Netflix, and Apple can achieve) https://cloud.google.com/bigtable/pdf/SunGardCATCaseStudy.pd...

Hey. I'm one of the "guys from SunGard", although I'm no longer there. The longer version is this: https://cloud.google.com/bigtable/pdf/ConsolidatedAuditTrail... . A lot of it is related to the use case, but yeah, Bigtable handled pretty much whatever we wanted to throw at it. No other cloud provider can offer this sort of scale and performance right now without a ton of manual management or significant compromises, something that seems to have yet to sink in (although few companies need the scale we went up to).

It did take a lot more work than "a couple weekends" though :).