AWS Web Console Down?

[typicalrunt]

https://console.aws.amazon.com/console/home

It appears to be down, saying "Website temporarily unavailable". But it's intermittent for other employees at my company so it might be failing app services behind a load balancer.

CLI access or EC2 services doesn't appear to be affected, at least from us-east-1.

And, of course, the status health dashboard shows Green across the board. https://status.aws.amazon.com/

UPDATE

HatchedLake721 mentions that it appears to just be us-east-1 Web console that's down. Use this instead for now: https://us-west-2.console.aws.amazon.com/console/home?region=us-west-2

Thanks @HatchedLake721!

[whatthesmack]

It seems that the issue may be specific to us-east-1...

https://console.aws.amazon.com/console/home?region=us-east-1

...doesn’t work, but:

https://us-west-2.console.aws.amazon.com/console/home?region...

...works.

[codecrane]

Both of those are working for me. Much appreciated!

[fiveoak]

It's annoying how often status pages for various services (when they even exist at all) show that things are working when they really aren't

[EnFinlay]

I want status pages to show traffic to the status page itself over the last 24 hours (or some time period). A sudden uptick in traffic but green across the board would indicate that there is an issue, but they just haven't updated the page yet.

[dogecoinbase]

Yes! I have yet to convince anyplace I've been working to do this, but I do make increases in traffic volume to status pages generate non-critical alerts. Has caught out a few problems much earlier than they otherwise would have been discovered.

[MadVikingGod]

Sounds like an interesting way to make developers (or ops) life hell. Just point a traffic cannon at the status page and bingo bango non-critical page.

[brianwawok]

If you can write a perfect automated status page - can't you basically write perfect integration tests and make sure no bad code gets deployed?

Bit of a chicken and an egg problem :)

[chabreck]

Do you write every single test for your codebase? Do you have control over every line of code that gets deployed? The purpose of monitoring is to detect issues, because preventing them entirely is next to impossible.

By your logic if you can write perfect integration tests, can't you write perfect application code that doesn't need to be tested?

[dahfizz]

I think you've missed the point.

OP is arguing that a perfect codebase is not possible, therefore it's a bit unfair to complain that the status pages do not work perfectly. Hence the chicken and egg problem of "if I could write a perfect status page, I would have skills such that the status page would not be necessary".

[brianwawok]

There are two ways to have a status page.

1) A guy with a button. It's right, but it requires the guy to notice or get told - and remember to push the button.

2) Automation. But your automation has to somehow catch bugs that you can't even dream of. And not be triggered randomly. And somehow be able to be automated but not fixed at the source (as if you know how something will fail - why not just fix that over adding more status checks?)

[jcwayne]

That may be true for small failures, but the AWS Status Page has routinely failed to report large scale outages.

[loudtieblahblah]

It's bc they're manually updated.

[xur17]

I've always wanted to build a public status page for services like Google Cloud and AWS with real tests against different services they offer. Not sure there is a good way to monetize though.

[kureikain]

https://statusgator.com/ does that.

[xur17]

> StatusGator monitors the service status pages of more than 410 cloud services.

It looks like they are just an aggregator for status pages. I want something that does its own testing to determine if a service is up or down.

[skrebbel]

I've always wondered about this strategy. I'm a developer and a business owner. I'm in AWS's core target demographic. I know three things about AWS:

    - Their products have weird names
    - Their web console has horrible UX
    - The lie about uptime and server status

I'm not looking to switch to AWS soon.

[philg_jr]

I take it you have never used Azure's UI? AWS is much better. I can't comment on Google Cloud because I've never used it.

[KZeillmann]

Some of the AWS UX is okay but the parameter store UX for the web console is some of the worst I've seen. It's difficult to navigate and paginate, and the search function is barely worth using. It would be amazing if they had fuzzy search on the names.

[philg_jr]

I do agree that the CloudWatch console search is like this, where you need to type out the entire string from the beginning to get it to match. But for the CloudFormation and Lambda consoles, I can just type a piece of the string I'm trying to find and all matching will just show up.

[gtsteve]

The UX is a bit poor and there are long standing issues like the S3 console timeout bug and the SWF console not obeying any known law of web design but serious teams don't really use the console, they use tools like Terraform, CloudFront or they roll their own automation.

I use the AWS APIs almost every day but probably log in once a week or so.

[zedpm]

FWIW, if you're already logged in to the console, it seems to be working just fine. In fact, if I try to go to a specific service page, e.g. https://s3.console.aws.amazon.com/s3/home?region=us-east-1# instead of just browsing to the console home page, that also appears to work.

[PDFormat]

This way worked for me too. I was able to go straight to the Lambda console.

[DonHopkins]

At least AWS doesn't start spamming you weeks after servers go down!

I canceled a dedicated server with IBM SoftLayer (nee ThePlanet), and a few weeks later I started receiving hourly IPAlerts about it being offline!

The server was canceled so there was nowhere in the interface for me to turn them off!

I opened a ticket, and they said other users were experiencing it too, and they though they had it fixed, and asked it I was still getting them. I was.

Their only suggestion was for me to make an email filter to ignore the IPAlerts, but what about the IPAlerts for servers I hadn't canceled that I actually want to see?

We went several rounds of this, each time they thought they had it fixed, and asked if I was still receiving them, and of course I was, like clockwork.

It's been more than a week and a half, and I'm STILL getting them!

I kept posting the raw email bodies so they could tell by the headers where it was coming from.

I even begged them to deploy one of their most powerful firewalls around the offending legacy nagios server to protect me from it, but they wouldn't do that.

I'm afraid if I cancel my other two servers and move to AWS, they's start spamming me with TWO MORE never-ending sets of IPAlerts about canceled servers!

What a passive-aggressive way of punishing long time customers for canceling their servers!

Has anybody experienced anything like this with AWS?

    Received:  from ipalert05.dllstx6.inside.theplanet.com
        by mx.softlayer.com with esmtps (TLSv1:DHE-RSA-AES256-SHA:256)
        (Exim)
        (envelope-from <[e-mail]>)
        for [e-mail]
        id 1hJhFu-0003RG-Tm; Thu, 25 Apr 2019 11:29:50 -0500
    Received: (from nagios@localhost)
        by ipalert.theplanet.com (8.13.6/8.13.6/Submit) id x3PGSB17029986
        for [e-mail]; Thu, 25 Apr 2019 11:28:11 -0500 (CDT)
        (envelope-from nagios)
    Date: Thu, 25 Apr 2019 11:28:11 -0500 (CDT)
    Message-Id: <[e-mail]>
    To: [e-mail]
    From: <[e-mail]>
    Subject: PROBLEM: xxx.xxx.com

[kylek]

I think the thing I miss most about AWS was sipping coffee while watching the #wtf peanut gallery on wednesday mornings and during events like this

[an_account_name]

!fq cockroaches

[amazon_throw]

In the post-nuclear-holocaust wasteland where only cockroaches survive, the cockroaches will use IRC to rebuild.

[rosencrantz]

Looks like it is not reported having a look at the status dashboard https://status.aws.amazon.com/

[sgt]

They follow the industry standard of hard coding their status dashboards.

[mulka]

Individual service pages seem to still be up. So, you can go to the EC2 page for example, click on the services dropdown on the top to select the service you want. https://console.aws.amazon.com/ec2/v2/home?region=us-east-1

[busterarm]

Can confirm that I am having this issue.

API requests/CLI work fine though.

[HatchedLake721]

You should be able to login through https://us-west-2.console.aws.amazon.com/console/home?region...

[bdcravens]

And then switch to us-east-1 services?

[crystite]

https://status.aws.amazon.com/

Shows AWS Web Console:

9:50 AM PDT We are investigating increased error rates when loading the AWS Management Console.

[crystite]

AWS keeps their status page up2date these days. It's probably easier to check that page than combing through posts on HN, but to each their own.

This makes me wonder... do people keep posting and looking here because they want an element of control, they don't trust or know about status.aws.amazon.com ... or why?

[nerdjon]

The status page was stating all green for about 20 or so minutes after this post was made.

Not really "up2date", so this post was valuable for everyone that was seeing errors and unsure how widespread it was.

[ulysses]

The date on the status page is 9:50 AM PDT, the original poster posted well before that.

It isn't historically uncommon for AWS outage information to be posted and commented on here well before the status page is updated.

[TheGRS]

I'll just report that we had an issue with an Elastic Beanstalk deployment that seems to have resolved itself now. Not sure if its related, but simply retrying the deployment resolved it without making any changes, so it sure seems related.

[crystite]

According to the AWS Status Page

Alternate link is available at: https://us-west-2.console.aws.amazon.com/ec2/v2/home

[coffeesn0b]

Is this the apocalypse?

[zonywhoop]

DynamoDB access via api in us-east-1 is down for me as well. Also, access to any region or resource through SSO is failing too.

[slenk]

I am having issues with SAML authentication as well

[goodfight]

Strange, I'm experiencing this issue as well. Wonder if it's an attack or just normal failure

[actionowl]

Also having issues, the AWS status page is oblivious...

[cortesoft]

Same, came here to check if there was any info on it.

[giffyUnicorn]

confirmed, us-west works for logging into the console.

[jshaw3]

us-east-1 is down for me

[giffyUnicorn]

well this is embarrassing.

[hsk823]

So you can get to AWS resources going direct to the respective URLs, such as https://s3.console.aws.amazon.com/s3/ or https://console.aws.amazon.com/ec2/v2/ or even changing regions works. It seems to just affect https://console.aws.amazon.com/console/

[tus87]

Everyone take the day off.