Meanwhile, anyone in Boston who only read the headline is completely unsurprised to hear that the red line isn't running and is wondering how this merits making the front page of HN.
I received an email for every paypal authorized connection I had indicating that it has been terminated. Related?
This email confirms that you have canceled your payment agreement with ###### No further payments will be made from your PayPal account to this merchant. If you have any further questions about the agreement, or wish to reinstate it, please contact ###### directly.
The trend is is to downplay the issue in status messages to obscure the real problem. That message could mean anything from an extra 1% of rejections to 99% of transactions are failing.
The vagueness is the point, because they want to avoid admitting serious problems.
We had this problem with some devops hires who came from a big company. They’d delay updating the status page as long as possible, then update with the weakest language that was technically correct. “Some customers might experience degraded performance” was their go-to message for nearly complete outages. They’d argue that it was technically correct because some requests were getting through in some logs somewhere.
It was a side effect of working in an environment where their bonuses depended on downtime and the severity of outages. The game was to admit as little as possible to keep those bonus numbers high. We didn’t calculate bonuses that way but they had ingrained the behavior from years of BigCo performance reviews.
>We had this problem with some devops hires who came from a big company.
Amazon.
All you have to do is look at their status page of green lights when us-east goes down completely to lose complete faith in their status page reflecting anything but wishful thinking.
Seems unlikely, bonuses are not an Amazon thing, and iiuc status pages aren’t a decision such people would be making anyway. A dedicated “devops” person at Amazon (to the extent that’s even a thing, mostly engineering teams own their own ops) would be unlikely to benefit from minimizing issues. The status page issue you’re discussing is real but I don’t think it’s the fault of lower level engineers.
Updating the status page in the middle of the incident is always an art. Sometimes you can truly define impact and update the status page without weak language but other times you can't.
You still want to notify customers they may be seeing issues even if you aren't confident on the percentage of impacted customers yet.
For merchants it is, I worked on a marketplace before and having checkout flows with higher than usual declines will eat on your sales. People don't tolerate it so well and will either drop the purchase completely if it's a "want" and not a "need", or will go to the competition to finish the purchase.
If a site is being DDoSed and only 10% of legitimate traffic is going through, is it "up"? I think you'd be hard pressed to call that "not down". So if the proximate cause is fraud instead of network requests, how is it any different?
I absolutely despise this kind of language that is becoming so commonplace now and is obvious BS. I wish I could pay my bills to these same companies using language like this. "It's not that I didn't pay my bill, it's just that some dollars may experience longer-than-usual time to get to your bank."
on the Ethereum blockchain, where, yes, the service is not technically down but unavailable to anyone who isn't paying a hundred bucks for a simple transaction.
I've seen people try and do it. It involves each service replicating copies of the data from every service it's dependent on (via CDC streams on kafka/whatever) in its own datastores and reading directly from its copy of data rather than actually calling the source-of-truth service.
There are some instances where that kind of thing is necessary, but...insane amount of clear downsides so your problem statement for doing it had better be pretty compelling.
This is typically referred to event carried state transfer for anyone curious.
The premise being high service availability and low latency to serve requests (no need to talk to source of truth) with the tradeoffs of higher storage / processing costs and from my experience higher complexity.
I don't think that's the most common cause. Deleting the production database is obviously disastrous but there are many other reasons for extended downtime, and they're usually about the distributed-ness of the underlying architecture.
We had to disable the fraud protection on our account to be able to accept transactions temporarily, luckily we have alternative fraud protection in play too.
I could name plenty of other systems nobody wants and speak to their uptime, too? My Plex server probably has better uptime than GitHub lately but I’m not gonna start pushing my code there. It does its job serving the barbie movie, just like defi tells me who to unfollow.
I think you may have missed a bit of their point... a plex server is a video hosting server.. like a personal Netflix. They wouldn't push their code to their Plex server because it's not a server that accepts code pushes.
Not on average. On average, with proper failover and redundancy, you can get five nines, which is 5 minutes of downtime per year. I have that much every time I reboot my machine.
But it is illegal depending on the country,
Or a scam depending on the receiver,
Or centralized if you use a hosted wallet,
DeFIs merits on paper sound life changing, but reality is far from it.
It's an unorganized, volatile mess.
Not a great message when billionaire tech-heads are getting scammed for seven-figures and are entirely defenseless. If Mark Cuban can't get the vendor/tech mix right, what chances do an average Joe have?
Assuming because someone is successful and competent in one area means they must be competent in other, related areas has a habit of going poorly, and happens a lot in tech.
people in the comments are getting nasty. Sure the ecosystem is plagued with scammers, but financial system in general is filled with scammers. We just have insurance products or the faith of the US government to bail you out lol
I still believe in digital currency as the future.
The -highly entertaining- comments are rightfully bashing the de facto state of defi in the context of GPs comment.
Remind me when a digital currency actually exists, because I haven't seen one yet. The problem isn't just that the ecosystem is filled with scammers, its that the tech doesn't solve any existing problem but creates new ones instead. Web3 is a joke.