Hacker News new | ask | show | jobs
by luso_brazilian 3558 days ago
From the article:

> The "Paranoids," the internal name for Yahoo’s security team, often clashed with other parts of the business over security costs. And their requests were often overridden because of concerns that the inconvenience of added protection would make people stop using the company’s products.

That's the best summary of the problem for the industry as a whole, not only tech but any industry where failures are uncommon but with grave consequences.

A quote from Fight Club that illustrates that problem:

> Narrator: A new car built by my company leaves somewhere traveling at 60 mph. The rear differential locks up. The car crashes and burns with everyone trapped inside. Now, should we initiate a recall?

> Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of-court settlement, C. A times B times C equals X.

> If X is less than the cost of a recall, we don't do one.

That's the current mindset of the technological world, estimating whether the cost of atoning for the problem is lower than the cost of securing the systems.

19 comments

Calling them the 'paranoids' probably seemed like a fun idea at the time, but I wonder if it set up a subconscious bias against their work. I wonder if they had been called 'The Guardians' or 'The Defenders' there would have been a different outcome.

Seems trivial, but words matter.

The Yahoo Paranoids chose their own name. It was designed to be light-hearted in a way that didn't make them seem stuffy so that engineering teams would be more receptive to their work. In my experience, this is incredibly important from the outset.

Anyone who has worked in information security for a month knows that the relationship between product engineering and security engineering defaults to antagonistic. It takes a lot of work to make it friendly and productive, and as a security professional I think "Paranoids" is much better for overall collaboration than something like "Defenders", which in my opinion reeks of self-importance.

The more pertinent issue here is management not fostering the culture enough.

Where I'm working now, we've got security engineers assigned to seating in each development team.

They're not managed by, or working for, our teams. They have their own manager and security work that they're getting on with.

Having them sitting amongst the team, however, is resulting in a much different narrative than any I've been around before. There's a much higher quality, and less antagonistic kind of engagement going on. They've become someone you chat with at the watercooler, or at their desks, instead of having to file tickets, or wait for scheduled reviews to raise things.

People can quickly consult with them and deal with a whole heap of small potential risks way early on in the development process, and it's paying serious dividends down the road.

That approach Works well with Q&A too.
You're talking about Squads basically. Bring different people in the same group. And yeah, QA is very similar to Security in some points, but if you think straight QA should include security. Weird to say that a software has quality without security included, but the truth is that security is specific that the regular QA usually can't handle.
You've capitalized Squad, but it's hard to Google. Where did you get that term, and where is it defined outside your head?
Security engineers are seen as experts you consult about something you don't know. QA are not seen this way. Some QA engineers actually are experts that can give good advice on structuring an application in a more testable way, but that's not the norm.
Most QA guys only check that something meets the spec/story requirements, not that the code is sane or testable... many don't even go beyond UI testing. That said, I think GP was referring to having a QA embedded as part of a team.
You know... I keep thinking that with source control systems like Bitbucket enterprise, etc... why aren't more mid-large sized orgs requiring a security signoff for every pull-request with a pull request to master/release branches being the trigger point.

I do a lot of PR reviews, and while I may not catch everything, I will catch a few things here and there... someone with that mindset would be in a better position to handle that from the start...

Having a few security guys that do PR reviews for about half their workload would go a long way to improving things.

We're going through an audit for an internal application now... there's 1 major flaw (SSL2/3 is enabled), a minor (session cookie isn't https only) and a couple trivial (really non-issue) concerning output caching on api resources and allowing requests with changed referrers (this can be spoofed).

In any case, having auditing earlier on and as a potential blocker would make each minor change easier to deal with than potentially much larger changes... the app in question was developed for the first 8 months without even a Pull Request check in place... by then many issues regarding code quality are already too late to fix completely. :-(

Nobody wants this.

No "security guy" who has a choice wants to spend half their workload waiting for PRs to come in so they can chime in with feedback about default configurations.

No product programmer wants to deal with some "security guy" parroting the results of an automated tool to them over a code review platform.

No product manager wants to see progress stall because the product programmer and "security guy" are arguing over whether or not a call to strncpy should be replaced with a call to strcpy_s.

In the immortal words of my generation, ain't nobody got time for that.

Honestly, someone should have time for that, it's part of the problem... I go out of my way to comment on as many PRs as I can, because I'll catch things that will become problems later far more than other peers who just click approve.

The same can be said for security guys... they spend their day needing to work as well, and seeing a bunch of smaller things fly by is just as valid as a big audit periodically. It's easier to catch a lot of things before they become big as well...

There are plenty of times I'll comment (Okay, letting this through, but in the future revise to do it this way), sometimes I'll push back, but not always, that's what the review process is for. I'm just suggesting multiple approvers for PR, where one is someone who is security minded.

It's funny how many issues I'll see from other systems where someone does something per the spec, that has a flaw because they were completely compliant. Someone crafts an exploit, and I'm interested because I'd usually be more pragmatic in implementation. Last year there was a huff about JWT allowing cert overrides in some frameworks, as they don't ensure the origin cert matches a whitelist... when I'd implemented JWT, I only checked against our whitelist and ignored the property.

Sometimes security guys will see things and think of things in a way others won't... for me, one thing I often catch that others don't are potential points for DDOS target viability. Some of that comes from using node, where you do NOT want to constrain your main event loop thread. Others don't think about putting limits on JSON size, or compute heavy tasks, etc.

And, frankly, I'm tired of fixing related bugs to patterns that were broken from the start.... turtles all the way down, but the turtles are eating all the errors.

In the immortal words of every other generation :), "someone is going to find your issues. It's either you or your customers."

You don't seen to have an appreciation for the difference between a secure and an insecure product. Yahoo didn't either.

Much more valuable to have the security folks a critical part of reviewing the _frameworks_, and then pushing adoption of those frameworks. Human reviewers won't catch everything no matter what, but you can make entire classes of problems go away by making them impossible to commit.
Does that mean we can kill angular 1.x because it encourages points of disconnect, undiscoverable code, too much pfm (pure fucking magic) and failure?
I understand what you are saying, but having been around similar dynamics in the past I think such deprecation is a little like starting off the relationship apologizing for what they're supposed to be doing.
Indeed, regulatory and security are the two parts of the company that are supposed to be antagonistic in order to keep the company out of trouble. How that plays out in practice has a lot to do with the personalities involved.
I used to slip in words like 'awesome', 'clever' and 'amazing' when talking to colleagues from other teams about the work that I was doing in the hope that it would influence their perception of the work. I've no idea if it worked though.
That was my first thought when I read it. A better name would have been "Tron", "Patronus", or "Endor".

Calling it "Paranoids" or "Inquisition" is just giving it another reason for people to loathe it.

Infosec isn't the ones doing the defending or guarding or any of that. They're typically working with the teams to do build and maintain to ensure their policies and procedures lead to and maintain a secure posture.
As a developer who is not very much into security, I am guilty of this crime. Infosec teams are very important and deserve respect and attention.
My experience with Yahoo (admittedly ending more than a decade ago, so I'm sure much has changed) was that cost probably was a huge deal. I ran engineering for the European billing platform. We processed many millions of dollars worth of transactions a year.

Yet when I had to ask for a new database server, I had to submit a written request to a committee in Sunnyvale, with graphs and other supporting documentation to demonstrate that the load of the server we already had was high enough to justify it. Then I had to join a hardware review meeting, that included maybe a dozen people. One of them being either Jerry Yang or David Filo (Yahoo founders; I've forgotten which one of them it was that did these).

The people in the meeting, even excluding whichever one of the founders, easily cost Yahoo more in salaries for the time they spent discussing my request for one lonely server than the fully loaded amortised cost of operating it for a couple of years.

It's not that I have an issue with reviews, and cost controls - on the contrary, but some degree of delegation and trusting staff with budgets would have been nice. I mean, I could have trivially cost Yahoo millions of dollars with a few keypresses if I wanted to or didn't pay attention - they trusted me with the ability to mess up their entire European payments platform with basically no oversight, yet I couldn't approve a single cent of hardware expenditure for the production platform, and neither could my manager, nor, I believe, could my managers manager, who was responsible for all of engineering across Europe.

I suspect a structure like that may have created a lot of resistance to recommendations from the Paranoids even when engineering (they seemed generally very well respected; one of my old developers is part of the Paranoids now - he'd wanted to for years) would like to accommodate them for the simple reason that getting approvals would be a massive hassle and slow things down.

Marcus Aurelius specifically talks about how important it was to have governors that he could trust, because the empire was so large that he could not possibly know everything about the empire in its current state. His lesson about task delegation is timeless. Well, his lessons are timeless, full stop.
I've been thinking a lot about how ancient empires operated and functioned, and what institutions they required.

Realise that Egypt, Greece, Macedonia, Rome, Persia, and China each spanned a thousand miles or more, the most effective transportation was over water, either along rivers or across seas or oceans, that ocean travel was impossible for much the year (Roman vessels were restricted to port from November through May, this lasted until the 1300s in Europe), and the minimum time for a message to traverse a thousand miles was easily ten days, if not months.

You needed autonomous lieutenants in place who could be given general orders (much like goal-seeking AI, now that I think about it), be trusted to be only modestly corrupt, not collude with enemies or others against the centre (a frequent problem), and to truthfully report what they'd experienced, in words -- writing existed, but not photography, video, audio, etc. Testimony, that is, someone's testement or attestation of fact, was all you had, though multiple testimonies could be compared against one another.

I find it interesting that every major empirical power had some intrinsic religion, probably serving as a moral check and guidance, a role that's often underappreciated today. Also that other than a set of strictures, the religions themselves often had little in common with one another: polytheistic vs. monotheistic, theistic vs. meditative, commandments vs. ancestor worship or reverence.

It's a topic on which I'm almost wholly ignorant, but find fascinating.

Communication delays were a big part of this.

I think unappreciated problem with modern communications tools is that by default, they enable and encourage micromanagement.

And as a consequence, deprecate trust.
That works... before the ipo. After that relentless success is the expectation - delegation has built in risk so is very hard to justify.
His column is pretty cool too.
> That's the current mindset of the technological world, estimating whether the cost of atoning for the problem is lower than the cost of securing the systems.

And for the record, this will always be the mindset of corporations whose only concern is the bottom line. Until we as a culture accept that the market does not solve all problems, we're not going to solve these kinds of problems.

  > > That's the current mindset of the technological world, 
  > > estimating whether the cost of atoning for the problem 
  > > is lower than the cost of securing the systems.
  >
  > And for the record, this will always be the mindset of 
  > corporations whose only concern is the bottom line. 
  > Until we as a culture accept that the market does not 
  > solve all problems, we're not going to solve these 
  > kinds of problems.
My immediate reaction is "Of course". A return on investment or risk analysis should drive activities on both the corporate and the government level.

This is particularly true in the security space, because no system is 100% secure. And since resources aren't infinite, where do you stop? 90%? 99%? 99.9%? What if addressing that incremental 0.9% costs as much as the rest of the security apparatus combined? As much as the rest of the product combined? As much as your total revenue?

What's the other option? It can't be "not release anything", so a middle ground is found. We're arguing about shades of grey.

And sure, the government can help. Either by bearing some of the cost (e.g., investment, tax breaks, etc.) or increasing the impact of an incident (e.g., penalties, etc.).

But this isn't a big, bad, greedy corporate problem. This is a broader issue about how much risk we're willing or unwilling to absorb, and how efficiently we can address that risk.

> My immediate reaction is "Of course". A return on investment or risk analysis should drive activities on both the corporate and the government level.

You're looking at this in only monetary terms, or at least Yahoo is. But frankly, I don't give a fuck about whether Yahoo succeeds financially--I want my life and the lives of other people to be better. And I want that to be the goal of my government.

> But this isn't a big, bad, greedy corporate problem.

Of course it's a big, bad, greedy corporate problem. The reason "return on investment" matters in a financial sense is because big, bad, greedy corporations only care about their bottom line. And quite frequently Yahoo's bottom line is in direct opposition to improving my life and the lives of other people.

>... But frankly, I don't give a fuck about whether Yahoo succeeds financially--I want my life and the lives of other people to be better. And I want that to be the goal of my government.

In this situation it doesn't matter that yahoo is a private corporation - the same cost/benefit analysis essentially needs to be done no matter what the structure of the organization. Let's pretend that email had been created by a government agency and that agency has to decide how much of the budget to spend on security. If it costs X dollars to make something 90% secure, 10X for 95% secure and 10,000X for 99.9999% secure, etc etc eventually you have to choose how much to spend - resources aren't infinite for that government agency either. (And to make it much more difficult, they just have a guess that X dollars will make their product N% secure.) It isn't as black and white as you are trying to portray it.

I think it is fair to criticize yahoo for how the prioritized security but the same kind of issue has happened with non-profit companies and with government organizations, so no, it isn't just a "big, bad, greedy corporate problem."

You're the one trying to make it black and white, he's simply saying that unlike private industry, government can have another motive be primary rather than profit, i.e. help it citizens as the primary goal. Yea, budgets aren't unlimited, but not having to be profitable makes a huge difference in which actions can be taken. Profit is not the correct goal for every action that can be taken by an organization, government isn't a business.
If "profit" is defined as: "generating more value than is consumed in the production process"...

Then yes, we damn well better demand that profit be the correct goal for every action regardless of organizational structure.

If our system is distorted to inaccurately measure profit locally, without properly accounting for negative externalities, then that's a legitimate problem, but the way to solve it is by factoring those hidden costs back into the profit calculation, not giving up on "profitability" properly defined.

  > ... government can have another motive be primary 
  > rather than profit, i.e. help it citizens as the 
  > primary goal.
But there's still ROI here, and there's still a budget (no matter how big the deficit gets). So the question remains: how do I spend that money? Do I spend all of it on security apparatuses, or do I have to scale back and spend some on other social services? How much? What's the best bang for my buck?
> budgets aren't unlimited, but not having to be profitable makes a huge difference in which actions can be taken.

Profits are still required for gov't spending, but they are just made by someone else in the country and transferred to the gov't via taxation. Even deficit spending is just the choice to spend money today that will be obtained from taxation at a later date.

I know this is snarky, but: tell it to the OMB.

Corporations do not have any sort of exclusive lock on cost-benefit analysis.

Edit: including bad cost-benefit analysis.

I'm looking at this in quantitative terms. Money is one measure. Effort, time, security, and others may be harder to quantify, but they're still important factors. "Security at any cost" quickly becomes simply impossible.

This is the general sense. Yahoo is probably on the "wrong" side of average.

But in some sense, you can vote with your feet. Companies who don't value security won't get your business. If enough people feel as you do, then the ROI calculation changes. And the same applies to politics as well: if you think more money should be spent on security and there's a societal good here, write to your congressman, or elect one who's receptive. Again, if enough people feel as you do, the political ROI makes this an imperative as well.

The fiction of markets is that costs and value can be reasonably determined. The truth is that in far too many instances, they cannot. Surface appearances or gross misbeliefs drive costing or valuation models and behavior, and as a consequence, goods are tremendously disvalued.

That's on top of the problems of externalities in which the costs or benefits aren't fully contained to the producer or consumer of a particular good or service.

A misprioritisation of values is what the drunk waking up with a hangover, the sweet-tooth spending 40 years dealing with systemic effects of diabetes, or the smoker suffering 20 years of emphysema and COPD comes to realise. The externalities are the drink-driving victim, the socialised medical costs (and privitised profits of the sugar firms), and the 2nd and tertiary smoke victims.

There are rather larger issues far more fundamental than these in the modern industrial economic system, but I'll spare you that lecture.

The point being that trusting on "the market" to offer corrections simply doesn't work.

>The reason "return on investment" matters in a financial sense is because big, bad, greedy corporations only care about their bottom line.

I would argue that it's ALL corporations that only care about their bottom line. The entire reason a corporation exists is to make money, any other considerations like employee well-being, care for the environment, etc are driven entirely by either legal requirements or a need to retain talent in order to make that money. Any corporation who successfully projects an image of being "different" just has a good marketing team.

Or they’re just a small-to-medium-business with a consistent set of ethics? Ever thought about that?
Externalities are a word we use to describe costs we find hard to model, but I find that most externalities do cost corporations real money. They just often aren't aware of it and haven't developed enough sophistication in their business cases to account for it. The best companies who support their security teams understand this. They understand that broken things lose them trust, customers and goodwill and those things are, even from a purely monetary and numerical perspective, incredibly valuable for a successful business in the long term.

The problem is not merely whether or not a profit motive exists to do right, but whether or not a business is insightful enough to model the full costs and include what we normally let go unexamined as mere "externalities".

Externality != "hard to model". Rather, it means difficult to internalise.

Garrett Hardin's "Tragedy of the Commons" gives a quite simple model of what an externality can be (overgrazing). The problem isn't in the modelling, but rather in the mutual enforcement of a collectively beneficial behavior.

That isn't to say that there aren't costs which are hard to model, but that's an orthogonal issue, and can apply just as well to internalised effects (e.g., the goodwill loss of a massive security breach) as to externalities.

Goodwill loss is not an externality.

I agree, adamantly, with your comment that businesses are frequently not enlightened or intelligent enough to model full costs. I'm seeing the issue of the long-term development of both cost and benefit awareness as a pressing issue, general to economics. It undermines many of the assertions of market efficiency.

I'd argue it >is< a corporate problem, and the article we are looking at shows exactly why. There should be consequences for running a company in this manner, and there are not. The people who made this decision did it because they were protected from the damage they did.
> There should be consequences for running a company in this manner, and there are not

And the consequences should be users choosing another company and they don't. So the core problem are users.

No, that assumes people are rational actors and they are not; preying on human psychology doesn't alleviate you of guilt, the companies are the problem, not their victims for not leaving.
It's similar to a company selling defective products or contaminating a city's water supply. The market response is too late to deal with those types of problems, and undervalues individual lives.
I don't think you need to even concede the idea that users are rational actors--there are plenty of reasons why a rational actor would prioritize another factor over security. For example, many people got Yahoo email addresses a long time ago, and built a personal contact list of people who only know their Yahoo email. A rational actor might value keeping in contact with those people over their privacy. That doesn't mean that it's okay to expose that person's data.
The consequences should be that the company loses its ability to run a business. You've arbitrarily decided that the only acceptable mechanism for this happening is users choosing a different company. There are a whole host of reasons that doesn't work, and simply shifting the blame onto users for not making it work doesn't solve the problem.
> The consequences should be that the company loses its ability to run a business.

Or gains ability to run it properly.

> the only acceptable mechanism for this happening is users choosing a different company.

I didn't state it should be the only mechanism. There could be others. Those class action lawsuits mentioned in the article prove there are some. But the primary mechanism is users' responsible choice.

> shifting the blame onto users for not making it work

Actually I think the blame is on us, techies. We should create a culture where security matters as much as performance, pleasant design or simple UI. Both among users we live with and companies we work in.

And one fundamental problem of security for the masses is not solved yet: how a user can see if a product they use is secure without being a security expert.

Security people grade issues from two simultaneous yet different perspectives, security risk and business risk. It sounds like you are describing accountants not security people.
But what's the concrete proposal?

The default "better idea" seems to be "let the government do it", but if you've been keeping up with the news in the past few years, "the government" doesn't exactly have a stellar track record either. Where a corporation may prioritize making money over security, government prioritize politics over security, wanting to spend money on things that visibly win them political points or power, not on preventing things that don't happen, which aren't visible to anyone. It's the same problem in a lot of ways. And both corporations and governments have the problems that specific individuals can be empowered to make very bad security decisions because nobody has the power to tell them that their personal convenience must take a back seat to basic operational security.

Even the intelligence agencies have experienced some fairly major breaches, which count against them even if they are inside jobs.

"The market screws this up!" isn't a particularly relevant criticism if there isn't something out there that doesn't screw this up.

> "The market screws this up!" isn't a particularly relevant criticism if there isn't something out there that doesn't screw this up.

My usual reply to this is that we use government to nudge market incentives, which is also what I think would be reasonable here: simply create a class of records related to PII, and create HIPPA like laws regarding those records that certain kinds of information brokers keep on people.

You then provide a corrective force to the market by providing penalties to violations, which raises the costs of breaches, and shifts the focus of the corporation towards security.

HIPPA or financial systems aren't perfect, it's true, but they're at a standard above what most of our extremely personal data is stored at, so we know we can do better, if we choose to as a society.

These laws would also be a lot more effective if you held the executive staff accountable as opposed to the shareholders. The model that corporations seek profit doesn't work in some cases, it's a group of individuals all seeking personal profit.
s/HIPPA/HIPAA/rg
So adjust the market.

There's a worthwhile conversation to be had about the corporate liability shield, and whether A) major security/privacy breaches should have some sort of ruinously high statutory damage award rather than requiring people to prove how they were harmed, and B) more suits -- not just over breaches -- should be able to pierce the corporation's protective structure and cause personal liability for corporate officers who make careless or overly-short-term decisions.

Adjusting the incentive structure of the market in which companies operate could do a lot.

It was already done by DOD under Walker's Computer Security Initiative. It succeeded with numerous, high-assurance products coming to market with way better security than their competitors. Here's the components it had:

1. A clear set of criteria for information security for the businesses to develop against with various sets of features and assurance activities representing various levels of security.

2. Private and government evaluators to independently review the product with evidence it met the standard.

3. Policy to only buy what was certified to that criteria.

Criteria was called TCSEC with Orange Book covering systems plus "rainbow collection" covering the rest. IBM was first to be told no in an embarrassing moment. Many systems at B3 or A1, most secure, were produced with a mix of special-purpose (eg guards) or general-purpose (eg kernels or VMM's). The extra methods consistently caught more problems than traditional systems with pentesting confirming they were superior. Changes in policy to focus on COTS not GOTS... for competition or campaign contributors I'm not sure... combined with NSA's MISSI initiative killed the market off. Got simultaneously improved and neutered afterward into Common Criteria.

Summary here:

http://lukemuehlhauser.com/wp-content/uploads/Bell-Looking-B...

Example of security kernel model in VAX hypervisor done by legendary Paul Karger. See Design and Assurance sections especially then compare to what OSS projects you know are doing:

http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A...

Best production example of capability-based security was KeyKOS. Esp see "KeyKOS NanoKernel" & "KeySAFE" docs:

https://www.cis.upenn.edu/~KeyKOS/

So, that was government, corporations, and so-called IT security industry threw away in exchange for what methods and systems we have. No surprise the results disappeared with them. Meanwhile, a select few under Common Criteria and numerous projects in CompSci continued to use those methods with amazing results predicted by empirical assessments from 1970's-1980's that led to them being in criteria in first place. Comparing CompCert's testing with Csmith to most C compilers will give you an idea of what A1/EAL7 methods can do. ;)

So, just instituting what worked before minus the military-specific stuff and red tape would probably work again. We have better tools now, too. I wrote up a brief essay on how we might do the criteria that I can show you if you want.

I posted this elsewhere, but I think I intended to post it in response to your post:

Well, there are a few possible solutions, and they don't all involve corporate incentives:

1. Government regulation

2. Technical solutions (alternatives to communication that have end-to-end encryption, for example)

3. Eschew corporations entirely when it comes to our data and communications (i.e. open source and personal hardware solutions)

Personally, I think some combination of 2 and 3 is my ideal endgame, but we aren't there technically yet. 1 isn't really a great option either, because government is so controlled by corporate interests, and corporations will never vote to regulate themselves. But we can at least make some short term partial solutions with option 1 until technology enables 2 and 3.

However, none of these options will happen while people hold onto the naive idealism that the free market will solve all our problems.

> Eschew corporations entirely when it comes to our data and communications (i.e. open source and personal hardware solutions)

Unless you're proposing a large rise in non-profit foundations, these are mostly funded by for-profit corporations operating in a market.

> However, none of these options will happen while people hold onto the naive idealism that the free market will solve all our problems.

I don't think most people beyond libertarians or knee jerk conservatives believe that. Heck most economists don't really believe the market is "self regulating", there's just too much evidence that it's not.

However, most do believe that a regulated market solves the problem of large-scale resource allocation better than planning in most cases. In same cases, no: healthcare is a well-studied is a case of market failure and why centralized / planned players fare better. It's not clear to what ends data/communications/security is a case of market failure and warranting alternative solutions.

> Unless you're proposing a large rise in non-profit foundations, these are mostly funded by for-profit corporations operating in a market.

You bring up a deep problem that I admit I'm not sure how to solve. I'd love to see a large rise in non-profit foundations, but I'm not actually convinced even that would solve the problem.

I think the solutions proposed by i.e. the FSF where up-front contractual obligation to follow through with their ideals may be a better solution, but we're beginning to see very sophisticated corporate attacks on that model, so it remains to be seen how effective that will be.

> I don't think most people beyond libertarians or knee jerk conservatives believe that. Heck most economists don't really believe the market is "self regulating", there's just too much evidence that it's not.

> However, most do believe that a regulated market solves the problem of large-scale resource allocation better than planning in most cases. In same cases, no: healthcare is a well-studied is a case of market failure and why centralized / planned players fare better. It's not clear to what ends data/communications/security is a case of market failure and warranting alternative solutions.

This argument is purely sophistry. You take a step back and talk about a more general case to make your position seem more moderate, admitting that the free market isn't self-regulating, but then return to the stance that the free market solves this problem because a regulated market (regulated how? by itself? In the context of free market versus government regulation, "regulated market" is a very opaque phrase) solves most cases, and on that principle, we don't know whether the very general field of data/communications/security warrants alternative solutions (now we can't even say "government regulation" and have to euphemistically call it "alternative solutions" as if government involvement is an act of economic deviance?).

We're speaking about a case where the free market didn't work, de facto: Yahoo exposed user data, hid that fact, and likely will get the economic equivalent of a slap on the wrist because users simply aren't technical enough to know how big of a problem this is.

So let's not speak in generalizations here: the free market has failed already in this case, and you admit that the free market doesn't self-regulate, so you can't argue that the free market will suddenly start self-regulating in this case. Regulation isn't an "alternative solution", it's the only viable solution we have that hasn't been tried.

" (now we can't even say "government regulation" and have to euphemistically call it "alternative solutions" as if government involvement is an act of economic deviance?)."

I presume an unregulated market is preferable to regulation at the outset, yes. Government regulation should be done in the face of systemic failures while retaining Pareto efficiency.

Put another way, I think the market can be very effective with well thought out regs. but I don't believe there are better general/default alternatives than to start with a free market and use the empirical evidence to guide policies...

"likely will get the economic equivalent of a slap on the wrist because users simply aren't technical enough to know how big of a problem this is."

I disagree that this is a case of market failure.

This is a case of "you know better than the market" and you want to force a specific outcome through regulation. But I'm not sure that's what people want.

what if people don't really care about their data being exposed all that much? It's a risk they're willing to take to use social networks. The penalty is that people might move off your service if you leak their information (as is likely to some degree with Yahoo). That to me seems to be the evidence here. That's not a market failure, that's a choice.

With legisilation, we can change the market. In the Fight Club example legislation can make C be ten times as big, and change the equation. For Yahoo, legally mandated fines, or restrictions on what they can do in future[1] could make them wake up.

[1] Maybe if you run email and you get hacked, you're not allowed to run email again for a few years? That'd've have woken them up.

And until we figure out how to incentivize this behavior (or discourage malicious behavior), corporations won't be willing to solve these kinds of problems.
If only we had some sort of structure in our society that could solve the problem but wan't profit-driven. Maybe something that could oversee these corporations. We could call it "government" or something.
Pretty sure the US has one of those, doesn't seem to be working. In fact, it often acts against that (preventing sharing of encryption algorithms, trying to force inclusion of backdoors..).
If only Hobbes, Locke, Rousseau et al. were around today..
Maybe you can try Bernard Stiegler. His english wikipedia page is a little poor in information, so I put a link about his last book (not yet translated).

https://en.wikipedia.org/wiki/Bernard_Stiegler

http://www.samkinsley.com/2016/06/28/how-to-survive-disrupti...

bookmarked this interview link for later. Thanks!
Not sure why you're getting down-voted - if the market is set up to incentivize a certain behavior, then someone will do it eventually. People can get mad all they want, but they should be furious that the government has laws in place to incentivize that type of behavior (or at least allow it to happen.)
Well, there are a few possible solutions, and they don't all involve corporate incentives:

1. Government regulation

2. Technical solutions (alternatives to communication that have end-to-end encryption, for example)

3. Eschew corporations entirely when it comes to our data and communications (i.e. open source and personal hardware solutions)

Personally, I think some combination of 2 and 3 is my ideal endgame, but we aren't there technically yet. 1 isn't really a great option either, because government is so controlled by corporate interests, and corporations will never vote to regulate themselves. But we can at least make some short term partial solutions with option 1 until technology enables 2 and 3.

However, none of these options will happen while people hold onto the naive idealism that the free market will solve all our problems.

The market seems to be solving the problem just fine: nobody uses Yahoo anymore and companies with solid security practices (e.g. Google, Apple, Facebook) are thriving. If Google had a serious security breach, you can bet the market would respond to it and Google knows it.
I mean, i am not agreeing with Yahoo here... but isn't that a reasonable thing to do?

Every act of securing or ensuring quality has a cost, and there is a line. I think most of us would agree that the line is very broken currently, but it appears you're citing a problem with the line in general, not the location of said line.

Everything has a cost, from a recall to better security to even a human life, the debate should be what we think should be paid, not whether or not we should worry about costs at all.

(If i misunderstood your intent, apologies)

The problem here is that the people who pay the costs of security are different from the people who are hurt when security is breached.
Loss of user trust hurts Yahoo.
Not enough that it isn't in Yahoo's favor to take that risk (you can't argue this--this is what happened).
And who are you to decide that that isn't a legitimate decision made by the users? If people cared more about security, they'd move away from Yahoo after something like this, and Yahoo would be more incentivized to keep this from happening.

Your problem is that you disagree with other users - but that's totally legitimate, not everyone has to care about the same things you care about.

> And who are you to decide that that isn't a legitimate decision made by the users? If people cared more about security, they'd move away from Yahoo after something like this, and Yahoo would be more incentivized to keep this from happening.

Who said this wasn't a legitimate decision by users? Certainly I didn't and wouldn't say that. There are a lot of reasons why a rational actor would choose to stick with Yahoo--that doesn't mean Yahoo exposing their private data is okay.

The other thing to realize here is that users aren't rational actors. My grandma is senile--is it okay for Yahoo to expose her private data because she doesn't know they aren't secure?

You've arbitrarily decided that users have to take all the responsibility here, and that the only way we can judge or punish Yahoo is by users leaving. But a) in many cases Yahoo is the only actor with agency to make a decision, and b) there are other ways Yahoo could be punished for using that agency to make decisions that harm users.

> Your problem is that you disagree with other users - but that's totally legitimate, not everyone has to care about the same things you care about.

No, I don't think that I disagree with other users--I think that many people care about their privacy, they simply a) don't know enough to make pragmatic decisions on how to protect their privacy, or b) have other priorities. And this is beside the point--none of this makes it okay for Yahoo to endanger their users' privacy.

> If people cared more about security, they'd move away from Yahoo after something like this

That's why Yahoo's failure to disclose this immediately bothers me so much.

Maybe the long-run solution is to make the coupling explicit: publicly post the value the company places on an account not being breached. (Ideally, this would work in tandem with some insurance policy that pays out for that amount, to validate that they really do so value it.)

Then, you can choose the provider with a high enough value to make you feel comfortable, in the understanding that higher-valued accounts will cost more.

This would work in many more contexts: The window sticker on my car can include the value they placed on passengers' lives when making cost-benefit trade offs.
It is reasonable, when your estimates are good and you're honest with regulators and customers. Sometimes your estimates are off by a factor of 10.

https://en.wikipedia.org/wiki/General_Motors_ignition_switch...

And you kill over 100 people, lie to regulators, lie to consumers, and end up spending billions trying to rectify the situation (recalls, settling suits, fines).

Yes, using the outcome of a formula to determine your actions generally relies on the formula being accurate.
It also relies on whomever is modeling the reductive, simplistic "cost model" to know the effect of all the other variables that factor into the companies success. Do these people really think that the legal/compensation costs are the only effect? How many sales did Ford miss out on because they were labeled as the "There is a known issue in this car that might kill you but until your life is worth more than a replacement part we wont repair it" car company? Did they factor in those costs into their revenue model projections? Did they factor in the sag in price point demand "Boss I wouldn't bid the same on that contract because they've shown themselves to sell a known defective product and we'll open ourselves to legal issues if one of their cars kill one of our customers we're transporting in their vehicles"

Despite what an MBA will tell you, the world is more complicated that X<Y*Z

There will always be things you can do that increase safety at a cost, but some of them will necessarily not be worth the effort, or you're forced to spend without bound on ever-more safety to the point that it's not worth using (and which may push people into still-riskier alternatives).

>How many sales did Ford miss out on because they were labeled as the "There is a known issue in this car that might kill you but until your life is worth more than a replacement part we wont repair it"

If you're turning down a company for making such a tradeoff, that's like saying "I'll buy a Ford rather than a GM because people might die in GMs."

You're right that you can legitimately criticize a company for failing to include certain things as costs, but it's not fair to fault them for somehow making this inevitable tradeoff, especially in the belief that you have some alternative provider that isn't.

(And example of such a cost -- that they can legitimately be expected to but don't -- would be something like "impact on general perception of risk", "impact on reputation of the car industry".)

>Despite what an MBA will tell you, the world is more complicated that X<Y*Z

It sounds more like you're agreeing that it's that simple, but that Z (events worthy of consideration) is not as simple as in typical models.

Which is why actuarial reports have around 2 pages of conclusions and 20 pages explaining the assumptions underlying them.
I also agree with this, security is always in a balancing act with convenience. Yahoo fell to far into the convenience side on this one but that debate on security vs convenience is happening in everywhere. The issue I've seen is that many companies are bad at doing risk analysis about these choices. That's the bigger issue in my view.
> security is always in a balancing act with convenience

I don't think that's always the case. A whole lot of security can be had with little or no inconvenience, given an appropriate mindset, though one might argue that such a mindset is an inconvenience in itself. :)

> many companies are bad at doing risk analysis about these choices Amen to that!

I think that having a basic, security aware mindset goes a long way, even if there is very little 'budget' or 'ability' to do inconvenient things.

Philosophically speaking, you cannot improve security without sacrificing usability. What I mean by usability is the capability for someone to do something, not simply convenience for the users themselves. No amount of security can be added without a concurrent decrease in usability, even if that usability is something you didn't expect or want to do.

For example, the user might not see a capability decrease if you use MD5 or bcrypt, but you certainly see a capability decrease because you can no longer see their passwords and you have to do extra work to maintain them securely. Sometimes security decisions are easy, like hashing passwords, because these days no one wants that capability. But sometimes they are not easy decisions.

You can pass a lot of convenience savings on to users by assuming the capability sacrifice yourself (for example, choosing the password hashing algorithm behind the scenes), but you can't do this for everything (for example, mandating two-factor authentication or password resets be masse).

This might come across as pedantic, but it's very important to maintain a mental model this way because it helps you understand risk analysis for more complicated security and usability tradeoffs. Starting from the premise that you can have any security without a decrease in usability is not helpful in that regard.

Your argument is assuming something that I don't believe is true, which is that we're already on the Pareto optimality frontier for security/convenience. It is certainly true that you can not forever increase security without eventually impacting usability, but I don't think many people are actually in that position.

I've improved a lot of real-world security by replacing functions that bash together strings to produce HTML with code that uses functions to correctly generate HTML, and the resulting code is often shorter, easier to understand, easier to maintain, and would actually have been easier to write that way in the first place given how much of the function was busy with tracking whether we've added an attribute to this tag yet and a melange of encoding styles haphazardly applied. What costs you can still come up with ("someone had to create the library, you have to learn to use it") are generally trivial enough to be ignored by comparison, because the costs can be recovered in a single-digit number of uses.

"Your argument is assuming something that I don't believe is true, which is that we're already on the Pareto optimality frontier for security/convenience. It is certainly true that you can not forever increase security without eventually impacting usability, but I don't think many people are actually in that position"

That's true that we aren't at the sweet spot yet but that what I meant by companies being bad about doing the risk analysis judgement of security versus usability.

On you second point languages have gone through that cycle. Look at Java doing boundary checks. That helps avoid a whole class of security issues but at the cost of making things that C was able to do easily more difficult. These tradeoffs happen at every layer.

> No amount of security can be added without a concurrent decrease in usability, even if that usability is something you didn't expect or want to do.

It seems strange to describe this this way for something like fixing a memory corruption bug or switching from a vulnerable cryptographic algorithm to a less vulnerable one. The capability that you're giving up is ... potentially breaking your own security model in a way that you weren't even aware was possible?

I think I might not be conveying my point very well. Let me clarify this as succinctly as I can.

Usability doesn't just mean things users want to do. Usability means things anyone (users, developers) can do. By definition, "securing" things means limiting the capability of certain users or developers to do (hopefully) specific things. How efficient you are at this determines whether or not you'll also reduce the capability users or developers want to have when you reduce the capabilities they don't want to have.

To give a concrete example: using a cryptographic algorithm immediately impacts usability along performance and capability axes. Previously, you could arbitrarily read and manipulate that data because it was plaintext. Afterwards, you could not. Now you need to be careful about handling that data and spend developer time and resources implementing and maintaining the overhead that protects that data and reduces its direct usability.

It doesn't matter if you wanted that capability - it's gone either way. That was a trade-off, and it is an easy decision to make, but not all decisions are easy to make. Every security decision can be modeled as a trade-off.

Ok, let's not talk philosophy and talk capability-based security with CapDesk instead:

http://www.combex.com/tech/edesk.html

They already demonstrated that integrating POLA at language and security level with simple, user authorizations could knock out most problems automagically. Did a web browser that way, too. KeyKOS previously used that model for whole systems that ran in production on IBM's mainframes with checkpoints of apps and system state every 30 seconds on top of that.

Still think you have to screw usability to improve security? And does it matter that it might be true in an absolute sense of some sort if in practice it might be no different (eg File Dialog on Windows vs on E/CapDesk)?

The point is that not ensuring security also has a cost, one which is harder to see.
> Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of-court settlement, C. A times B times C equals X. > If X is less than the cost of a recall, we don't do one.

This is a reasonable expected value calculation - and not really that controversial. The real issue is that the model for cost isn't quite accurate; if you ask an actuary, whose livelihood is based on accurately measuring and accounting "risk," he/she will tell you that you would need to account for the probable loss in future revenues due to negative customer sentiment. Once you account for that, the cost of recall is a _much_ better proposition.

I know it's not good economics, but if you see a human life as priceless, the numbers don't work out quite the same. I think that's what 'Fight Club' was about. I guess the conversation departs the realm of economics at that point, and becomes one of philosophy and/or religion.
I think I'd challenge the assertion that Fight Club in either medium was about human life being priceless (but I understand you probably meant the quote). Quite the opposite, I'd think.
Seems to me that the character, the author, and the audience find something tragic in a life measured only in calculation, and that they all think there ought to be more to life than what's apparent. 'Priceless' may be a stretch, I agree.
If you've built an organization on numbers based decision making, you can no longer consider anything priceless, because an infinity (especially if there are two competing infinities) will cripple your ability to decide.

Companies run on strictly utilitarian ethics, which is why so many ethical complaints are invisible to them. For example: (Customer) ad tracking is bad for me! (Company) But tally the value of our services, we're clearly in the black!

I would contend that those equations are a bit more nuanced than you give credit.

Let's say I hypothetically give you that the Customer sees ad tracking as "bad" (whatever that means ...let's just accept it for argument.)

(1) Then the [Customer] utility function is: (Value from free services) - (Negative experience from ad tracking) + (Possible positive experience from learning about a new product or service from better targeted ads)

(2) The [Company] utility function is: (Value from ad revenue alone) - (Negative feedback on ad targeting) + (Revenue gained from higher ROI on marketing spend resulting in more purchases/subscriptions/whatever.)

In (1), I think people on average don't care about "privacy" related news because users don't see the negative experiences outweighing the other parameters. In (2), the negative feedback on ad targeting isn't really that large at the [Company] level to warrant much change (at least if you leave the echo chamber of HN every now and then.)

In the case of Yahoo, I still hold the hypothesis that they underestimated the (Negative feedback on a breakdown in security) as well as (Positive revenue gained from trust in security.) Then again, I doubt myself because if this were true, Box would be lightyears ahead of Dropbox; sometimes the coefficient on UI _really is_ larger than that of security...?

> In the case of Yahoo, I still hold the hypothesis that they underestimated the (Negative feedback on a breakdown in security)

Yahoo's stock is up (+53% since February, with a small dip in late June). Where is the miscalculation?

Volkswagen is back to positive sales growth, and their stock has recovered 50% since their discovery last September. Their calculation was correct, too.

Ah good point - at least for Volkswagen...Yahoo has other confounding factors (their sale, etc.) but overall the impact is probably a short-term shock with few longer-term lagging effects.
And that's why, if the problem was obvious/known, we need to fine companies enough that x becomes way bigger than a recall.
If you make cost X so high that X is an existential risk, people/companies will chance it because security isn't binary and "Either way we're fucked if we get a breach".

So then companies just never disclose.

Or it makes the cost so high that the underlying product becomes impractical. I'm pretty happy to live in a world where I can buy a car for less than $100k, even if that car ends up being much less safe than an S-class.
That's true, but a company can only play that game so many times before it catches up to them. "Never disclose" isn't a workable policy because eventually someone will leak the data.

It's also worth noting that you're talking about a hypothetical, but there are real life examples of this sort of security working despite your claim that it won't work. I've worked for HIPAA-regulated companies. It's certainly difficult to meet their requirements, but it's not impossible, and the regulations do have a real impact on the security of the data.

I'm also not convinced that security isn't a binary. You're either secure or your not, and you're only as secure as the weakest link in your system: that seems pretty binary to me.

A more accurate statement might be that perfect security is prohibitively expensive in many cases. But in many of those cases, data is actually not needed, and is collected because business wants visibility into users, even if that means compromising user security. This divides companies into three camps:

1. Companies where security is cost-effective.

2. Companies where security is cost-prohibitive, but which don't need to collect data.

3. Companies where security is cost-prohibitive, but which need to collect data.

I'd posit that the vast majority of companies are in categories 1 and 2, and that it would be a net benefit to people if all companies in category 3 stopped existing.

> I'm also not convinced that security isn't a binary. You're either secure or your not, and you're only as secure as the weakest link in your system: that seems pretty binary to me.

You cannot use the phrase "as secure as your weakest link" and then assert that security is binary. You're using terms that indicate varying levels of security.

More to the point, security is clearly not binary. You can support login over HTTP, which is quite insecure. You can support login over TLS which is much more secure. You can support only more recent algorithms over TLS which is more secure still. You can enforce two factor authentication, which adds more security. You can make your clients use certificate pinning which makes you more secure yet. You can allow easy access only from known clients and otherwise make the clients go through some extra authentication steps (secret questions, email verification, etc.). You can do the same for known locations.

Each of these options provides different levels of security. None of them are "secure" in any binary sense.

I think the missing piece in what you are saying is that there's an unspoken question here: "Secure against what?"

Let's use your examples to explain:

> You can support login over HTTP, which is quite insecure. You can support login over TLS which is much more secure. You can support only more recent algorithms over TLS which is more secure still.

Secure against what? If it's password exposure you're worried about, then HTTP is definitely not secure unless some other security is used. But given the attacks I know of against older versions of TLS, I don't think it makes sense to say that older versions of TLS are less secure against password exposure than newer versions of TLS, because the vulnerabilities I know of in old versions don't leak passwords[1]. So HTTP: not secure, TLS: secure, for password exposure. It's a binary whether it's secure for password exposure.

If, however, it's unauthorized access we're worried about, the CRIME and BREACH attacks are usable against all versions of TLS for session hijacking, so we could say that neither HTTP nor TLS is secure against unauthorized access. Again, it's a binary whether you're secure for unauthorized access.

So yes, actually each of these options is secure in a binary sense, when you ask what it's secure against.

Security, as a whole, as I see it, is a big `&&` of all the smaller binary pieces of security that matter for a given product. In reality, for most products, you have to be secure against password exposure and unauthorized access. It doesn't matter if you're secure against one if you're insecure against the other--that's what I mean when I say you're only as secure as your weakest link. So when talking about your security as a whole, it really is a binary: either you're secure or you aren't.

[1] This is for the sake of argument--don't take my word that older versions of TLS are secure against password exposure, as I haven't investigated that claim fully.

You're trying really hard to fit this into your binary model. Security is all about managing risk. It's not absolute. TLS didn't change when the CRIME attack was revealed, but it suddenly became less secure because the risk profile changed. But before CRIME, TLS wasn't perfectly secure. There was always the risk that the protocol could have undiscovered flaws, that an attacker could guess the private keys, that a cert authority could issue a valid cert to an attacker, etc.

In a world of imperfect security, talk of binary security is meaningless.

That isn't something made up for the film, in real life at least one "no recall" decision has been made using exactly that sort of cost/benefit analysis: https://en.wikipedia.org/wiki/Ford_Pinto#Cost-benefit_analys...
Isn't this the case of any business? Strictly speaking, they're profit generating machines. That's the purpose of regulation is to offset this equation by some amount that makes the equation balanced where society collectively deems it reasonable. That's the intended purpose anyways.

  Narrator: A new car built by my company leaves somewhere traveling at 60 mph. The rear differential 
  locks up. The car crashes and burns with everyone trapped inside. Now, should we initiate a recall?

  Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by 
  the average out-of-court settlement, C. A times B times C equals X.

  If X is less than the cost of a recall, we don't do one.
The example reminds me of this discussion[0] between Milton Friedman and a student.

[0]: https://www.youtube.com/watch?v=jltnBOrCB7I

Note that:

1. Friedman positions the student's view as wrong. And changes the question.

2. Friedman argues himself to the student's argument. Without acknowedging this.

3. Friedman never once acknowedges that the problem was that Ford was aware of the risks but chose to conceal them from the public, such that the public was fundamentally unable to make an informed choice.

4. That allowing people to bargain with their own lives leads to numerous other slippery-slope and logically-constrained tragic inevitabilities. Individuals almost always think they can beat the odds. They're almost always wrong.

What cost-benefit analysis almost always fails to consider are the moral and goodwill costs of making a decision which is intrinsically harmful to the customer. Most especially when not informing the customer of the full risks.

That specific clip is among the more prominant reasons I find Friedman an entirely unfaithful and bad-faith debater. He keeps moving the goalposts and using equivocations just enough that unless you're quite attuned to the fact, you'll miss it completely. And that is where he's not lying outright. Curiously enough, his son David does pretty much precisely the same thing.

Neither seem capable of admitting error either, which is the final loss of credibility.

But cost is one - pretty good - way to figure out which branch of that tree to take. You can decide to (arbitrarily?) weight the decision towards the cost of securing the system.

I don't see how it is that security isn't totally analogous to a lighthouse, which is the classic example used to explain public goods. Yet we're expecting Yahoo to underwrite security on its own?

And how is it that we simply let the attackers off the hook? Using naval piracy as a metaphor, the response was rather violent suppression by (primarily the British) Naval forces.

The seas were commons, and pirates were hung from yardarms as a public service.

I could cynically project that "security" is being used somewhat as a make-work program for engineering staff. Te concern is that language systems of inappropriate weight may be used simply because they're "secure". Granted, the hardware ecosystem certainly makes this less than problematic.

And I am sure these metaphors break down at some point, but they work for me, for now.

  That's the current mindset of the technological world,
  estimating whether the cost of atoning for the problem is lower
  than the cost of securing the systems.
That mindset can be changed if the companies are fined heavily for breach of customer data.
This was so fking stupid because it's not even how this works. Yes, the manufacturer can initiate a voluntary recall, but there are other paths to recall. Insurance companies are the point of the spear for payouts -- if they think that a particular model or manufacturer has a problem they're going to say something if only to reduce their costs.
That's not just the tech industry, that's more or less the foundation of society: maximizing value. The only aberration here is that Google, not Yahoo, sees that there is a second component of X: [how much the average customer believes they will lose in future security breaches] * [total number of customers].
that fight club example is amusingly cynical, but a true cynic might think it idealistic to believe that high-level decisions are made according to any formula. if they were, what calculations could explain the decision to allocate resources to Katie Couric?
A long of the good ones left a long time ago.

Y! been stuck in a rut, coasting, like AOL for a long time... hence Verizon sees their old white grandaparents with email as a stable user base. Most old people won't change email addresses no matter what happens.

To this add what I call "pre-breach failures of imagination". Most often manifesting itself as "Why would anyone hack us?"
It sounds to me like Fight Club just repurposed the Ford Pinto lawsuit.
I'm not a lawyer but if the "probable rate of failure" passes a certain threshold, criminal negligence should be a consideration. Certainly in the cases of vehicles and also in the case of computer security where lives are stake, hospital systems for instance.