[iancmceachern]

They are, and it's generally good practice. The difference is now they spend much more time and effort on the penetration testing.

An example. In the past it would be OK to use security bit screws for this. Yes you can buy the bits online, but it was at least one layer of perceived security.

This doesn't fly anymore.

The real challenge is they implemented these new requirements on devices that were already in the submission process. Also, these things aren't written down anywhere in standards etc. so you know them ahead of time when you design. You have to just wait until the penetration testing and find out.

Ultimately the new rules aren't the challenge, it's the fact that you don't get to know them when you start and finish the design, you find out later.

[MichaelZuo]

Isn't the intention that everyone submitting their devices for approval have done in-house penetration tests extensively? Or at least laid out specific claims as to what it can endure and what it cannot?

The third party test seems to be just the last verification stage to reassure the FDA the company is not making unsupportable claims.

[iancmceachern]

Not really because these things aren't defined.

There is no definition or standard to which you would do your in house tests to. It's not like other things where you design it to comply with iso whatever and then you test to that.

Here the standard so to speak is defined by the penetration test itself.

An example in safes. No safe is untraceable. Safes are spec'd by number of minutes to resist a tool attack. Then when a safe company goes to UL or whatever to certify the safe, UL technicians get the best commercially available tools and try there best to break into the safe and time themselves. If it takes them more than the spec, it passes.

Here there is no spec. There is no defined time. There is no standard. It's just up to what you can get the penetration test house to agree to write.

[MichaelZuo]

But the company has to submit in writing an application laying out their claims?

I'm not really sure why the lack of such a standard definition prevents people from writing that down and then being willing to back up their words?

I can see a time efficiency argument, cost reduction argument, etc., for standard definitions here, but at the end of the day, they're not necessary.

The companies that offer the most credible products, verified via third party testing, get FDA approval. Everyone else gets weeded out.

[iancmceachern]

>"But the company has to submit in writing an application laying out their claims?"

How so?

[iancmceachern]

I'm saying the written submission doesn't contain this, and even if it did there is no one reviewing it that actually knows the technical details enough to provide meaningful oversight.

It's similar to that quote from a Boeing insider that came to light "they (Boeing airplanes) are designed by clowns suprivised by monkees".

Note - these are not my words or opinion, just a quote from another guy

[MichaelZuo]

You don’t think they contain… written claims?

I’m not saying they contain foolproof technical specs, but broad claims certainly.

If you genuinely refuse to believe this is possible or is currently done by some fraction of folks, then I guess I’ll leave it at that.

[MichaelZuo]

Are you unsure about the meaning of submitting a written application?

Or is there some other confusion here?

[iancmceachern]

Please see my reply immediately above.