Hacker News new | ask | show | jobs
by SlowRobotAhead 2685 days ago
Automotive EE here.

In the last 5-8 years there has been a DRAMATIC change in the way vehicles are tested and released. Everyone has become much more focused on electronic features - while at the same time almost everyone is completely outsourcing electronics design and firmware. Cars hit the lots with 10 new flashes waiting. When I started, it was a BIG DEAL if you had a lot held up somewhere in Detroit because of a pending flash.

I've worked with two of the big three, and right now, neither own their traction control, radio, trans, body, ignition, steering, etc controllers. Basically anything. Even engines are being outsourced. I don't just mean the module, but the code on it. One of the mfgs, their engineers can't even SEE radio code or anything much more than general documents. They used to have compiler access to most things, now it's extremely rare.

The mfgs wanted to outsource the development, and now when they need an adjustment to traction control it's a $50, 100, 250, 500,000 charge, BUT, they've also moved the responsibility to the mfgs like Bosch, ConTevis, etc.

Short version is everyone is playing the game to get hot new electronics in cars, to make them seem as advanced as phones - but the work is not being put in besides surface level customer view. I've found obvious glitches in zero mile vehicles this year, but none I can think of from 2000-2013

I would not purchase a 2019 anything right now. I'd allow for two-three years worth of flashes before considering a new car. And that doesn't mean you're "safe" with your old car. You think your 2001 Subaru Outback is going strong? Well, surprise, you're likely to fall victim to a different issue.

As to the article, yes, luxury cars are worse. Most people that don't realize you can't leave your touring Ferarri in the garage not hooked up to a trickle charger, or also picking on Ferarri that the LaFerarri when runs battery dead may require to be loaded on a truck and shipped to a dealer before it'll charge again, or that the new Range Rovers will attempt to void your battery and electronics warranty if they find a radar/laser detector hooked up because THAT is how finicky the electronics system is.

It's funny to me the amount of work that your average Toyota or Chevy vehicle gets relative to lux models that skirt by with less engineering and more features.

As to Cadillac and Lexus, those are just GM and Toyota, all the same things plus some. So fundamentally they're the same, but yes, add features add problem areas.

8 comments

With the exception of safety features (think blind spot monitoring, backup cameras, lane keep assist, parking sensors) and mobile connectivity features like bluetooth, frankly I'm getting tired of all the crap they're throwing into vehicles these days for many of the reasons you mention.

It seems every manufacturer or parent brand has their own infotainment system, NAV, etc. and you pay the price for it when you buy the car. And god forbid you want the upgraded safety features or leather seats, but don't want any of the other bells and whistles, because you can't have it without tacking on $4000 for the "technology" package.

My ideal car these days is one with the safety bells and whistles, limited luxury features (automatic and heated/cooled seats, climate control), but without any complicated infotainment beyond bluetooth/Android auto/Apple Car Play connectivity. There's no reason to have redundant technology built into cars by companies that aren't as good at it when we have perfectly capable, and generally more up-to-date mobile devices with us at all times.

Is anyone making reasonably well-appointed vehicles without throwing everything and the kitchen sink in there?

>Is anyone making reasonably well-appointed vehicles without throwing everything and the kitchen sink in there?

No. Honda, Toyota, Fiat, GM, everyone seems to have the same problem. For what it's worth, FiatChrysler and GM have at least had the sense to set higher requirements for vendor/suppliers than Toyota.

Specicially, after Toyota's "unintended acceleration" issues (mostly floor mats, some APPS/Pedal issues) it was discovered that their ECU taking accelerator pedal position (I'm not sure if it was the PCM/ECM specifically) had over 12,000 GLOBAL VARIABLES! Let that sink in for second.

After a code audit by iirc UPenn the bug was not found, but that wasn't a glowing review, it was ad admission it was impossible to prove/disprove.

Edit: I take that back. There are Chinese brands I drove in Malaysia and export-only Nissans that while being wholly illegal in the US or Europe are still barebones vehicles.

I've read the audit and basically it was done by a guy with no clue about the automotive world. He compared the code with the one made at nas. In the end they indeed did not find the root cause, but they had to say something as they are the "experts". So they pointed at "bad practices" like global variables.

What if i tell you that a pretty big chunk of the cars on the road are running software which are using global variable as means to exchange data between modules ? Well, that's how it is done and it isn't less safe. The last SW i worked with had 20000 global variables and 60000 parameters that the calibration guys could fiddle with.

Serious suppliers are applying safety standard and methodology and are not rushing the FMEA. Countless times my customer cursed at the safety guys for postponing the SW because they were not finished with testing, but this very same customer never had to stand in court for a safety issue with the SW, and so are many other carmaker.

When the Toyota pedal issue came out, we just could not believe that Toyota did not have the gas pedal override by brake safety in their SW.

> over 12,000 GLOBAL VARIABLES

How many global variables do each of Toyota's competitors have?

honestly, in the context of an embedded system, I'm not bothered by this.

I'd expect to see very little dynamic resource allocation, so you could probably put a permanent name to almost every byte of data in the system.

Remember, once it's compiled, everything's a global anyway, especially on chips small enough to not be set up with a strong multi-tasking and memory management model.

Does it matter if all your friends jump off a bridge?

Toyota got caught with inexcusably bad software practices. There is a reason Chrylser, GM, Ford, and others are pushing it out of house to suppliers.

Look for trims of vehicles designed for fleet use. Those remain pretty spartan.
They look it, but they aren’t. The RAM truck that has manual windows and manual door locks - has the same ABS/ESP, Radio Hub, body controller, battery modules, 7 CAN buses (2018+), three or more LIN buses, etc etc as the full load models.

Same body controller guts in as an Alfa Romeo.

He wants heated and cooled seats. You're not going to find that in any fleet vehicle.
Those aren't necessarily mandatory, but it certainly seems a lot of the safety tech is not available unless you also get things like that.

For reference, my wife and I bought a 2014 Mazda 6 Touring, which seems to be one of the few cars with a well-balanced mid-tier that doesn't cost an arm and a leg and still has good safety features (and did not come with heated/cooled seats).

Yeah, I got a 2015 Mazda 3 Grand Touring so I could get the heated seats, among other things (like xenon headlights, which are an important safety feature; halogen headlights should be banned, and the IIHS agrees with me).

But of course it also comes with a clunky and nearly useless nav system that costs a fortune if you look at the line-item cost. They only recently, finally, started supporting Android Auto.

Yep, I've been mulling some DIY work on replacing the headlights and adding foglamps... Despite appearing bright on my garage door, the actual street visibility of the standard headlights is pretty awful on this car.

I'm not sure about 2015 Mazda 3, but I know 2016 and later cars can be upgraded to support Android Auto for a fee at the dealer (I want to say ~$500, which may or may not be worth it).

I completely despise the new consumer facing electronics in cars. Not because I find anything inherently wrong with electronics, but because I find the software/electronics in every car, in literally every car made today, cheap or expensive, to be an epic disaster both from an industrial design, and security standpoint.

Sadly, I think "normal" cars will go extinct the way non-smartphones and non-smart TVs went extinct. It's impossible to buy a TV these days that is not infected with Internet of Shit.

Where's the 70 Series Toyota Land Cruiser of today? The 70 series is still being made, it's just not sold in developed countries. Why can't they just make it emission compliant and sell that car in the EU and US? That's the car I want!

> It's impossible to buy a TV these days that is not infected with Internet of Shit.

It's possible, they are just called monitors or commercial displays; there's a significant premium for them at typical living room TV sizes compared to things marketed as “TVs”, though.

Yes, the "dumb" version is much more expensive than the "smart" version.

But that is not all of it. Most monitors approximate (some better than others) sRGB, while TVs approximate (again some better than others) Rec. 709. Rec. 709 and sRGB share color primaries, so they have the same gamut, but Rec. 709 uses a different transfer function. If you use the (profiled) monitor with a computer that does color management, and use a color-managed video player, all is well, but if you use the monitor as a "TV", without a computers, there's no profiling and no color management, so the broadcast and movies will not appear colorimetrically correct.

But wait, now we have Rec. 2020. I don't know if Rec. 2020 TVs are common now, but they will become common soon enough. Computer monitors are very different than Rec. 2020, even wide gamut ones. You MUST use color management to display Rec. 2020 on a computer monitor. Without a computer involved, computer monitors can't display Rec. 2020 properly.

But that is not the final story still. TVs can generally do both 23.976 fps, and 24 fps. Computer monitors very rarely can do this, generally reserved for expensive models designed for professional media creation.

So even if you pay the huge "dumb monitor tax", you still can't get the same result.

Buy a J5005 NUC and be done with it. There's your computer.
You can sit around and wait for the next Steve Jobs to release the iPhone of cars but it won’t happen. Too much regulation, some for good reason.

If you buy an iPhone and it crashes, no big deal. Not the same for your car.

>Where's the 70 Series Toyota Land Cruiser of today?

Wouldn’t pass emissions, homologation / pedestrian impact, safety, etc. Everyone loves regulations until they don’t. Jeep Wrangler is the closest you can get, and the new Wrangler has two or three microcontrollers in the steering wheel alone even on the base model.

> The mfgs wanted to outsource the development, and now when they need an adjustment to traction control it's a $50, 100, 250, 500,000 charge, BUT, they've also moved the responsibility to the mfgs like Bosch, ConTevis, etc.

I can see where this would complicate and slow delivery of bugfixes, but if you ask me who I'd rather have mucking around with my ABS firmware, a GM engineer or a Bosch engineer, that's an easy answer.

Except Bosch is doing almost all of their real engineering in India right now. GM has their testing ground with cars and data on hand.

It's not as simple as you make it out to be. There are times I would want the system to be looked at holistically not just as components.

But that's been that way for some time now. My 2002 (E46) BMW 3-series has mostly Bosch parts inside. The transmission is made by... GM. The only thing that's BMW is the engine and the body. All the sensors on the engine are Bosch. It's a fairly smart way of developing a car in my opinion: concentrate on what you're good at. But to add to your point: you can also tell from the problems what BMW had outsourced -- it's often the accessories that don't age well. The engine in the E46 BMWs are fairly reliable. I had to open the valve cover once after 10+ years of driving and everything looks pristine inside. The biggest defect I had to deal with was the VANOS valves, which only required some bearing replacement, and gasket replacements.

My 2013 (F30) BMW 3-series is similar in its parts sourcing. The turbo is made by Mitsubishi and the transmission is made by ZF (which is pretty much industry standard). Other than the engine, body, and maybe the steering components, everything else is again outsourced. So far it's been pretty damn reliable though.

That's been the case for a very long time. I have a 1980 BMW 528i, and it's the same. BMW motor, but everything electric is Bosch, including the L-Jetronic fuel injection. Transmission is ZF as well.
Even my 1975 bmw 2002 has a getrag 4 speed manual. The m10 engine I have was still being made in the late 80s for the 318i (with electronic ignition and fuel injection), but still mated to a 5 speed getrag.
That's some interesting insight... I wonder where it'll all end up... it's kind of difficult for markets to function well with things so utterly opaque to the average consumer. I wonder if it'll end up regulated, or if the fear of lawsuits or something else will knock people in line.

That said, the best answer is clearly to ride bikes more! A lot better for the environment, your wallet, and your own health!

http://www.mrmoneymustache.com/2011/04/18/get-rich-with-bike...

You almost touched on something car manufacturers aren't doing, and that's providing software updates.

My Subaru's infotainment system has strange intermittent bugs with the Bluetooth system. I get annoyed that even if the developer fixes it, I'll never see an update without buying a new car.

It's one of the reasons I'm such a Tesla fanboy. They do software updates. What's really great too is that they're done over the air. You don't need to bring it to a dealer and pay a ridiculous fee for it.

It'd be really nice if the other manufacturers could get on that.

My Porsche and my mom's Acura can both do OTA updates, though they both require user acknowledgement to install. Honestly, nothing coming OTA should be urgent enough that it can't wait until the yearly service.
> My Porsche and my mom's Acura can both do OTA updates

Are those updates for vehicle control software, or the entertainment/nav system? I would be surprised if you could, for example, change the engine control software or the ABS software, or the anti-collision safety system via OTA updates.

Updating the entertainment/nav console is more common. My 2018 Ford does that too.

Has there been a trend to bring controls back in house? I used to work in another mature industry, HVAC. It also went through a period of outsourcing all electronics/software, and then they realized that all the IP/secret sauce going forward will be in the controls and related software.

My former employer had plenty of unpleasant experiences outsourcing firmware, which contributed to bringing it all back in house.

>Has there been a trend to bring controls back in house?

Best I can tell it's still going out, not pendulum back in.

Part of the issue, and the thing almost no one realizes about Carpocolypse 2008 is that all these suppliers are 100% dependant on supplying to 5 vendors and that if one stops buying X module for 6 months, they fold. Blame in on JIT strategy, or regulation, or whatever, but Johnson Controls would have folded if GM and Chrysler weren't bailed out. Ok, but that means no seats for Toyota, Honda, Ford, etc. Very few people outside of Germany and Detroit realize what was really at stake there.

> Part of the issue, and the thing almost no one realizes about Carpocolypse 2008 is that all these suppliers are 100% dependant on supplying to 5 vendors and that if one stops buying X module for 6 months, they fold.

This was widely pointed out at the time, including, IIRC, by the CEO of Ford, who used it in pointing out why the bailouts, which Ford did not directly receive, were essential for Ford.

>Part of the issue, and the thing almost no one realizes about Carpocolypse 2008 is that all these suppliers are 100% dependant on supplying to 5 vendors and that if one stops buying X module for 6 months, they fold

They didn't have to wait until 2008 to learn this lesson. They could have learned it in the 90s if they were watching.

This is exactly why it's so hard to source vehicle parts for old Chrysler products. Every time they go bankrupt their suppliers go bankrupt and 3/4 of the tooling winds up turned into tin cans.

I did at the time, I'd have to dig around to find my comments then - I looked at the bailout as cheap, compared to unemployment for the big three, and their dependent manufacturers.
Yes, Autosar enables the carmaker to develop code in house for their cars, but it still needs the suppliers for the HW and the controls of specialised parts like injectors, rpm or lambda sensors.

The trend is for the carmaker to develop vehicle functions like start stop or cruise control, and give the object code to the supplier for integration.

> I would not purchase a 2019 anything right now. I'd allow for two-three years worth of flashes before considering a new car

Do you mean a model that is all new for 2019 or do you mean literally any car assembled in 2019?

The latter basically. I have seen no one that has their shit together right now.