[abalone]

What's really interesting about this is not just that Lattner is brilliant and liked, but that is highlights just how critical software correctness and reliability is to autonomous vehicles. Naively one might have expected some machine learning expert to take over the reins at Tesla. But fast-moving Silicon Valley needs a fundamental shift in quality standards when it comes to safety-critical software, and if you look closely, Lattner has been leading this charge at the language and compiler level.

His work has been distinguished by the melding of language safety, reliability and clarity, that is, not merely having sophisticated constructs that help the guarantee correctness, but also making code simple, beautiful and easy to read. Ultimately writing safe code depends on the ability of the programmer to comprehend it, so creating a programming environment that's successful on all fronts is a foundational achievement.

A notable example: LLVM enabled ARC, a beautifully simple approach to memory management that removed much (not all) of the need for the developer to implement details in code, while providing high efficiency and, perhaps even more importantly, predictable performance (no garbage collection pauses). These are all essential for safety-critical realtime software.

[pjmlp]

> LLVM enabled ARC, a beautifully simple approach to memory management that removed much (not all) of the need for the developer to implement details in code, while providing high efficiency and, perhaps even more importantly, predictable performance (no garbage collection pauses). These are all essential for safety-critical realtime software.

A GC algorithm known since the early days of Lisp GC research and used in languages like Mesa/Cedar in the late 70's.

I can think of lots of other examples, like how VB, Delphi and C++ Builder interfaced with COM in the mid-90's.

Yes, ARC PR made automatic memory management to those without background in compiler research more easy to accept, specially if they weren't aware of the stability issues that trying to implement a tracing GC in Objective-C semantics meant.

However it goes without saying, that Lattner is really brilliant and his work on LLVM as a compiler building tool, as well as, clang in regards to improving the status quo of C static analysis tooling and compiler error messages, is really notable.

In regards to ARC, maybe also in terms of PR, as now there is a whole generation of developers that thinks reference counting isn't garbage collection.

[vintagedave]

> I can think of lots of other examples, like how VB, Delphi and C++ Builder interfaced with COM in the mid-90's.

One curious thing is how similar Swift is to modern Pascal, ie Delphi. It has many other influences, and it is certainly not Pascal with another syntax, but reading the Apple guide when Swift was released gave frequent moments of deja-vu.

On the subject of COM, yes. If you think COM means ATL you should try it in a language designed to work with interfaces and reference counting inbuilt - Delphi has COM interfaces as first-class language primitives, and a heap of classes and other code making COM quite straightforward. Much of this even spills into C++Builder, though since it's a different language it's not as clean as in Delphi. Still miles past ATL though.

Re ARC: I think the thing that makes it appealing is it's conceptually simple, completely deterministic, and can be traced by reading code rather than understanding an environment's implementation. Delphi does ARC now too, and if you've ever wanted ARC in C++, C++Builder optionally supports it for some classes. ARC is not yet on Windows for either language, we're talking just iOS, Android and soon Linux here.

(Disclosure: I recently started working at Embarcadero on C++Builder. This is a personal comment only. But liking the languages Embarcadero makes was one main reason I joined.)

[abalone]

By PR you mean public relations? Note, I did not claim that Lattner invented the general idea. I'd suggest that dismissing the careful selection, refinement, integration and popularization of design elements as "PR" underestimates how great products are made. It's like saying Tesla's doing nothing new or the Macintosh was invented by Xerox.

[devsquid]

ARC is a whole debate, but the one thing it is not is simple and I would argue its more error prone than a traditional GC. I've used it for most of my career and I have seen what sloppy/unaware coding can do to it.

Lattner is a well known expert on compilers. Having used Swift since its inception, I would call into question the reliability of the Swift LLVM compiler. In its current state (3.0.2) its absolutely terrible and does not back up the sentiment; "But fast-moving Silicon Valley needs a fundamental shift in quality standards when it comes to safety-critical software, and if you look closely, Lattner has been leading this charge at the language and compiler level".

[abalone]

I'm not claiming Swift 3.0.2 is the most reliable language ever. All systems require time to mature and it is very young historically speaking. I am saying Lattner's design principles should realize better results for safety-critical software in the long run. Compiler bugs can be fixed; programmer error is tied to design. It might be Swift, it might be something more pared down, but he's a good guy to have in charge of an autonomous driving software platform.

[devsquid]

Well I'm sure he's a very experienced manager for technical projects and also brings a large amount of experience with compilers to the table.

His online presence always has that Apple Arrogance™ to it. Thats coming from someone who was born and raised an Apple fan.

[webjprgm]

I'm not sure I would say Swift is the kind of robust that is needed for safety critical software, but it is a nice step forward for application code. It mostly forces you to deal with things safely while still allowing Objective-C dynamic behaviors when you need to get around the restrictions (often to interact with other Obj-C code).

So, yes I can see why one would call Lattner at the forefront of making more reliable compilers and pushing shifts in quality standards in fast-moving Silicon Valley. It is an awesome achievement to create a new language that improves both readability and safety and even more awesome to get it mainstreamed so quickly.

There are a few people who I would like to trade places with. Lattner is one of them, Musk is another. They both fulfill different parts of my long-held dreams. So I consider them to both be quite awesome. Its cool that they'll be working together too I guess.

[devsquid]

Sure Lattner and Musk are interesting people, but I find the level of hero worship in the tech industry to be sickening.

Having used compilers for a few new languages (Rust, Go, Dart, Kotlin, Swift). Swift is the only one I've had any issues with as well as Swift seems to be the only language to have adopted the "move fast and break things" philosophy of Silicon Valley. I dunno, I just don't see the argument.

[exDM69]

Lattner is best known for starting the whole LLVM project. Swift is just a small side project in comparison, it just got adopted by Apple for some reason.

LLVM is one of the most influential pieces of software of the past decade. Hero worship isn't good but credit where credit is due.

[MrZipf]

Are you sure no one else was involved in starting LLVM? PhD supervisor, fellow students, etc. Just saying...

[devsquid]

Oh yea LLVM and Clang are incredible pieces of software! He is truly a master.

[mpweiher]

clang was mostly steve naroff.

[abalone]

You're confusing language development with autonomous vehicle development. Think of the long term goal. It's desirable to move fast and fix things with language development in the near term, to achieve a more perfect design and accelerate its maturation at the temporary cost of more volatility. After this process achieves a high level of maturity, said design principles may offer a safer, more reliable programming system that would be better suited to safety-critical applications.

Additionally I'm sure we can all agree there is no substitute for maturation through time and usage in the field. Which frankly is an argument for more popular languages over obscure ones. None of the ones you mentioned are ready for safety-critical system development (including Swift 3), but which one is most likely to achieve widespread adoption and field testing in the long run?

[devsquid]

No I'm not confusing them. I'm responding directly to the comment that Chris lattner represents a more measured approach to software development than is tradition in the tech industry.

I don't think Swift stands to gain wide spread traction outside of Apple orientated app development. Aside from a lack of stability, Apple is to well known for boxing its competitors out. I've used and loved their products my entire life and I know how annoying it is to go against Apple's grain.

[jolux]

>I don't think Swift stands to gain wide spread traction outside of Apple orientated app development.

It already is though, there are several Linux web frameworks etc. It's open source and community run so I'm not sure how they're planning to box out competitors from it.

[hamstergene]

ARC is a tradeoff between manual and automatic memory management. Requring a little bit more care from programmer is intentional, not a disadvantage as you picture it, it is a price for not having, you know, GC. GC is less error prone not for free but at the price of eating CPU and memory, which in the world of mobile devices equal less battery life, so it is quite desirable for iPhones and MacBooks software not to have it.

[pjmlp]

ARC is automatic memory management.

"The Garbage Collection Handbook", chapter 5

http://gchandbook.org/

[devsquid]

Yup but people love to argue over the small shit. Chris Lattner even refers to ARC as a form of GC.

[hamstergene]

Different approaches to memory management differ in extent of how much of programmer's job they automate.

Garbage collectors are fully automatic and rarely if ever require to mind anything; automatic RC does almost everything but requires programmer to analyze and annotate some things as 'weak'; manual RC requires a lot more programmer's effort while still technically being "automatic"; and manual memory management means the programmer does everything.

Automatic/manual is a scale, not a boolean yes/no, and the point is that ARC lies on it a bit closer to manual than garbage collectors.

[pjmlp]

The thing is, ARC is a garbage collection algorithm.

There isn't anything like ARC vs GC, that is layman knowledge and just wrong from CS point of view.

[hamstergene]

ARC is not an algorithm, it a language-level feature that generates retain/release calls automatically so that the programmer does not have to. Unlike GC systems where the resulting program does run an algorithm (and wastes CPU on that), with ARC the generated program is no different as if retain/release were written manually and runs no extra code.

[solidsnack9000]

The right comparison for ARC is with manual memory management -- not GC.

[adevine]

Depends on what your point is. Both ARC and GC are approaches to limit the complexity and difficulty of memory management. As such, I think it's very reasonable to compare them, because they're different approaches to the same underlying problem.

FWIW, as someone who was a Java programmer for over a decade before learning Objective C right after ARC came on the scene, I greatly prefer ARC over garbage collection. I find the things you have to remember to think about with both ARC and GC (e.g. circular references and unintentionally strongly reachable references) to be about the same cognitive load, but the deterministic, predictable behavior of ARC means you won't have to try to debug random GC hangs that only happen in prod under heavy load and the subsequent fiddling with a million GC options to get performance to be acceptable.

[pjmlp]

ARC is a GC implementation algorithm, you probably mean tracing GC algorithm.

"The Garbage Collection Handbook", chapter 5

http://gchandbook.org/

[rimantas]

I think it is time for you to buy more books.

[pjmlp]

I have a very good collection of CS books and papers about programming languages and compiler design...

[abalone]

I'm actually excited about server-side Swift for exactly this reason. It's early days though.

[brokencode]

Not really. They are both valid comparisons, since ARC is much easier to work with than manual management and can offer more predictable performance than GC. That said, it's also slower than manual management and can be trickier to work with than GC.

[pjmlp]

ARC is a GC implementation algorithm, you probably mean tracing GC algorithm.

"The Garbage Collection Handbook", chapter 5

http://gchandbook.org/

[nicky0]

Just to point out that the Swift LLVM compiler is written in C++. So it being unreliable doesn't necessarily say much about the reliability or goals of Swift as a language.

[steeve]

It looks like you haven't used C++ in a while.

C++14 is a whole other world, and can be written with most (if not all) the safety guarantees you would expect from Swift or Rust.

I had to use it for a project and was very surprised about this too...

[autoreleasepool]

You're lucky if you get to use C++14 in the real world. My last C++ job was maintaining a 2 millions of line of legacy MFC code.

[pjmlp]

Even there one can write quite safe C++ code with help of CArray, CString, CComPtr, ...

I used MFC like that in the late 90's/early 2000.

The problem are the team mates that write Win32/C like code, instead of MFC/C++ code.

[nicky0]

I wasn't trying to suggest that C++ was the cause of the compiler issues, just that Swift wasn't.

[9935c101ab17a66]

Can you explain why you call into question the reliability of the Swift LLVM compiler?

[nekitamo]

It would be apparent if you were a regular user of the compiler. Random crashes while compiling well-formed code and performance problems with larger files/projects are quite common.

Still, I would say it's mostly usable now. It used to be a lot worse.

[devsquid]

You find it better than before? I find that Swift 3.+ is drastically worse than Swift 2.

[devsquid]

Do you use Swift at all?

-edit- I meant it as a serious question. But the person who responded to me sums up the issues.

[chris_7]

This is downvoted presumably for lack of information, but it's pretty much true.

The Swift compiler segfaults very frequently. I do find this amusing in that it's the compiler for a theoretically largely-memory-safe language (yes the compiler is written in C++, it's still funny). The syntax highlighter in Xcode, which is driven by the same stuff, also crashes, which breaks autocompletion and even indentation. Using Xcode, you just have to get used to it. It frequently reports the wrong error message - just something that isn't even close to related. Sometimes object files just become 0 bytes and you either need to clean (and experience the Swift compiler's blazing performance again) or go and modify that file so that the incremental compiler will pick it up.

I've found most of these to be triggered by using a lot of closures and possibly type inference. Shaking out the incorrect errors or segfaults is... not fun.

[nodamage]

The most annoying one is the incremental compiler is broken under Xcode 8, leading to full recompiles every time a line of code is modified.

https://forums.developer.apple.com/thread/62737?start=0&tsta...

[devsquid]

Oh? I meant it as a srs question. I wasn't sure what parts I should include. But thanks for essentially saying most of it.

I should mention I also find the community to be sorta toxic. They are so focused on Swift being the one language to rule them all and they use terms like "Swifty".

[pjmlp]

All language communities do that.

[72deluxe]

Yes I can agree with these.

I wonder if we'll have refactoring in Xcode for C++ now Lattner has gone. I wonder why they never added it.

[freyir]

Swift, XCode, and Apple software in general don't scream reliability to me. No doubt Lattner is a smart guy, but I'd be more comfortable with a smart guy from NASA or somewhere similar with experience developing survival-critical software systems.

[noobermin]

This is my feeling too. Yes, I'm a scientist, so I'm from that side of the pond, but does it make sense to hire a developer type for engineering where lives could be at stake?

Not that dev'ing should rise to that level, but it isn't the typical level most developers work at. Also, if he is more of a "software correctness and reliability" guy as abalone says he is, then yes, that is the right direction.

[shiny]

Pretty much none of Apple's current software is written in Swift though, from what I can tell.

[dwaite]

Actually a fair bit is in Sierra. Apple decided apparently not to invest resources in targeting 32-bit platform support, so they have had to hold off shipping software relying on it until the platforms drop 32-bit support. Sierra did that last year, and I'd put $5 on iOS 11 doing that as well.

[dwaite]

Correct, I should have said "the platforms dropping 32-bit processor support".

The ability to run 32-bit applications means that preexisting libraries cannot incorporate Swift code yet.

The "application may slow down your phone" warnings that users are getting with 32-bit apps this year is a pretty strong indicator that Apple is going to remove support for running 32-bit apps completely for iOS 11 or 12. They previously had a deadline for apps to have 64-bit version submitted, but backed off the ultimatum for now.

[gurkendoktor]

> They previously had a deadline for apps to have 64-bit version submitted, but backed off the ultimatum for now.

Apps have had to support 64-bit since June 2015 (February 2015 for new apps), and Apple hasn't backed off that deadline. But there are still 32-bit apps on the store that haven't received an 64-bit update, and I don't think Apple has ever stated what it is going to do about them (other than showing the warning).

[mikeash]

Swift supports 32-bit iOS, just not 32-bit Mac.

Apple can't write Swift libraries (at least not ones that are publicly exposed) until the ABI stabilizes, so that will not happen until at least Swift 4. Apple can write Swift apps as long as they don't need to support 32-bit Mac, which they haven't needed to do for years.

[valleyer]

Sierra did not drop support for 32-bit apps. It doesn't support 32-bit hardware, but that change was made many releases ago (Mac OS X Lion IIRC).

[rarepostinlurkr]

Sierra did not drop support for 32-bit applications. Pretty sure Microsoft Office just released a 64-bit version of their product a month or two ago. Good luck dropping 32-bit on the Mac and godspeed.

[chris_7]

Including the Swift compiler itself - though at least that might prevent the segfaults.

[devsquid]

If I remember right, its just the calculator app. Lol....

[pjmlp]

The dock and launchd were rewritten in Swift for Sierra.

There is a WWDC session talking about it.

[devsquid]

Well that doesn't bode well ;) Sierra's dock is sooo buggy. I have to restart my machine numerous times a day because the dock stops working, literally doesn't work lol. I'm sure theres a way to restart just the dock app, but eh i've been quite busy. Sierra has been one of the buggiest version of Mac I can remember using.

I would surprised if they actually did tho. I know someone did some static analysis of the apps for mac and iOS and found Swift was barely used at all.

[amiga-workbench]

The dock is a separate process, as is Finder, you can just kill them.

I have to stomp on coreaudiod a few times a month because my USB DAC stops responding.

[ryanmarsh]

NASA has a culture of safety and redundancy ad nauseam because people are indoctrinated into it, not because their hires have an innate proclivity for it.

It is not fair to judge Lattner's ability or commitment to safety based upon Xcode and Swift. One of these predates him, the other is the result of his decisions (no doubt) but also countless decisions of others, including those above his pay grade at Apple.

Xcode is basically the evolution of something designed for NEXT. Swift is a solution to various problems in application development. I'm not aware if he (et.al.) had real time processing or safety critical devices in mind with 3.0.2.

[freyir]

> NASA has a culture of safety and redundancy ad nauseam because people are indoctrinated

I'm not claiming Lattner is missing some innate ability -- it just appears they're putting someone in charge who has never been exposed to this mindset. Maybe they have a culture or other leaders already in place who can foster this within the team.

> It is not fair to judge Lattner's ability or commitment to safety based upon Xcode and Swift.

To be fair, we cannot judge Lattner's ability or commitment to safety at all, because he has no publicly-known experience with safety-critical systems.

Perhaps he has relevant experience that's not public. And if it turns out he has no relevant experience, I'm not saying he can't learn. It's just strange for Tesla to put someone in charge who will be learning on the job.

[pjmlp]

Probably not, but the WWDC session about real time audio used Swift on their presentation.

http://devstreaming.apple.com/videos/wwdc/2016/507n0zrhzxdzm...

[ryanmarsh]

"Real-time" audio doesn't involve human critical systems.

[pjmlp]

Of course not, but part of the sentence said

> I'm not aware if he (et.al.) had real time processing

[devsquid]

I know the first thing I thought about was does this mean Tesla's going to start writing its software in Swift. If so they are fucked :P. (BTW I write non-critical consumer software in Swift as my day job)

[webjprgm]

Swift has its problems but keeps getting better. I only get to use it for my at-home projects, since at work the portion of the codebase that is for iOS is Objective-C with no plans to switch any time soon. Old Obj-C too, and started by people used to programming in VB on MS platforms. So count your blessings.

[devsquid]

I find Swift to be getting worse each year. While sure i'm blessed that I don't have to write applications in assemble, I have grown jaded towards Swift. If it wasn't because I am an iOS developer, I would happily not use the language. I have been slowly positioning myself away from doing iOS development. I feel like a massive corporation like Apple can provide better tools to write apps for their walled garden than what they are providing me.

[72deluxe]

What would you recommend instead?

C#? C++?

C++ has become incredibly great recently after languishing in the C++2003 period for too long.

[nicky0]

SPARK ADA of course.

[tma-1]

Well considering that most of the autopilot software is driven by black box deep neural networks, I don't think hiring someone with a strong coding background is going to make that much of a difference when it comes to safety.

[tawaycozstoned]

Sure it is. NNs can behave non-deterministic (i.e. they produce different output for same input because AFAIK some operations on GPUs are undeterministic) and it makes sense to invest into software systems that clearly take a closer look on software correctness.

One idea to get correcter code is clean code layout (I heard good things about the LLVM code base), simple to read code and compilers that exploit theoretical knowledge we gained over years of compiler and type theory research. I think Chris Lattner has expertise in all of them (or at least knows about their importance, contributions and drawbacks), and if want to build a full-blown self-driving car it is important to have no indeterminism in your car, and advanced languages and compilers help to guarantee specific statements about your code.

So it absolutely makes sense to invest in your compiler research team for safety, as our security, reliability and correctness expectations will rise (which is good) in particular for self-driving cars (they are not just DNNs).

[tma-1]

It ultimately depends on how much of the car's action is driven by DNNs, a miss-classification in the CNN can easily lead to a car crashing on a highway and causing a pileup. DNNs are so complex, you can't really write unit tests like you typically do for deterministic code.

That's why some of the big banks flat out refuse to implement any form of deep learning for risk analytics. They're much more reliant on simpler ML models like random forests and logistic regression that are easier to analyse and diagnose by model governance teams.

[tawaycozstoned]

Yeah and it will happen. I am all for self-driving cars simply as they increase the life-quality for our civilization (more independency, cheaper ways of transportation, less fatalities), but there will be accidents and then you have to analyze them (which is good).

In the end you want to know what did go wrong (the public will demand it and they are right) and it might be misclassification.

But that is not enough. You want to know why did it classify situation X wrong? So the answer is because the inference network (which was created via the training network) computed its weights because the input was Y. Now you might throw your hands in the air and say "oh it's complicated, the network is undeterministic, blame NVidia", but you can also go farther and build your networks deterministic (which is possible and AFAIK not a performance penalty). Compiler research helps in at least guaranteeing that certain parts of code are deterministic which makes it easier to debug and maybe avoid complex NN misclassification scenarios, but the way to do it doesn't have to do much with NN's itself but more with language design and (real-time, in particular deterministic) OS research.

So the statement oh, that's so complex, we do not no why we misclassified is no excusion, we can do better.

For starters we have to publish NN papers with implementations that describe how to make a particular NN out of given trainigs data (provide that too). We already publish the code and network structure (see caffe, etc.), but often with pre-trained models that have been build on a cluster with many forms of training-data going through network structure, etc.

Now at the moment you read a paper, head to the published code (often the case, again a desirable property of the ML community) and try to reproduce some examples by training on the data.

However it is hard to say in the end, if your network is really as good as the published, as a simple

  $ diff my-net-binary-blob.dat tesla-net-binary-blob.dat

might fail, if the stack (training etc.) to build the network is undeterministic. However if you have a good (i.e. deterministic) stack you might be able to reproduce NNs bit-for-bit, which makes it simpler to answer the question "why did we missclassify".

[Smaug123]

> However if you have a good (i.e. deterministic) stack you might be able to reproduce NNs bit-for-bit, which makes it simpler to answer the question "why did we misclassify".

It does make it simpler, but surely not usually simple enough to answer the question "why did we misclassify". It's like saying we will finally understand consciousness once we simulate the quantum mechanics of a certain cubic metre of space with perfect accuracy - which need not be true even if that cubic metre happens to contain a functioning brain.

[simonh]

So would you say that the reliability and correctness of the non-NN part of the system, and the code that implements the NN itself, is not worth bothering with because the system includes a black box NN in it's design? What a bizarre conclusion to draw!

[davidgay]

I like LLVM. But in the world of programming languages/compilers, someone like Xavier Leroy (a real, proved correct C compiler) is much more a name that I would associate with a drive for "software correctness and reliability".

[PinguTS]

If you want to talk about safety and reliability in automotive, then we talk about Autosar[1].

One of the concepts is to prevent the use of any dynamic behavior, which means you don't need any garbage collection at all. Because garbage collection is not predictable. It can be good and bad, but even a good one is not predictable.

Embedded design and development is completely different to classical IT. For IT that sounds archaic. But it always proved right at the end.

[1] https://en.wikipedia.org/wiki/AUTOSAR

[Matthias247]

True, AUTOSAR is the current standard for embedded automotive systems. Regarding safety I heard different things about it, some parties would only use it for up to ASIL A, others to C, others also to D. My personal opinion is that AUTOSAR (and even it's basics like C as a programming language and the OSEK OS) are not the best solutions that we should come up with for all the safety critical tasks in the autonomous driving world. Someone like Lattner would clearly have the potential to improve the current state of the art there a lot - but it's uncertain if Tesla would share any advantages with the remaining industry.

And regarding memory allocation in reliable automotive systems: Yes, the best practice would be not to allocate at all to get to some deterministic behavior. However I've seen lots of projects where "don't allocate" is implemented as "don't allocate with malloc", and you find dozens of custom memory allocators and pools throughout the code. Some of those designs are probably less reliable and safe than using a garbage collected language would be.

[therockhead]

I often wonder would Erlang be a good language for embedded automotive systems. It's seems to have the right traits - no GC pauses, built for reliability etc.

[Matthias247]

I think it would fit great from the semantic. Message based communication is exactly what a lot of automotive systems are about (CAN, FlexRay, ...). The downside of the actor model from an implementation point of view is that you need potentially unbounded memory for the incoming message queue of each actor. For each message that is sent to an actor you need to allocate a message, populate it and place in that that actors queue. If the actor can't process fast enough the queue will fill hap.

It might work out with a very careful system design, but determining statically how much queued messages and memory is needed seems like a very hard task.

[Cyph0n]

That's not very surprising honestly. ML/AI algorithms alone are useless if they are written incorrectly or are not executed properly by the underlying OS/firmware.

[thatfrenchguy]

"predictable performance (no garbage collection pauses)"

No garbage collection means more fragmentation though which can be an issue at some point perf-wise, there's no free lunch.

[gigatexal]

software correctness eh?

so build the autopilot and other systems in haskell?