> These “you should switch language” remarks are strangely enough from the backseat drivers of the Internet. Those who can tell us with confidence how to run our project but who don’t actually show us any code.
This is an usual problem with people participating in discussion over the Internet. I am sure those people mean well, but often they don't put enough effort into assessing the situation and underestimate its complexity before they decide to hit the keyboard. I've myself done it a number of times, but I am working on not doing it.
Advice for the people who are making all these suggestions and: A good way to see if you're talking out of your ass—unintentionally, of course—is to try to do what you're saying someone should do. Just try and re-write some parts of curl in Rust (you can go with a hip name like rurl). It's a good exercise to learn about the project you're rewriting, the language you're writing it in, and the problem domain. So, at the very least you'll get something out of it, and if your project is successful: even better!
Hear hear! At Recurse Center this is an explicit (lightweight) social rule:
NO BACK-SEAT DRIVING.
If you overhear people working through a problem, you shouldn't intermittently lob advice across the room. This can lead to the "too many cooks" problem, but more important, it can be rude and disruptive to half-participate in a conversation. This isn't to say you shouldn't help, offer advice, or join conversations. On the contrary, we encourage all those things. Rather, it just means that when you want to help out or work with others, you should fully engage and not just butt in sporadically.
Ironically you are violating another recurse center social rule in saying this: no feigning surprise, i.e. just because something is obvious to you doesn't mean it is obvious to anyone else, so don't act surprised when someone doesn't know or understand something.
Because everyone has had their own experience that they have learned from. Most people who do this are not trying to be a troll. They really have learned A lesson and want to help you by sharing. The problem is few people have the context to know all the important issues, they just know they were burned badly one time and want to save you not realizing that the solution they found has downsides that will burn the other guy.
I think that you express a fallacy. You're saying that nobody can provide thoughtful criticism unless they take up the task themselves. So the next time you criticize politics, are you going to run for office? The next time you criticize a meal, are you going to become a chef? The next time you criticize a car, are you going to become an engineer? Yes, it's one thing to make thoughtless and uninformed criticism. But providing thoughtful and useful criticism does not necessarily require actively creating something from scratch. Otherwise, we would all only be starting new projects and never collaborating!
No, he's spot on. The one thing that keeps happening over-and-over again is that people who have never put together a system of any complexity will yell from the sidelines how others should build their stuff.
This goes for operating systems, application software and advanced rocketry alike.
The reason we have so much C code powering the world is that (1) there wasn't anything better when that code was written, (2) time has weeded out most of the bad bugs to the point that the old codebase will be substantially more stable than anything new no matter what language you would write it in.
If someone is willing to throw a few hundred million or so at re-building an operating system from the ground up that is 100% safe and reliable and that does not have more bugs than what is available right now then they are free to do so. They're also free to donate their own time and to try to enlist their buddies. But they're not free to yell at others how those others should do their jobs.
> If someone is willing to throw a few hundred million or so at re-building an operating system from the ground up that is 100% safe and reliable and that does not have more bugs than what is available right now then they are free to do so.
But at the same time - we are spending those billions. We keep throwing billions at Apple by buying their new phones, yet they release lists of security issues which this time were basically all:
I'm not sure if I have to be able to write a font parser and rendering subsystem in a memory safe language myself to be allowed to complain that Apple is using my money for something other than doing just that. To me it's completely mind blowing that someone uses a C library to parse complex third party binary data from the internet.
Any lang that easily calls C and can be called by C, or can be compiled to C should be fine to use today.
Re: kernel - it's a lot of pretty simple subsystems. Each system is usually simpler (easier to verify and polish) than a lot of common libs doing really complex things (ssl, video codecs, ttf, ...). Complexity + raw pointers quickly add up to guaranteed problems.
Rust uses llvm. Use it for new development on any platform that llvm supports. A few microcontrollers might be left out.
>The reason we have so much C code powering the world is that (1) there wasn't anything better when that code was written...
Right, but the criticism he received for his first post was not primarily about choosing the wrong language or about refusing to rewrite in a different language now. It was primarily about downplaying the role of C in the security issues that were found.
The rest of the debate, at least as I read it, was not so much criticism as it was a generic debate about the pros and cons of various programming languages like we're having one every other day on here and on reddit.
Choosing C back then was clearly justified for a project like curl given all the constraints and considerations. And for a rewrite to be a net positive in terms of security probably takes years, not to mention the huge effort that someone would have to put into it.
I have put together systems of considerable complexity (>100k lines), and am working professional programmer for almost 2 decades. Most of my work does not involve C, but various "safe" languages.
There were better alternatives already available at the beginning of the 70s, and certainly when most of the software was written that we are using today. Just to name a few I am familiar of with:
- Pascal being as old as C, and later on
- Modula 2, Oberon, the Oberon System was completely implemented in Oberon
- Eiffel
and of course, today:
- Rust
- Go
So, even back then, there were more safe alternatives to C around. C gained more popularity, and consequently the tooling gave it some edge, but there would have been alternatives. Of course, when you have a nice debugged C program, that is something worth keeping. And as a consequence, for mostly historic reasons, a lot of our current infrastructure is built on top of C - but that alone does not mean that C was the perfect or even the right tool to build all of this upon.
> (2) time has weeded out most of the bad bugs to the point that the old codebase will be substantially more stable than anything new no matter what language you would write it in.
This seems to assume that the codebase is stable and in maintainance mode. However, the reality for the vast majority of open-source projects is that they're either being actively developed, adding new code and new bugs, or they're abandoned.
I wouldn't put it as strongly--I think valid criticism can come from someone who hasn't "put something together"--but at the same time I think a lot of the criticism in this area isn't all that valid. Some of it is, but mostly it seems to me just people parroting "c is unsafe" as though that is itself the end of the discussion.
Exactly, show don't tell. And when I point to the fact that the Rust codebase itself has a lot of "unsafe" places I get the explanation "yes, but it's an old code and we haven't had the time to rewrite it to use the newer features that would allow more safe code."
Think about it: the people who design Rust don't have the time to rewrite their own Rust code to be safe.
When there's no time to rewrite the code of Rust to be demonstrably safe, why should any other project have more time than that, where the programmers know Rust much less or not at all?
Maintaining the successful project is hard and takes a lot of time, even without switching to something new.
Second point: look at the Benchmark game, a fast Rust code example also typically has an "unsafe" code. So it's still not even demonstrated that "you can have it both fast and safe" holds for these examples. Fast as in "as fast as C" and "safe" as in "provably safer than C."
Finally, I believe that the language should simply implement a language-level no-cost check for integer overflow, as the machine languages already have, and not complain that there aren't some special mechanisms in CPUs to do it differently. It's your language's failure if you can't use what already exists. The CPUs already can do it, the high level languages don't provide the high-level access to it. Which doesn't matter for C which is "unsafe" by definition (and where it's natural to call some asm when needed) but it should for Rust which goal is to produce a more provably "safe" code.
Writing unsafe Rust instead of unsafe C is not giving you any real advantage for the existing projects, as long as the state is as it is.
Also, look how Rust uses TLS. Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?
I like the perspectives of what Rust could bring. But I'd like to see a real life case of its success too.
> Writing unsafe Rust instead of unsafe C is not giving you any real advantage for the existing projects, as long as the state is as it is.
Your argument hinges pretty heavily on this. It's incorrect. The reality is that unsafe code in rust is not like unsafe code in C. First, it's bounded to a module. Second, you can grep for it (huge). Third, you still have less UB in unsafe rust than in C.
I've never written code in rust that required unsafe, after many thousands of lines.
> Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?
> when I point to the fact that the Rust codebase itself has a lot of "unsafe" places
One of the criteria for being in the standard library is "is this something which needs a lot of unsafe." This is so it can receive extra auditing, etc.
> yes, but it's an old code
This may have been true a while ago, but in my understanding (I'm not on the library team) it isn't really any more.
> why should any other project have more time than that
Because they do not need to write their own unsafe code, since it's already done in the standard library.
Generally speaking, applications in Rust should almost never need unsafe. Libraries may.
> a fast Rust code example also typically has an "unsafe" code.
This is not really true anymore; in fact, recently some programs were contributed that are 100% safe yet are faster than their previous, unsafe-using versions.
> believe that the language should simply implement a language-level no-cost check for integer overflow
This is not viable. Not enough machines have this.
> Writing unsafe Rust instead of unsafe C is not giving you any real advantage for the existing projects, as long as the state is as it is.
Unsafe Rust still has many, many more safety checks than C.
> Also, look how Rust uses TLS. Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?
Does it have all the features that curl needs, or does curl still have to use a Rust wrapper around openssl, inheriting all the potential unsafety problems of openssl anyway? What is worth then, in percentage, having anyway less exposed part of the code in Rust? How many other wrappers curl has to use? Have you tried to check and estimate if it's still worth?
Once again, Rust people should first demonstrate that they can manage to make any relevant library "cleanly" safe, then complete the feature set, and only then expect big non-trivial programs consider using some Rust.
First make the basics safe, then let us use that basics. Once you have save openssl equivalent, even C programs can link to it and be much less exposed to the potential problems than they are now. It's that direction that only has sense.
It's not what I tried to say, but I see your point. I merely pointed out that the people who're saying saying "Why not rewrite curl in X" probably didn't stop to think how challenging it would be.
> The next time you criticize a meal, are you going to become a chef?
Well, they're not criticising the meal here, they're criticising the kitchen it was prepared in. :P
But joking aside, they don't need to rebuild all of curl in <language of choice>, just get it started, and if enough people think it's a worthwhile project, they will contribute.
> But providing thoughtful and useful criticism does not necessarily require actively creating something from scratch.
I agree. However, a lot of these comments are along the lines of "C is unsafe. Go rewrite curl in Rust!"[1] without realising how much effort and time that would take (and I'd not be surprised if some portion of this group never worked on a project in either language). I personally don't find those types of comments very useful.
> Otherwise, we would all only be starting new projects and never collaborating!
This is what the people I'm referring to are essentially asking. Going hybrid would probably introduce subtle bugs, and increase complexity and maintainability down the line.
[1]: I love Rust, and a few of my recent pet projects are being written in Rust (one of which is being ported from C). So, I'm not trying to bash Rust. I'd even encourage anyone to use Rust over C, if they were to start a new project.
It's one thing to criticize somebody over a missing feature or a bug. It doesn't take an engineer to author a valid complaint when a car door handle falls off. It's another thing entirely to criticize an entire methodology or to try to dictate exactly how something should be done.
I don't think this generalises that well. We're learning some general truths that should be known to many people with enough time. You can certainly criticise whole architecture and methodology based on those ideas. For some ideas it took a long time before they became something obvious - we may be at a time when the scales are slowly tipping.
For example imagine telling someone in 2000 that they should be running an automated continuous integration pipeline rather than purely manual testing. Now it's a norm. Then, we were still discussing whether Extreme Programming was a waste of time. But we also didn't have the tools to support the CI properly.
Memory safe compiled languages seem to me like the same thing. We had things like Cyclone, OOC, and other experiments which barely anyone knew. Now we have Rust which is still hard to use, but it's gaining ground. In a decade, I hope to be pointing out to this post when someone complains that you can't criticize an entire methodology. "Remember when people said the same thing about enforced memory safety?"
Curl will not be rewritten of course. But I'm happy people push this idea. The next curl will not be written in C.
> It doesn't take an engineer to author a valid complaint when a car door handle falls off. It's another thing entirely to criticize an entire methodology or to try to dictate exactly how something should be done.
But if a car manufacturer consistently uses a production methodology which causes door handles to keep falling off, again and again, even after they've been re-glued on for the Nth time...
Then I think it's fair game to say that you think there's something wrong with the methodology. People suggesting alternate approaches without putting in the actual work involved may very well be "backseat drivers of the Internet"-type commenters, but that doesn't mean they're not on to something.
If my comment could be construed in any way as an attack on people for suggesting others use different languages such as Rust, it was not intended. I'm a fan of Rust myself and I would recommend it to anyone who wants to start a new project.
My comment was meant to pertain to the suggestion floating around that Daniel ought to undertake a total rewrite of curl in Rust. That's an entirely different ball of wax. It's like telling the car maker to tear down and build a new factory because the door handles keep falling off.
>But if a car manufacturer consistently uses a production methodology which causes door handles to keep falling off, again and again, even after they've been re-glued on for the Nth time...
We know the characteristics of glue, and have switched to better glues. All our assembly jigs work with glue. When the handle falls off it is easy and cheap to glue it back on.
If we switch to screws we need to redesign the parts and the jigs; and reorder the entry assembly line. [Not realized by the engineers at the time is if they use screws dissimilar metals of the screws corrode and result in the handle falling off as well - and it is a much more expensive fix because you have to replace the handle and the part it attaches to as both have nothing to screw into)
If I say: "There are all these bugs in curl, you should rewrite it in rust and it will be better (but I've never done anything similar, nor there is any scienetific proof for what I am saying)" then it's not thoughtful criticism, but pure personal opinion. There is a very low value in that.
On the other hand if there is a project like curl rewritten from C to Rust and its author says: "We have switched from C to Rust in a year and after two years we have less new bugs", there is at least some value.
I think the parent is correct, and the situation is more nuanced than your argument would suggest.
Politics: Are you critiquing the distribution of funds to the school your child attends? You probably have some insight there. Are you instead expressing a strong opinion on a complex subject that you likely don't have an iota of experience with? Well, then you should realize that you're probably not in a position to have such a strong opinion.
Meal: You're the user, so you have experience here. All you're commenting on is how it tastes to you, so that's perfectly valid, no need to become a chef.
Car: Again, you're the user. Are you critiquing a UX problem? Well, then you're in a perfect position to do so. If you're remarking about e.g. the torque to weight ratio then you should probably understand some of the engineering that goes into creating a vehicle.
Your argument is the fallacious one here. You're making an all or nothing comparison and life rarely works that way. If you're going to make a statement like "you should rewrite curl in X language" then yes, you should know what the hell you're talking about because you're making an engineering level statement.
I wasn't making the argument that curl should be rewritten in Rust. It's too late to edit my comment to clarify it, so please see my response here: https://news.ycombinator.com/item?id=13996154
It's not that anyone who has an opinion or criticism has to be an expert or be in the trenches. But in the same token not every tom, dick and harry has valuable opinion or criticism that puts them on par with people doing the work or legitimate experts. There's certainly a fine line to walk between ignoring people or wasting your time trying to filter out a cacophony of opinion and belief from anyone willing to utter it.
No, it is not a fallacy - when it comes to matters of opinion on the feasibility of difficult tasks, experience has some relevance.
Your objection follows a common pattern - if we took every nuanced situation and turned it into a universal black-and-white dichotomy, we would end up with half the arguments on the internet.
If a person was claiming it should be much bigger and more successful than it is, and that claim is primarily opinion-based, one might take into consideration that person's achievements when evaluating their claim.
Replying to myself, since I can no longer edit what I posted: I'm not advocating the developers of curl rewriting it in Rust! I think that a lot of the debate here is around a few different points, and that we're generally talking past one another. My primary concern was that the initial response seemed to disregard the vulnerabilities that can occur due to unsafe memory management. To the extent that I read people say arguments along the line of "you can only make that criticism if you write a curl replacement yourself", I felt that that was unfair.
D is compatible enough with C/C++ that I have been able to take largish multifile projects and convert them one file at a time to D. The full regression suite was run after each file was converted. This helped ensure both an accurate conversion and did not leave the project in a broken state for an extended period.
I've successfully converted many projects from one language to another using this method. One was a huge assembler program I converted to C back in the 1980s (ironically, so it could be ported to other platforms!).
> Just try and re-write some parts of curl in Rust
I don't doubt that some suggestions have come from backseat drivers... but I also don't believe that's relevant. First, people are implementing curl's functionality in Rust (e.g. hyper, reqwest) and it's working out well. Second, you don't have to be the one implementing something to push for improvements- in the general case, that's absurd.
> ... people are implementing curl's functionality in Rust (e.g. hyper, reqwest) and it's working out well
That is good, and it's the reasonable thing to do. New tools provide the same functionality, get used where curl would have been used otherwise, and when the time is right, curl can enjoy a well-earned retirement in the hall of fame of famous ancient projects.
There is a certain type of person (I once fit the mold) who thinks new languages could significantly improve software development. The evidence is to the contrary, and many improvements turn out to be double-edged swords, capable of doing damage in the wrong hands. I am still interested in new languages, but I am no longer looking for the holy grail.
I've actually grown extremely bored of programming language debates and yet continue to be fascinated with the act of programming. Each language has something new to offer and also some limitation(s). Why not try them all?
I believe opening our minds will be the only way to achieve meaningful improvement in software development.
You only get these kind of advices, not when you need them, but when your project is successful ... Everyone want to help a successful project. But while struggling, you get very little help.
Bang on point. All the people who complain "oh but C is so dangerous you should rewrite in $HotNewLanguage" do nothing to actually help with their "expert" recommendation.
I used to program in $BoringOldTurboPascal, before C became the $HotNewLanguage in systems programming, so excuse me if I feel like I know one thing or two about C security exploits.
C was a UNIX only language, it had no relevance outside UNIX in the 80's.
Those of us using CP/M and other systems had access to K&R C dialects like Small C, not fully compatible with C.
In those days C was just another systems programming language, looking for a spotlight.
That spotlight came thanks to universities adoption of UNIX, which given AT&T prohibition to profit from it, it was sold at symbolic prices and a few AT&T researchers did provide tapes to universities. Which then started to adopt UNIX instead of paying commercial licenses for other OSes.
So when workstation startups like Sun and SGI started to look for OSes, they based their software stack on UNIX.
Before the widespread of UNIX out of university labs into the workstation market, C wasn't that relevant.
You see the same happening nowadays with Browser == UNIX and JavaScript == C.
Also C is actually 10 years younger than Algol or PL/I, both used to implement several mainframe OSes.
C was the system programming language of Unix. Many people wanted Unix, so they started programming in C, because that was the best supported language. In the late 70s, byte code interpreted Pascal was also popular on Unix because it allowed for bigger programs on the then current 16-bit systems.
It's pity that Modula-3 never got much traction. I guess that's part do to Modula's overly verbose syntax.
C works as long as the platform is sufficiently like Unix.
For example, in the MS-DOS world with near and far pointers, programming in C becomes utterly confusing.
These days, linker scripts and ELF hide a lot of stuff C cannot express.
For any sizable project in the public, I think switching languages is never the right choice. That is at best confusing for people, and at worst kills the project. The right way to replace a library or application is to build something better. If it matters, possibly with the same, or a close enough API.
This is the only thing I've seen work in the past.
I would also like to see these arm chair developers bootstrap a new set of compilers, operating system and tool chain with a native system interface written entirely in there super cool new languages that are going to replace C and C++.
For all the talk of 'a better system programming language' and replacing C/C++ there is curiously little interest or code written to prove that any of these new languages can be used to write systems from scratch and that the language and runtime can scale across architectures.
Thats before you even get to a comparison of efficacy and speed vs C/C++.
The problem is catching up with 40 years of history.
The quality of code generated by C compilers used to be pretty lame in the 80's, any junior Assembly programmer could do better.
As for ubiquity, C was a UNIX only language, with K&R C incomplete dialects available in other systems, where C was just "yet another language".
It was the adoption of UNIX, an almost free ($$) with source code available, by the 80's startups for creating the workstation market (like Sun and SGI) that lead to UNIX and C's adoption in the market.
Had UNIX been as expensive to get as the other OS of the time, and we wouldn't be talking about C's ubiquity and execution speed of generated code.
How much history can you try to rewrite in one comment?
A vast amount of software in C was written for other platforms than UNIX, both SGI and SUN were companies that were focused on selling hardware rather than software. If anything the workstation market was created by companies like Xerox and commercialized by companies like Apollo.
UNIX was for a long time a very expensive operating system on X86, it only became affordable with SCO/Xenix and it became free with 386BSD by Bill and Lynne Jolitz and later Linux by Linus Torvalds.
C definitely wasn't a UNIX only language, it was available for an enormous range of computers from 8 bitters to top of the line machines. In fact that (and not UNIX) was a major factor in C's long time dominance.
Companies buying UNIX weren't buying X86 systems to run it.
We were using CP/M (on Spectrum +3 A), MS-DOS, Atari, Amiga, C64, ZX Spectrum, MSX.
C was just yet another programming language, nothing special about it.
In fact the only reason to bother having a C compiler on those systems for my group of acquaintances, was to be able to bring home the work done in expensive UNIX computers at school.
C supporters are the ones that like to rewrite history by selling UNIX and C as if they were the genesis of systems programming and OSes.
Well when I started programming Windows 3.1 it was with Microsoft C 7 and the help of the Petzold book. At that time it seemed to be the main way of writing Windows programs, at least until Visual Basic 3 came out anyway. It wasn't just another programming language in the Windows world. It was the programming language.
I think you're not looking hard enough. There's lots of operating systems being written in the weirdest languages. So are the toolchains and compilers. People do it for fun - just look at osdev forums. It's not a new thing either, I was doing some development on an AOT CIL compiled, single memory segment system a decade ago. Kind of like Singularity.
But it takes time. Unless you're actually looking for those projects, don't expect to even know they exist during the first 5 years.
There are many reasons why curl uses C, and should continue to do so. Yes, C allows a combination of portability, performance, and accessibility of your source code by others that is not matched by any language. And yes, C was a perfectly valid choice throughout the vast majority of its life and still is. Switching is not a realistic option at all.
But I think Daniel's original attempt at explaining away C's security issues has been debunked. And I think he owns it well by singing a slightly different tune in this blogpost.
The lesson learnt here is that if you use C, don't try to explain away the security issues you have just opened for yourself. It will not hold up. Accept that the safety of C is a very serious issue, and that there are other reasons you choose it. Choosing C over more secure options does not mean you don't care about security, but it means that you had to make pragmatic choices. Choosing C over more secure options and subsequently attempt to minimize the security impact of this decision (not talking about Daniel here) will lead you to not addressing those issues effectively.
> Choosing C over more secure options and subsequently attempt to minimize the security impact of this decision (not talking about Daniel here) will lead you to not addressing those issues effectively.
When you start a new C/C++ project in 2017 you can effectively and confidently address security and 'safety'. And it is done on a regular basis.
There is an entire infrastructure of tools and introspection refined over the past 30+ years for C/C++ that is part of the project from the beginning. Address sanitizes, Better compilers, Valgrind, Intel's tools etc.
All of the potential problems are known and there are tools to deal with them. But the responsibility is put in the hands of the programmer like many other things in C/C++.
I do get a vibe of tone deafness coming from such a reply.
1) You're stating that automated tools are a solution in a thread in a blogpost for a heavily scrutinized tool that has a zero tolerance policy for Coverity problems, uses Valgrind and address sanitizers and still owes more than half of its CVE's to various language and memory safety issues.
2) "The responsibility is in the hands of the programmer" - I think we have argued enough that putting this responsibility in the hands of very imperfect programmers is precisely the problem and we should do the best we can to stop it.
You could make the case that the reason why we have C/C++ and its still in use is because of the tone deafness of academic language designers.
Language designers have been arguing (mostly with themselves) since the 1970's that pointers are dangerous and memory management is bad and that programmers should not have those things.
We can go on using C/C++ and language designers can go on creating there new version of Pascal. We all know how that turned out.
> I think we have argued enough about that putting this responsibility in the hands of very imperfect programmers is precisely the problem and we should do the best we can to stop it.
System designers and those writing performance critical code program hardware. Not an abstract machine that the language designer thinks is good enough for lowly programmers.
Convince Intel and AMD, Nvidia etc, that they should make chips and instruction sets that have no access to memory, cores, vectors and so forth because 'we should do the best we can to stop it'. Lets see how many of there clients want to buy useless blobs of silicone.
A computer is a tool for computation not an adult diaper designed to prevent leaks.
We've gone from arguing "the security issues for C are overrated" to now arguing either "security issues don't have solutions I like" or "real programmers should be able to deal with these things" (I'm not sure which).
In addition, you're arguing a strawman. Nobody says that direct memory access should be taken away permanently. Many 'safe' languages allow you to directly access memory when you need it.
> A computer is a tool for computation not an adult diaper designed to prevent leaks.
This attitude is why we can't have nice things. It would be great if you cared more about the security and safety of the things you make than whether you look cool while making them.
> We've gone from arguing "the security issues for C are overrated" to now arguing either "security issues don't have solutions I like" or "real programmers should be able to deal with these things" (I'm not sure which).
I don't think anyone was arguing any of those points. What was being explained is that C/C++ is designed to be flexible and simple and it was know from its inception that that flexibility lets the programmer 'shoot them selves in the foot'. Building out tooling to address any error patters in usage and keeping that basic simplicity and flexibility is desirable. More desirable then moving to a more complicated less flexible language that may eliminate some possible errors.
There are literally millions of C/C++ programmers and projects writing secure reliable code. Picking out a few open source projects and turning there bug reports into an internet drama is not going to convince anyone the sky is falling.
> In addition, you're arguing a strawman. Nobody says that direct memory access should be taken away permanently.
I think its fairly clear that the sentiment was.
"putting this responsibility in the hands of very imperfect programmers is precisely the problem and we should do the best we can to stop it."
This has been the justification for every misguided attempt to 'make things better' for a very long time.
A short list of unnecessary things that needed to be stopped thought the years.
* Pointers
* Manual memory management
* Unconditional Jumps
* direct access to machine words ie bytes
* assembly instructions.
etc..
I am fairly skeptical of arguments that suggest 'the programmer does not need that'.
> System designers and those writing performance critical code program hardware. Not an abstract machine that the language designer thinks is good enough for lowly programmers.
Oh, so you say they don't use C, right? It's defined in terms of abstract machine which has very little in common with how actual hardware works nowadays...
You can also drive your car without ABS, ESC, and other safety mechanisms. The responsibility is put in the hands of the driver.
Yes, there are reasons to sometimes disable those mechanisms. Yes, it is not a good idea to force everybody right now to abandon cars without it. Still, if you build a school bus today, I would criticize you, if you don't include the safety technology.
Developers, on average, don't even use the free tools.
I mean, speaking as one of the 1% who fully voluntarily uses these sorts of tools (and can generally only get free ones either because the price is prohibitive for the expensive ones, or I'm working in languages that expensive ones don't exist for), I think you're all bloody loonies for not doing it too. Offload to the tools everything you can, to save your precious brain cycles for the stuff that matters. Makes you look like even more of a hero.
(The excuse I see all too often that by doing that you'll atrophy your own judgment is, by the way, itself just proof you've never used these tools, or at least not used them properly. If you use them frequently, i.e., at commit-time, rather than quarterly-release time, they train you by telling you what you did wrong, and you learn how to write better code that meets their criteria fairly quickly. They make you a better programmer. These things should be used early and often; I'm sure a lot of the reasons people tend to not enjoy using them is that they want to write a quarter million lines of code without ever consulting the tool, then run it once at the end. That is a terrible plan. The tool will scream bloody murder and especially in C you are very likely to discover that your entire architecture is fundamentally impossible to fix now, making it very discouraging. Use the tools early and often. Seriously, as soon as you've created the new project and have hello world going, hook up the tools. They both provide more value and are way easier to use that way.)
That's a fair point. And for a large portion of software the benefits of C/C++ do not outweigh the potential costs and code should be written in a more protective language.
But that does not change the fact that C/C++ is preferable and often the best choice in certain domains.
> All of the potential problems are known and there are tools to deal with them.
This is untrue. There are tools out there to deal with a lot of the issues, but not all of them. Due to what C allows and the halting problem, you cannot create such tools. Static analysis and dynamic sanitizers are still useful of course, but let's not pretend they're a solution.
Arguably someone adhering to the C++ Core Guidelines, using GSL and using the new static analysis tools (currently only in VS?) can be reasonably confident about choosing C++ for a new project in 2017.
I wonder how much experience people yelling "OMG you're using C(++) - you have to rewrite everything!" actually have. Some of them can motivate their claims with good examples (at least to some extent) while others just show a serious lack of understanding of project scope and/or the reason something is written the way it is.
Ahah nice example! "I can't code, but if I were, I would like to be productive in something that doesn't require me to make any effort in understanding the problem, please advise"
I don't know where all the C hate is all about. I love C, it's a great tool. Can be quite sharp, be careful with your remaining fingers.
I spent the better part of last week studying rust and go, and my conclusion was 'meh, I'll have another look in another few years" and kept on trucking.
I don't get all the C "hate" either. Sure, C is a source of many "interesting" things - but it's also a natural part of control systems in avionics, space craft and nuclear power plants. I don't want someone who just read that "Rust is a great, safe language" to rewrite those. I really don't.
I do. We have compilers that can verify seemingly perfectly (as far as we can tell) that a program won't do something silly like reference memory it doesn't own or try to use memory after freeing it (and not just languages like Rust, but any language that doesn't make you allocate and free memory). The best C programmer out there will still make mistakes because it's simply impossible to ensure that large programs won't do something they shouldn't. It boggles my mind that programmers seem to love better tooling and better abstractions, but when it comes to a certain class of languages, everything stopped improving in the 1970s with the PDP-11 and we just can't get past it.
And I do. Planes crashed and people died because of buggy software. We have lost Mars climate orbiter due to not using type-safe language. In the cases of software that works fine, we are just relying on ridiculous amount of manual verification, and this is both risky and expensive.
> We have lost Mars climate orbiter due to not using type-safe language.
One company creates a closed-source component that produces a float, another company creates a closed-source component that receives a float. Units differ. Crash. No type-safe language will prevent that bug from happening. Let 100 programmers write these programs in your type-safe language of choice and all will be vulnerable to this.
"Oh no", I hear you say, "we'll give every unit its own type" and it still won't stop your programmers from deserializing a 4 byte value in English units into the metric value they think it is. "So we'll make all the IO typed as well!" You don't need a type safe language to do that. And why would you do it in the first place? If your API spec says you receive a float in metric units, you write your program accordingly. When you're writing mission critical software, would you really want to replace a
> read(sensor, &value, sizeof(float))
with 100k lines of even buggier code to prevent a bug from happening that shouldn't happen in the first place?
All bugs "shouldn't happen in the first place", the use of types is to reduce the surface area for where errors can be introduced. If you read some value from an external source and subsequently use measure types then you've isolated the scope for the introduction of errors to the read function instead of every location in the program that uses the value.
Mars Climate Orbiter was lost because they forgot to convert units between different systems. That's an error that can (and probably would) be done in all type-safe languages I know of.
It could be prevented if you could express units in the type system. Declare numeric_value_in_meters and numeric_value_in_yards as types and problem solved. Sure, it still possible that someone would not do that, but having this option is statistically better than not having.
No amount of 'type safety' will prevent idiots from being idiots, and with all the other languages, the problem is very often sitting on the chair. People will screw up creatively in any language, and trying to 'hide' that behind a language runtime overhead is idiotic. The good thing with C is that it will catch the worst idiots quickly, so you know not to hire them to write avionics for you.
What is NOT idiotic iswhat you discribe as 'ridiculous amount of verification' as it's often the only thing that will really work to catch mistakes.
> No amount of 'type safety' will prevent idiots from being idiots
But it will limit the damage they do considerably. Having bit of space or trees between lanes on a road saves a lot of lives, even if people drive as idiotically as they would without it.
> What is NOT idiotic is what you discribe as 'ridiculous amount of verification'
Designing systems that rely only on that is idiotic when there are better options.
> Ahah nice example! "I can't code, but if I were, I would like to be productive in something that doesn't require me to make any effort in understanding the problem, please advise"
Except that's not what the commenter said, and what the commenter did say is, although glib, not inconsistent with the motivation behind Bernstein's entreaty to just create a boringcc already.[1]
Also, generally, just don't make up fake quotes to attribute to people. It's poor form.
This is not making fake quotes, it's called paraphrasing. And "just don't" is impolite, I can just do pretty much as I like, as it is my own interpretation, Mr PC.
"Paraphrasing" is not a convenient word for writing something that tries to masquerade as a direct quote—things that are paraphrased don't try to hide the fact they're being paraphrased.
And you've failed to address the parts of my comment that are most relevant to the discussion. Why?
Well, seems we can't seem to agree on a simple thing, I don't want to waste my time attempting to discuss anything more complicated than a word. You went straight for the cool-aid bottle on the previous comment, and you wonder why I'm ignoring the rest?
Beside, my reply wasn't related to that particular post in detail it was paraphrasing (<-- hint) a behaviour that is pretty common across many projects, wether they are open or closed source.
Sorry to come out as a bit flippant, but I'm slightly overdosed on the 'fake .' argumentation these days.
I am one of those people who doesn't have any relevant experience, but I think it would be amazing if curl and openssl were written with formal verifications, and with complete memory safety. And maybe in a purely functional style. It would be nice if lots of critical software was written to a much higher standard.
>I think that’s missing the point I was trying to make.
I think Daniel Stenberg missed the point of some of the comments. (At least the prominent ones I saw.)
>We use C for a whole range of reasons as I tried to lay out there in spite of the security challenges the language brings.
The availability and practicality of old ecosystems are valid justifications but it's orthogonal to the "memory bugs caused by the language vs the human".
There's nuance and you have to distinguish multiple conversations:
1) curl should have been written in a memory safe language (past tense)
2) curl should be rewritten in a memory safe language (future tense -- e.g. maybe use Rust)
3) setting aside curl's 19 years of wide deployment challenges, D.S. could have been a better spokesman for the benefits memory safe languages
Daniel's response is mostly to #1. However, some of the criticism is really conversation #3. Understandably, #3 is somewhat harder for D.S. to adopt because it requires distancing himself from the 19 years of curl as a successful utility and instead, consider how a language's design affect programmers' bug count. Many felt that D.S. citing curl's bugs as "human logic errors" instead of "language-induced errors" undermines his point when conversation #3 is the framework.
So maybe some of the new nitpicking of counting CVE bugs is more about conversations #2 & #3. The #2 may not be realistic for another 10 years... or maybe never because of various deployment hurdles. However, we can still discuss #3.
I think this is the disconnect. One can understand why curl is written in C, not expect Daniel to write it in another language, appreciate that the overall vulnerability count being impressively low considering how old the project is, and still worry about the section about vulnerabilities in the original post looking dismissive when it comes to vulnerabilities that are attributable to C and perhaps getting trotted out in the future by others contemplating new projects as an excuse not to use a safe language.
Yeah, I absolutely think that curl should stay a C project until the day it dies. I also think someone should write a curl-alternative in rust. And at some point, people may start using the Rust version instead of the c version.
I don't think the reactions and comments were meant as an attack on curl, but more as an attacks on C.
Most programmers who attack C have seen the consequences of a memory-unsafe, type-unsafe language dominating systems programming throughout the years. I remember the period during the early 2000s when "buffer overflow exploits" were an entire industry. I also remember the strange allure of C and C++ in beginner programmer circles. The logic goes something like this: "they are powerful languages, all of the big and popular software projects are written in them, so they must be the best". Their next step was to go on to start a new generation of unsafe and vulnerable programs and libraries, and so on.
Now that we have the tools and technology to do better, we should really be making some efforts to push for them. The goal isn't change right now, but starting the process early seems like a good idea.
Most programmers who attack C at the moment are frontend web javascripter kids and rust/functional language snobs, who both never used C and keep repeating the same stuff.
You just have to witness the number of "OMG, you did that in C? Wow!" that are posted every time someone posts a basic C program on a forum. There is a whole crowd that thinks C is some arcane stuff only to be used by by magicians, and if you're not a guru it's gonna kill you.
And then you have the Rust people who chime in and point a list of C problems that are in fact C++ problems.
And then you get the "but anyway modern processors do not execute the code you tell them to execute" guys. Sigh...
> Most programmers who attack C at the moment are frontend web javascripter kids and rust/functional language snobs, who both never used C and keep repeating the same stuff.
Having half-baked information on how C sucks is better than having illusions that its somehow awesome. This:
> There is a whole crowd that thinks C is some arcane stuff only to be used by by magicians, and if you're not a guru it's gonna kill you.
is actually a very reasonable starting point when deciding whether to start a substantial new project in C. Well, minus the arcane part - part of C's success can be attributed to how simple it is to learn how to shoot yourself in the foot.
It also means the number of new programmers that are fascinated with / desire to learn C continues to drop; hopefully that will be enough to break the cycle at some point in the (not so near) future.
Everyone will just have to accept that Daniel is not going to rewrite curl in an other language. If you want curl in Rust, a Rust programmer will have to step up and do it.
curl is currently one of the most distributed and most widely used software components in the universe, be it open or proprietary and there are easily way over three billion instances of it running in appliances, servers, computers and devices across the globe. Right now. In your phone. In your car. In your TV. In your computer. Etc.
If we then have had 40, 50 or even 60 security problems because of us using C, through-out our 19 years of history, it really isn’t a whole lot given the scale and time we’re talking about here.
I think there's a disconnect here. Given something that's has a thousand instances running and averages 20-30 security problems a year and something else that has over three billion instances running and averages 2-3 security problems a year, which one do you think I care more about? I care about the one running everywhere, and on devices which may never see updated code in the rest of their lifetime.
If curl were some unused project this wouldn't be a conversation. At the point where your code is on all those devices, you will get scrutiny. Expect it. Your choices may have some impact on a significant portion of the Earth's current population.
At the same time, it's not fair to expect curl to rewrite their project in some other language. Some other project can start with the goal of providing the same API as curl (or not) that attempts to mitigate security problems though a different code architecture or language, and maybe curl devs will contribute. But expecting curl devs to start over from scratch is not feasible.
The phrase "worse is better" was coined precisely to explain the success of C, and I consider that paper required reading for entering this discussion.
"Worse is better" has nothing to do with it. Stop saying it every time anything you think is bad comes up. It's incredibly arrogant, the way people basically spam "worse is better" like they're in Twitch chat every time C, Unix, HTTP, or anything else that they perceive as imperfect comes up.
But I do consider that article a good explanation of the tremendous success of C and Unix, and I think its perspective should be part of these discussions.
The best feature of Curl is that you can use it as a child process, driving it through the command line interface of its principal executable. Quite a lot of the API-level bells and whistles are covered.
A few years ago I looked at the API and internals and went, "gack!" No way am I integrating this dung heap at that level; let the users just wrap the process with a curl function that returns a process pipe stream if they want to use this.
I wouldn't advocate for the curl project to experiment with rewriting some code in [Zig](http://ziglang.org/) yet, due to lack of maturity, but it is exactly in the target niche of the project. That is, near-seamless integration with a large pre-existing C codebase.
So you could, for example, take a 400 line C file, convert it to Zig, and have it import the .h files that the previous C file was including, and export an .h file for the rest of the codebase to look at. The compiler can turn this Zig file a .o file that integrates with the rest of the build. The part of the codebase written in Zig could benefit from the safety features, generics, and error handling features of the language while presenting the same interface to the rest of the codebase.
Now of course, if you already have a codebase written in 1 language, it's more complicated to introduce another. This is where the Zig build system comes into play. I'm working on this now, and it's a competitor to autotools, scons, cmake, make, and all those players. It's the Zig programming language with a declarative build system API built into the standard library with some basic tooling around it, and it ships with the compiler.
So if the Zig build system gets to the point where it is high quality enough to attract users and ubiquity in package managers, then a project which already uses Zig for its build system can start using it for its actual code without the penalty of adding a dependency.
In conclusion, once Zig has reached a stable enough state, I would challenge the libcurl devs to consider, perhaps it could make sense to slowly, carefully, methodically transition to a newer language, if that language was friendly enough to the process.
I don't know anything about Zig, but I believe that a good reason for cURL to be written in ANSI C is that even your toaster ships with a C compiler able to build the source code.
If that is in fact the reason for sticking with ANSI C then I would like to advocate for a language that compiles to C. In which case I would like to suggest Nim[1].
Of course, I don't think that curl should be rewritten in Nim. But since we are advocating for other languages I thought I would pitch in. If anything, writing new curl functionality in Nim would be the way to go. That is assuming that curl's author is unhappy with C, which does not seem to be the case.
If you depend on Zig then you only depend on Zig to produce a binary or library for any system.
If you depend on Nim then you depend on Nim, a C compiler, and a linker. And I hope your C compiler supports cross compiling out of the box. It probably doesn't.
Also, it might be useful to know that HN's comment syntax isn't Markdown, though it shares a few features like indenting for code blocks and asterisks for italics. Markdown-style links are not supported.
Of course, not only are markdown style links unsupported, nothing but raw urls are supported as links. For whatever reason, the majority of people here choose to relate links to locations in text using a particular footnote style. Since this format is no more blessed than the common markdown style, they could have easily chosen that instead. Presumably, they didn't because it breaks flow when using long links. For short links, like the one in the GP, inline links work just as well as footnote based ones, and provide the extra benefit of not requiring scrolling to follow them.
If there's no need to tell a parser where the link text begins and ends, then putting the URL in parentheses after the relevant text (without square brackets) would be less fussy.
The "HN style" of citation/linking is basically the IEEE citation style. You could call it "blessed".
I think most of the points he made in his original post are very valid ones, it's just the fatalist "devs are going to create bugs, such is life" approach to errors that I think I glimpsed in the original post that prompted my response in the previous thread. I'm tempted to call it the "C Stockholm syndrome".
The reason I say that is that I could actually have written something like that a few years ago myself. I think it's a pretty common mindset amongst the "low level" C crowd. And there's a good reason for that, we've been burned before.
People have been advertising "safe" alternatives to C since forever. Use Perl! Use Java! Use Lisp! Use Haskel! Use Python! Use Go!
But none of these languages can replace C in all of its use cases. You have big runtimes, garbage collection, interpreters, you name it. You can't just bootstrap a stack pointer in 3 lines of ASM and call into them like you can with C. And even if we don't say it out loud there's a bit of snobbism to writing low level "unsafe" code. "Safe languages are for noobs and webdevs, real coders debug with oscilloscopes".
And so I had this mindset that you couldn't have your cake and eat it too: you could be safe or you could be low level. Segfaults and NULL pointers were just a normal part of dealing with low level "unmanaged" environments and we'd just have to deal with it. People who complain about it are just not worthy.
In my opinion the biggest success of Rust so far is in proving that you could be safe without needing a GC, a big runtime or complex abstractions. You could design a better C without having to compromise. Even if Rust fails in the long run, it would still be a very important "proof of concept". It showed that C was unnecessarily unsafe, that you could achieve the same feature set without throwing safety completely out of the window.
That being said if there really are folks bugging devs by asking them to consider switching to language X or Y then I think it's fair to say that these people are idiots. Might as well ask JK Rowling to rewrite Harry Potter in Esperanto because "it's totally the future!".
I mean, let's face it, at this point Curl probably has a much bigger userbase than Rust itself by at least an order of magnitude. Which one is more likely to be irrelevant in 10 years? Rust or Curl (and by extension, C)? If you answered the latter you should read less hacker news.
And even if Rust ends up being successful it's not like C will go away any time soon, or even in our lifetimes. Curl (as a C library) will probably still be useful and relevant for decades to come no matter what.
The problem is that the author listed specific reasons, which opened him up to specific challenges.
He should have just said: "curl is an established project. New languages are not being seriously considered for curl."
If you want to close down a conversation, it's counterproductive to make new dubious claims like "C is not the primary reason for our past vulnerabilities".
Note that closing down a conversation isn't bad. Sometimes they are annoying or distracting. I love rust but here was my comment when someone wanted to rewrite postgres in rust:
As a meta-comment - it's interesting to see how religiously-intense this discussion has became. Why are we - programmers - need to hold such a strong opinions about languages? Especially considering a fact that the playground is completely open.
My personal opinion is that if he was wrong - we wouldn't even be reading his argument - it wouldn't make it to the top of HN - everybody would be using Xurl library instead (written in X-lang, where X is not C).
At the same time Rust is shaping to be fantastic language and I personally hope that soon we'll see Rurl cargo together with kernel and what not in Rust. Unless in this decade something better will appear, who knows?
Are there any automated tools that you can help you convert a C codebase into Rust? And is there any reason why Rust couldn't be as portable as the C code that makes up curl?
Is it very difficult to transpile C into Rust?
As an aside, I not like the curl style guidelines [1]. Especially "'else' on the following line", and "No space before parentheses". That doesn't look nice at all.
Last I heard, Corrode can convert many C syntactic structures into their Rust equivalents, they usually compile and sometimes they even work. Even when the conversion works flawlessly, the resulting Rust does exactly what the original C code did—there's no safety benefit (but you can begin refactoring to clean things up).
And is there any reason why Rust couldn't be as portable as the C code that makes up curl?
In theory no, in practice yes. Rust is using LLVM project to generate machine code, and that only supports certain architectures. There are however other safe languages that can generate C, if this is a requirement.
> Is it very difficult to transpile C into Rust?
If safety guarantees could be added to C by some conversion or analysis, there would be no need for Rust to exist.
Rust (and other languages with similar goals) is safer for many reasons, not limited to languages itself. Stdlib and runtime play their part too. Automatic conversion that takes all that into account is either impossible or impractical. There is a tool for converting C into unsafe Rust (corrode), which might be useful for people attempting to port C code, but in itself doesn't improve code safety at all.
https://github.com/jameysharp/corrode is a C-to-Rust translator. It won’t be idiomatic Rust, though; not by a long shot. (Rust is much more restrictive in what it allows, for reasons of safety, though more expressive.) And it may not be correct, either; corrode is getting pretty good, but it’s not perfect yet.
Note: I am discussing using safe languages in general and reasoning for doing or not doing that. If you believe rewrite requests are unreasonable no matter what,
don't bother reading that. Or perhaps consider this to be addressed to people deciding how to choose libraries to depend on, not to specific library authors.
> curl is currently one of the most distributed and most widely used software components in the universe, be it open or proprietary and there are easily way over three billion instances of it running in appliances, servers, computers and devices across the globe. Right now. In your phone. In your car. In your TV. In your computer.
Congratulations. So how many of those have unpatched vulnerabilities right now that were caused by using unsafe language? In my phone? In my computer? In my car? How many accidents of data or money theft it caused? Not to mention common non-security related bugs.
> If we then have had 40, 50 or even 60 security problems because of us using C, through-out our 19 years of history, it really isn’t a whole lot given the scale and time we’re talking about here.
Curl had multiple unpatched security problems for several years before they were discovered. As the author admits, it most likely still has many.
dpkg -l | grep '^ii lib'|wc -l lists over 2000 libraries I have installed, lot of them written in C. An estimate of 2000 multiplied by 60 is ... scarily to many. Large amount of them avoidable by using safer languages.
> Using another language would’ve caused at least some problems due to that language
If it solves more problems than it creates, it is worth doing.
> none of the memory safe languages anyone would suggest we should switch to have been around for 19 years.
Neither have been mobile phones (at least as we know them), but they are useful, so I am using one.
> We will continue to work hard on minimizing risks, detecting problems early by ourselves and work closely together with everyone who reports suspected problems to us.
If you believe that you can rewrite any of those libraries or tools in a nice modern language with no security issues and all the portability that they had (including operating systems and embedded systems), please do grace us with your work.
In the meantime, it appears rather difficult to replace the last 40 years of work overnight in a novel, untested language, with support for a tiny fraction of the targets that C has been supporting all that time, while maintaining reasonable performance.
Or in short: It's easy to work smart when you're not working hard in your arm chair.
> If you believe that you can rewrite any of those libraries or tools in a nice modern language with no security issues and all the portability that they had (including operating systems and embedded systems), please do grace us with your work.
I can point you to numerous examples of people writing and rewriting software, there is no reason for it to be my work to prove its doable and often worth the effort. And all languages that generate C code are as portable as C is. I can also point you to countless examples where people replaced their dependencies with something written in another language. Also please go back to first line of my comment and read it again.
> In the meantime, it appears rather difficult to replace the last 40 years of work overnight in a novel, untested language, with support for a tiny fraction of the targets that C has been supporting all that time, while maintaining reasonable performance.
Who suggest it can be done overnight? Why not use something that compiles to C if you need this compatibility? There is no need to replace all of it at once - you can pick particularly sensitive parts first, one file or one library at a time. There are many ways to introduce safer languages if you want to do it. Also difficult or not, all code will be replaced eventually.
As for performance - its irrelevant in large amount of networking code. I'd have no issue whatsoever with adding few cpu instructions for bounds checking, when you wait several milliseconds for IO operation, which is the case with curl.
> there is no reason for it to be my work to prove its doable and often worth the effort.
Why? You don't have expertise needed to do that yourself? It seems you think you have enough to lecture others and write how easy it is (just generate some C code).
> I can point you to numerous examples of people writing and rewriting software
Ok, point me to this software that was rewritten from unsafe C code and was replaced with memory safe language. I don't want any hobby projects, real software used by millions. Give me 10 examples.
I too am curious to know of large and prominent C projects rewritten in another language successfully. One recent atypical example that I can think of is the Go compiler.
Good point. A project with the full financial support of Google with a team including Rob Pike, Ken Thompson, Russ Cox and others, in a language that they designed. And it still took a few years and was a huge and stressful process.
> You are taking code criticism to personally. Keep it professional.
What? I am asking if what you are writing is based on your experience or it's just theory that you've read somewhere. How is this personal?
I am not asking for myself. I already know the answer for that question after reading your comments (the last one only confirmed that), but people that are reading your comment may not, so why don't you just answer the question ?
I am talking about projects that NEEDED C for it's time/space efficiency and cross platform support. Not any C project. Do you see OS kernels in production written in python used by millions ? Do you see video/audio codecs written in Java/python/.net used in production by millions? No. This is where C is used because it's needed.
Coreutils in rust, remacs, ruby extensions - this is not used by millions.
> Objective-C to Swift migration
What? This are different languages. It's not SAME software rewritten from C.
> Various parts of firefox
Whole software, not parts.
So the only valid example you gave is Go (Go compiler is not used by millions by the way). So you gave 1 example instead of 10 and Go compiler is now twice as slow after rewrite.
> dpkg -l | grep '^ii lib'|wc -l lists over 2000 libraries I have installed, lot of them written in C
That '2000' number is not the relevant one here. The reason that security is important wrt Curl is that Curl is used to access the internet. Security in the library used to print colours to your terminal is a bit less important.
It's those widely used, public facing libraries that need a good security story (and using a memory-safe language is a good start there).
It's really easy to criticize. Why don't you rewrite curl yourself? Instead of talking do something. In language that is memory safe + supports all the platforms that curl support (good luck with that) and same time is as efficient.
Yes, that's the point of this discussion. We are not criticising some code for having one bug a decade that requires 24 steps to reproduce, we are criticising code for having numerous bugs that are easily preventable with existing technology. Its way easier than it should be.
I have no personal need for rewriting curl. Instead I am using libraries written in safer language. You are misaddressing my criticism. I am not criticising curl author for using C - just some of his arguments for doing so the way he does. They are not valid in my opinion. There might be reasons for distributing code as C source, there is no reason for networking library to be written in C directly.
> In language that is memory safe + supports all the platforms that curl support
There are languages safer than C that compile to C, supporting all platform C does. No luck is needed.
> There are languages safer than C that compile to C, supporting all platform C does. No luck is needed.
I think you are living in theory world, but you don't have any practice. Show me 10 examples of C code (>10KLOC) that was generated by those "safe languages" that compile to C which is safe, as time and space efficient as equivalent hand written, highly optimized C code.
> curl is currently one of the most distributed and most widely used software components in the universe, be it open or proprietary and there are easily way over three billion instances of it running in appliances, servers, computers and devices across the globe. Right now. In your phone. In your car. In your TV. In your computer. Etc.
And every one of those three billion devices is vulnerable, due to the use of C.
> I feel a need to underscore the fact that none of the memory safe languages anyone would suggest we should switch to have been around for 19 years
False: Common Lisp has been around since 1994 (23 years), and in substantially the same shape for longer still. Standard ML has been around since 1990 (27 years). OCaml has been around since 1996 (21 years). Smalltalk has been around since 1984 (33 years). Each of those languages is more memory-safe than C and has facilities which help prevent other C-like errors. Each is capable of speeds approaching that of C, esp. for a problem like URl fetching (e.g. I just tried fetching http://www.google.com/ with both curl & DRAKMA — a Common Lisp package for URL fetching: curl reliably ran in about .065 seconds & DRAKMA reliably ran in about .14 seconds; I have no reason to believe that DRAKMA is particularly well-optimised; no doubt it could get even faster if desired).
I think this really is a textbook example of the Blub Paradox: someone using C thinks that for the most part it's a reasonable choice in order to achieve certain goals, while someone used to a better language is able to see that C is simply unfit for purpose: programs written in it will inevitably have security flaws which will inevitably cause harm — particularly when three billion devices, many unpatched, are running them.
We should stop using Pythagoras' a^2 = b^2 + c^2. The last commit to that was like 2500 years ago. (Core engine only, not counting some syntactic sugar).
Not to rewrite is a very sensible decision due to the fact that the current way of programming is un-rewritable. For example, people don't write FORTRAN in every language. The choice of programming language often fundamentally change the way we approach programming. Switching language not only involves addition or reduction of language constructs, it fundamentally alters programmer's highest level thinking process. That is not a merit.
What we need is a meta-layer programming interface that isolates our higher thinking from lower language concerns to some degree. If you can keep the higher level understanding, then rewriting become less of a barrier. We need write pseudocode in every language.
I have been exclusively programming with a meta layer, MyDef, for quite a while, and I routinely re-write my Perl program to C, not without effort, but always a worthwhile effort trading for the goal of rewriting (speed in this case). I am a researcher, and I rarely write production code. But for production code, I think most software should consider rewriting in Rust.
Prototype in Perl/Python, develop in C, and produce in Rust -- makes good sense. It makes good sense in our current non-software industry, where researchers, developers, and producers are people with different focus and use different material and tools. This is not true for our software industry, where people who are not interested in researching or developing are forced to research and develop, and people who are only interested in research are forced to develop and people who only interested in developing are forced to research and produce.
This is an usual problem with people participating in discussion over the Internet. I am sure those people mean well, but often they don't put enough effort into assessing the situation and underestimate its complexity before they decide to hit the keyboard. I've myself done it a number of times, but I am working on not doing it.
Advice for the people who are making all these suggestions and: A good way to see if you're talking out of your ass—unintentionally, of course—is to try to do what you're saying someone should do. Just try and re-write some parts of curl in Rust (you can go with a hip name like rurl). It's a good exercise to learn about the project you're rewriting, the language you're writing it in, and the problem domain. So, at the very least you'll get something out of it, and if your project is successful: even better!