[SKILNER]

Great post - the one word missing, "maintenance." Most programming work is done maintaining/enhancing existing code. The greenfield work is a piece of cake by comparison.

You want to do the hard stuff? Maintain existing code you're not familiar with.

The problems around maintaining unfamiliar code are huge, largely unsolved, expensive and risky. There's a little branch of computer science called Program Comprehension and no one pays any attention. Though most programming money is spent on maintenance.

[danielvaughn]

I always tell my clients - the difference between writing code and maintaining it, is the difference between raising your hands and keeping them raised indefinitely.

[deepakkarki]

Or the difference between conceiving a child and raising one :)

[metaltyphoon]

I always smile when a green field project starts and then they claim its “Clean Code”. No, you won’t known if it was clean code until years down and the system will need updates. Then and only then you can reflect and see how hard it was to changes things in it.

[tasuki]

Fully agreed. No matter how "clean code", the next person or team is immediately going to label it "legacy" and complain endlessly about all the choices made by the original author(s).

[Jensson]

Many dependencies? Updating them is hell, remove dependencies and just write the utilities we need so we can update to latest easily!

Few dependencies? Too much reinventing the wheel, delete all that code and add dependencies!

There is no perfect code, can always make different trade-offs and move things around.

[tacitusarc]

My metric for clean code is how quickly a developer unfamiliar with it can understand it

[agumonkey]

It seems that anything successful, for any given era, is the one thing that minimize time to understanding better than others.

[simonblack]

Much as we denigrate COBOL, that is still its greatest advantage. Yes, it's wordy, yes it's old. Yes, it needs to be really updated. But it's still easier for a new hire to understand the COBOL old code than any other old code.

[Ygg2]

I've never had any problem digging into any Python codebase.

Even magic stuff like `@attributes` is easily searchable.

[tomphoolery]

I mean, a green field project is about as clean code as you can get. It's (reported) bug-free!

[agumonkey]

raising someone else's that is already 4yo and emotionally harmed

[pojzon]

This hits home. You win.

Maintaining and cleaning up a mess someone left behind are two different things.

[sdashner]

Whenever I do a big refactor it’s so that it will be messed up my way instead of being messed up their way.

[danielvaughn]

Ha, love this one.

[baq]

great analogy. totally stealing this.

[iovrthoughtthis]

powerful image!

[agumonkey]

I sincerely don't know why uni don't make more classes on that only.

  - pick any software
  - try to change something
  - give a precise impact analysis
  - generate a few potential implementation paths
  - measure how fast you did all that and what failed / worked

[AndyMcConachie]

I agree.

A fundamental problem with how we teach programming is that we focus on writing, instead of reading, software. To borrow terminology from the language arts; we don't focus enough on reading comprehension.

[salawat]

This. So much this. I have devs I oversee vomiting forth code with nary a thought of how it integrates with anything else.

[johncessna]

At the University I'm associated with, there was a discussion about why more 'practical' classes weren't taught. The answer was 'We're not a trade school.'

[schaefer]

exactly, Just like getting assigned book reports, but for software nerds!

[vlunkr]

I've also wondered if it's time to back off on agile a bit. Or at least "agile" as it is implemented generally, which means we get to make up new requirements every two weeks. In my experience, the hardest maintenance problems occur because we're trying to re-shape code into something that the developers never knew would be coming down the line. Spending lots more time up-front deciding requirements would go a long ways towards more maintainable code.

[bmitc]

It's interesting to me that people are so into Agile when it simply doesn't work all by itself. How many books, conferences, certifications, practitioners, evangelists, etc. does it take to make a paradigm, supposedly the paradigm, of working actually work? Every company I have worked for that used Agile basically had broken processes and were not productive. The one job where we did not explicitly use Agile was actually a place where I produced the most useful work.

If the number one answer to we're using Agile but it's not working is "you're doing Agile wrong", then maybe Agile isn't the solution?

The way I worked at the place that did not have an explicit process was one of switching between agile and waterfall methods. Early on in the projects, the process was primarily waterfall, defining things up front, building prototypes, laying out the project, etc. Then once passed that stage, projects would enter into a more agile or iterative process. Then as new big features came in, back to waterfall and then switching to agile once that stabilized. This worked quite well.

[samsquire]

I think engineering effectiveness comes down to a people and communication problem.

Writing software well is difficult by itself and people are all different in understanding, experience and skill level. Throwing a group of people together and trying to build something that works for every scenario is a miracle given the complexity of software and the complexity of interpersonal relationships and organisations (see Conway's law). I'm impressed by every multiplatform language or tool or software. It's a lot of work!

If everybody did things the same way i.e the agile way and if agile was proven to work and everyone followed it the way it was intended and designed, then maybe we could all be interoperable and easily work together and produce projects that don't fail. That's the fantasy.

To be fair I've been 6 years worth of agile projects and I was never on a project that failed.

[CodeMage]

Maybe I'm just an old curmudgeon, but it seems to me there's way too much emphasis on speed of development in general. Time to market seems to be more important than quality, robustness, security, performance, or any other concern.

Another thing that rubs me wrong is the recurring notion that we need to get rid of the text as a representation of code. I've yet to meet a mathematician who wants to get completely rid of formulas because coming up with a proof is slow and cumbersome.

I understand that the incentives are drastically different between academia and business, but perhaps we've gone too far in this particular direction. Perhaps it's okay to admit that programming isn't as easy as we all want it to be and it's okay to take the time to change things carefully instead of moving fast and breaking stuff.

[_gabe_]

> Another thing that rubs me wrong is the recurring notion that we need to get rid of the text as a representation of code.

I completely agree with this notion. However, I feel like we're sorely missing out on some form of visual exploration. I feel like the majority of my time is spent trying to understand the flow of execution of a program I'm trying to maintain that was written by other teams that are long gone.

It would be so amazing to be able to "zoom out" from the text and get a graph view of execution flow through different files. And then being able to highlight a particular execution flow that you're studying would be great. I know we already have some flavors of this with "find all references", or something like Visual Studio's profiling tools that highlight flows of execution, but none of these have ever felt like they improve the exploration of the codebase very much.

It would be very interesting to see some tools that allow a more fluid exploration of an unknown codebase. More akin to zooming out of a Google map view and tracing flow from point to point, instead of diving headfirst into a million files looking for the correct information.

[samsquire]

Yes, like crocodile clips for a circuit board, I can insert a probe into a circuit and see what's on the wire.

When your callstack is 50 methods deep and there's 20 layers involved and marshalling and it's difficult to see what is going on.

I want to mark two pieces of code and see the data structures passing through them, similar to a debugger but more like a log file or trace like Jaegar or Kibana. But the actual POJO or JSON objects themselves.

[_gabe_]

This is a great analogy and captures what I meant very clearly! It would be awesome to pinpoint two pieces of code and ask for the execution flow visualizer for that path :)

[samsquire]

Thank you for sharing your idea with us :-)

[midoridensha]

I've felt like this for a very long time, and it's disappointing that there aren't really any tools to do this yet. It would make it so much faster to come up to speed on a new codebase, and also to see where some potential problems might lie.

[KronisLV]

Sourcetrail actually tried to do that for a select few languages https://github.com/CoatiSoftware/Sourcetrail

Sadly, they retired the entire project a while back.

[_gabe_]

This looks very cool and pretty similar to what I was visualizing. I'll have to download it and play around with it for a bit :)

[vlunkr]

Seems like you’re describing architecture or sequence diagrams.

[bmitc]

> Another thing that rubs me wrong is the recurring notion that we need to get rid of the text as a representation of code. I've yet to meet a mathematician who wants to get completely rid of formulas because coming up with a proof is slow and cumbersome.

I see what you're saying, if I understand correctly, in terms of the hype around "low or no code" environments, if that's what you mean. However, I do disagree with the notion of equating code with text. It seems myopic in the same fashion of equating tape or punch cards with code.

Also, I think the mathematics example isn't quite apt. Mathematicians are very willing to use visual representations to both illustrate and prove ideas. The way mathematicians work is actually a great example of moving between visual and symbolic representations and also prose.

In many significant ways, text is a highly limiting form of expression. In my opinion, we have nearly tapped out the available expression that can come from purely text-based programming languages. One example of this is the limitations of the innovations in syntax. There's been basically no significant innovation in this area aside from small iterative improvements, and I think that's a fundamental limit we've hit. I feel the future of programming is likely to hinge on a highly hybrid environment. In some sense, Smalltalk and its ilk, Emacs and Common Lisp, TouchDesigner, LabVIEW, and vvvv are precursors for what could come.

[BurningFrog]

The part of Agile a lot of people ignore is the phase when, after your code works, you spend as much time as it takes to make it well structured and readable to others.

[BurningFrog]

BTW, this is the Design Phase in XP!

You first write the code, then you design it. Genius!

[andsoitis]

If you think of code writing as a process of organizing your thoughts, exploring/understanding a domain, and articulating it iteratively (for code/run/debug loop), then your pithy comment seems perfectly rational.

[commandlinefan]

> The problems around maintaining unfamiliar code are huge, largely unsolved, expensive and risky

Not to mention massively underestimated and unappreciated by non-programmers.

[samsquire]

A software engineer, coder and developer's responsibility is to manage technical complexity and solve technical, data or business problems.

I'm not sure if we are respected for this capability. Do people want to pay for what they do not understand why it is so hard?

[dlojudice]

the problem is that a large amount of code produced is to support the accidental complexity of the software (let's call it infrastructure code) while the value generated by it comes from its essential complexity, that is, the application domain, as cited by the article .

while Domain Driven Design and Clean Architecture are a first step in the right direction, languages and frameworks are still limited in supporting these ideas.

It's almost insane to think that once you have a modeled domain you need to replicate them in resolvers, types, queries, etc (for graphql), resources, responses, endpoints (for rest), etc, etc.

to reinforce the point of the article and the one brought by @SKILNER, I believe that the great transformation that is to come is in maintainability through a focus on the domain

[galdosdi]

I think, as OP alludes, ultimately explorability is the heart of the problem that stops people from writing code this way.

If you don't care about other people being able to read and explore your code without a lot of preperation, you can go full hog creating layers of DSLs and metaprogramming, and with enough dedication you can end up with all your real domain level business rules in one place separate from the "infrastructure"

But if you do this, you end up with a codebase that is hard for a newcomer to ask simple maintenance questions about like "What are all the places XYZ is called from?" or "What are all the places to write to ABC" etc

So an experienced developer learns to limit their metaprogramming so their code retains easy explorability at the expense of long term sustainability. Golang is kind of an epitome of this kind of thinking. Lisps are kind of the epitome of the opposite I guess.

This is what's behind the paradox where a good dev working alone or maybe with one very like minded person can produce a level of productivity you can't match again as you add developers, till you have way more developers. The dip represents the loss due to having to communicate a common understanding of the codebase, that doesn't get adequately compensated for till you've added a lot more people.

[pendar747]

> This is what's behind the paradox where a good dev working alone or maybe with one very like minded person can produce a level of productivity you can't match again as you add developers, till you have way more developers. The dip represents the loss due to having to communicate a common understanding of the codebase, that doesn't get adequately compensated for till you've added a lot more people.

Wow I couldn't agree more with this point. You put it perfectly. I worked alone and later with one other developer on a project and it felt way more productive than my current team of 5 developers!

[fleddr]

A long time ago, I presented to my manager a carefully crafted application, a stereotypical multi-layered architecture and then some more.

His response: wow, that's a lot of crap just to run a sql query.

He was fully right.

[samsquire]

I agree with you!

I find other people's code difficult to follow due to all the abstractions that I wouldn't have inserted.

When I write Clojure code for example, the code just works. I somewhat understand the code I write.

But when I read other people's Clojure, I find it difficult to understand. I think it's my maturity in understanding Clojure, the mental model isn't there yet. Which is funny because I've been on two projects where we used Clojure.

I don't have the same problem with less metaprogramming languages such as C, Java or Python.

[bmitc]

I would argue that a huge or maybe even the primary reason why maintenance is so hard, though, is because a lot of software was originally not written "correctly", that is without maintenance and longevity in mind.

The hardest part of maintenance, in my experience, is that programs were developed under needless or incorrect constraints and then expected to be magically maintained. There is a huge downstream effect of decisions made early on in a software system's life.

The story of "just get it working" that evolves into "now that's it's working, don't change it but add these new features" repeats itself over, and over, and over. It's not surprising why maintenance in systems developed like that is hard.

[bsedlm]

yes. I wish I could somehow get to work on a blend between: a decompiler, debugger, emulator, static analyzer, memory profiler, and so on.

The idea being some kind of a runtime for assembly code which does not actually execute the program but allows one to understand it in different sematinc levels, or dunno.. this is a very raw idea. needs a lot of work (and a lot more knowldedge) to set down.

too bad none of the professors that I was able to meet were really interested in this kind of thing

[Banana699]

I think dynamic analysis is incredibly powerful and criminally underused in IDEs and other dev tools.

I have thought of an idea about 6 months ago that has been fermenting in my mind since then : what if (e.g.) a Python VM had a mode where it records all type info of all identifiers as it executed the code and persisted this info into a standard format, later when you open a .py file in an IDE, all type info of the objects defined or named in the file are pulled from the persistent record and crunched by the IDE and used to present a top-notch dev experience?

The traditional achilles's heel of static analysis and type systems is Turing Completeness, traditional answers range from trying and giving up (Java's Object or Kotlin's Any?), not bothering to try in the first place (Ruby, Python, etc...), very cleverly restricting what you can say so that you never or rarely run into the embarrassing problems (Haskell, Rust,...), and whatever the fuck C++'s type system is. The type-profiling approach suggests another answer entirely : what if we just execute the damn thing without regard for types, like we already do now for Python and the like, but record everything that happens so that later static analysis can do a whole ton of things it can't do from program text alone. You can have Turing-Complete types that way, you just can't have them immediately (as soon as you write the code) or completely (as there are always execution paths that aren't visited, which can change types of things you think you know, e.g. x = 1 ; if VERY_SPECIFIC_RARE_CONDITION : x = "Hello" ).

You can have incredibly specific and fine-grained types, like "Dict[String->int] WHERE 'foo' in Dict and Dict['bar'] == 42", which is peculiar subset of all string-int dictionaries that satisfy the WHERE clause. All of this would be "profiled" automatically from the runtime, you're already executing the code for free anyway. Essentially, type- checking and inference becomes a never-halting computation amortized over all executions of a program, producing incremental results along the way.

I have ahead of me some opportunity to at least have a go at this idea, but I'm not completely free to pursue it (others can veto the whole thing) and I'm not sure I have all the angles or the prerequisite knowledge necessary to dive in and make something that matters. If anyone of the good folks at JetBrains or VisualStudio or similar orgs are reading this : please steal this idea and make it far better than I can, or at least pass it to others if you don't have the time.

[RyanCavanaugh]

This is how JavaScript intellisense in Visual Studio used to work, except that the program was executed "behind the scenes" using a trimmed-down VM that could execute without side effects or infinite loops. It was eventually abandoned due to poor performance, predictability, and stability.

The problem is this dilemma: If you have to wait for a "real" execution of a program, then very reasonable expectations like "I can see a local variable I just declared" doesn't work. If you try to fake-execute a program, you have problems like trying to figure out what to do with side-effecting calls, loops, and other control flow problems.

Trying to reconcile a previous type snapshot with an arbitrary set of program edits was tried by an early version of TypeScript and wholly abandoned because it's extremely difficult to get right, and any wrongness quickly propagates and corrupts the entire state. The flow team is still trying this approach and is having a very hard time with it, from what I can tell.

[int0x80]

Something very close to this can be done and it's in fact done already by static analysis. Static analysis doesn't have any Turing complete problem. Static analysis has a limitation of providing complete answers because of the halting problem, but it can provide instead sound answers. That is, static analysis can provide all possible runtime types for any given (e.g. python) expression. This can be accomplished by doing Abstract Interpretation, or Constraint Analysis and Dataflow Analysis (kCFA), which is in way similar to what you suggest of running the program in a profiler, but instead it runs the program in an abstract value domain. With static analysis you will get some imprecisions (false positives) but no false negatives, so it can effectively be very useful for a developer in an IDE. The precision (amount of FPs) and performance are mutually dependent, but good performance and reasonable (read useful) precision can be achieved, although it's a non-trivial engineering problem.

Additionally, some programs cannot be easily and/or quickly executed to cover all possible paths, so parts of the program will remain uncovered by the profiler. That is one place where static analysis becomes very powerful, because you can cover all the program much faster (in linear time making largish precision tradeoffs, but analysis still yielding a usable result).

source: I work on a flagship static analyzer.

[Banana699]

Indeed, Abstract Interpretation is powerful and beautiful as heck. Type Systems can be seen as just a special case of general Abstract Interpretation, where the types are the abstract values computed from each expression. The problem is that it requires ingenuity to devise abstract value systems that don't degrade into useless generality as unknown branches in the code 'execute'. Even types only succeed as much as they do because they require extensive and invasive (language-design-changing as well as program-changing) cooperation from both the language designer(s)\implementer(s), as well as the language developer(s), without this cooperation you will leave open very wide gaps. Symbolic Execution is another much much more powerful technique but, predictably, also very inefficient in general.

My way of looking at this is just frustration at the misallocation of resources. Dynamic programs already have tons of typing information available, just not in the right time and place. Your brain spends an aweful lot of time "executing" the code of a dynamic language while it's writing the code, just like the language runtime only far more slowly and with a sky high probability of error. So all what I'm saying is, why this immense suffering, when the information that your brain is in dire need of is already right there in the interpreter's data structures, just in opaque and execution-temporary forms ? It strikes me as incredibly obvious to try to bring in those typing information from the runtime into persistent transparent records available at dev-time.

Abstract Interpretation or Symbolic Executation are (fancy) tools in programming language theory's toolbox that can help us, but the simplest possible thing to try first is to make use of the already-available information that are just lying around elsewhere. To make a somewhat forced analogy : if we have a food crisis at hand, we could try fancy solutions like genetic engineering of super crops or hydroponic farming, but the dumbest possible solution to try first would be to simply bring in food from other places where its plentiful. Typing info in dynamic languages is very plentiful, it's just not there when we need it.

[int0x80]

The problem is what I mentioned at the end of my other comment -- complex programs will get inputs from web APIs, console, database, so it's not possible to have complete runtime coverage for all paths in the interpreter on the general case.

[bsedlm]

do you know any textbooks on abstract interpretation?

[int0x80]

"Principles of Program Analysis" covers the subject very well. It's not an easy or beginner book because it's very formal, but very complete and with plenty of examples.

[rileyphone]

The idea of runtime type feedback was originally explored by the SELF group for optimization [0], and is currently used by JS runtimes like V8. The obstacle your vision faces today is that program editors are almost all completely separate to the languages they are used for, and thus such communication with the runtime is enormously painful and complex. Time to return to the truly integrated development environments of Smalltalk/Lisp with modern niceties like gradual types and multicore processors, methinks!

[0] https://dl.acm.org/doi/pdf/10.1145/178243.178478

[SKILNER]

I developed a dynamic analysis/IDE tool for old IBM systems. I was told by VC's that IT management never spends significant money on programmer productivity. I think they're mostly right. Maybe because no one knows how to actually measure it.

[chowells]

I think if you have to run a whole program to understand a small part of it, you've already lost. The most valuable tools are a REPL that can execute units of your code, and a language that enforces purity and immutability so that local reasoning is more likely to be sufficient.

[mst]

> The traditional achilles's heel of static analysis and type systems is Turing Completeness, traditional answers range from trying and giving up (Java's Object or Kotlin's Any?), not bothering to try in the first place (Ruby, Python, etc...), very cleverly restricting what you can say so that you never or rarely run into the embarrassing problems (Haskell, Rust,...), and whatever the fuck C++'s type system is.

This stanza was maybe not your primary point but it was beatifully written and multiple programmer friends of every variety mentioned have been cackling at it.

[sankha93]

There has been attempts as you describe before. I can specifically point to work done in Ruby by my PhD advisor using the exact profiling approach, and then static typing from that: http://www.cs.tufts.edu/~jfoster/papers/cs-tr-4935.pdf

> you're already executing the code for free anyway

Based on my experience of working on similar domain of type systems for Ruby (though not the exact approach you describe), this turns out to be the ultimate bottleneck. If you are instrumenting everything, the code execution is very slow. A practical approach here is to abstract values in the interpreter (like represent all whole numbers are Int). However, this would eliminate the specific cases where you can track "Dict[String->int] WHERE 'foo' in Dict and Dict['bar'] == 42". You could get some mileage out of singleton types, but there are still limitations on running arbitrary queries: how do you record a profile and run queries on open file or network handles later? How do you reconcile side effects between two program execution profiles? It is a tradeoff between how much information can you record in a profile vs cost of recording.

There is definitely some scope here that can be undertaken with longer term studies that I have not seen yet. Does recording type information (or other facts from profiling) over the longer term enough to cover all paths through the program? If so, as this discussion is about maintaining code long term, does it help developers refactor and maintain code as a code base undergoes bitrot and then gets minor updates? There is a gap between industry who faces this problem but usually doesn't invest in such studies and academia who usually invests in such studies but doesn't have the same changing requirements as an industrial codebase.

[bramblerose]

https://github.com/instagram/MonkeyType can perform the call logging, and can export a static typing file which is used by mypy, but also e.g. PyCharm. It doesn't expose such fine grained types, but you could build that based on the logged data.

[mst]

Shit. Also. This has actually been attempted.

Urbit "infers" types from functions by executing them with type objects so the relevant bindings themselves return type objects.

If you can skip over both the syntax and the creator, there are some intesting things to be found in there.

[mtVessel]

This is only feasible if the program takes no input, or a very limited set. Once you open it up to arbitrary input, no single run (or even a large set of runs) can capture everything the program might be expected to handle.

How does the type profiler know that the variable that only contained values like "123" or "456" was handling identifiers, not numbers?

[Banana699]

>This is only feasible if the program takes no input, or a very limited set.

One of the insights that people making tools for dynamic languages discover over and over again is that most uses of dynamic features is highly static and constrained. In general, yes, a python program can just do eval(input("enter some python code to execute :) \n>>")), but people mostly don't do this. People use extremly dynamic and extremly flexible constructs and features in very constrained ways. This is like the observation that most utterances of human languages are highly constrained and highly specific, even within syntactically-valid and semantically-meaningful sentences, not all utterances are equally probable, and some are so improbable as to be essentially irrelevant. People who try to memorize the dictionary never learn the language, because the vast majority of the dictionary is useless and mostly unused, and even the words that are used are only used in a subset of their possible meanings.

>Once you open it up to arbitrary input, no single run (or even a large set of runs) can capture everything the program might be expected to handle.

Anything is better than nothing, right ? if your program keeps executing with some set of types over and over again (and it will, because no program is infinitely-generic, the human brains that wrote the code can't reason over infinity in general), wouldn't it be better to record this and make it avilable at static write-time ?

Human brains are finite, how do we reason over the "infinite" types that every Python program theoretically deals with ? We don't! like I said, most dynamic features are an illusion, there is a very finite set of uses that we have in mind for them. Here is an experiment you might try, the next time you write in a dynamic language, try to observe yourself thinking about the code. In the vast majority of cases, you will find that your brain already has a very specific type in mind for each variable (or else how can you do anything ? even printing the thing requires assuming it has a __repr__ method that doesn't fail.).

>How does the type profiler know that the variable that only contained values like "123" or "456" was handling identifiers, not numbers?

It doesn't. I think you misunderstood the idea a little, the type profiler makes no attempt whatsoever at discerning the "meaning" of the data pointed to by variables, it will only record that your variable held strings during runtime. If the number of string values the variable held was small enough, it might attempt to list them like this "Str WHERE Str in ["123","456"]". If the number of values the variable held was larger than some threshold but some predicate held for it consistently it can also use that, i.e. "Str WHERE is_numeric(Str)". If a string variable was always tested against a regex before every use, it will notice that and include the regex into the type. No additional "smart pants" than this is attempted, just the info your VM or interpreter already knows, just recorded instead of thrown away after each execution.

The profiler will not and can not attempt to understand any "meaning" behind the data nor it needs to in order to be useful, it's just a (dynamic) type system. No current type system, static or otherwise, attempts to say "'123' is a numeric, would you like to make it an int ?", that would be painful, absurd in most cases I can think of and misguided in general.

[mtVessel]

> wouldn't it be better to record this and make it available at static write-time

I think you misunderstand my position. It's better for the creator to simply specify it when they write the code - i.e. static typing.

> the type profiler makes no attempt whatsoever at discerning the "meaning" of the data

Your examples are waaay beyond what I need or expect. What I need is for the program to recognize that, for a number, 123 and 456 are valid, but "abc" will never be. Conversely, for an identifier, no matter how many runs use values like 123, someday someone might provide the value "abc" and that's ok. Also, any code that attempts to sum up a collection of identifiers should not be runnable, even if the identifiers in question all happen to be numbers.

This is something that static typing provides, and no amount of profiling will ever be able to divine.

> In the vast majority of cases, you will find that your brain already has a very specific type in mind for each variable

Sure, for the code I write. But I've seen plenty of code written by juniors where, upon inspection, it was completely inscrutable whether a given parameter expects an integer, a string, a brick wall or a banana.

Which is all to say that my default position is unchanged: dynamic typing is unhelpful for anything larger than small, single-purpose scripts.

[samatman]

> No current type system, static or otherwise, attempts to say "'123' is a numeric, would you like to make it an int ?"

SQLite's column type affinity will in fact do this, if you tell SQLite that a column is an INTEGER it will turn '123' into 123 but will happily take a string like "widget" and just store it.

I also wanted to add that your thoughts on this subject are well-stated and align with some work I've been patiently chipping away at, in the intersection of gradual typing and unit testing. I'll have more to say on that subject someday...

[herdrick]

Correct, that's why you use the large dataset available to you -- tons of runtime data from production.

[herdrick]

It would seem like that's getting the types too late to be useful, but in practice most code goes into production in stages, so you could start getting this probabilistic typing data from code that has rolled out in a limited way, for example features that are hidden to most users.

[leksak]

Sounds smalltalk-y with a little more bolted on.

[di4na]

There have been interesting progress in this space from a different angle of attack. o11y (Akita, honeycomb, ebpf, prodfiler, ...) and "Learning from incidents" are two of these angle that have really built on this idea that "understanding what the system does in prod" matters far more than "writing it right the first time".

It is also the thinking we can see supporting a lot of the early devops movement.

[the_duke]

I see this largely as a language failure.

Alot of popular languages make it really easy to write code, but hard to maintain it .

It's one of the reasons I like Rust so much,even for relatively high level code. It has one of the srictest type systems and std/library ecosystems of any language in existence, which tends to make code very robust and easy to refactor.

I often wish for a language that has a similarly strict type type/ecosystem but is higher level. Swift is quite close, but very Mac OS centric. Haskell has exceptions, which are often used for error handling, making code less robust (plus the ecosystem is small). Other languages like Ada/Spark or Idris are way too fringe...

[thrown_22]

>Alot of popular languages make it really easy to write code, but hard to maintain it .

The language you're writing in has next to no meaningful impact on long term maintenance. When you're trying to do software archeology on why a function even exists the types it has will not help you figure out it's there because between 1996 and 2002 there was a tax rebate on capex for rocket development.

You need human understandable documentation, not fancy language features. This is no where near as sexy so no one does documentation and we're in the middle of a digital dark age.

[kinjba11]

I disagree. Language does matter for long term maintenance. For instance with a tax rebate.

Static typing - a fancy language feature - can immediately tell you the function deals with money. +1 for maintenance. Static typing with a solid IDE will immediately reveal everywhere the function is used. +1.

Languages shepherd you into certain designs or mistakes by making some things easier than others, such as when class inheritance can be easily abused in OO languages, cryptic one liners are easy in Perl, point-free cleverness in Haskell, monkey patching in dynamic languages, etc. +/-1 depending.

Per documentation, languages come with tools and best practices: Javadoc, Godoc, Rustdoc; C# even has a dedicated comment syntax for documentation - a fancy language feature. +1. No one does documentation, until the language tooling forces them to.

That said, BG (Before Git) and widespread use of source control, as in '96 to '02, was a dark age. We know more now.

[Sohcahtoa82]

> The language you're writing in has next to no meaningful impact on long term maintenance.

I disagree.

If you abandon a large block of code for a year, then come back to it, depending on the language, it may be difficult to grok what the code was doing and how it worked.

If it's Java, the tendency to create 100 levels of abstraction make it hard to reason about. If it's Python without type annotations, then your IDE may not be able to determine the types of variables, so it's difficult to determine what can be done with an object.

If it's Perl, then God help you. May He have mercy on your soul.

[kinjba11]

What about F#? Often seems underappreciated

[kanzenryu2]

Also possibly including: install very old OS version, get libs and dependencies to some old version, and maybe update to a slightly newer compatible version, but not the latest. Create mock services of complex back end systems for test cases. Convert https connections to http for testing, or use unsupported TLS versions and protocols for some connections. Then you can get started!

[tppiotrowski]

Working on old but popular software is where legends are made. You need to be methodical about changes. How do you make changes in a million line+ codebase without breaking anything for millions of existing users? This challenge is reserved for the finest engineers on the planet. These are the people you want your desk near when you start your professional career as a programmer.

[baq]

this is simpler than it sounds - the process trumps any engineering stardom. the process is king. the process is love, the process is life, quite literally. getting any change in there isn't so much fine engineering as it is wrestling with layers and layers of process, where every layer has been added due to a monumental f-up in the past. it's an environment where getting any change committed into a repository usually takes weeks, unless one of processes for sidestepping the process is invoked.

[synergy20]

A huge opportunity here: ways to assist developers to figure out how existing code base works by generating code flow, class hierarchy, etc.

yes there are a few tools on the market, nothing really standout though, maybe it's for AI/ML to innovate in this field.

Linux kernel is well designed, in that I can add code relatively easy into its subsystems, it's those Object oriented or FP code base that bothers me the most, especially when they're large, often times they just made me feel hopeless, good tools desperately needed.

[gitgud]

A similar concept is the maintenance of physical world machinery, it's also often overlooked, but a huge industry even compared to manufacturing the machinery it self