[rosariom]

This is the experienced developers dilemma: "To engineer or to not engineer"; engineering usually turning out to be over engineering. A more experienced developer friend of mine would always tell me: "build for today's requirements". I try hard to fight the design/architect voices in my head that always want to imagine this made up future where we will need X or else we cannot go live. These voices usually only stall real actual work, instill fear, and serve very little to no purpose.

I once had an argument with a Wall Street Java developer who was made the "lead" of one of our team projects. He decreed that every single class have an interface so that we can be generic and not tightly couple any of the components to concrete classes. I agreed that in some instances where functionality, i.e. methods that can be represented by different classes with the same method signatures made sense but not every single class needs an interface (if that is the case just go with a beautiful dynamically typed language like Python and avoid the code bloat). He got management on his side and we went off and built an overly engineered Straight Through Processing solution. It was a sheer nightmare to debug and the code bloat made me scream one day when we had a serious production issue. Even our manager (who finally had to look at the code when most of us were out on vacation once to answer some user questions) was flabbergasted at the amount of code he had to read through in order to answer the most trivial of questions. One extreme example was an interface for trade references. Our trade references were always strings with a date and some numeric value concatenated to it. The "engineer" decided that we needed an interface for this and added one interface and concrete class for our trade references. I told him that all classes needing trade references instance variables could just have a String instance variable named tradeReference or something like that and he went on to give me a design pattern lecture. We argued for nearly 20 minutes about this silly thing as he kept insisting that the future was unknown so we have to future proof the code from unforeseeable changes. When he said this I asked him to remove the Crystal ball plugin he had in Eclipse for predicting the future and get real. He got angry and we had a team call to waste yet another hour of developer time to discuss this. In the call I mentioned that our trade references scheme had not changed in 8 years and was unlikely to change... I lost the debate anyway. The ratio for most of the code base from interface to concrete class was largely 1-1 thus not justifying this code bloat approach.

Experienced developers (at least I think) seem to have these crystal balls in their heads or IDEs and usually try to be clairvoyant when it comes down to building a product. We need to get out of the business of overly engineering and just do as my friend said: "build for today's requirements". It is called software for a reason: it is soft. It can change (most likely will), can be refactored, redesigned, and/or incrementally made better or more abstract to accommodate changes. I am in no way saying no design, just limit it and get to work. A successfully built product is more satisfying then the imaginations of your head and the "perfect" engineering/scaling solution that never materializes. Users will like you, you will like you, and the team will get an andrenaline boost with each and every release keeping the spirits high. Remove the Crystal ball plugin from your head/IDE and stop trying to be clairvoyant and be a developer.

[dan00]

There's a reason why this happens more often in languages like Java. Because of the verbosity of the language, it's just painful to rewrite anything, even if the current solution isn't that much over engineered. So there might be a greater tendency to over engineer at the beginning, just to avoid any rewriting, which at the end isn't possible.

I'm a full time C++ developer, which might be a bit better in this regard than Java, but not that much, and a hobby Haskell programmer, and one of the greatest things about Haskell is it's brevity. It makes rewriting a lot less painful, so you're not avoiding it that much.

[Sandman]

I disagree with your point that the verbosity of Java is the reason for over-engineering. The refactoring capabilities of modern-day IDEs help immensely with reducing the amount of work one has to do to make syntactic changes over the whole codebase. So the argument that developers tend to over-engineer when coding in Java to avoid any pains that may arise because of its verbosity doesn't hold, IMO.

I think that the over-engineering happens simply because it's a pain to do serious refactoring when working on large enterprise software in general, never mind what language it's written in. The sad truth of our profession is that the customer requirements may change quickly and drastically, requiring us to rewrite large portions of our code, and very often we find ourselves thinking "If I only engineered it that way instead of this way, I wouldn't have so much trouble right now". This is why we strive to create the most robust, flexible solution that will be able to handle any future customer requirement. So we basically turn our code into a framework that, we hope, will allow us to respond to change quickly. Unfortunately, we can never predict everything that the users might want, so this whole approach falls down like a house of cards when a user requirement comes in and we need to change a large portion of the code. I believe this is true for a sufficiently large app written in any language, Haskell included.

[dan00]

"The refactoring capabilities of modern-day IDEs help immensely with reducing the amount of work one has to do to make syntactic changes over the whole codebase. So the argument that developers tend to over-engineer when coding in Java to avoid any pains that may arise because of its verbosity doesn't hold, IMO."

Refactoring tools might be nice and will help you here and there, but there's a difference in the abstraction abilites of a language like Java compared to language like Haskell.

It's not only about the amount of code, but also about the complexity of the code, when building abstractions.

Yes, a refactoring tool might help you dealing with the complexity, but it's still there and makes it more difficult.

I never understood the point of using a less capable language and then using a tool to compensate it, e.g automatically generate code for it.

[Sandman]

I never understood the point of using a less capable language and then using a tool to compensate it, e.g automatically generate code for it.

I definitely agree with you on this one :). Sure, it's better to use a language that lets you have less complexity even as your codebase grows quite large. You mentioned Haskell. Since I don't have any experience with it, what do you think is the reason that it's not used very often for building large enterprise applications (or maybe it is, and I just don't know about them)?

[dan00]

"Since I don't have any experience with it, what do you think is the reason that it's not used very often for building large enterprise applications (or maybe it is, and I just don't know about them)?"

Well, I don't know if it's even clear why other languages are used for enterprise software?

I don't think that their technical or whatever superiority was the main reason. Sometimes it seems that everything that is needed is to push it with a lot of marketing into the mainstream and then just let it go.

At some point there're more libaries for a language, most people use that language, universities are teaching it, so that's then the main reason to use a language.

Java might been there, pushed into mainstream, at the right time, with the right features, which made it less complex and less error prone (garbage collection, no memory pointers) to use, compared to C/C++.

But perhaps there's something about "object orientation", how it's implemented in Java, which makes it for people easier to grasp, if I read all the hate about these strange scheme/lisp courses in universities, but perhaps they're just already used to much to other languages.

On Haskell, at the beginning it looks very strange, especially compared to languages like C/C++, Java or C#, but I think that most of the felt strangness is a matter of habit, because most of the mainstream langugages aren't that different.

I don't think that learning Haskell was that much harder for me than learning to program in C++ or Java. Sometimes people seem to forget the challenges they had, when they learned programming for the first time.

[kahuna]

Depends if you are engineering an enterprise system/product, or building a smaller one off project.

In the case of the enterprise solution that will be around for the next 10 years, I would agree with your tech lead.

Every time you make a code change for something that is out of dev budget, you face a budget overrun in the project that was interrupted.

If you choose to deliver something that just works and as soon as possible, your total costs tend to balloon over the lifetime of the system.

Question to ask is: how long would it take for a new dev (someone who has never seen the code) to change the tradeReference naming convention to include the asset type, or some conditional tag, lets say to conform to reporting regulations or even an expanded business mandate?

Interfaces do help here, because the new guy can make a localized change, write a small unit test, and commit the code to source control before you can say "rebuilding search index".

Keep in mind that you do not know if something is well architected until AFTER its been in production for at least a year and had features built into it for another year or two, and has added new members to the team, and has lost a few of the original team members in that time,

I believe that most people who are able to look back and claim that they have delivered at least three large projects (>500k LOC c++ or >200k LOC java) or product releases that meet the above criteria, would agree with your tech lead.

[rosariom]

Good points; we were building an in-house solution for an Investment management wing in a bank.

To give some context on the trade reference I referred to, it was an internal tracking within the STP system used solely for tracking state and for communication between IT and the business. It was a simple date + numeric value used in our STP system for users to use in our Struts web and C# front end to check trade state through the STP flow and communicate with us if issues arose.

We did have lots of interfaces where it made sense and relied highly on object composition to represent financial concepts more richly and for inject-ability via Spring and Unit testing (makes writing tests easier when you mock things out). Asset types, security identifiers, etc, were represented correctly from an OO perspective and were a part of the xsd layer/interface between us and the trading systems. To us these were read-only values we just passed through for STP.

You are right about interfaces and unit testing but this is one case I highlighted of many where I think the lead was going over board. The internal trade reference was the same for 8 years and still the same to this day (which gives it another 3 years since I left for a total of 11 years). It never had more than one concrete class. It is no big deal on its own, but when combined with the other interfaces that only have one concrete class, it just bloats the system for no good reason.

Design and architecture is good for the reasons you have mentioned and more but it can go over board as is the case here IMO. There were other instances of that in our code base but that would take an entire blog post to cover some of the atrocities this engineer created because of his forecasting ability.

[brazzy]

This. A hundred times this. It is truly mind-boggling how many crimes against maintainability are committed in the name of future-proofing.

[rumcajz]

The rule of the thumb: If an extension point is used by only a single extension, get rid of it. It won't fit the next extension anyway so the work needed to maintain it in the meantime is a waste of time.