[mhomde]

Nice article! I'd like to a software methodology more supportive of what I'd call "Continuous refactoring" or a more "kaizen"-like mindset. "Agile" isn't always as agile as I'd like, but becomes rigid in peoples interpretation of the rules.

It's a constant fight to keep down the number of lines of code and complexity, in large projects. Once a code-base passes beyond a certain point it becomes difficult to see redundancies and patterns. Bloated code breeds bloated code.

If you on the other hand keep trimming and keep refactoring deep and wide as the code and requirements change you can often keep things manageable and understandable. "bad code" get's easier to spot, it's a little bit like the broken windows theory.

Scrum IMO and a lot of the agile methodologies, at least in their implementation, in my experience, tends to treat projects like brick-building and lacks a way to support and enforce constant re-architecturing and removal of code smells.

If code quality and refactoring is not supported by the actual methodology it can become an bureaucratic problem where product owners doesn't see the value and doesn't prioritize it (even though it leads to quicker development time and less bugs in the long run)

Unit testing is supposed to make refactoring easier, but can become an impediment to refactoring as well, as the cost of rewriting tests can be seen as too high.

We're currently experimenting with having one hour meeting each week where you can discuss things like that, and then dedicating some time to refactor, we'll see how it works out :)

[zwischenzug]

That sounds like the 'healthy' team I talk about in the article. The problem with the 'continuous refactoring' (in my experience) is that the people that hold the purse strings cannot grasp the benefit, so cut this early.

[arethuza]

In my experience, the easiest way to get 'continuous refactoring' done is to build it into all estimates without telling whoever controls the 'purse strings' ....

[mhomde]

yeah, that's what's often is being done. But it prevents the methodology to have some useful best practices built into it, and a methodology could be a force in improving code quality and keeping it intact over time, which is from all the projects I've seen severely needed

[nradov]

Scaled Agile Framework (SAFe) explicitly has that useful best practice built into it. http://www.scaledagileframework.com/refactoring/

[user5994461]

The problem with refactoring is that developers think it's a task of it's own. They want time to do it, they want special estimate, they want it to become a thing of its own.

Refactoring is low level work that is part of every task, it should not even be mentioned.

When you ask your mechanic to change wheels, you don't expect him to explain that he will have to take off the previous wheel and change the screws, that will be an item on the bill consuming half of the budget.

[ehnto]

It often is a task of it's own. It's not always "low-level work", sometimes it's directly replacing key components in a huge chain of events that all need to be modified to accept your refactor. It really depends on the codebase, features you're developing, nature of the business engagement and so on. There is also risk in a refactor, especially if it's heavy in business logic.

If I'm doing a bugfix task on a legacy site, I might note that it would be good to spend ~4h on a refactor, when the bug may have taken 0.25h to fix. In that scenario the benefits for refactoring are low, so we don't do it.

Alternatively, if I'm doing a 6h feature, and understanding and working with the bad code would take 6h, versus spending 4-6h on a wholesale rewrite, of course I would just slip the refactor inside the original task.

You have to keep your expectations realistic and your project managers informed. If your refactor suddenly blows out in time because you didn't fully comprehend your problem space, then you both haven't completed the feature or achieved the refactor. Maybe in your business a day extra is fine, in some businesses running over by a couple of hours can mean the ROI doesn't make sense on that task anymore.

The key is to be sensible and pragmatic. There is no one solution to all thinking. If there's anything any of these methodologies are trying to do, it's provide a framework for quickly communicating and reacting to the nuance of software development.

[user5994461]

>>> It often is a task of it's own. It's not always "low-level work", sometimes it's directly replacing key components in a huge chain of events that all need to be modified to accept your refactor. It really depends on the codebase, features you're developing, nature of the business engagement and so on. There is also risk in a refactor, especially if it's heavy in business logic.

And that's why refactoring has a terrible reputation and developers can't be trusted. Now you're talking about replacing key components and altering the business logic under the pretext of refactoring.

This should NOT be called refactoring, this should be called replacing key components and revisiting the business logic.

Of course, management doesn't want that to be done without justification.

[watwut]

Refactoring may change how business logic works internally without changing anything in input/output or features. If you need to change how key components are implemented, without changing what they do, then you are refactoring them. Changes may be big internally, but it is still called refactoring if features don't change.

[adrianN]

If your code developed a "god class" that does everything and should be ripped apart there isn't much you can do about it except changing core parts of your project.

[ehnto]

I don't mean changing business logic, refactors have to retain business logic still, and the risk is that the developer gets that wrong.

It sounds like you're not getting good communication from your developers or you're misunderstanding them when they say they need to refactor. Refactor is still the right word for it, but you need to find a way to communicate the extent of the refactor.

[mhomde]

Some developers think that, and unfortunately a lot of methodology enforces that thinking.

Cleaning up code as you go should be a habit I agree. It's slightly trickier when code reaches a "boiling frog" point and you need to take a look at the bigger picture and perhaps re-architecture. It often easy to keep chugging away at the tree's and not seeing the forest.

The downside with the scope of those kind of structural refactoring is that they touch a lot of code, so they're perceived as "risky".

Also developers get attached to the architecture they have learnt and know, and don't want to re-learn, even if the simplified code would be easier to learn for someone new

[aninhumer]

Not all refactoring fit neatly alongside existing work.

Sometimes you get a small change request that suggests a broader refactoring. You're pretty sure it will be valuable in future, but it would be a lot more work and isn't that useful in the short term. So if you try and do that refactoring upfront, you get pushback for wasting time on a small change. And then again on the next relevant change.

Developers often want to schedule these kinds of refactorings, because otherwise they get pushback on every chance they might otherwise have to fix these problems.

[user5994461]

Not all refactoring fit neatly alongside existing work... because every refactoring fits nearty alongside the new work.

In your example, you don't refactor for fun, you refactor to integrate new things. Integrate the refactoring as part of the new work.

Guess what. The mechanic too has to deal with the old wheels, and don't get him started on the old screws that broke and the assembly that had to be cleaned just to work in decent conditions.

[maxxxxx]

" The problem with refactoring is that developers think it's a task of it's own. They want time to do it, they want special estimate, they want it to become a thing of its own."

I always tell people to constantly refactor on a smaller scale and not ask management. It's just part of the job. A lot of the "big" refactors end up with just another set of problems but don't really make things better.

[laythea]

If you limit yourself to refactoring within the scope of the task at hand, then you end up with a mess after a certain size of software project. In my opinion, that is why software leads will normally at some point steer the success of future development by refactoring project wide, when and where his/her team can. It's also necessary for performance reasons sometimes.

[specialist]

Let’s check back in 10 years, see if you still think refactoring (eg mitigating tech debt) is trivial.

But I could be wrong too. I’m starting to think “devops” just means tech support with continuous rework.

[marcosdumay]

> I’m starting to think “devops” just means tech support with continuous rework.

Well, I think it's nothing more than giving the ops access and responsibilities to the same devs that created the mess to start with.

[specialist]

Every writer needs an editor.

[marcosdumay]

That is probably much better done by code reviews and tests. The siloing of attributions (and particularly of responsibilities) isn't great.

[crdoconnor]

Sometimes they're right. I'd prefer a methodology where the people who hold the purse strings have to explicitly declare their current desired emphasis on dev velocity / quality.

At the moment every single methodology requires devs make that decision even though it's really a bizdev decision.

[mhomde]

I don't know, when it comes to code quality and things like that I'd rather prefer a methodology that insulates it against bureaucracy rather than involves it. Keep the Dilbertian forces at bay.

Product owners should prioritize feature set, requirements and sometimes bugs, but the less they have to do with the intrinsics of reaching them the better.

[crdoconnor]

I think their involvement should be strictly limited to telling developers what % of their time they're to spend on refactoring/tooling.

This actually avoids dilbertian forces - the PM can't weasel out of responsibility for pressuring you to deliver shit quickly. That % would stick out like a sore thumb if it was permanently under, say, 15%.

[mhomde]

Unfortunately IMO that kind of thinking seldom works. It's very managerial to assign specific time and percentages to things but in practice I've never known development to be managed that way. Also you underestimate their weaselness :)

Refactoring needs to be an integral and constant part of the development, in the long run that will decrease time needed for refactoring, and time required to fix bugs. But that time is variable, and might come in large chunks depending on how much technical debt you've accumulated

[crdoconnor]

Precision isn't the issue. Story pointing isn't supposed to be 100% accurate either. It's supposed to shine a light on an otherwise opaque process. It works at that. This is the same.

I could quite easily spend all of my time refactoring and none on features and vice versa. There is no "right" amount as you seem to think, just a whole bunch of trade offs.

I think you underestimate the power of making something measurable and exposing it to upper management. That often changes behaviour in a quite radical way, especially among weasels.

And, you've never known development to be managed in this way because it's a new idea.

[mhomde]

Yeah totally, but if was intrinsic to a trendy methodology you might be able to shield it better in the same way you do stuff in the name of being ISO-whatever compliant :)

[TheCoelacanth]

It is intrinsic to XP. One of the rules is "Refactor Mercilessly"[1].

[1] http://www.extremeprogramming.org/rules/refactor.html

[s73ver_]

""Agile" isn't always as agile as I'd like, but becomes rigid in peoples interpretation of the rules."

I think a lot of this is because, if you don't enforce the rules rigidly, they tend to get cast by the wayside, especially by management.

[nradov]

Scaled Agile Framework (SAFe) places a heavy emphasis on continuous refactoring. http://www.scaledagileframework.com/refactoring/

[virgilp]

That's a lousy abbreviation - SAFr ("safer") would've been so much better.

[exelius]

Technically it's SAF-e -- Scaled Agile Framework for (e)nterprise.

[nradov]

No the official abbreviation is "SAFe", not "SAF-e".

[exelius]

I realize that, I was separating for effect. :)

[specialist]

The Agile methodology is to argue about the Agile methodology.

PMI style critical path, planning backwards from the release, is the only approach I’ve ever seen that was clear, concrete, actionable, high functioning.