[CharlieDigital]

> I know I need to, at least, read a CSV, parse a CSV row into some data structure and then categorise it. So I TDDed my way through to complete these “sub problems”. However, when I was about to apply the categorisation rules I realised that the data structure I created didn’t help, was just bad and didn’t work at all

TDD is kind of a forcing function for modular design (whether OOP or functional programming).

In your scenario, you'd want:

1) a module to read the CSV -> test that it can load a file correctly given a path string, correctly handles invalid paths (format, file does not exist, etc)

2) a module to parse a row -> given a string, test that it can parse a 0 row file, a 1 row file, a 2 row file, and correctly handles an unparseable file (e.g. binary file); test that it correctly handles incorrectly formatted rows

3) a module to categorize -> given a set of rows, test that it can categorize a sample set correctly with expected output

If you try to make 2 depend on 1 or 3 depend on 2 and 1, it's no longer a unit test and becomes more of an integration test.

[rubicon33]

It is indeed a forcing function for modular design but what you get with that if you aren’t careful is heavily over engineered code.

Take that first module, the CSV loader… if you build it in a truly modular way, it can be hard to stop with the features that just support your end user application.

[crazygringo]

> it can be hard to stop with the features that just support your end user application.

I mean, it's not hard, you just choose to stop.

All it really takes is an awareness that you're programming to solve a specific limited defined task, not programming just for fun.

Of course, if you're doing it for fun, then fine, don't stop.

But nothing about stopping is hard. It's just a choice you have.