[ebolyen]

I don't know, that sounds like a complex kind of ingest which could be arbitrarily subtle and diverge over time for legal and bureaucratic reasons.

I would kind of appreciate having two formats, since what are the odds they would change together? While there may never be a 3rd format, a DRY importer would imply that the source generating the data is also DRY.

[Stratoscope]

Good point. This may be a case where domain knowledge is helpful.

One of the reasons they brought me in on this project is that besides knowing how to wrangle data, I'm also an experienced pilot. So I had a good intuitive sense of the meaning and purpose of the data.

The part of the data that was identical is the description of the airspace boundaries. Pilots will recognize this as the famous "upside down wedding cake". But it's not just simple circles like a wedding cake. There are all kinds of cutouts and special cases.

Stuff like "From point A, draw an arc to point B with its center at point C. Then track the centerline of the San Seriffe River using the following list of points. Finally, from point D draw a straight line back to point A."

The FAA would be very reluctant to change this, for at least two reasons:

1. Who will provide us the budget to make these changes?

2. Who will take the heat when we break every client of this data?

[ebolyen]

I see, so it's a procedural language that is well understood by those who fly (not just some semi-structured data or ontology). This is a great example of the advantage of domain experience. Thanks for sharing!

[Stratoscope]

> a procedural language that is well understood by those who fly

That is a great way to describe it!

Of course it is all just rows in a CSV file, but yes, it is a set of instructions for how to generate a map.

In fact the pilot's maps were being drawn long before the computer era. Apparently the first FAA sectional chart was published in 1930! So the data format was derived from what must have been human-readable descriptions of what to plot on the map using a compass and straightedge.

I just remembered a quirk of the Australian airspace data. Sometimes they want you to draw a direct line from point F to point G, but there were two different kinds of straight lines. They may ask for a great circle, a straight path on the surface of the Earth. Or a rhumb line, which looks straight on a Mercator projection but is a curved path on the Earth.

You would often have some of each in the very same boundary description!

For anyone curious about this stuff, I recommend a visit to your local municipal airport and stop by the pilot shop to buy a sectional chart of your area.

[tass]

Paper charts are great (they're fairly cheap and printed quite nicely in the USA at least) but you can get a good look at these boundaries through online charts.

https://skyvector.com is a good way to view these.

[Stratoscope]

Thank you! I was trying to remember the name of that site and it slipped my mind. Yes, SkyVector is great.

[hamasho]

I think if you know the domain well, it's not "premature" at all.

[TeMPOraL]

In such case I think I'd go for an internal-DRYing + copy-on-write approach. That is, two identical classes or entry points, one for each format; internally, they'd share all the common code. Over time, if something changes in one format but not the other, that piece of code gets duplicated and then changed, so the other format retains the original code, which it now owns.

[chii]

I believe this very method is very common in games - you have similar logic for entities, but some have divergences that could occur in unknown ways after playtesting or future development.

Tho if done haphazardly by someone inexperienced, you might end up with subtle divergences that might look like they're meant to be copies, and debugging them in the future by another developer (without the history or knowledge) can get hard.

Then someone would wonder why there are these two very similar pieces of code, and mistakenly try to DRY it in the hopes of improving it, causing subtle mistakes to get introduced...

[cies]

I prefer the FP approach of separating data and logic. you could end up with a box of functions (logic) that can be reused by the different "entities".

Last time i checked the FP world is slowly producing ECS frameworks that are needed to make the game performant. They used to be nearly C++ (or OO) exclusive.

[trealira]

Is an entity component system really functional programming? I had the sense that functional programming was more about writing functions that are pure and referentially transparent, and making data immutable normally, which can make code simpler and more modular. It tends to use higher-order functions (recursive operators) more than direct recursion, because it's easier to verify correctness then. Rather than imperative loops and mutation, the meat of the program consists of the composition of many different functions, both small and large.

Entity component systems are pretty cool, as is functional programming, but I don't see the relation.

In addition, object-oriented languages seem well-suited to making entity component systems. There are some tutorials on them in different object-oriented programming languages:

C++: https://austinmorlan.com/posts/entity_component_system/

C#: https://matthall.codes/blog/ecs/

Common Lisp: https://edoput.it/2023/11/19/data-oriented-clos.html

[chipdart]

> In such case I think I'd go for an internal-DRYing + copy-on-write approach.

I agree. The primary risk of presented by DRY is tight coupling code which only bears similarities at a surface level. Starting off by explicitly keeping the externa bits separate sounds like a good way to avoid the worst tradeoff.

Nevertheless I still prefer the Write Everything Twice (WET) principle, which means mostly the same thing, but following a clear guideline: postpone all de-duplication efforts until it's either obvious there's shared code (semantics and implementation) in >2 occurrences, and always start by treating separate cases as independent cases.

[jeremyjh]

This is really good advice and a great way to think about it.

[ebolyen]

I like that approach.

[Spivak]

I've had the mantra "inheritance is only for code reuse" and it's never steered me wrong.

[jameshart]

Inheritance is only good for code reuse, and it’s a trick you only get to use once for each piece of code, so if you use it you need to be absolutely certain that the taxonomy you’re using it to leverage code across is the right one.

All ‘is-a so it gets this code’ models can be trivially modeled as ‘has-a so it gets this code’ patterns, which don’t have that single-use constraint… so the corollary to this rule tends towards ‘never use inheritance’.

[Spivak]

Single use? No way that's what multiple inheritance and mixins are for. Inheritance being only for code reuse is explicitly about not creating a taxonomy. No more is-a just, "I need this code here." Hey this thing behaves like a mapping inherit from the MutipleMapping and get all the usual mapping methods for free. Hey this model needs created/updated_at, inherit from ChangeTracking and get those fields and helper methods for free.

Has-a doesn't make sense for code like the literal text reuse. It makes sense for composition and encapsulation.

Edit: I'm now realizing that Python has one of the only sane multiple inheritance implementations. It's no wonder the rest of y'all hate it.

[zeroCalories]

It seems like OP is describing a shared interface, not necessarily inheritance.

[bcrosby95]

I don't know. I've seen this approach for projects before go bad - people didn't want to DRY because they might diverge. Except they never did. Our 3rd+ scenarios we abstracted.

But what basically ended up happening was we had 2 codebases: 1 for that non-DRY version, and then 1 for everything else. The non-DRY version limped along and no one ever wanted to work on it. The ways it did things were never updated. It was rarely improved. It was kinda left to rot.

[chipdart]

> But what basically ended up happening was we had 2 codebases: 1 for that non-DRY version, and then 1 for everything else. The non-DRY version limped along and no one ever wanted to work on it. The ways it did things were never updated. It was rarely improved. It was kinda left to rot.

It sounds to me that you're trying to pin the blame of failing to maintain software on not following DRY, which makes no sense to me.

Advocating against mindlessly following DRY is not the same as advocating for not maintaining your software. Also, DRY does not magically earn you extra maintenance credits. In fact, it sounds to me that the bit of the code you called DRY ended up being easier to maintain because it wasn't forced to pile on abstractions needed to support the non-DRY code. If it was easy, you'd already have done it and you wouldn't be complaining about the special-purpose code you kept separated.

[bcrosby95]

In my experience, once you copy code its bound to diverge, intentional or not. Bugs become features and you can never put the cat back in the bag without a monumental amount of work.

Undoing an abstraction is way easier. Eventually, they all turn bad anyways.

[jononor]

Why wasn't the original implementation swapped for the new one? The unwillingness/inability to do that seems to be most likely the core of the issues here?

[bcrosby95]

The majority of our business was through the 1st implementation. Because of that it was the base we used to refactor into a more abstract solution for further scenarios. It was never deemed "worth it" to transition the 2nd non-DRY version. Why refactor an existing implementation if its working well enough and we could expand to new markets instead?

[jononor]

Yes, why do it? :p I mean, there are pros and cons - costs and benefits. And I can see both scenarios where it is better to spend the time on something else (that has better chance of bringing in money), and cases where it would be the right thing to do the cleanup (maybe original is just about to fall apart, or the new has straight up benefits to the business, or the act of doing it will greatly improve testing/QA in a critical area, etc).

Writing it DRY in the first place would also have costs, including the alternative costs. Would it have been better to take those there and then?

[kiitos]

I have seen far more problems caused by premature or over-use of DRY than the opposite.

Undoing an abstraction is always far more difficult than the pains caused by not having one in the first place.

[mannykannot]

I vaguely recall Fred Brooks, in The Mythical Man-Month, using a somewhat similar situation, but involving the various US states' income tax rules and their relationship to the federal tax, as an example in order to make some sort of point (that point being 'know your data', IIRC.)

In a situation where there is a base model with specific modifications - which is, I feel, how airspace regulation mostly works - then I suspect that a DRY approach would make it easier to inspect and test, so long as it stays that way.