[eviks]

> spent weeks in the early days following AI down dead ends, exploring designs that felt productive in the moment but collapsed under scrutiny

> I paid for that with a total rewrite.

With so much waste and not a single example of the "brilliant at giving you the right answer to a specific technical question"

> The takeaway for me is simple: AI is an incredible force multiplier

Seems more like a feel multiplier, rather than force.

> 500 tests, many of which I felt I could reuse

Indeed, feeling is the only saving grace for a mountain of random unreviewed tests

[lalitmaganti]

> With so much waste and not a single example of the "brilliant at giving you the right answer to a specific technical question"

In my opinion, "giving me a better understanding for the architecture of the project" is reasonable technical compensation.

> Indeed, feeling is the only saving grace for a mountain of random unreviewed tests

I think I say a line or two above that this was after a review of the codebase so I did review these tests.

[eviks]

How many of 500 tests were actually reviewed/tested and found good? The code review results were: don't understand code and/or it's pretty bad. Then 0 of those 500 tests were used due to the full rewrite. So nothing to extrapolate usefulness from, all that's left is a feel...

> giving me better understanding

Examples of that would also be nice (I don't doubt the personal feel that waste was justified)

> JOURNAL before: ...

> JOURNAL after: ... > was wrong here, learned this

[lalitmaganti]

> How many of 500 tests were actually reviewed/tested and found good?

Essentially ~all of the tests were found to be useful but in a more "smoke test" capacity i.e. they provided good "basic" coverage but it was clear that it was also not sufficient.

Which is why in the rewrite: 1) I built a TCL driver that run the upstream SQLite tests and verified we accepted or rejected the SQL in the same way as SQLite.

2) I wrote a test runner which checked for "idempotence" i.e. run the formatter over all the SQL from all the other types of tests then verify that the AST was identical in the input and output.

3) I also wrote a script which ran the formatter over the PerfettoSQL standard library [1], a real world SQLite-based codebase that I knew and deeply understood so I could go through each file and manually check the output.

> Examples of that would also be nice (I don't doubt the personal feel that waste was justified)

Some things learned concretely:

1) C was not going to work for the higher level parts of the project, even the formatter was not pleasant to read or write in C, the validator was much worse

2) Doing the SQLite source extraction in the same language meant that I could ship a really cool feature where the syntaqlite CLI could "generate dialect extensions" without people needing to download a separate script, run their own extraction on the SQLite source code, or worse yet, need to fork syntaqlite. This actually makes it technically possible for people in the web playground to dynamically build extensions to SQLite (though I haven't ended up plumbing that feature through yet)

3) Having a DSL [2] for extensions of SQLite (that e.g. PerfettoSQL could use) was the correct way to go rather than using YAML/JSON/XML etc becaue of how much clarity it provided and how AI took a lot of the annoyance of maintaining a DSL away.

4) I need to invest much more in testing from the start and also more testing where the correctness can be "proved" in some way (e.g. idempotence testing or SQLite upstream testing as described above)

[1] https://github.com/google/perfetto/tree/main/src/trace_proce... [2] https://docs.syntaqlite.com/v0.2.15/guides/custom-dialects/