Hacker News new | ask | show | jobs
by yeti-sh 962 days ago
Indeed, they are out of scope. But I still would argue this does have something in common with Make — just as Make, jeeves can be used as a command runner, in which role Make is oftentimes used as well.

Instead of `make` as entry point for all project-specific commands (like `make lint`, `make build`, `make deploy`), users can rely upon `j` — both locally and CI.

Regarding your particular points:

* dependency graphs aren't presently at scope, I haven't yet come up with the method of expressing them in Python which would have entirely satisfied me;

* I am not so sure I would want to implement pattern rules due to complexity they bring about. Maybe just writing explicit Python code would be enough for cases where they're used. No strong opinion though.

1 comments

Having worked with Makefile and pushed it to limits, I'd say there are some deficiencies in the system you may try to tackle:

1. Databases. We ended up calling `psql -f some_script.sql && touch $my_task_name` and tracking changes with touch files. (Putting this into database on views or materialized views proved to be unsustainable.)

2. Datasets. If you just open a sqlite file, it's changed, and GNU Make thinks you must rebuild everything downstream. Datasets are mostly treated as row-order-independent, so hashing them as is does not always work.

3. Very expensive tasks that shouldn't be called always. Like my makefile had a script that parsed a million web pages, going around captchas via Tor, and touching the upstream files was to be avoided -- or if it happened, I had to manually touch the target, to avoid re-running that part.

4. Some targets can be updated and the result will always be new -- e.g. run a query to a live database, or news website. Some may produce the same. Would appreciate if any system has such a distinction.

5. Surprisingly, lots of alternative build systems don't do partial update. They only update every item in the deps graph.

If you manage to get any of these right, you'd be praised.