[brandonbloom]

I think that the immutable data structure of lein is largely superficial. Once the final project map is built, all bets are off for the effectful behavior of downstream tasks/plugins. The only clear benefit from it is that you can capture a "complete" configuration with `lein pprint`. However, it does introduce significant cost in terms of the declarative configuration wack-a-mole, where you never quite know what magic ^:replace or :special-key to include to get what you want.

If you're not going to attempt to fix this superficiality by pushing immutable data deeper in to the system, then it makes perfect sense to discard it completely.

[michaniskin]

Totally agree.

Also, in boot we don't want to obtain a complete configuration before tasks run, because tasks can participate in the process. In boot a task can add dependencies or call other tasks, etc. This is why we need only one abstraction (tasks).

Just like in a Clojure program you don't have a complete configuration of values in variables before the program runs, because it's the program that creates those values. Boot figures so much out on its own that `boot pprint` isn't even a thing you would want.

Consider the hoops that needed to be jumped through to get the Maven wagons system implemented in Leiningen. In boot it's a non-issue–the environment is dynamic so you can just install wagon deps and then in the next expression install the deps and repositories that depend on that wagon. We didn't need to make any changes to boot to accommodate them.

[brandonbloom]

I still don't understand what the "task abstraction" is or what it provides. It seems to me that it's simply a Clojure function with a corresponding command line interface. Is that fair? Does it do something else too?

If it's just a function, I don't know why command-line support is valuable. For interactive use, the Clojure REPL is just fine. For automated use, you only need a single shell utility like perl or awk to evaluate an expression or run a script.

[michaniskin]

Yes, in the post I didn't get too technical with the treatment of tasks; I'll elaborate a little here. First, the command line thing.

You're right that the command line isn't strictly required to use boot, you can do everything at the REPL or perl/awk etc., as you pointed out. But for me it's really useful just ergonomically to be able to use command line arguments to configure ad-hoc builds because they can be very concise. Just like I probably wouldn't be super excited to use a Clojure shell instead of Bash, because Lisp is usually more verbose for the kinds of things I do on the command line. Consider:

    $ boot cljs -usO none

vs

    (boot (cljs :unified true :source-map true :optimizations :none))

the command line version is just nicer for that. When it's time to automate is when you'd put that in your build.boot and make it a new task.

This brings us to tasks. We used to describe them as "middleware factories" but Rich has provided us with a way cooler name: stateful transducers. The build process can be imagined as a transducer stack applied to a file set instead of to an async channel or sequential thing. The principal value the task abstraction provides is their process-building power.

A typical task definition looks like this:

    (deftask foo
      "This task does foo."
      [...]                  ; kwargs/cli-opts
      (let [state ...]       ; local state
        (fn [continue]       ; middleware
          (fn [event]        ; handler
            ...              ; build something, do work
            (continue event) ; call continuation
            ))))

Please ignore "event", it's there for historical reasons. But like transducers, we have powerful ways to build processes from tasks that don't need to know anything about each other now. For example:

    (deftask bar
      "This task does bar."
      [...]
      (comp
        (fn [continue]
          (fn [event]
            ...
            (continue event)))
        (foo :bar "baz" :baf "quux")
        (omg :hello "world")))

A key property of transducers is that they can also perform process control flow duties. The boot `watch` task, for instance, is a totally general-purpose way to to incremental-anything in boot. The `cljs` task doesn't have a file watcher in it, none of the other tasks do. They don't need it.

Another example is the `cljs-repl` task, which emits ClojureScript code when you start the CLJS REPL. This requires recompiling the JS file and reloading the client. This all happens automatically because the cljs-repl task can call its continuation whenever it likes, so it does that when you start the REPL. This means that your webapp code doesn't contain any REPL connecting code, so you don't have to think about removing it for production builds etc. The REPL connecting code is in there when you use the task, and not when you don't. Very clean.

Another interesting property of tasks is that they accept only keyword arguments. They do not take positional parameters. This means that partial application of tasks is idempotent, and that last-setting wins. For instance, given a function f that takes no positional parameters, we have:

    (-> f (partial :foo "bar")) ==
    (-> f (partial :foo "bar") (partial :foo "bar"))

and

    (-> f (partial :foo "bar")) ==
    (-> f (partial :foo "baz") (partial :foo "bar")).

This is pretty interesting because it gives us a nice way to manage global preferences. We have a macro called task-options! which can be used to globally apply options to tasks:

    (task-options!
      foo [:bar "baz"
           :baf "quux"]
      omg [:hello "world"])

This macro actually does some currying and alter-var-root, replacing the value of the task var (deftask defines a var, of course) with a curried version. A cool thing about this is that the last-setting-wins property means that you can override these settings on the command line or in the REPL:

    (boot (foo :bar "not-baz") (omg))

which would override the :bar option, but not the others.

This is probably long enough, hahaha! I'll hand the mic back to you now :)

[brandonbloom]

> it's really useful just ergonomically to be able to use command line arguments to configure ad-hoc builds

It's largely also what contributes to "works for me" build environments... It's better to have a just one way to do it interface and discourage excessive tinkering with parameters. The more parameters, the more likely for your dev env to be unstable across individual checkouts or developers. I know it's idealistic, but I think we should strive for zero-arg builds, which oddly means not making it easier to configure them.

I'll have to think on all the other stuff you wrote, since it's not totally clear to me yet. I may ping you again after I noodle a bit.

[michaniskin]

Your point about repeatability is valid. As a policy matter we would never advocate building a project without codifying the process as zero-arg tasks in the build.boot file. You can see this in our own projects (eg. https://github.com/tailrecursion/boot-useful/blob/master/bui...). This project has one way to build the project jar file:

    (deftask build-jar
      "Build project jar file."
      []
      (comp (pom) (add-src) (jar) (install)))

When you build the jar file you just do

    boot build-jar

This ensures repeatability, etc.

But we don't only use boot for repeatable builds! Boot is in a unique position in that it's on the intersection of application and environment. That is to say, boot can be used to "bootstrap" the application. With boot we can create sort of self-configuring applications, where the entry point of the application is the build.boot file. This is a very clear win, for example, when running Clojure applications in docker on Elastic Beanstalk, etc. (We'll write that up, too.)

[bsima]

> running Clojure applications in docker on Elastic Beanstalk, etc. (We'll write that up, too.)

Yes. I would love to see a writeup on that.