[monsieurbanana]

I can't blame you for thinking it was a GPT reply these days. Anyway I'm also not GGP but he's right, it is Hot Module Replacement.

You don't really need specific tools for HMR in clojure, it's kind of baked in the language thanks to:

- immutability (not impossible but hard to do HMR without it)

- REPL workflow

When developing on clojure, there's a long-lived clojure process and you continually send expressions to be evaluated there.

Let's say you start a server, you make a change to one of the functions that handle a route, just after making the change you will evaluate that function (from your editor) and the update will be "live" in the running server.

It's honestly hard to describe and it doesn't help that we call that "repl-based workflow" when in fact we are not using the REPL directly, but rather through plugins available for various editors. People might think it's the same as using nodejs or python repl, when it's nothing like that. If you're interested, you could check "Parens of the Death" or look for "Clojure CALVA" (VSCode's clojure plugin) videos on youtube.

https://www.parens-of-the-dead.com/s2e1.html

Edit: It is also not limited to development. Nothing prevents you from hooking into the clojure process in your prod server from your editor, editing a function and evaluating it.

You just monkey-patched a live server to fix a time-critical bug with roughly the same workflow as editing your code locally. Or brought production down, 50/50, but what's life without some risks eh?

[beepbooptheory]

I have a lot more experience with common lisp, but currently doing AOC with Clojure right now to try and position myself better for my dream job where I can work with any kind of lisp. Thought I could maybe ask a quick question I have been having trouble answering? I find that with emacs/cider any evaluation is blocking. I am a little spoiled on SLIME/SLY, where it has the nice baked-in feature that any evaluation you do interactively becomes its own process and you can carry on evaluating other things if the process is long running. Is there any way I can replicate this in clojure? Is it just a matter of using `future` explicitly? Or is there something I am missing with cider?

[Volundr]

If you want a long running calculation to not block the REPL you'd need to use a future, or some other method to get it off the REPL thread.

I don't remember what your describing being a feature of SLIME, but it's been 10+ years so I may just not remember.

[monsieurbanana]

I don't think you are missing anything, I am not aware of anything that would automatically launch new threads when evaluating something.

That does seem like an interesting, although in practice I rarely have to evaluate something that would take more than a a couple of seconds, so manually wrapping it in a future seems acceptable.

On a semi-related note, sometimes I mistakenly evaluate something that will take a long time, or there's a bug and I'm stuck in an infinite loop. CIDER has a command to interrupt the on-going evaluation but I'm my experience it only works about half the time. In those cases having it wrapped in a future would certainly be helpful.

[38529977thrw]

Just for the record, in Java (JVM) this is done (routinely circa first 'bean containers' of ancient history) via interfaces and class loaders. Using Java I can have n distinct versions of a class each implementing the same interface in the same process as long it is the interface reference that is passed around.

> immutability

Make a case for immutability being so critical to HMR. How does an immutable data allow for swapping of code.

[groestl]

AFAIK, Java's native HMR is not built on dynamic classloaders, but using a separate technology built into the JVMs debugger support, called HotSwap. You can only replace (certain) function bodies with it. To get around these restrictions, more elaborate HMR can be achieved with classloader magic, such as OSGi, but you subscribe to a world of ClassCastException-pain with it.

[kaba0]

It is still a very good system, and you can get quite a far away with only method-swaps, I love to use them in case of Spring apps, where changing a backend controller and doing another api call is a very fast REPL cycle basically.

But objects don’t give themselves very easily to hot reloading, there are a bunch of edge cases that are not well-defined in a semi-swapped state (e.g. this new field always gets a value in the constructor, but was no just added. Should you rerun the constructor, or have it be in an inconsistent state?)

Immutable/FP concepts are a much better fit, especially if the building blocks are as small as they are in case of Clojure.

[everforward]

My limited understanding of Clojure's HMR is that there aren't n distinct versions of a class/function. It's actually removing the old ones from memory and replacing them.

Also, it is worth noting that everything is immutable, not just data. Functions/protocols/records/etc are also immutable.

Functions/classes being immutable makes it much, much easier to reason about the dependencies of that function/class. E.g. check out this Python code:

    import someotherpackage
    import threading
    import random

    class MyLogger:
        def __init__(self, name, path):
            self.path = path
            self.name = name
        def log(self, message):
            with open(self.path, 'w') as fd:
                fd.write(message)

    my_logger = MyLogger("init-logger", "init.log")
    someotherpackage.logger = my_logger
    someotherpackage.logger.error = print

    print = someotherpackage.logger.error
    my_logger.thingy = "thingy"

    def change_logger():
        def actually_do_change():
            time.sleep(random.randint(1, 100000))
            new_logger = MyLogger("main-logger", "main.log")
            someotherpackage.logger = new_logger
            someotherpackage.logger.error = my_logger.error
        time.sleep(random.randint(1, 10000))
        threading.Thread(target=actually_do_change).start()

    time.sleep(random.randint(1, 10))
    threading.Thread(target=change_logger).start()

Now imagine that this isn't our `main`, this is some library that's a transitive dependency of something we're importing.

Trying to hot-swap this would be awful. The big issues related to mutations are that a) there's no indication in someotherpackage that it's behavior depends on this package, and b) because mutations are allowed, the order that things are reloaded in matters, and c) some mutations are time-bounded.

If I change MyLogger and the VM/interpreter wants to hot-swap MyLogger, it has to recognize that it can't just re-load the instances of MyLogger with the same state. It has to re-load those instances, then re-mutate them, then re-mutate someotherpackage, and finally re-mutate the print function. It might also need to do the stuff that's delayed by that thread. Maybe. Depends on a couple of random integers that probably got GCed a while ago.

If you want to add an extreme layer of annoying, consider that this package could be a late import and only happens if a plugin is enabled, so the earlier code might rely on `print` actually being `print` or on someotherpackage.logger behavior that changes over time.

None of that applies in an immutable world. Nothing can mutate the someotherpackage package nor the `print` function, ergo nothing can depend on an earlier or different version of them. Dependencies are easy to track because the only way to introduce them is to import/directly reference them, or have them passed as parameters.

I don't know that it even needs any dependency tracking, though (provided they're using a pointer to a pointer, or maybe something smarter than that I can't think of). Immutability means that a) everything can be passed as pointers safely, b) those pointers can't be modified by the code, and c) there is never a reason to have more than one copy of a pointer to the bytecode for a function.

Each function gets a pointer to its bytecode, let's call that p1. Every reference to that function is a pointer to p1, which we'll call p2.

When you want to hot-swap code the language pauses the VM/interpreter, recompiles any function with a diff (easy with an AST), puts the new functions in memory somewhere new, changes p1 to point to the new bytecode, then marks the old bytecode for GC.

If you tried something that simple in Python for the code I wrote above, it would explode. It's an infinitely harder problem.

> Using Java I can have n distinct versions of a class each implementing the same interface in the same process as long it is the interface reference that is passed around.

It's been a while since I worked with those app containers, but from my recollection it shares virtually nothing with hot reloading. E.g. is state preserved between those? If I'm caching stuff in RAM, and I change that class out, does it keep the cache? HMR does, as I recall.

My understanding is that app containers basically just run N versions of your app/class and allow you to choose which one you route execution to.

App containers are more similar to blue-green deployments with a load balancer. Each instance is totally separate, and the load balancer lets you choose which one you route to. App containers just do that process inside the JVM.

That's not bad, but it's also not as good as being able to repeatedly tweak a function without ever clearing caches or re-connect to downstreams or etc.

[roenxi]

> My limited understanding of Clojure's HMR is that there aren't n distinct versions of a class/function. It's actually removing the old ones from memory and replacing them.

It is almost both. Clojure creates n distinct versions of the function (which are in fact objects and subject to garbage collection). The symbol of the function is rebound (technically this language is wrong) to point to the most recent one. Then, usually, the Java garbage collector deletes the old object.

So it is possible (likely under some code styles) that old versions of a function can hang around in a REPL environment if they ended up embedded in a data structure. However, if you make the call by resolving the function's symbol then that will reliably call the most recently def-ed version.

For example:

(defn poor-style [] 1)

(def object {:fn poor-style :call (fn [] (poor-style))})

(defn poor-style [] 2)

(prn ((:fn object))) ; => 1, calls the old embedded function

(prn ((:call object))) ; => 2, calls a function that resolves the symbol, finds the new version.

(prn (poor-style)) ;=> 2, just call the new function