With CL if you have a bug on your web server you can just connect a repl into the running server, inspect the state of the system, modify that state and update code on the fly if necessary.
I guess the key here for me would be inspecting the state of the system. I tried learning common lisp a few times and it never clicked.
When clojure came around, I tried lisping again, and for whatever reason i was ready for it, and clojure is my goto language.
That said, I'd never connect to a repl on a server and do any kind of bug fixing unless it was really dire. It's too easy to get the run state out of sync with the text source files, and I don't see why this would be different with common lisp.
I think the repl is great for development and figuring things out, but the whole idea of "program as state" seems antithetical to the ideas of functional programming. With the main issue for me being that I have to somehow keep track of this program state in my head once I start tinkering with it.
For example, say I find a function has a bug and I go into the repl on the server and fix it, in situ. Say I spot some other issues, or had them on my list of things to fix when I had time... I start modifying functions here or there. Now my "program state" no longer matches my source code, and I have to remember to fix it so that it matches. That's error prone. I could enforce my own methodology of fixing the source and copy-paste to the repl or send it in whatever way, still error prone.
I have to remember what changes I made, to what functions, and if I iterated, now I have to remember which iteration was final. Multiply that by any and all functions that are touched. It's a lot to keep track of.
So when you say "inspect the state," I have to think common lisp must have some feature for state inspection that is missing in clojure. I know there are a lot of features of common lisp, especially in stack-tracing, that clojure has no parity to. But I'm curious if common lisp has some way of completing a "round trip edit" such that an edit/correction made on a server process, in situ, are able to be easily propagated back to your master source?
> But I'm curious if common lisp has some way of completing a "round trip edit" such that an edit/correction made on a server process, in situ, are able to be easily propagated back to your master source?
In SLIME you just connect to the Lisp process, open up your versioned-controlled file, edit it, hit C-c C-c for slime-compile-defun or C-c C-k for slime-compile-and-load-file. Isn't that how CIDER works also?
True. If you are going purely functional I guess you wouldn't have much state to inspect. Plus in a stateless web server you generally want to avoid keeping state around on the server.
I remember one time, it was actually with Clojure, I had to deal with Paypal. Since paypal only sent results back to the server and not to localhost I couldn't debug the problem locally. So I connected to the server, bunged all the debug information into an atom and made a few requests. Then I could inspect the atom, make some changes and find out what the problem was and fix it right away. It took just a few minutes.
At about the same time I was also working on a C# server. There was a bug there that could only reproduce itself on the server. To fix that bug I had to put a load of logging into the source. Check the file into source control. Wait for the CI system to build and deploy. Then kick off some requests. Inspect the log files. Make some changes. Check in.. wait.. etc.. It took days to fix that bug.
Admittedly the Clojure project was mine on my own server and the C# project was on some corporate enterprise behemoth so there were other factors that hindered that project, but it did highlight to me how much quicker you can get stuff done when you have that instant feedback.
When I say inspect the state, generally I just mean inspecting the values of global variables. In CL it is possible to put breakpoints in the source and inspect all the variables up the stack. Mind you, this can be a bit hairy on the server if there are multiple people connected as there is no guarantee as to whose Slime connection will pick up the breakpoint.
The round trip edit is pretty similar to Clojure. You would generally have your source file open in Emacs, change the source code and press Ctrl+CC with the cursor in your function definition to apply your edits to the server. Then you save the file and check it in to source control. You don't have to type directly into the repl.
You don't have to work like this if you don't want to. Without mentioning any names, it has been known for the server image to be completely out of sync with the state of the source files to the point where there weren't really any source files. The work around was to just treat the server image as the master source of truth. Luckily in CL you can just dump the image to create a new executable with its current state. So for several years this one server existed only as a constantly updated binary image. I wouldn't recommend working like that, but it has been done.
Depending on your implementation - I know Clozure CL at least allows this - you can extract the source code out of the running image. So it is certainly possible to propogate back to master source. It was eventually how this one server was restored to a more orthodox source code tree..
Thanks for the reply. It seems a bit fragile and convoluted to me, but I suppose if you are used to it and are comfortable with the pitfalls it would be fine.
Yes, working directly on the server is definately fragile and fraught with all kinds of dangers. I try to avoid it as much as possible, but it is very handy having the option there as a last resort.
If you wanted, you could have any bug in a running web server pop up a window in your editor, fix the bug within the call that's buggy, then resume execution from that point at the call stack, successfully returning results to the client. Doing so doesn't necessarily make a lot of sense, but it's a very powerful technique if you choose to use it.
Not quite. In Lisp, when you get an error, the system pauses and lets you resolve the issue before resuming the program. Sometimes this even means typing in a function really quickly before a request times out (which isn't such an insane suggestion in the context of development). In Clojure, errors like that unwind the stack and tear everything down.
Reminds me of Paul Graham's essay about how at Viaweb, they sometimes used to fix bugs (in the Lisp code) while the customers who reported them were still on the phone, and then the support staff would tell them to check whether they were sure it was a bug, by trying again - and since the bug was now fixed in the server, when they did that operation again, it would work.
I think it was in his Beating the Averages essay - the one in which he talks a lot about the advantages Lisp gave them at Viaweb, over the competition.
I think I had not described it fully (since had read it long ago). Searched for the article just now. The Beating the Averages article [1] does not have the point I mentioned. But a more detailed article [2] linked from it, does:
Lisp's interactive toplevel is a great help in developing software
rapidly. But the biggest advantage for us was probably in finding
bugs. As I mentioned before, with Web-based applications you have
the users' data on your servers and can usually reproduce bugs.
When one of the customer support people came to me with a report
of a bug in the editor, I would load the code into the Lisp
interpreter and log into the user's account. If I was able to
reproduce the bug I'd get an actual break loop, telling me exactly
what was going wrong. Often I could fix the code and release a
fix right away. And when I say right away, I mean while the user
was still on the phone.
Such fast turnaround on bug fixes put us into an impossibly tempting
position. If we could catch and fix a bug while the user was still
on the phone, it was very tempting for us to give the user the
impression that they were imagining it. And so we sometimes (to
their delight) had the customer support people tell the user to
just try logging in again and see if they still had the problem.
And of course when the user logged back in they'd get the newly
released version of the software with the bug fixed, and everything
would work fine. I realize this was a bit sneaky of us, but it
was also a lot of fun. ]
Incidentally, I seem to remember the following: some years ago, after playing around some with Lisp (Allegro CL from Franz, and the Lisp IDE from Lispworks, both of which supported this feature or something like it), I was trying out a then-current version of Visual Studio, and saw that it had a new feature called Edit-and-Continue, which I thought was similar to the Lisp feature.
When clojure came around, I tried lisping again, and for whatever reason i was ready for it, and clojure is my goto language.
That said, I'd never connect to a repl on a server and do any kind of bug fixing unless it was really dire. It's too easy to get the run state out of sync with the text source files, and I don't see why this would be different with common lisp.
I think the repl is great for development and figuring things out, but the whole idea of "program as state" seems antithetical to the ideas of functional programming. With the main issue for me being that I have to somehow keep track of this program state in my head once I start tinkering with it.
For example, say I find a function has a bug and I go into the repl on the server and fix it, in situ. Say I spot some other issues, or had them on my list of things to fix when I had time... I start modifying functions here or there. Now my "program state" no longer matches my source code, and I have to remember to fix it so that it matches. That's error prone. I could enforce my own methodology of fixing the source and copy-paste to the repl or send it in whatever way, still error prone.
I have to remember what changes I made, to what functions, and if I iterated, now I have to remember which iteration was final. Multiply that by any and all functions that are touched. It's a lot to keep track of.
So when you say "inspect the state," I have to think common lisp must have some feature for state inspection that is missing in clojure. I know there are a lot of features of common lisp, especially in stack-tracing, that clojure has no parity to. But I'm curious if common lisp has some way of completing a "round trip edit" such that an edit/correction made on a server process, in situ, are able to be easily propagated back to your master source?
If you read all that, thanks :)