[haberman]

> Say you want to resume continuation K, which has a stack of some size N.

The size of the continuation's stack doesn't matter for the problem I described, it's the size of the stack "underneath" your continuation that matters (ie. C(1) and C(2) above).

If C(2) > C(1) there is no way to shrink C(2) such that the continuation's stack can be copied into the right place.

> I like doing unacceptable things in my programs. It's the best part of programming, really.

What you do in your programs is up to you! But nobody else is going to use a C library that messes with the execution state of its callers (unless that is the point of the library, which it isn't with Lua).

[sillysaurus3]

I don't understand, but I'd like to.

To create a continuation, we need to copy the entire stack, by definition. But "the stack" is just an array of bytes. It's all the bytes between the current stack pointer and the "root" stack frame. So to create a continuation, copy these bytes and stash them somewhere, then set up a longjmp target to the current instruction.

To apply a continuation, i.e. to restore the stack, we overwrite the current stack starting from the root frame. Then we longjmp to where the continuation was originally created.

It seems like this scheme should work in any situation, but perhaps I'm missing something?

loeg pointed out getcontext(3) / setcontext(3), which seems promising. It looks like a standard way to sidestep all of this bookkeeping. It appears to be a high-level interface to the operations described above.

Lua is just a language, though. It's not "for" anything in particular.

[haberman]

The Lua implementation is a C library. You invoke it by calling C functions like lua_call().

Imagine you have a C program like this:

    #include <fancylib.h>

    int main() {
      for (int i = 0; i < 10; i++) {
        printf("val[%d] = %d\n", i, fancylib_calculate(i));
      }
      fancylib_cleanup();
    }

Now imagine that internally, fancylib uses Lua. So fancylib_calculate() calls lua_call().

Now imagine that the Lua function run by fancylib decides to use continuations. When you call fancylib_calculate(0), it creates a continuation. And when you call fancylib_calculate(1), it decides to call the continuation.

If you restore the entire C stack to resume the Lua continuation, it will reset the loop in main() to i=0! Your program might end up printing val[0] over and over, in an infinite loop. This would be extremely surprising to you as the author of main(), because you were just trying to write a normal old for() loop. The Lua continuation should just restore the Lua-related stack, not the stack of the functions calling Lua!

[sillysaurus3]

If you restore the entire C stack to resume the Lua continuation, it will reset the loop in main() to i=0!

That's the point of continuations, though. That's a feature, not a bug. When you create a continuation, you're saying "whatever happens after this, allow me to do it again at some later time." If the calling library happened to be in a loop, then the goal is to serialize that loop so that it can be invoked again, at a later point.

If you keep applying the continuation in a loop, then you'll get an infinite loop. But if you invoke the continuation once, (and if subsequent calls to fancylib_calculate() don't), then you'll get the ability to print

  val[0] = 42
  val[1] = 99
  ...
  val[9] = 7

on demand. By invoking the continuation, you cause the loop to happen again.

In fact, you gain the ability to prevent the program from terminating, in a controlled fashion. Since you have access to the continuation, you can choose to invoke it on the 10th call to fancylib_calculate(), up to 3 times in a row. That would produce output like:

  val[0] = 42
  val[1] = 99
  ...
  val[9] = 7
  val[0] = 42
  val[1] = 99
  ...
  val[9] = 7
  val[0] = 42
  val[1] = 99
  ...
  val[9] = 7

then the program would exit.

Does that make sense? Apologies if we're talking past each other. I appreciate the patience.

[haberman]

> That's the point of continuations, though. That's a feature, not a bug.

If that's what you're after, then by all means implement that. :) But I think most people would expect the Lua interpreter state to be self-contained, and not to affect the state of the surrounding C execution environment.

[a_t48]

The lua library can already affect the surrounding C state (either through FFI with LuaJIT, or having a lua function call back into C code), so I don't see that as a real argument.

Unless you are allowing arbitrary code execution, you are likely the one passing in scripts to the interpreter - these sorts of interactions should be well documented by the lua scripts themselves...and I don't recommend allowing arbitrary lua anyhow - os.syscall and friends say hi - need to block access to those carefully. Thus, we have two situations - either you know and trust the code not to do unexpected things (or to do them in an expected manner :) ), or you've set up your lua environment in such a way to disallow such calls, and it doesn't matter.

[daurnimator]

When using lua_pcall (in 5.2 and above) you can provide a continuation function. Would that solve the dilemma?