[kazinator]

TXR Lisp also fails this test:

  1> (len
       (with-stream (s (open-file "/usr/share/dict/words"))
         (get-lines s)))
  ** error reading #<file-stream /usr/share/dict/words b7ad7270>: file closed
  ** during evaluation of form (len (let ((s (open-file "/usr/share/dict/words")))
                                      (unwind-protect
                                        (get-lines s)
                                        (close-stream s))))
  ** ... an expansion of (len (with-stream
                                (s (open-file "/usr/share/dict/words"))
                                (get-lines s)))
  ** which is located at expr-1:1

The built-in solution is that when you create a lazy list which reads lines from a stream, that lazy list takes care of closing the stream when it is done.

If the lazy list isn't processed to the end, then the stream semantically leaks; it has to be cleaned up by the garbage collector when the lazy list becomes unreachable.

We can see with strace that the stream is closed:

  $ strace txr -p '(flow "/usr/share/dict/words" open-file get-lines len)'
  [...]read(3, "d\nwrapper\nwrapper's\nwrappers\nwra"..., 4096) = 4096
  read(3, "zigzags\nzilch\nzilch's\nzillion\nzi"..., 4096) = 826
  read(3, "", 4096)                       = 0
  close(3)                                = 0
  fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
  write(1, "102305\n", 7102305
  )                 = 7
  exit_group(0)                           = ?
  +++ exited with 0 +++

It is possible to address the error issue with reference counting. Suppose that we define a stream with a reference count, such that it has to be closed that many times before the underlying file descriptor is closed.

I programmed a proof of concept of this today. (I ran into a small issue in the language run-time that I fixed; the close-stream function calls the underlying method and then caches the result, preventing the solution from working.)

  (defstruct refcount-close stream-wrap
    stream
    (count 1)

    (:method close (me throw-on-error-p)
      (put-line `close called on @me`)
      (when (plusp me.count)
        (if (zerop (dec me.count))
          (close-stream me.stream throw-on-error-p)))))

  (flow
    (with-stream (s (make-struct-delegate-stream
                      (new refcount-close
                           count 2
                           stream (open-file "/usr/share/dict/words"))))
      (get-lines s))
    len
    prinl)

With my small fix in stream.c (already merged, going into Version 292), the output is:

  $ ./txr lazy2.tl
  close called on #S(refcount-close stream #<file-stream /usr/share/dict/words b7aecee0> count 2)
  close called on #S(refcount-close stream #<file-stream /usr/share/dict/words b7aecee0> count 1)
  102305

One close comes from the with-stream macro, the other from the lazy list hitting EOF when its length is being calculated.

Without the fix, I don't get the second call; the code works, but the descriptor isn't closed:

  $ txr lazy2.tl
  close called on #S(refcount-close stream #<file-stream /usr/share/dict/words b7b70f10> count 2)
  102305

In the former we see the call to close in strace; in the latter we don't.