[aphyr]

I'd like to contrast this: if your application is heavily dependent on state, especially state which is shared between threads, Clojure offers a concise, safe, and uniform way to understand and organize state transformations--at the cost of performance. Where fast mutability is required, it's easy to drop down to explicit locks, atomics, java.util.concurrent collections, et al.

I find the majority of my time in writing stateful programs goes towards reasoning about safety, and having access to STM lets me write correct code with less thinking. Then I focus on optimizing the parts where performance is critical.

[billsix]

At ILC 2009, Guy Steele asked Rich Hickey how Clojure's STM interacted with I/O. To which, Rich replied that it doesn't. No further discussion continued on the subject.

I have found that this rarely enters discussions regarding Clojure, although the STM is frequently lauded. GHC's STM also does not work with the IOMonad, which is clear because of the types, and is articulated in books such as such as Real World Haskell.

http://book.realworldhaskell.org/read/software-transactional...

Are people really getting benefits from transactions which do not involve I/O?

[aphyr]

Riemann is a giant ball of mutable state and IO, and makes extensive use of the STM: https://github.com/aphyr/riemann/blob/master/src/riemann/str...

The critical thing, as you observed, is to separate IO from retryable STM transactions with a barrier; e.g. by receiving data, doing an idempotent computation inside dosync or swap!, and then emitting results. Haskell has an advantage in that this separation is provable by the type system, whereas in Clojure you need to remember.

A contrived example:

  (let [pizza (get-from-fridge)
        meal  (dosync
                (alter eaten-foods conj pizza)
                (deref eaten-foods))]
    (tell-friend "So far I ate" meal))

where get-from-fridge and tell-friend are IO operations, and our list of eaten foods is mutated by pure functions.

In practice I don't find this particularly limiting: dosync and swap! are almost always short operations for performance reasons anyway. Then you return a consistent snapshot of the updated state from dosync--where multiple pieces of state are involved, you can use vectors or maps along with destructuring bind. Sometimes it's a tad unwieldy, but typically much shorter than the equivalent mutex dance.

There are other aspects of Clojure's concurrency libraries, like agents, futures, and promises, which are useful in IO. Specifically, agents give you asynchronous serializability, futures give you asynchronous concurrency, and promises allow for synchronous handoff of delayed values. Those are quite useful when working with IO, though for heavy lifting you may be better off using explicit queues and worker pools from java.util.concurrent.

Sometimes I think of Clojure (as an imperative language) as the dual of Haskell's lazy evaluation model. Clojure uses futures, promises, and lazy sequences to provide explicit laziness, where Haskell uses the IO monad to provide explicit ordering. Both have an STM for serializable, atomic mutability between threads.

[jrheard]

That's odd, that seems a little inaccurate to me. Clojure's agents are useful for encapsulating I/O-related resources, and are integrated with the STM system. Per http://clojure.org/agents -

    Agents are integrated with the STM - any dispatches made in a transaction
    are held until it commits, and are discarded if it is retried or aborted.

[craigching]

I'm a clojure newb myself, but I thought STM transactions weren't allowed to have side-effects and IO would be a side-effect, no?

[jrheard]

Refs and atoms shouldn't have side effects, but agents are great for performing side effects. From page 214 of Clojure Programming:

    Unlike refs and atoms, it is perfectly safe to use agents to coordinate 
    I/O and perform other blocking operations. This makes them a vital
    piece of any complete application that use refs and Clojure’s STM
    to maintain program state over time. Further, thanks to their semantics,
    agents are often an ideal construct for simplifying asynchronous
    processing involving I/O even if refs are not involved at all.

Further:

    Agents are integrated into Clojure’s STM implementation such that
    actions dispatched using send and send-off from within the scope
    of a transaction will be held in reserve until that transaction
    is successfully committed. This means that, even if a transaction
    retries 100 times, a sent action is only dispatched once, and that
    all of the actions sent during the course of a transaction’s
    runtime will be queued at once after the transaction commits.