[vincentdm]

Can someone explain the benefits of Datomic Ions? What problems is it supposed to solve and how does it compare to existing solutions?

Also, what is the story regarding local development? (I also have this concern about AWS Lambda; it seems you can only realistically run code in the actual cloud environment)

We are investing heavily in the Clojure-ecosystem (but not yet Datomic). It seems Cognitect is strongly headed into a direction which involves some kind of holistic vision about a new stack, but they don't seem to explicitly communicate this vision anywhere.

Edit: I asked the above question also on Clojure Reddit and got an interesting response: https://www.reddit.com/r/Clojure/comments/8p3d5s/datomic_ion...

[jeremiahwv]

My take is that one of Datomic's original value props was that it ran in your Java app's process. That plus immutability plus cache-foo allowed you to write your Java app as-if your entire db was not only "in-memory", but "in-your-app's-process-memory." That is, um, "pretty dope".

Then they introduced Datomic Client, which is more of the "normal" db paradigm. Your app includes the client lib, which connects to the running db process "somewhere else". I don't know why they did this (easier to include?), but it meant that the "db-in-your-app-process" value prop went away.

Initially when they deployed Datamic Cloud (a sort of managed Datamic service deployed in AWS), they only supported the client model.

That is context for your question. The problem Ions solves is how to get that "db-in-your-app-process" value prop while using Datamic Cloud. It seems to me that it effectively reverses the original method. Original method was add the DB as a lib into your app, Ions method is push your app into the db (which is running in a configured-for-you cluster on AWS).

(It also allows you to not have to provision/configure servers to actually run your app code in addition to the db servers, but that seems secondary to me.)

Note, that I'm not affiliated with Cognitect, and I've never used Datamic, or Clojure in a production system. So have a salt shaker.

[derefr]

> I don't know why they did this (easier to include?)

I wanted to use Datomic, but none of the languages I work in run on the JVM. Datomic Client is really the only way I could. (Though I also considered, briefly, reimplementing the entire "transactor" library for this other platform, since the transactor living "inside" your process really is a selling point.)

[hellofunk]

What is the diff between “in memory” and “in-your-app's-process-memory”?

[derefr]

An "in memory" DB is just a DB that can answer queries quickly (e.g. in O(1) time.) You still speak to it over a network protocol (even if it's going over the loopback interface), where everything gets serialized into long binary strings and then deserialized again. You can maybe achieve zero-copy access into the packets the DB is sending you if the DB is communicating using something like Capn Proto, but that's still two copies of the data: one in your app process, and one in the DB process.

A database whose data is in your process (like, say, BDB, or Erlang's ETS, and not like SQLite†) allows you to hold pointers to data (usually either "tuples" or "objects") that are in the DB, from your process, and treat them as if they were just regular data sitting on the stack/heap. Depending on the DB's persistence architecture, this can mean that you can even mutate the data in object form, and that mutation will be automatically persisted—because what you're mutating is, after all, the DB's copy of the data. Unlike e.g. Firebase, there's no secondary synchronization between "your local" copy of an object and "the database's copy" of the object; your process is holding the database's copy of the object.

Or, if you like, you can think of an "in-process-memory" database like a server in a distributed-RPC stack, ala CORBA or Windows COM. You ask the DB for an object, it gives you an RPC handle to the object. Except, since you and the COM server share memory, this handle doesn't need to do any marshalling or IPC or locking; it's an optimized-path "direct" handle.

Another potential path to enlightenment: if you create a "flat-file database" by just mmap(2)ing a file—and then you write structs into that file-backed memory, and keep pointers to those structs—then you've got an "in-process-memory database."

† SQLite the library resides in your process's memory, and it can even allocate tables that reside only in-memory (which is also "your process's memory") but SQLite still communicates with your process through the SQL protocol, as if it was on the other side of a socket to your process, with no way to "get at" SQLite's internal representation of its data.

[jonahbenton]

Datomic Ions are a cloud-native version of stored procedures (in Datomic terms). I have not fiddled with Ion but the local dev story with Datomic is fine. One does not need cloud connectivity for local work, and ordinary functions can be run locally in the write path in Datomic. I would guess/assume that these same functions can be deployable as is in Ion- modulo the CI/dependency machinery required for cloud deployments.

[rockostrich]

I can't speak for Datatomic Ions and Clojure specifically, but there's a bunch of tooling options around "running" AWS Lambda locally. The most popular last time I checked was Serverless (https://serverless.com/)

[scarface74]

Why do you say that you can only run lambdas in the environment? A Lambda function just calls a function with an event and a context parameter.

I run and test my Lambda methods locally just like I do my Controller classes - create "unit tests" (not real unit tests just a method that calls the Lambda handler with the event body I want to use). Your Lambda handler should be skinny and just translate the message to your business methods just like a controller.

[vincentdm]

Yes, I understand that, but it doesn't account for all the moving parts of the Lambda service (like container re-use, warmup time, deployment steps...) that have no local equivalent AFAIK. But maybe I'm overestimating their importance...

[eikenberry]

You can replicate some of the live-like environment using SAM[1] and SAM-local[2]. Combined with services like ngrok[3] you can even test remote hooks and such.

[1] https://github.com/awslabs/serverless-application-model [2] https://github.com/awslabs/aws-sam-cli [3] https://ngrok.com/ or http://serveo.net/

[scarface74]

Deployment to a production environment is usually different than running locally. I have a CloudFormation script to handle deployments.

Well, I also have a yml file that builds my Lambda package with CodeBuid since I develop on Windows and Python packages with binary dependencies don't work on Linux based lambdas.

The CodeBuild step will package the Linux versions of the packages.