[geokon]

Sorry for the delay, I've been thinking about how to respond :)

> For me it is.

if you can have a running example I'd love to see it

> I want some good guidelines

Yeah, I totally understand. I don't have a very clear mental model myself as of yet.

> try to pack related entities into their own nested collections

I'd try to just be wary of trying to map OO concepts to Pathom. I get the impulse, but my gut feeling is that this won't be fruitful or will make things muddled.

> you can make bridges between that resolver

tbh I haven't had to really use bridges. The few times I did, I then rearchitected till they disappeared. I do sometimes have different ns's using keys from each other though .. which feels wrong. No clear picture here yet.

My mental model at the moment is a bit fuzzy but goes like this:

Pathom is fundamentally a web of resolvers. The resolvers are best thought of as functions that can only be run once. This is the fundamental constraint you're working under - but you get a bunch of benefits - and I wouldn't try to fight this fundamental paradigm

1.

How do we deal with resolvers that conceptually have the same I/O keys? Say we have two resolvers:

    :filename -> [readFileJPG] -> :image
    :filename -> [readFilePNG] -> :image

(I'm going to use this much simpler example b/c it's sufficient to illustrate the problems)

You have two choices/branches. Pathom can't make that choice for you:

In your systems either the provider has to decide:

    :filenameJPG -> [readFileJPG] -> :image
    :filenamePNG -> [readFilePNG] -> :image

.. or the consumer has to decide:

    :filename -> [readFileJPG] -> :imageJPG
    :filename -> [readFilePNG] -> :imagePNG

I think this is the part where you really have to think hard about who knows which branch should be taken. Who knows you need a fat db request or a skinny one. Is that the part providing the :id and whatnot, or is it the one receiving the result of the db query?

But bear in mind that Pathom is a "compile time" tool in a sense. If it's something determined from a db query.. then it's not part of the program control flow and needs to be decide at "runtime". It's not a Pathom problem! It has to happen inside the resolver code and it can't be part of the web of resolvers.

In that previous :fat-pack example, the equivalents would been :id-FAT (provider) or :customers/phone_number-FAT (consumer)

2.

Okay, but say I have readFileJPG and readFilePNG provided by some library/module/ns. I can't modify it's I/O keys. Then hopefully the keys are namespaced:

    ::jpg/filename -> [jpg/readFile] -> ::jpg/imageJPG
    ::png/filename -> [png/readFile] -> ::png/imagePNG

Now you can bridge keys either on one side or the other, effectively re-creating either a provider or consumer driver flow. But you can't bridge both sides, cus then you're back to the original problem

3.

Okay, but say I have a pipeline that is filetype-agnostic and for some reason :filename are coming in from "outside" (maybe user is typing it into a text box) and the output really just wants :image keys for some subsequent processing. What are your options?

- let the provider/upstream decide: The :filename is coming in from somewhere, but it know the file type a priori. So it can supply a flag that the resolvers require - ex: :is-jpg , :is-png

    ::filename + :is-jpg -> [jpg/readFile] -> :image
    ::filename + :is-png -> [png/readFile] -> :image

- let the consumer/downstream decide: nested outputs {:jpg [:image]}

    ::filename -> [jpg/readFile] -> {:jpg [:image]}
    ::filename -> [png/readFile] -> {:png [:image]}

Both options leave a bad taste the mouth.

- The first feels fragile (you live in fear of it taking a wrong branch), esp if you have "default no-flag" case like in your db request example.

- The second creates a packed thing that needs to be unpacked. (this is what I showed with :fat-pack). I actually think this is the worse solution b/c it breaks composability. The packed space is completely isolated from the wider soup of keys, which can severely limit what you can do (as I'll illustrate).

EDIT: Part 6 I show the better solution for the consumer

4.

The second solution also conceptually clashes with another tool in the toolbelt. Nested inputs and outputs have a fundamental, completely unrelated, different usecase. They allow you to create a mini-context in which you can re-execute your Pathom resolvers. So if you have two images in your application, you can read them both in. These two transformation can co-exist:

    {:profile-pic [:filename]} => {:profile-pic [:image]}
    {:background-pic [:filename]} => {:background-pic [:image]}

You can of course create mini-contexts arbitrarily if you want. The downside of these nested mini-contexts is that they are aggressively isolated and completely oblivious of the larger world, so you need to shove in all the keys you may use. If you have a resolver somewhere else that processes the image:

    :image + :username -> [addUsernameOverlay] -> :image-with-username

Then you'll need to jam all the necessary keys in to the mini-context:

     {:profile-pic [:filename :username]} => {:profile-pic [:image-with-username]}

5.

So for the consumer side solution:

    ::filename -> [jpg/readFile] -> {:jpg [:image]}

You're a bit screwed. You've lost the natural extensibility of Pathom. You now CANNOT do

    ::filename + ::username => :image-with-username [XXXX - doesn't work!]

Nor can you do

    ::filename + ::username => {:jpg [:image-with-username]}

One hack is to have jpg/readFile pack the :jpg container (ie. forward the :username into it)

    ::filename + :username -> [jpg/readFile] -> {:jpg [:image + :username]}

This makes the second call work, but this is a bit gross.. B/c now the jpg/readFile resolver requires a :username that it actually doesn't use. The resolver is now coupled to downstream requirements. Eww

6.

After chewing it over, I think the best solution is to pack everything when possible:

    {:jpg [:filename]} -> [jpg/readfile] -> {:jpg [:image]}

Now the jpg/readfile resolver doesn't take a :filename, but instead takes a :jpg as {:jpg [:filename]} and unpacks it. Here.. I think technically only consumer must know the type.. But now you can add the username transparently and this transformation works without jpg/readfile needing to forward anything.

    {:jpg [:filename + :username]} -> {:jpg [:image-with-username]}

The downside is that you may end up having to shove in a lot of variables in to the mini-context. If it's all using resolvers from a library/ns that are kinda doing an isolated set of stuff then this can be very viable though!

When to create these mini-contexts is not something I have a clear mental model for. It's related to the fundamental downstream output