[geokon]

Okay - actual code is good :))

But I can't replicate the behavior. Here is the code:

https://github.com/kxygk/ednless/blob/master/pathom-prectest...

I didn't know what jdbc was so I just plugged in dummy values - but I think I'm matching your logic one2one

Let me know if you can tweak it to have the weird results trigger

At a high level I'd say a couple of things stand out.

- Your resolvers take in an id and return an id.. That sets off a bunch of alarm bells for me. I wouldn't ever do that. There is no good reason to have that, even if the values are identical (if the values change then that's even more dangerous). I can't point to exactly what will go wrong, but you're exacerbating the problem of having multiple resolvers providing an input. Worse yet, in this example you are guaranteed that they are all running. So where is it going to take the ID from..? I don't even know

- The :fat-key isn't doing anything here. If you wanted to do the nested request, it'd be a solid way to guarantee the fat branch is always taken:

    (pco/defresolver get-user-with-customer
      [{:users/keys [id]}]
      {::pco/output [{:fat-pack [:users/id
                                 :users/email
                                 :users/customer_id
                                 :customers/id
                                 :customers/billing_number
                                 :customers/phone_number]}]}
      (println "get-user-with-customer FAT triggered")
      {:fat-pack {:users/id                 66
                  :users/email              "[e-mail]"
                  :users/customer_id        id
                  :customers/id             id
                  :customers/billing_number 666
                  :customers/phone_number   6666}})

    (def env
      (pci/register [get-user
                     get-customer
                     get-user-with-customer
                     user-customer-bridge]))

    (p.eql/process env
                   {:users/id 1000}
                   [{:fat-pack [:customers/phone_number
                                :users/email]}])
    ;;{:fat-pack {:customers/phone_number 6666, :users/email "[e-mail]"}}

I think you can see you get the same results, but there is no way for it to go wrong. Admittedly here I have the EQL request "unpacking" it, but it can also be unpacked transparently from a different resolver using nested inputs.

    (pco/defresolver fat-eater
      [{:keys [fat-pack]}]
      {::pco/input[{:fat-pack [:users/id
                               :users/email
                               :users/customer_id
                               :customers/id
                               :customers/billing_number
                               :customers/phone_number]}]
       ::pco/output [:response]}
      (println "fat-eater triggered")
      {:response (str "yum, just ate: "
                      (:users/id fat-pack))})

    (def env
      (pci/register [get-user
                     get-customer
                     get-user-with-customer
                     user-customer-bridge
                     fat-eater]))

    (p.eql/process env
                   {:users/id 1000}
                   [:response])
    ;; {:response "yum, just ate: 66"}

This is very explicit and about equivalent to your original thought of "why not just have explicit imperative function calls". The consumer (the `fat-eater` or the user making the EQL request) knows a priori that he wants the fat call.

EDIT: At the bottom of the .clj I added a bunch of other tests for the pipeline branching (the A->B->C thing)

- Adding an XXX resolver to override step B worked when "forced" with an extra input key.

- Adding an YYY resolver to override step B by outputting an extra key ends up always overriding B (whether you request the new dummy output key or not!)

I'm guessing here.. but it's probably b/c YYY has the same input requirements as B, but YYY outputs two keys instead of one - so it's preferred.

In a way both seem useful. The first lets you use a flag to select when you take a branch. The second make you always override the branch

[Capricorn2481]

> Your resolvers take in an id and return an id.. That sets off a bunch of alarm bells for me. I wouldn't ever do that. There is no good reason to have that, even if the values are identical (if the values change then that's even more dangerous). I can't point to exactly what will go wrong, but you're exacerbating the problem of having multiple resolvers providing an input. Worse yet, in this example you are guaranteed that they are all running. So where is it going to take the ID from..? I don't even know

You mean don't return :users/id when that's what was passed in? That makes sense, but I don't think I ran into issues with it.

> The :fat-key isn't doing anything here

For me it is. The absence of it means only one of the resolvers is run regardless of data order.

> This is very explicit and about equivalent to your original thought of "why not just have explicit imperative function calls"

I've been playing with this more today. My big thing is I want some good guidelines for writing resolvers to begin with so query time is not an exercise in auditing the whole codebase. Here are some guidelines I played around with, let me know what you think.

- When a resolver gets an entity and related data (like `get-user-with-customer`), try to pack related entities into their own nested collections. I.E. don't return `[:users/email :customer/name]`, return [:users/email {:users/customer [name]}`. This makes it less ambiguous, as putting the customer information flat with the other user data can complicate queries later on. Like if you select a user and their last order, what does `:customer/name` refer to? The customer of the user or the customer of the order?

- If a resolver is really expecting to operate on a users customer id, make that explicit in the input with `{:users/customer [id]}`.

- If you want a resolver to stay "open" and not specify where its inputs come from (just takes `:customer/id`), you can make bridges between that resolver and the entity resolvers that translate to the right keys (A bridge that turns `{:users/customer_id}` into `:get-billing-method/customer_id)

- Careful seeding multiple entities. If you feed your query the params `{:users/id 16230 :admins/id 9}`, it's not clear what the requested data is applying to (is `:roles-and-permissions` applying to the admin or the regular user?)

[geokon]

Sorry for the delay, I've been thinking about how to respond :)

> For me it is.

if you can have a running example I'd love to see it

> I want some good guidelines

Yeah, I totally understand. I don't have a very clear mental model myself as of yet.

> try to pack related entities into their own nested collections

I'd try to just be wary of trying to map OO concepts to Pathom. I get the impulse, but my gut feeling is that this won't be fruitful or will make things muddled.

> you can make bridges between that resolver

tbh I haven't had to really use bridges. The few times I did, I then rearchitected till they disappeared. I do sometimes have different ns's using keys from each other though .. which feels wrong. No clear picture here yet.

My mental model at the moment is a bit fuzzy but goes like this:

Pathom is fundamentally a web of resolvers. The resolvers are best thought of as functions that can only be run once. This is the fundamental constraint you're working under - but you get a bunch of benefits - and I wouldn't try to fight this fundamental paradigm

1.

How do we deal with resolvers that conceptually have the same I/O keys? Say we have two resolvers:

    :filename -> [readFileJPG] -> :image
    :filename -> [readFilePNG] -> :image

(I'm going to use this much simpler example b/c it's sufficient to illustrate the problems)

You have two choices/branches. Pathom can't make that choice for you:

In your systems either the provider has to decide:

    :filenameJPG -> [readFileJPG] -> :image
    :filenamePNG -> [readFilePNG] -> :image

.. or the consumer has to decide:

    :filename -> [readFileJPG] -> :imageJPG
    :filename -> [readFilePNG] -> :imagePNG

I think this is the part where you really have to think hard about who knows which branch should be taken. Who knows you need a fat db request or a skinny one. Is that the part providing the :id and whatnot, or is it the one receiving the result of the db query?

But bear in mind that Pathom is a "compile time" tool in a sense. If it's something determined from a db query.. then it's not part of the program control flow and needs to be decide at "runtime". It's not a Pathom problem! It has to happen inside the resolver code and it can't be part of the web of resolvers.

In that previous :fat-pack example, the equivalents would been :id-FAT (provider) or :customers/phone_number-FAT (consumer)

2.

Okay, but say I have readFileJPG and readFilePNG provided by some library/module/ns. I can't modify it's I/O keys. Then hopefully the keys are namespaced:

    ::jpg/filename -> [jpg/readFile] -> ::jpg/imageJPG
    ::png/filename -> [png/readFile] -> ::png/imagePNG

Now you can bridge keys either on one side or the other, effectively re-creating either a provider or consumer driver flow. But you can't bridge both sides, cus then you're back to the original problem

3.

Okay, but say I have a pipeline that is filetype-agnostic and for some reason :filename are coming in from "outside" (maybe user is typing it into a text box) and the output really just wants :image keys for some subsequent processing. What are your options?

- let the provider/upstream decide: The :filename is coming in from somewhere, but it know the file type a priori. So it can supply a flag that the resolvers require - ex: :is-jpg , :is-png

    ::filename + :is-jpg -> [jpg/readFile] -> :image
    ::filename + :is-png -> [png/readFile] -> :image

- let the consumer/downstream decide: nested outputs {:jpg [:image]}

    ::filename -> [jpg/readFile] -> {:jpg [:image]}
    ::filename -> [png/readFile] -> {:png [:image]}

Both options leave a bad taste the mouth.

- The first feels fragile (you live in fear of it taking a wrong branch), esp if you have "default no-flag" case like in your db request example.

- The second creates a packed thing that needs to be unpacked. (this is what I showed with :fat-pack). I actually think this is the worse solution b/c it breaks composability. The packed space is completely isolated from the wider soup of keys, which can severely limit what you can do (as I'll illustrate).

EDIT: Part 6 I show the better solution for the consumer

4.

The second solution also conceptually clashes with another tool in the toolbelt. Nested inputs and outputs have a fundamental, completely unrelated, different usecase. They allow you to create a mini-context in which you can re-execute your Pathom resolvers. So if you have two images in your application, you can read them both in. These two transformation can co-exist:

    {:profile-pic [:filename]} => {:profile-pic [:image]}
    {:background-pic [:filename]} => {:background-pic [:image]}

You can of course create mini-contexts arbitrarily if you want. The downside of these nested mini-contexts is that they are aggressively isolated and completely oblivious of the larger world, so you need to shove in all the keys you may use. If you have a resolver somewhere else that processes the image:

    :image + :username -> [addUsernameOverlay] -> :image-with-username

Then you'll need to jam all the necessary keys in to the mini-context:

     {:profile-pic [:filename :username]} => {:profile-pic [:image-with-username]}

5.

So for the consumer side solution:

    ::filename -> [jpg/readFile] -> {:jpg [:image]}

You're a bit screwed. You've lost the natural extensibility of Pathom. You now CANNOT do

    ::filename + ::username => :image-with-username [XXXX - doesn't work!]

Nor can you do

    ::filename + ::username => {:jpg [:image-with-username]}

One hack is to have jpg/readFile pack the :jpg container (ie. forward the :username into it)

    ::filename + :username -> [jpg/readFile] -> {:jpg [:image + :username]}

This makes the second call work, but this is a bit gross.. B/c now the jpg/readFile resolver requires a :username that it actually doesn't use. The resolver is now coupled to downstream requirements. Eww

6.

After chewing it over, I think the best solution is to pack everything when possible:

    {:jpg [:filename]} -> [jpg/readfile] -> {:jpg [:image]}

Now the jpg/readfile resolver doesn't take a :filename, but instead takes a :jpg as {:jpg [:filename]} and unpacks it. Here.. I think technically only consumer must know the type.. But now you can add the username transparently and this transformation works without jpg/readfile needing to forward anything.

    {:jpg [:filename + :username]} -> {:jpg [:image-with-username]}

The downside is that you may end up having to shove in a lot of variables in to the mini-context. If it's all using resolvers from a library/ns that are kinda doing an isolated set of stuff then this can be very viable though!

When to create these mini-contexts is not something I have a clear mental model for. It's related to the fundamental downstream output