[geokon]

I'll be honest, I work in a very different area (scientific computing) so my code is a lot more exploratory and I don't ever deal with DB access for instance. If you have a very stable interface and clear objectives then coupling isn't really a concern b/c there won't be anything to refactor and extend.

> avoiding intermediate keys at all can be helpful for performance, and graph querying encourages people to break their queries into small, atomic units that can fire in any order

EDIT: Reading the other comments, I realize here query is a DB query and not a EQL query.. so nevermind :)

I'm a bit fuzzy on what you're saying, but I think you may be misunderstanding an aspect (I could be wrong here). You typically have one complex top-level query and the engine builds the sequence/graph of resolvers that need to be run to derive the requested query. In that graph key values can be reused and branches can be run in parallel. You don't run a series of small queries and manually build up anything.

In my limited experience the order in which the resolvers are run is pretty clear (unless they're independent branches of the graph being run concurrently) and if you have a non-branching pipeline there isn't really any incentive to break it up. From a performance perspective I'm guessing you mean in terms of DB access? Because calling a series of functions or a series of resolvers is going to be quite similar performance wise - though you have some engine and destructuring overhead (can be significant in tight loop situations).

> Is that exceptionally harder to do with regular functions though?

It's hard to make a general statement here b/c it really depends on how you've set up your functions. But yes, generally if you are just playing with functions it can be harder to plug in a different "backend" or step in the middle unless you've somehow planned for it. Is it very hard? Generally not super difficult - but you generally need to refactor code to make it happen and explicitly handle the branching logic - so the code usually gets uglier

If you want to do a mock or try injecting some step, with resolvers you can do that without touching your code

[Capricorn2481]

> Reading the other comments, I realize here query is a DB query and not a EQL query.. so nevermind :)

I have been using them interchangeably and it's confusing. I am talking about how differing queries to the Pathom environment may trigger different resolvers, but the resolvers themselves also have DB queries.

In Pathom, as far as I have seen, their query planner will try to fulfill the requested keys with the least amount of resolvers. That means if you have the following resolvers, each their own DB query

- GetEmployee

- GetCompany

- GetEmployeesAndTheirCompanies

Then querying Pathom for a users general information + their company data should only trigger the third resolver, preventing a redundant DB fetch from happening. So as your schema evolves and new entities emerge, when you find your routes do not have optimal resolvers, you can try to make a new one that fulfills a previously unexpected combination of keys.

But that, to me, feels like undoing the things that make graphs appealing. Instead of just querying whatever you want, you now have to remember if the resolvers you've written up to this point can meet the query efficiently, or if it's an non-optimal combination of resolvers. That feels kinda leaky to me, and I'd rather just explicitly code the flow for each route than write Pathom queries and hope the key combination is performant enough. Caching absolutely helps, but doesn't eliminate this.

I also don't like that your only tool to guide which resolver is chosen is just priority. You don't know which two resolvers might be competing against each other, so giving any one resolver a single number for its priority feels very wrong. I can't guarantee how it shakes out without tracing the wrong code. When I am writing explicit procedural code, I do have to write a lot more, but I can just go to definition and see where everything is being used.

I do think this would be less important if your resolvers aren't particularly expensive, or if they are in memory DB calls.

> It's hard to make a general statement here b/c it really depends on how you've set up your functions. But yes, generally if you are just playing with functions it can be harder to plug in a different "backend" or step in the middle unless you've somehow planned for it. Is it very hard? Generally not super difficult - but you generally need to refactor code to make it happen and explicitly handle the branching logic - so the code usually gets uglier

I think I see what you're saying. You compared resolvers to an interface. I would use an interface in procedural code, like for a repository that gets users, and I can plug in a different implementation if I want to replace it. But if I wanted to change the interface itself, that would require refactoring code. You're saying this would be as simple as making the resolver in Pathom, and it can plug in anywhere now.

[geokon]

great example.

Some of this is out of my bailiwick, but on a high level I agree with you. I think your intuition is right. If you have behavior that's dependent on priority, this is a code-smell. It feels like you're just sort of #yolo'ing and hoping the right resolver is called. So far.. In these situations I usually pause and reconsider my architecture. There are probably several solutions here.

(Do bear in mind that I'm still learning the Pathom kungfu here, so I can't guarantee these are the best solutions..)

1.

So in your example the first step in isolating the behavior would be to re-think of it as three keys

- ::employee-ID

- ::company-ID

- ::employee-company-id-pair

and make more resolvers

- ::employee-ID -> ::employee

- ::company-ID -> ::company

- ::employee-ID + ::company-ID -> ::employee-company-ID-pair

- ::employee-company-ID-pair -> ::employee-company-pair

You can then just request an ::employee-company-pair and it should be disambigious. The problem is that you've now have a dense pair that doesn't hook back up with the rest of your resolvers. But this can be addressed with ...

2.

isolating behavior using "nested inputs/outputs". They allow you to go from a soup of keys to a narrow subset

Again:

- ::employee-ID

- ::company-ID

- ::employee-company-id-pair

first you can just have the original three 1-to-1 resolvers

- ::employee-ID -> ::employee

- ::company-ID -> ::company

- ::employee-company-ID-pair -> ::employee + ::company

At this point, as you illustrated, you have a bit of a priority issue. With a ::employee-ID and ::company-ID keys it's unclear which path is taken.

The trick is to now use nested input to disambiguate things.

You make a resolver that returns the results wrapped in a key (nested output):

- ::employee-ID + ::company-ID -> {::packed-request [::employee-company-ID-pair]}

The "consumer" resolver that only wants that efficient db call has on input a ::packed-request and just "unpacks" the request using nested inputs. Furthermore on input it will directly requests {::packed-request [::employee ::company]} and the engine handles the ::employee-company-ID-pair -> ::employee + ::company conversion. This nested input scope (ie. the inside of ::packed-request) doesn't have ::employee-ID and ::company-ID keys, so the request is always unambiguous.

The Pathom docs could be a bit more clear on this. They just show the basics and unfortunately don't walk through these tricks. But you can be explicit about both input and output map shapes and the engine uses these to do conversions. This allows you to narrow the set of inputs. So here one resolver outputs a {:packed-request [::employee-company-ID-pair]} and another takes a {::packed-request [::employee ::company]} - and the conversion is implicit. The engine finds only one resolver that returns a ::packed-request and it sees that it internally it will have a ::employee-company-ID-pair key. It then looks for a path from ::employee-company-ID-pair to the requested ::employee + ::company pair and it finds the corresponding resolver(s). Sometimes you need to forward other keys into this inner context, but you just provide them in parallel to the ::employee-company-ID-pair - it's all explicit.

I'll admit.. this looks weird. As I said elsewhere.. it's a real paradigm shift in how to think about and organize code. But I find after some adjustment it's actually worked really nicely for me so far.

[Capricorn2481]

This is an interesting way to tackle it, but in my scenario, the company id is data from the users table. So without fetching the user first, you don't know what their company ID is.

If we don't have the users data, we can't pass the company id with the user ID. This is not a problem if you are doing an explicit query because you can fetch user + company data at the same time. But if you're using the three resolvers above and your only way to guarantee the path chosen is to already have the company id, that won't be possible.

Here's another thing I ran into with Pathom. Adding keys may make a previous desired path choice change. Let's say we have our three resolvers return the following.

- GetUser - provides :user/email

- GetCompany - provides :company/phone_number

- GetUserWithCompany - Provides :user/email :company/phone_number

I query the environment by giving it a user ID and asking for the phone number that belongs to that users company.

    (p.eql/process env
        {:users/id 16230}
            [:company/phone_number])

It pings the third resolver. That's great! It got the information I want in one node.

But now what if we also want the users email?

    (p.eql/process env
        {:users/id 16230}
            [:company/phone_number
             :users/email])

You would think this would just use the same resolver because it provides both of these keys. But it doesn't. It will call the other two resolvers.

The reason it does this in my example is I had made a bridge between user and company before I made the more efficient resolver that gets all of that data at once.

    (def user-customer-bridge
      (pbir/alias-resolver :users/customer_id :customers/id))

We needed this bridge before, otherwise we just had two separate user and company queries that couldn't connect at all. The bridge lets that data be joined. But when the bridge is still here, Pathom, for whatever reason, will get the users data first and use the cached result to get the rest of the data, instead of just using our new resolver that gets it all at once. The only solution here is either to set priorities on the resolvers, or to remember to remove the bridge when adding new resolvers.

You might think this is petty and arbitrary, and maybe it is. Maybe I am holding it wrong. But it is exactly the kind of thing I ran into just doing test scripts on my own with very simple schemas. Imagine working with 12 people and having hundreds of tables.

I suppose the nice thing about Pathom is if you really want to override the path, you can just fetch that stuff manually and then query the environment with the data you got manually.

[geokon]

It doesn't seem petty at all. These are the fundamental primitives of how you want to decouple and organize code. You have to look at them through small examples.

In the first case, the situation looks largely the same. I mean you can either uses the same nested inputs strategy but have a special key that triggers the fat-query resolver. Something like :employed-user-id.

The other alternative is using nested outputs. You have the resolver returned a keyed bundle. Something like {:fat-request [:users/email :company/phone_number]}. The downstream resolver then consumers a :fat-request and unpacks it using nested inputs.

As for the second example. I'm a little confused on some of the specifics. Writing out the resolver mappings more explicitly.. I'm inferring this is what's going on:

- GetUser - :users/id -> :users/email :company/id

- GetCompany - :company/id -> :company/phone_number

- GetUserWithCompany - :users/id -> :users/email :company/phone_number (<- this is a shortcircuit bypassing :company/id)

From this I can see why the first request triggers number 3. You could of course make the third resolver return a bundled output which may simplify things.

The second request.. I get a bit confused here.

1. I'm a bit confused about the bridge. Maybe there's a typo? Or are you saying `customer` is some completely separate namespace that's interfering with the behavior? In the text you seem to imply you're aliasing company/id and user/id.. but that would be a bit crazy :)

2. You then say "and use the cached result to get the rest of the data". So you have cached resolvers and there is memory from previous requests?

Is it internally, after the first request, remembering the :company/id associated with this :users/id? So it triggers the first two resolvers instead of the third one (but why was the :company/phone_number and users/email not in the cache?)

> Imagine working with 12 people and having hundreds of tables.

Yeah, I think at this point in time there is no sense of best-practices or common programming patterns. From reading the docs and issues, I don't even get the sense Pathom's author has a good sense of how best to hook things up. So we're in the `goto` era of using Pathom and you can end up with a messy web of resolvers. Playing around with the system.. I'm left feeling like there is some emergent logic and ways to organize code. But maybe there are corner cases where it all breaks down and you start to miss imperative programming. My gut reaction is that if I see two separate paths on the same inputs/outputs .. then I'm immediately thinking - "can I redesign my system to avoid this?"

As a simple thought experiment. Say you want to inject stuff in to a pipeline (So some A -> B -> C -> D becomes A -> B -> ZZ -> D). Pathom's author suggests using the Priority attribute. https://pathom3.wsscode.com/docs/resolvers/#prioritization

But you have other alternatives...

- You can also add a dummy key :take-zz-branch. You have resolverC that takes :B and you have resolverZZ that takes :B and :take-zz-branch. Precedence rules .. should .. make it take the branch (unless it for some reason requires fewer inputs?).

- You can make resolverZZ output some dummy key :zz-was-run. If you request :zz-was-run then I think it should also take the branch? (or maybe it runs both branches).

Maybe there are other methods I've not considered. But at this point I'm not clear which method is best!

[Capricorn2481]

On local it's customer, but I renamed it Company for you and forgot to rename it on the bridge. I will call it Customer below.

When I say cached result I misspoke, I mean Pathom, rather than running the one resolver that gets everything, will first see the resolver that gets only the user and grab that. Then it will consider :user/email satisfied and look for other resolvers. It seems to not look for the resolver that solves all the requested data, and instead goes first come first serve.

> In the first case, the situation looks largely the same. I mean you can either uses the same nested inputs strategy but have a special key that triggers the fat-query resolver. Something like :employed-user-id.

> The other alternative is using nested outputs. You have the resolver returned a keyed bundle. Something like {:fat-request [:users/email :company/phone_number]}. The downstream resolver then consumers a :fat-request and unpacks it using nested inputs.

> You can make resolverZZ output some dummy key :zz-was-run. If you request :zz-was-run then I think it should also take the branch? (or maybe it runs both branches)

I think that these tricks can work sometimes, but you might be surprised what odd behavior can come up if you rely on this. In your pathom query, the order of your requested data matters.

    (pco/defresolver get-user [{:users/keys [id]}]
      {::pco/output [:users/id :users/email :users/customer_id :users/budget]}
      (println "get-user triggered")
      (-> (jdbc/execute-one! ds ["SELECT * FROM users WHERE id = ?" id])))
    
    (pco/defresolver get-customer [{:customers/keys [id]}]
      {::pco/output [:customers/id :customers/billing_number :customers/phone_number]}
      (println "get-customer triggered")
      (-> (jdbc/execute-one! ds ["SELECT * FROM customers WHERE id = ?" id])))

    (pco/defresolver get-user-with-customer [{:users/keys [id]}]
      {::pco/output [:users/id :users/email :users/customer_id :fat-key
                     :customers/id :customers/billing_number :customers/phone_number]}
      (println "get-user-with-customer triggered")
      (let [user (jdbc/execute-one! ds ["SELECT * FROM users WHERE id = ?" id])
            customer (jdbc/execute-one! ds ["SELECT * FROM customers WHERE id = ?" (:users/customer_id user)])]
        (merge user {:customers/id (:customers/id customer)
                     :customers/billing_number (:customers/billing_number customer)
                     :customers/phone_number (:customers/phone_number customer)
                     :fat-key true})))

    (def user-customer-bridge
  (pbir/alias-resolver :users/customer_id :customers/id))
    
    ;; ---- test1
    (p.eql/process env
                   {:users/id 1000}
                   [:fat-key
                    :customers/phone_number
                    :users/email]) 
    
    ;; ---- get-user-with-customer triggered
 

    ;; ---- test2 
    (p.eql/process env
                   {:users/id 1000}
                   [:customers/phone_number
                    :fat-key
                    :users/email])
    
    ;; ---- get-user triggered, 
    ;; ---- get-user-with-customer triggered

I couldn't tell you why it shakes out this way. Something about the bridge and requesting `:customers/phone_number` first must be coercing Pathom to take the longer path to :customers/phone_number, even though the other resolver can get everything in one shot. This is the kind of subtle behavior that is unintuitive and could blow up production if you trigger the wrong resolver in the wrong hot path. So I don't see a reasonable argument for using Pathom in an application that needs reliability unless you make the resolved paths part of the test suite. Although, I still think it could be a powerful combination with something like Sqlite or Datalevin where many reads don't matter as much.

[geokon]

Okay - actual code is good :))

But I can't replicate the behavior. Here is the code:

https://github.com/kxygk/ednless/blob/master/pathom-prectest...

I didn't know what jdbc was so I just plugged in dummy values - but I think I'm matching your logic one2one

Let me know if you can tweak it to have the weird results trigger

At a high level I'd say a couple of things stand out.

- Your resolvers take in an id and return an id.. That sets off a bunch of alarm bells for me. I wouldn't ever do that. There is no good reason to have that, even if the values are identical (if the values change then that's even more dangerous). I can't point to exactly what will go wrong, but you're exacerbating the problem of having multiple resolvers providing an input. Worse yet, in this example you are guaranteed that they are all running. So where is it going to take the ID from..? I don't even know

- The :fat-key isn't doing anything here. If you wanted to do the nested request, it'd be a solid way to guarantee the fat branch is always taken:

    (pco/defresolver get-user-with-customer
      [{:users/keys [id]}]
      {::pco/output [{:fat-pack [:users/id
                                 :users/email
                                 :users/customer_id
                                 :customers/id
                                 :customers/billing_number
                                 :customers/phone_number]}]}
      (println "get-user-with-customer FAT triggered")
      {:fat-pack {:users/id                 66
                  :users/email              "[e-mail]"
                  :users/customer_id        id
                  :customers/id             id
                  :customers/billing_number 666
                  :customers/phone_number   6666}})

    (def env
      (pci/register [get-user
                     get-customer
                     get-user-with-customer
                     user-customer-bridge]))

    (p.eql/process env
                   {:users/id 1000}
                   [{:fat-pack [:customers/phone_number
                                :users/email]}])
    ;;{:fat-pack {:customers/phone_number 6666, :users/email "[e-mail]"}}

I think you can see you get the same results, but there is no way for it to go wrong. Admittedly here I have the EQL request "unpacking" it, but it can also be unpacked transparently from a different resolver using nested inputs.

    (pco/defresolver fat-eater
      [{:keys [fat-pack]}]
      {::pco/input[{:fat-pack [:users/id
                               :users/email
                               :users/customer_id
                               :customers/id
                               :customers/billing_number
                               :customers/phone_number]}]
       ::pco/output [:response]}
      (println "fat-eater triggered")
      {:response (str "yum, just ate: "
                      (:users/id fat-pack))})

    (def env
      (pci/register [get-user
                     get-customer
                     get-user-with-customer
                     user-customer-bridge
                     fat-eater]))

    (p.eql/process env
                   {:users/id 1000}
                   [:response])
    ;; {:response "yum, just ate: 66"}

This is very explicit and about equivalent to your original thought of "why not just have explicit imperative function calls". The consumer (the `fat-eater` or the user making the EQL request) knows a priori that he wants the fat call.

EDIT: At the bottom of the .clj I added a bunch of other tests for the pipeline branching (the A->B->C thing)

- Adding an XXX resolver to override step B worked when "forced" with an extra input key.

- Adding an YYY resolver to override step B by outputting an extra key ends up always overriding B (whether you request the new dummy output key or not!)

I'm guessing here.. but it's probably b/c YYY has the same input requirements as B, but YYY outputs two keys instead of one - so it's preferred.

In a way both seem useful. The first lets you use a flag to select when you take a branch. The second make you always override the branch