[keyless_]

A long held view of mine is that our current architecture where we keep logic on the server and data in the database is rather unfortunate, because it means you either have to

* marshall data to your server to process (which can be slow), or

* marshall code to your db to run (which can be messy, in part because you're translating from one programming language to your db's language - this is the main reason ORMs exist)

I always thought that the solution is to have all computation live in the DB (similar to how kdb does it), but this is pretty nice too. If you squint your eyes this looks like a much cleverer ORM.

It would be even better if the language supported SQL as a first class citizen, but that's not really an option here, since you're stuck with js.

[timeserious]

fyi kdb+ now supports ANSI-SQL. https://code.kx.com/insights/1.1/core/sql.html

Including support for the PostgreSQL wire protocol https://code.kx.com/insights/1.1/core/pgwire.html

[jnordwick]

KDB+ has always supported SQL92 I thought (minus the DDL stuff). I've seen and edited Arthur's files (and blind now). Much of '99 was there too when k had those alternate syntax libraries just translated down to k (may still be that way). How is this different?

When FD picked up Kx, I was kind of hoping for better admin tooling than support for poor query languages that you shouldn't even be writing in. I know it is there as a necessity to help move code to kdb, but the more sql the less kdb shines. Its strange to push something that makes the product inferior. sql is just a really bad fit for a column db.

[timeserious]

The implementation is similar to before where the sql support is loaded as a library that translates to k - however it goes a few steps further by supporting (not an exhaustive list):

- Intermingling q and sql in the same expression. e.g SELECT * from qt('marketSnap`IBM')

- extensive joins, aggregates, sub-selects, cte, parameters etc - whereas the SQL92/'99 support was v. limited

- Distributed support for executing/joining across multiple targets - allowing a gateway sql query to run against in-memory (rdb) and on-disk (idb/hdb) databases. We are also testing support for querying external databases through 'virtual' tables (as if the data was stored natively in kdb+). The sql support will never replace Q APIs but we're really keen to see what our users will try to do with it. e.g single-pane governance/data mesh patterns

- PostgreSQL dialect and wire support to allow sql/no-code users to connect from 3rd party tools using the ootb PostgreSQL connectors (tableau/powerbi/glue etc) without loading and maintaining special odbc/jdbc drivers for kdb+. Queries generated via tools will push down aggregates/predicates to execute directly against kdb+ (versus the legacy import model)

Specifics aside, I hear all your points loud and clear - and agree with sql being inferior to q-sql (opinions are my own!) - but the intention is not to replace q-sql here. We're simply opening/democratising the technology for less advanced users by enabling new options for them to get more value on day 1 without the typical on-ramp usually required to start using/writing analytics in KX.

With the ultimate aim being that they see huge performance benefits when using KX (even via SQL) vs. other databases with minimal changes to their applications.

From there we hope they will begin to use more q-sql and realise how much easier and succinct it is to express questions against their time-series data e.g window functions, order and temporal based queries.

For some colour on the topic, SQL is by far the biggest request we get from new users (especially outside finance) - so this will help us grow the community and create more opportunity for careers in new sectors for kdb+ developers.

p.s We've invested heavily in better tooling for kdb+ this past 2 years which you may not have seen but I'd mostly love to discuss your ideas/requests and share with Product Strategy who I spend a lot of time working with.