[anentropic]

Maybe I skim read it too fast, but I did not find any clear description in the blog post or website docs of how this relates to SQL databases

I was kind of guessing that it doesn't run in a database, that it's a SQL-like syntax for a visualisation DSL handled by front end chart library.

That appears to be what is described in https://ggsql.org/get_started/anatomy.html

But then https://ggsql.org/faq.html has a section, "Can I use SQL queries inside the VISUALISE clause," which says, "Some parts of the syntax are passed on directly to the database".

The homepage says "ggsql interfaces directly with your database"

But it's not shown how that happens AFAICT

confused

[thomasp85]

That is fair - it is somewhat of a special concept.

ggsql connects directly with your database backend (if you wish - you can also run it with an in-memory DuckDB backend). Your visual query is translated into a SQL query for each layer of the visualisation and the resulting table is then used for rendering.

E.g.

VISUALISE page_views AS x FROM visits DRAW smooth

will create a SQL query that calculates a smoothing kernel over the data and returns points along that. Those points are then used to create the final line chart

[georgestagg]

ggsql has the concept of a "reader", which can be thought of as the way ggsql interfaces with a SQL database. It handles the connection to the database and generating the correct dialect of SQL for that database.

As an alpha, we support just a few readers today: duckdb, sqlite, and an experimental ODBC reader. We have largely been focusing development mainly around driving duckdb with local files, though duckdb has extensions to talk to some other types of database.

The idea is that ggsql takes your visualisation query, and then generates a selection of SQL queries to be executed on the database. It sends these queries using the reader, then builds the resulting visualisation with the returned data. That is how we can plot a histogram from very many rows of data, the statistics required to produce a histogram are converted into SQL queries, and only a few points are returned to us to draw bars of the correct height.

By default ggsql will connect to an in-memory duckDB database. If you are using the CLI, you can use the `--reader` argument to connect to files on-disk or an ODBC URI.

If you use Positron, you can do this a little easier through its dedicated "Connections" pane, and the ggsql Jupyter kernel has a magic SQL comment that can be issued to set up a particular reader. I plan to expand a little more on using ggsql with these external tools in the docs soon.

[nojito]

Highly suggest leveraging adbc. I would love to use this against our bigquery tables.

[data_ders]

plus 1 for ADBC!

[gazpacho]

Another +1 for ADBC

[chatmasta]

So we could use this with Postgres by putting DuckDB in front with its Postgres extension, pointing to the source data in PG?

[georgestagg]

In principle, yes, that’s the idea! However I will say we have focused mainly on the grammar and using DuckDB for reading from local files for this alpha release, so I expect there may be some bugs around connecting to remote databases still to iron out!

[chatmasta]

Dunno, I expect if DuckDB works as advertised it might just work! That's the beauty of how they've separated the syntax parsing into frontend/backend from the rest of the engine.

[password4321]

Yes this was my question as well, an example showing all the plumbing/dependencies to generate a graph from an external database server would be very helpful.

[thomasp85]

We certainly plan to create a few videos showing how to set it up and use it. If you use it in Positron with the ggsql extension it can interact directly with the connection pane to connect to the various backends you have there

[anentropic]

Please just document the library itself before making a bunch of videos

I eventually found this readme https://github.com/posit-dev/ggsql/tree/main/ggsql-python which tells me far more than anything I found on the website

[tantalor]

> SQL databases ... confused

"SQL" and "databases" are different things

SQL is a declarative language for data manipulation. You can use SQL to query a database, but there's nothing special about databases. You can also write SQL to query other non-database sources like flat files, data streams, or data in a program's memory.

Conversely, you can query a database without SQL.

[philipallstar]

A SQL database is a database you can connect to and query with SQL.

[johnthescott]

> Conversely, you can query a database without SQL.

fond memories of quel.

[anentropic]

thanks Captain Obvious, where would we be without you