|
|
|
|
|
|
by danmaz74
5348 days ago
|
|
|
This looks interesting, thanks for sharing. But what kind of performance is it possible to get with complex queries on these external sources? Does the PG server need to load all the data in memory from the source to do filters, joins and sorts? |
|
|
The postgresql plan is parsed, and passed as a list of "quals", objects representing simple filters. As an implementer, you don't HAVE to enforce those, since postgresql will recheck them for you anyway, but they can be quite handy.
For example, you can look at the imap foreign data wrapper (https://github.com/Kozea/Multicorn/blob/master/python/multic...) to see how the conditions from postgresql are converted to an IMAP filter, allowing for server side filtering.
The required columns are also provided, so if you don't need the email payload, the foreign data wrapper will not fetch it.
For joins, it will depend on the execution plan. There is still plenty of work on parsing the postgresql execution plan into something more useful, but the current set of "optimizations" is sufficient for our main use cases.