Hacker News new | ask | show | jobs
by aleatorisch 743 days ago
I'm curious what errors you were running into? Mind posting an issue in the repo, or here? Thanks!
2 comments

Sorry for the late answer. I get the following errors if I try to execute queries on delta tables (accessing them with spark works fine)

InvalidInputException: Invalid Input Error: Attempting to execute an unsuccessful or closed pending query result Error: IO Error: Hit DeltaKernel FFI error (from: kernel_scan_data_next in DeltaSnapshot GetFile): Hit error: 2 (ArrowError) with message (Invalid argument error: Incorrect datatype for StructArray field "partitionValues", expected Map(Field { name: "entries", data_type: Struct([Field { name: "keys", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "values", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, false) got Map(Field { name: "key_value", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false))

Thanks, appreciate the reply. I believe this should have been fixed by https://github.com/delta-incubator/delta-kernel-rs/pull/210.

I'm guessing you were reading from a table with a checkpoint :)

DuckDB unfortunately hasn't pulled in the latest changes just yet, but as soon as they update to include that fix, I expect your queries will work.

If the table is public, I'd be happy to check that kernel now supports reading it.

My problem is probably already covered by:

https://github.com/duckdb/duckdb_delta/issues/18