|
Hello HN, Thank you for feedbacks on our earlier introductory post about SyncLite, and we have been incorporating it ! Earlier, we posted about a specific case of replicating app-embedded DuckDBs into a centralized PostgreSQL database. We would like further highlight SyncLite as a generic data consolidation framework to replicate/consolidate data from edge/mobile applications using various popular embedded databases: SQLite, DuckDB, Apache Derby, H2, HyperSQL into various centralized industry leading databases: PostgreSQL, MySQL, MongoDB, DuckDB SQLite, etc. We would love to get suggestions for improvements, new features/functionalities, new connectors etc. Brief summary of SyncLite's core infrastructure: SyncLite Logger: is a single Java Library (JDBC Driver): SyncLite encapsulates popular embedded databases: SQLite, DuckDB, Apache Derby, H2, HyperSQL(HSQLDB), allowing user applications to perform transactional operations on them while capturing and writing them into log files. Staging Storage: The log files are continuously staged on a configurable staging storage such as S3, MinIO, Kafka, SFTP, etc. SyncLite Consolidator: A Java application that continuously scans these log files from the configured staging storage, reads incoming command logs, translates them into change-data-capture logs, and applies them onto one or more configured destination databases. It includes many advanced features such as table/column/value filtering and mapping, trigger installation, fine-tunable writes, support for multiple destinations etc. |