[charlie-haley]

Hey HN, I wanted to show off my project Marmot! I decided to build Marmot after discovering a lot of data catalogs can be complex and require many external dependencies such as Kafka, Elasticsearch or an external orchestrator like Airflow.

Marmot is a single Go binary backed by Postgres. That's it!

It already supports: Full-text search across tables, topics, queues, buckets, APIs Glossary and asset to term associations

Flexible API so it can support almost any data asset!

Terraform/Pulumi/CLI for managing a catalog-as-code

10+ Plugins (and growing)

Live demo: https://demo.marmotdata.io

[wiredfool]

How does this get the maps of the data flows and so on? Does it require read credentials to each data silo, or is there a manual mapping process?

[charlie-haley]

It supports either, I didn't want to restrict people to just one method of getting their catalog populated. The CLI and Plugin system works on needing read credentials to a given Service, it then populates the catalog with those assets. Any lineage links currently need to be done manually (unless they're part of the same plugin). Otherwise, you can integrate with your existing IaC pipelines using Terraform or Pulumi to populate the catalog at deploy time instead of needing to scrape a bunch of services.