Combine technical, operational and business metadata to provide a 360 degree view of your data entities.
Apply “shift-left” practices to pre-enrich important metadata using ingestion transformers, support for dbt meta-mapping and other features.
Act on changes in metadata in real time by notifying key stakeholders, circuit-breaking business-critcal pipelines, propogating metadata across entites, and more.
DataHub was originally built at LinkedIn and subsequently open-sourced under the Apache 2.0 License. It now has a thriving community with over a hundred contributors, and is widely used at many companies.
Forward Looking Architecture
DataHub follows a push-based architecture, which means it's built for continuously changing metadata. The modular design lets it scale with data growth at any organization, from a single database under your desk to multiple data centers spanning the globe.
DataHub has pre-built integrations with your favorite systems: Kafka, Airflow, MySQL, SQL Server, Postgres, LDAP, Snowflake, Hive, BigQuery, and many others. The community is continuously adding more integrations, so this list keeps getting longer and longer.
A Modern Approach to Metadata Management
Automated Metadata Ingestion
Push-based ingestion can use a prebuilt emitter or can emit custom events using our framework.
Pull-based ingestion crawls a metadata source. We have prebuilt integrations with Kafka, MySQL, MS SQL, Postgres, LDAP, Snowflake, Hive, BigQuery, and more. Ingestion can be automated using our Airflow integration or another scheduler of choice.
Learn more about metadata ingestion with DataHub in the docs.
datahub ingest -c recipe.yml
Discover Trusted Data
Browse and search over a continuously updated catalog of datasets, dashboards, charts, ML models, and more.
Understand Data in Context
DataHub is the one-stop shop for documentation, schemas, ownership, lineage, pipelines, data quality, usage information, and more.
Trusted Across the Industry
- B2B & B2C
- Financial & Fintech
- And More