DataHub: The Metadata Platform for the Modern Data Stack
Built with ❤️ by Acryl Data and LinkedIn
🏠 Hosted DataHub Docs (Courtesy of Acryl Data): datahubproject.io
📣 DataHub Town Hall is the 4th Thursday at 9am US PT of every month - add it to your calendar!
✨ DataHub Community Highlights:
- Read our Monthly Project Updates here.
- Bringing The Power Of The DataHub Real-Time Metadata Graph To Everyone At Acryl Data: Data Engineering Podcast
- Check out our most-read blog post, DataHub: Popular Metadata Architectures Explained @ LinkedIn Engineering Blog.
- Join us on Slack! Ask questions and keep up with the latest announcements.
DataHub is an open-source metadata platform for the modern data stack. Read about the architectures of different metadata systems and why DataHub excels here. Also read our LinkedIn Engineering blog post, check out our Strata presentation and watch our Crunch Conference Talk. You should also visit DataHub Architecture to get a better understanding of how DataHub is implemented.
Features & Roadmap
Demo and Screenshots
Please follow the DataHub Quickstart Guide to get a copy of DataHub up & running locally using Docker. As the guide assumes some basic knowledge of Docker, we'd recommend you to go through the "Hello World" example of A Docker Tutorial for Beginners if Docker is completely foreign to you.
If you're looking to build & modify datahub please take a look at our Development Guide.
Source Code and Repositories
- datahub-project/datahub: This repository contains the complete source code for DataHub's metadata model, metadata services, integration connectors and the web application.
- acryldata/datahub-actions: DataHub Actions is a framework for responding to changes to your DataHub Metadata Graph in real time.
- acryldata/datahub-helm: Repository of helm charts for deploying DataHub on a Kubernetes cluster
- acryldata/meta-world: A repository to store recipes, custom sources, transformations and other things to make your DataHub experience magical
- dbt-impact-action : This repository contains a github action for commenting on your PRs with a summary of the impact of changes within a dbt project
- datahub-tools : Additional python tools to interact with the DataHub GraphQL endpoints, built by Notion
- business-glossary-sync-action : This repository contains a github action that opens PRs to update your business glossary yaml file.
Here are the companies that have officially adopted DataHub. Please feel free to add yours to the list if we missed it.
- Digital Turbine
- Expedia Group
- Haibo Technology
- PITS Global Data Recovery Services
- Saxo Bank
- Shanghai HuaRui Bank
Select Articles & Talks
- DataHub Blog
- DataHub YouTube Channel
- Optum: Data Mesh via DataHub
- Saxo Bank: Enabling Data Discovery in Data Mesh
- Bringing The Power Of The DataHub Real-Time Metadata Graph To Everyone At Acryl Data
- DataHub: Popular Metadata Architectures Explained
- Driving DataOps Culture with LinkedIn DataHub @ DataOps Unleashed 2021
- The evolution of metadata: LinkedIn’s story @ Strata Data Conference 2019
- Journey of metadata at LinkedIn @ Crunch Data Conference 2019
- DataHub Journey with Expedia Group
- Data Discoverability at SpotHero
- Data Catalogue — Knowing your data
- DataHub: A Generalized Metadata Search & Discovery Tool
- Open sourcing DataHub: LinkedIn’s metadata search and discovery platform
- Emerging Architectures for Modern Data Infrastructure
See the full list here.