Version: Next

Slack

Feature Availability

Self-Hosted DataHub

Managed DataHub

Slack


Status
Version Requirements

Overview

This Action integrates DataHub with Slack to send notifications to a configured Slack channel in your workspace.

Capabilities

Sending notifications of important events to a Slack channel
- Adding or Removing a tag from an entity (dataset, dashboard etc.)
- Updating documentation at the entity or field (column) level.
- Adding or Removing ownership from an entity (dataset, dashboard, etc.)
- Creating a Domain
- and many more.

User Experience

On startup, the action will produce a welcome message that looks like the one below.

On each event, the action will produce a notification message that looks like the one below.

Watch the townhall demo to see this in action:

Supported Events

EntityChangeEvent_v1
Currently, the MetadataChangeLog_v1 event is not processed by the Action.

Action Quickstart

Prerequisites

Ensure that you have configured the Slack App in your Slack workspace.

Install the DataHub Slack App into your Slack workspace

The following steps should be performed by a Slack Workspace Admin.

Navigate to https://api.slack.com/apps/
Click Create New App
Use “From an app manifest” option
Select your workspace
Paste this Manifest in YAML. We suggest changing the name and display_name to be DataHub App YOUR_TEAM_NAME but this is not required. This name will show up in your Slack workspace.

display_information:
  name: DataHub App
  description: An app to integrate DataHub with Slack
  background_color: "#000000"
features:
  bot_user:
    display_name: DataHub App
    always_online: false
oauth_config:
  scopes:
    bot:
      - channels:history
      - channels:read
      - chat:write
      - commands
      - groups:read
      - im:read
      - mpim:read
      - team:read
      - users:read
      - users:read.email
settings:
  org_deploy_enabled: false
  socket_mode_enabled: false
  token_rotation_enabled: false

Confirm you see the Basic Information Tab

Click Install to Workspace
It will show you permissions the Slack App is asking for, what they mean and a default channel in which you want to add the slack app
- Note that the Slack App will only be able to post in channels that the app has been added to. This is made clear by slack’s Authentication screen also.
Select the channel you'd like notifications to go to and click Allow
Go to the DataHub App page
- You can find your workspace's list of apps at https://api.slack.com/apps/

Getting Credentials and Configuration

Now that you've created your app and installed it in your workspace, you need a few pieces of information before you can activate your Slack action.

1. The Signing Secret

On your app's Basic Information page, you will see a App Credentials area. Take note of the Signing Secret information, you will need it later.

2. The Bot Token

Navigate to the OAuth & Permissions Tab

Here you'll find a “Bot User OAuth Token” which DataHub will need to communicate with your Slack workspace through the bot.

3. The Slack Channel

Finally, you need to figure out which Slack channel you will send notifications to. Perhaps it should be called #datahub-notifications or maybe, #data-notifications or maybe you already have a channel where important notifications about datasets and pipelines are already being routed to. Once you have decided what channel to send notifications to, make sure to add the app to the channel.

Next, figure out the channel id for this Slack channel. You can find it in the About section for the channel if you scroll to the very bottom of the app.

Alternately, if you are on the browser, you can figure it out from the URL. e.g. for the troubleshoot channel in OSS DataHub slack

Notice TUMKD5EGJ/C029A3M079U in the URL
- Channel ID = C029A3M079U from above

In the next steps, we'll show you how to configure the Slack Action based on the credentials and configuration values that you have collected.

Installation Instructions (Deployment specific)

Managed DataHub

Head over to the Configuring Notifications section in the Managed DataHub guide to configure Slack notifications for your Managed DataHub instance.

Quickstart

If you are running DataHub using the docker quickstart option, there are no additional software installation steps. The datahub-actions container comes pre-installed with the Slack action.

All you need to do is export a few environment variables to activate and configure the integration. See below for the list of environment variables to export.

Env Variable	Required for Integration	Purpose
DATAHUB_ACTIONS_SLACK_ENABLED	✅	Set to "true" to enable the Slack action
DATAHUB_ACTIONS_SLACK_SIGNING_SECRET	✅	Set to the Slack Signing Secret that you configured in the pre-requisites step above
DATAHUB_ACTIONS_SLACK_BOT_TOKEN	✅	Set to the Bot User OAuth Token that you configured in the pre-requisites step above
DATAHUB_ACTIONS_SLACK_CHANNEL	✅	Set to the Slack Channel ID that you want the action to send messages to
DATAHUB_ACTIONS_SLACK_DATAHUB_BASE_URL	❌	Defaults to "http://localhost:9002". Set to the location where your DataHub UI is running. On a local quickstart this is usually "http://localhost:9002", so you shouldn't need to modify this

note

You will have to restart the datahub-actions docker container after you have exported these environment variables if this is the first time. The simplest way to do it is via the Docker Desktop UI, or by just issuing a datahub docker quickstart --stop && datahub docker quickstart command to restart the whole instance.

For example:

export DATAHUB_ACTIONS_SLACK_ENABLED=true
export DATAHUB_ACTIONS_SLACK_SIGNING_SECRET=<slack-signing-secret>
....
export DATAHUB_ACTIONS_SLACK_CHANNEL=<slack_channel_id>

datahub docker quickstart --stop && datahub docker quickstart

k8s / helm

Similar to the quickstart scenario, there are no specific software installation steps. The datahub-actions container comes pre-installed with the Slack action. You just need to export a few environment variables and make them available to the datahub-actions container to activate and configure the integration. See below for the list of environment variables to export.

Env Variable	Required for Integration	Purpose
DATAHUB_ACTIONS_SLACK_ENABLED	✅	Set to "true" to enable the Slack action
DATAHUB_ACTIONS_SLACK_SIGNING_SECRET	✅	Set to the Slack Signing Secret that you configured in the pre-requisites step above
DATAHUB_ACTIONS_SLACK_BOT_TOKEN	✅	Set to the Bot User OAuth Token that you configured in the pre-requisites step above
DATAHUB_ACTIONS_SLACK_CHANNEL	✅	Set to the Slack Channel ID that you want the action to send messages to
DATAHUB_ACTIONS_DATAHUB_BASE_URL	✅	Set to the location where your DataHub UI is running. For example, if your DataHub UI is hosted at "https://datahub.my-company.biz", set this to "https://datahub.my-company.biz"

Bare Metal - CLI or Python-based

If you are using the datahub-actions library directly from Python, or the datahub-actions cli directly, then you need to first install the slack action plugin in your Python virtualenv.

pip install "datahub-actions[slack]"

Then run the action with a configuration file that you have modified to capture your credentials and configuration.

Sample Slack Action Configuration File

name: datahub_slack_action
enabled: true
source:
  type: "kafka"
  config:
    connection:
      bootstrap: ${KAFKA_BOOTSTRAP_SERVER:-localhost:9092}
      schema_registry_url: ${SCHEMA_REGISTRY_URL:-http://localhost:8081}
    topic_routes:
      mcl: ${METADATA_CHANGE_LOG_VERSIONED_TOPIC_NAME:-MetadataChangeLog_Versioned_v1}
      pe: ${PLATFORM_EVENT_TOPIC_NAME:-PlatformEvent_v1}

## 3a. Optional: Filter to run on events (map)
# filter: 
#  event_type: <filtered-event-type>
#  event:
#    # Filter event fields by exact-match
#    <filtered-event-fields>

# 3b. Optional: Custom Transformers to run on events (array)
# transform:
#  - type: <transformer-type>
#    config: 
#      # Transformer-specific configs (map)

action:
  type: slack
  config:
    # Action-specific configs (map)
    base_url: ${DATAHUB_ACTIONS_SLACK_DATAHUB_BASE_URL:-http://localhost:9002}
    bot_token: ${DATAHUB_ACTIONS_SLACK_BOT_TOKEN}
    signing_secret: ${DATAHUB_ACTIONS_SLACK_SIGNING_SECRET}
    default_channel: ${DATAHUB_ACTIONS_SLACK_CHANNEL}
    suppress_system_activity: ${DATAHUB_ACTIONS_SLACK_SUPPRESS_SYSTEM_ACTIVITY:-true}

datahub:
  server: "http://${DATAHUB_GMS_HOST:-localhost}:${DATAHUB_GMS_PORT:-8080}"
  

Slack Action Configuration Parameters

Field	Required	Default	Description
`base_url`	❌	`False`	Whether to print events in upper case.
`signing_secret`	✅		Set to the Slack Signing Secret that you configured in the pre-requisites step above
`bot_token`	✅		Set to the Bot User OAuth Token that you configured in the pre-requisites step above
`default_channel`	✅		Set to the Slack Channel ID that you want the action to send messages to
`suppress_system_activity`	❌	`True`	Set to `False` if you want to get low level system activity events, e.g. when datasets are ingested, etc. Note: this will currently result in a very spammy Slack notifications experience, so this is not recommended to be changed.

Troubleshooting

If things are configured correctly, you should see logs on the datahub-actions container that indicate success in enabling and running the Slack action.

docker logs datahub-datahub-actions-1

...
[2022-12-04 07:07:53,804] INFO     {datahub_actions.plugin.action.slack.slack:96} - Slack notification action configured with bot_token=SecretStr('**********') signing_secret=SecretStr('**********') default_channel='C04CZUSSR5X' base_url='http://localhost:9002' suppress_system_activity=True
[2022-12-04 07:07:54,506] WARNING  {datahub_actions.cli.actions:103} - Skipping pipeline datahub_teams_action as it is not enabled
[2022-12-04 07:07:54,506] INFO     {datahub_actions.cli.actions:119} - Action Pipeline with name 'ingestion_executor' is now running.
[2022-12-04 07:07:54,507] INFO     {datahub_actions.cli.actions:119} - Action Pipeline with name 'datahub_slack_action' is now running.
...

If the Slack action was not enabled, you would see messages indicating that. e.g. the following logs below show that neither the Slack or Teams action were enabled.

docker logs datahub-datahub-actions-1

....
No user action configurations found. Not starting user actions.
[2022-12-04 06:45:27,509] INFO     {datahub_actions.cli.actions:76} - DataHub Actions version: unavailable (installed editable via git)
[2022-12-04 06:45:27,647] WARNING  {datahub_actions.cli.actions:103} - Skipping pipeline datahub_slack_action as it is not enabled
[2022-12-04 06:45:27,649] WARNING  {datahub_actions.cli.actions:103} - Skipping pipeline datahub_teams_action as it is not enabled
[2022-12-04 06:45:27,649] INFO     {datahub_actions.cli.actions:119} - Action Pipeline with name 'ingestion_executor' is now running.
...

Is this page helpful?

Slack

Slack

Overview​

Capabilities​

User Experience​

Supported Events​

Action Quickstart​

Prerequisites​

Install the DataHub Slack App into your Slack workspace​

Getting Credentials and Configuration​

1. The Signing Secret​

2. The Bot Token​

3. The Slack Channel​

Installation Instructions (Deployment specific)​

Managed DataHub​

Quickstart​

k8s / helm​

Bare Metal - CLI or Python-based​

Sample Slack Action Configuration File​

Slack Action Configuration Parameters​

Troubleshooting​

Overview

Capabilities

User Experience

Supported Events

Action Quickstart

Prerequisites

Install the DataHub Slack App into your Slack workspace

Getting Credentials and Configuration

1. The Signing Secret

2. The Bot Token

3. The Slack Channel

Installation Instructions (Deployment specific)

Managed DataHub

Quickstart

k8s / helm

Bare Metal - CLI or Python-based

Sample Slack Action Configuration File

Slack Action Configuration Parameters

Troubleshooting