Skip to main content

Feast

There are 2 sources that provide integration with Feast

Source ModuleDocumentation

feast

This plugin extracts:

feast-legacy

This plugin extracts the following:

Note: this uses a separate Docker container to extract Feast's metadata into a JSON file, which is then parsed to DataHub's native objects. This separation was performed because of a dependency conflict in the feast module.

Read more...

Module feast

Certified

Important Capabilities

CapabilityStatusNotes
Table-Level LineageEnabled by default

This plugin extracts:

CLI based Ingestion

Install the Plugin

pip install 'acryl-datahub[feast]'

Starter Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
type: feast
config:
# Coordinates
path: "/path/to/repository/"
# Options
environment: "PROD"

sink:
# sink configs

Config Details

Note that a . is used to denote nested fields in the YAML recipe.

View All Configuration Options
FieldRequiredTypeDescriptionDefault
pathstringPath to Feast repositoryNone
environmentstringEnvironment to use when constructing URNsPROD

Code Coordinates

  • Class Name: datahub.ingestion.source.feast.FeastRepositorySource
  • Browse on GitHub

Module feast-legacy

Certified

Important Capabilities

CapabilityStatusNotes
Table-Level LineageEnabled by default

This plugin extracts the following:

Note: this uses a separate Docker container to extract Feast's metadata into a JSON file, which is then parsed to DataHub's native objects. This separation was performed because of a dependency conflict in the feast module.

CLI based Ingestion

Install the Plugin

pip install 'acryl-datahub[feast-legacy]'

Starter Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
type: feast-legacy
config:
# Coordinates
core_url: "localhost:6565"

sink:
# sink configs

Config Details

Note that a . is used to denote nested fields in the YAML recipe.

View All Configuration Options
FieldRequiredTypeDescriptionDefault
envstringThe environment that all assets produced by this connector belong toPROD
core_urlstringURL of Feast Core instance.localhost:6565
use_local_buildbooleanWhether to build Feast ingestion Docker image locally.False

Code Coordinates

  • Class Name: datahub.ingestion.source.feast_legacy.FeastSource
  • Browse on GitHub

Questions

If you've got any questions on configuring ingestion for Feast, feel free to ping us on our Slack