Skip to main content

SageMaker

For context on getting started with ingestion, check out our metadata ingestion guide.

Setup#

To install this plugin, run pip install 'acryl-datahub[sagemaker]'.

Capabilities#

This plugin extracts the following:

  • Feature groups
  • Models, jobs, and lineage between the two (e.g. when jobs output a model or a model is used by a job)

Quickstart recipe#

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:  type: sagemaker  config:    # Coordinates    aws_region: "my-aws-region"
sink:  # sink configs

Config details#

Note that a . is used to denote nested fields in the YAML recipe.

FieldRequiredDefaultDescription
aws_regionโœ…AWS region code.
env"PROD"Environment to use in namespace when constructing URNs.
aws_access_key_idAutodetectedSee https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
aws_secret_access_keyAutodetectedSee https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
aws_session_tokenAutodetectedSee https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
aws_roleAutodetectedSee https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
extract_feature_groupsTrueWhether to extract feature groups.
extract_modelsTrueWhether to extract models.
extract_jobs.auto_mlTrueWhether to extract AutoML jobs.
extract_jobs.compilationTrueWhether to extract compilation jobs.
extract_jobs.edge_packagingTrueWhether to extract edge packaging jobs.
extract_jobs.hyper_parameter_tuningTrueWhether to extract hyperparameter tuning jobs.
extract_jobs.labelingTrueWhether to extract labeling jobs.
extract_jobs.processingTrueWhether to extract processing jobs.
extract_jobs.trainingTrueWhether to extract training jobs.
extract_jobs.transformTrueWhether to extract transform jobs.

Compatibility#

Coming soon!

Questions#

If you've got any questions on configuring this source, feel free to ping us on our Slack!