File

Module file

This plugin pulls metadata from a previously generated file. The file sink can produce such files, and a number of samples are included in the examples/mce_files directory.

Install the Plugin

The file source works out of the box with acryl-datahub.

Quickstart Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide

source:
  type: file
  config:
    # Coordinates
    filename: ./path/to/mce/file.json

sink:
  # sink configs

Config Details

Options

Note that a . is used to denote nested fields in the YAML recipe.

View All Configuration Options

Field	Required	Type	Description	Default
filename	✅	string	Path to file to ingest.	None

Schema

The JSONSchema for this configuration is inlined below.

{
  "title": "FileSourceConfig",
  "type": "object",
  "properties": {
    "filename": {
      "title": "Filename",
      "description": "Path to file to ingest.",
      "type": "string"
    }
  },
  "required": [
    "filename"
  ],
  "additionalProperties": false
}

Code Coordinates

• Class Name: datahub.ingestion.source.file.GenericFileSource
• Browse on GitHub

Questions

If you've got any questions on configuring ingestion for File, feel free to ping us on our Slack