This guide assumes that you've already followed the metadata ingestion developing guide to set up your local environment.
The reporter interface enables the source to report statistics, warnings, failures, and other information about the run. Some sources use the default
SourceReport class, but others inherit and extend that class.
The core for the source is the
get_workunits method, which produces a stream of MCE objects. The file source is a good and simple example.
Declare the source's pip dependencies in the
plugins variable of the setup script.
Declare the source under the
entry_points variable of the setup script. This enables the source to be listed when running
datahub check plugins, and sets up the source's shortened alias for use in recipes.
Tests go in the
tests directory. We use the pytest framework.
Add the plugin to the table at the top of the README file, and add the source's documentation underneath the sources header.