DataLoader

class felis.tap_schema.DataLoader(schema, mgr, tap_schema_index=0, output_path=None, print_sql=False, dry_run=False)

Bases: object

Load data into the TAP_SCHEMA tables.

Parameters:

  • schema (Schema) – The Felis Schema to load into the TAP_SCHEMA tables.

  • mgr (TableManager) – The table manager that contains the TAP_SCHEMA tables.

  • engine – The SQLAlchemy engine to use to connect to the database.

  • tap_schema_index (int, default: 0) – The index of the schema in the TAP_SCHEMA database.

  • output_path (str | None, default: None) – The file to write the SQL statements to. If None, printing will be suppressed.

  • print_sql (bool, default: False) – If True, print the SQL statements that will be executed.

  • dry_run (bool, default: False) – If True, the data will not be loaded into the database.

Methods Summary

load()

Load the schema data into the TAP_SCHEMA tables.

Methods Documentation

load()

Load the schema data into the TAP_SCHEMA tables.

Return type:

None

Notes

This will generate inserts for the data, print the SQL statements if requested, save the SQL statements to a file if requested, and load the data into the database if not in dry run mode. These are done as sequential operations rather than for each insert. The logic is that the user may still want the complete SQL output to be printed or saved to a file even if loading into the database causes errors. If there are errors when inserting into the database, the SQLAlchemy error message should indicate which SQL statement caused the error.