feat: add new dbt artifacts support

[gaurpulkit]

Important

Add support for run_results and sources dbt artifacts in onboarding, including CLI updates, schema definitions, and tests.

• Behavior:
  • Adds support for run_results and sources dbt artifacts in onboard_file() in utils.py.
  • Updates onboard command in cli.py to handle --run-results-path and --sources-path options.
  • Validates run_results and sources files in onboard command.
• Schemas:
  • Adds run_results.py and sources.py to define schemas for RunResults and Sources using Pydantic.
• Utilities:
  • Implements load_run_results() and load_sources() in utils.py to parse and validate respective JSON files.
• Constants:
  • Defines SUPPORTED_ARTIFACT_TYPES in constants.py to include run_results and sources.
• Tests:
  • Adds tests for onboard_file() in test_utils.py to check supported and unsupported artifact types.
  • Adds tests for load_run_results() and load_sources() in test_artifact_loaders.py.
  • Updates CLI tests in test_cli.py to verify new artifact options in onboard command.

^{This description was created by for 6d2034f. You can customize this summary. It will automatically update as commits are pushed.}

[ellipsis-dev]

Important

Looks good to me! 👍

Reviewed everything up to 9358e35 in 26 seconds. Click for details.

• Reviewed 770 lines of code in 19 files
• Skipped 0 files when reviewing.
• Skipped posting 2 draft comments. View those below.
• Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. src/vendor/dbt_artifacts_parser/parser.py:254

• Draft comment:
  The error message in parse_run_results is misleading; it says 'Not a manifest.json' when it should indicate run-results.json.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

2. src/vendor/dbt_artifacts_parser/parser.py:324

• Draft comment:
  The error message in parse_sources is incorrect; it raises 'Not a manifest.json' instead of 'Not a sources.json'.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

Workflow ID: wflow_eK1ZtMY6XRJXBtYu

^{You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}

[ellipsis-dev]

Important

Looks good to me! 👍

Reviewed 6d2034f in 45 seconds. Click for details.

• Reviewed 366 lines of code in 12 files
• Skipped 0 files when reviewing.
• Skipped posting 12 draft comments. View those below.
• Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. src/datapilot/clients/altimate/constants.py:9

• Draft comment:
  Removed 'semantic_manifest' from SUPPORTED_ARTIFACT_TYPES. Ensure this breaking change is documented.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

2. src/datapilot/core/platforms/dbt/cli/cli.py:21

• Draft comment:
  Removed import of load_semantic_manifest. All references to semantic_manifest are cleaned up.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

3. src/datapilot/core/platforms/dbt/cli/cli.py:160

• Draft comment:
  Removed '--semantic-manifest-path' option and related validation from the onboard command. Confirm this removal aligns with overall artifact support changes.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

4. src/datapilot/core/platforms/dbt/schemas/semantic_manifest.py:1

• Draft comment:
  Deleted the semantic_manifest schema file. Ensure downstream consumers are updated, as this removal is breaking.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

5. src/datapilot/core/platforms/dbt/utils.py:132

• Draft comment:
  Removed the load_semantic_manifest() function. This cleanup is consistent with the removal in CLI and schema. Confirm no external dependency remains.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

6. src/vendor/dbt_artifacts_parser/parser.py:346

• Draft comment:
  Removed parse_semantic_manifest function. Verify that no external code depends on this parser.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

7. src/vendor/dbt_artifacts_parser/parsers/semantic_manifest/__init__.py:1

• Draft comment:
  Removed the semantic_manifest init file from the parsers. Ensure that no module import errors occur post-deletion.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

8. src/vendor/dbt_artifacts_parser/parsers/semantic_manifest/semantic_manifest_v1.py:1

• Draft comment:
  Removed the semantic_manifest_v1 parser implementation. Confirm that full semantic manifest support is no longer required.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

9. src/vendor/dbt_artifacts_parser/parsers/version_map.py:78

• Draft comment:
  Removed the SEMANTIC_MANIFEST_V1 mapping from ArtifactTypes. This is consistent with the removal of semantic manifest support.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

10. tests/clients/altimate/test_utils.py:8

• Draft comment:
  Updated expected artifact types in tests to reflect removal of semantic_manifest. Test coverage seems adjusted accordingly.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

11. tests/core/platform/dbt/test_artifact_loaders.py:35

• Draft comment:
  Removed tests for load_semantic_manifest. Ensure that removal does not affect backward compatibility if needed.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

12. tests/core/platform/dbt/test_cli.py:125

• Draft comment:
  Adjusted CLI help test: '--semantic-manifest-path' is no longer expected. Verify that help output remains clear for supported options.
• Reason this comment was not posted:
  Confidence changes required: 0% <= threshold 50% None

Workflow ID: wflow_P90yRDDddxLPfvJA

^{You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}