Skip to content

fix(ingestion): register Drive entities in ENTITY_REFERENCE_CLASS_MAP and ES_INDEX_MAP#28104

Merged
Khairajani merged 3 commits into
open-metadata:mainfrom
ulixius9:fix-automator-drive-entity-reference-map
May 15, 2026
Merged

fix(ingestion): register Drive entities in ENTITY_REFERENCE_CLASS_MAP and ES_INDEX_MAP#28104
Khairajani merged 3 commits into
open-metadata:mainfrom
ulixius9:fix-automator-drive-entity-reference-map

Conversation

@ulixius9
Copy link
Copy Markdown
Member

@ulixius9 ulixius9 commented May 14, 2026

Summary

Drive entities (directory, file, spreadsheet, worksheet) and the driveService were missing from two ingestion-side registries:

  1. ENTITY_REFERENCE_CLASS_MAP in metadata/utils/constants.py — used by the Automator app to resolve entity classes from resources.type. Selecting a Drive entity in the Add Automation UI (AUTOMATOR_DATA_ASSETS_LIST) failed at runtime in Automator.fetch_entities:

    AutomatorException: Can't get class from resource type: directory
    
  2. ES_INDEX_MAP in metadata/utils/elasticsearch.py — used to resolve the correct search index per entity class. Drive entities had no mapping, breaking search-index lookups during ingestion/automation flows.

This PR registers the Drive data entities and driveService in both maps.

Changes

metadata/utils/constants.py

  • Import Directory, File, Spreadsheet, Worksheet, DriveService
  • Add driveService to the service entities section of ENTITY_REFERENCE_CLASS_MAP
  • Add directory, file, spreadsheet, worksheet under a new "Drive Entities" section

ENTITY_REFERENCE_TYPE_MAP is derived from the class map, so it picks these up automatically.

metadata/utils/elasticsearch.py

  • Import Directory, File, Spreadsheet, Worksheet
  • Add entries to ES_INDEX_MAP mapping each class to its *_search_index

Test plan

  • from metadata.utils.constants import ENTITY_REFERENCE_CLASS_MAP resolves directory/file/spreadsheet/worksheet/driveService to their classes
  • from metadata.utils.elasticsearch import ES_INDEX_MAP resolves Directory/File/Spreadsheet/Worksheet to their corresponding search indexes
  • Automator workflow targeting a directory resource type no longer raises AutomatorException

🤖 Generated with Claude Code

The Automator app resolves resource types from resources.type against
ENTITY_REFERENCE_CLASS_MAP. Drive entities (directory, file, spreadsheet,
worksheet) and the driveService were missing, so selecting them in the
Add Automation UI failed at runtime with:

  AutomatorException: Can't get class from resource type: directory

Register the Drive data entities and driveService so the Automator can
paginate and act on them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 14, 2026 05:50
@ulixius9 ulixius9 requested a review from a team as a code owner May 14, 2026 05:50
@github-actions github-actions Bot added Ingestion safe to test Add this label to run secure Github workflows on PRs labels May 14, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Drive entity resolution in ingestion utilities by registering Drive-related generated schema classes in ENTITY_REFERENCE_CLASS_MAP, enabling Automator workflows to resolve Drive resource types.

Changes:

  • Adds imports for Drive data entities and DriveService.
  • Registers driveService, directory, file, spreadsheet, and worksheet in the entity reference class map.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 14, 2026

🟡 Playwright Results — all passed (13 flaky)

✅ 4057 passed · ❌ 0 failed · 🟡 13 flaky · ⏭️ 103 skipped

Shard Passed Failed Flaky Skipped
✅ Shard 1 299 0 0 4
🟡 Shard 2 744 0 6 25
🟡 Shard 3 782 0 2 7
🟡 Shard 4 789 0 1 18
🟡 Shard 5 708 0 1 41
🟡 Shard 6 735 0 3 8
🟡 13 flaky test(s) (passed on retry)
  • Features/BulkEditEntity.spec.ts › Glossary (shard 2, 1 retry)
  • Features/Glossary/GlossaryWorkflow.spec.ts › should display correct status badge color and icon (shard 2, 2 retries)
  • Features/KnowledgeCenterList.spec.ts › Knowledge Center List - Test infinite scroll/pagination (shard 2, 1 retry)
  • Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
  • Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
  • Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
  • Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
  • Features/UserProfileOnlineStatus.spec.ts › Should not show online status for inactive users (shard 3, 1 retry)
  • Pages/CustomProperties.spec.ts › Should search custom properties for apiCollection in right panel (shard 4, 1 retry)
  • Pages/ExplorePageRightPanel_KnowledgeCenter.spec.ts › Should remove user owner for knowledgeCenter (shard 5, 1 retry)
  • Pages/Lineage/LineageFilters.spec.ts › Verify lineage schema filter selection (shard 6, 1 retry)
  • Pages/Teams.spec.ts › Add and Remove User for Team (shard 6, 1 retry)
  • Pages/Users.spec.ts › Reset Password for Data Consumer (shard 6, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@gitar-bot
Copy link
Copy Markdown

gitar-bot Bot commented May 14, 2026

Code Review ✅ Approved

Registers Drive entities and driveService in ENTITY_REFERENCE_CLASS_MAP to enable proper Automator workflow resolution. No issues were found.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@Khairajani Khairajani changed the title fix(ingestion): add Drive entities to ENTITY_REFERENCE_CLASS_MAP fix(ingestion): register Drive entities in ENTITY_REFERENCE_CLASS_MAP and ES_INDEX_MAP May 14, 2026
Copilot AI review requested due to automatic review settings May 14, 2026 19:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

@sonarqubecloud
Copy link
Copy Markdown

Copy link
Copy Markdown
Contributor

@Khairajani Khairajani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Khairajani Khairajani merged commit 98e22c5 into open-metadata:main May 15, 2026
46 checks passed
ulixius9 added a commit that referenced this pull request May 15, 2026
… and ES_INDEX_MAP (#28104)

* fix(ingestion): add Drive entities to ENTITY_REFERENCE_CLASS_MAP

The Automator app resolves resource types from resources.type against
ENTITY_REFERENCE_CLASS_MAP. Drive entities (directory, file, spreadsheet,
worksheet) and the driveService were missing, so selecting them in the
Add Automation UI failed at runtime with:

  AutomatorException: Can't get class from resource type: directory

Register the Drive data entities and driveService so the Automator can
paginate and act on them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Adding missng drive entity to ES MAP

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Khairajani <himanshukhairajani8@gmail.com>
Co-authored-by: Himanshu Khairajani <46777429+Khairajani@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ingestion safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants