[FEATURE] - Publish only changed Lakehouse Shortcuts

[bckstrm]

Use Case / Problem

At the moment, the publish_all() in the ShortcutPublisher class (_lakehouse.py) republishes all shortcuts, even when a shotcut is already deployed and its definition is unchanged. In use cases when you have very many shortcuts, this causes unneccesary API calls and longer deployment times.

Proposed Solution

As the workflow already fetches deployed_shortcuts , we can easily make a comparision of the shortcut content and only republish shortcuts where we have changes. If the shortcut is already published at current state, we skip republish.

Alternatives Considered

No response

Impact Assessment

• This would help me personally
• This would help my team/organization
• This would help the broader fabric-cicd community
• This aligns with Microsoft Fabric roadmap items

Implementation Attestation

• I understand this feature should maintain backward compatibility (if applicable)
• I understand this feature should not introduce performance regressions for existing workflows
• I acknowledge that new features must follow fabric-cicd's established patterns and conventions

Implementation Notes

No response

[shirasassoon]

/triaged

@bckstrm Thanks for raising this feature request. We support a selective deployment feature for shortcuts where you can define a regex pattern that excludes specific shortcuts from publish - would this help in your scenario?

[shirasassoon]

/needinfo

[shirasassoon]

/triaged

[shirasassoon]

/needinfo

[bckstrm]

@shirasassoon
I would say that the regex exclude pattern is not sufficient for our use case. Since all shortcuts are defined within the same file, we would need to implement custom logic to compare our local shortcuts with what is already deployed via calls to the Fabric API. That’s something we’re trying to avoid, as we want to rely solely on fabric-cicd for this functionality.

Instead, I would suggest introducing a new feature flag, ENABLE_SHORTCUT_SMART_DIFF, which performs a field-by-field comparison and only publishes shortcuts that are either not yet deployed or have changes.

What do you think about this approach? If it sounds reasonable, I’d be happy to put together a proposal.

[microsoft-github-policy-service]

Thank you for replying with additional information, @bckstrm.
A member of the Fabric CICD team will continue reviewing this issue.

[bckstrm]

Proposal in #916