Skip to content

feat: emit metadata events for schema changes with actor context for webhooks#17622

Open
mabdullahabaid wants to merge 24 commits intomainfrom
metadata-events
Open

feat: emit metadata events for schema changes with actor context for webhooks#17622
mabdullahabaid wants to merge 24 commits intomainfrom
metadata-events

Conversation

@mabdullahabaid
Copy link
Member

Summary

This PR adds metadata eventing: when schema metadata (objectMetadata, fieldMetadata, view, viewField, etc.) is created, updated, or deleted, we now emit events that can trigger webhooks and future audit logs. It also adds actor context (userId, workspaceMemberId) to those events so subscribers can attribute changes to a user or API key.

What changed

1. Metadata eventing (first commit)

  • MetadataEventEmitter
    New service that emits batch events after successful workspace migrations. Event names follow metadata.{entity}.{action} (e.g. metadata.objectMetadata.created, metadata.fieldMetadata.updated).
  • MetadataEventsToDbListener
    Listens for metadata events and enqueues webhook delivery via CallWebhookJobsForMetadataJob.
  • Event types (twenty-shared)
    MetadataEventAction, MetadataEventBatch, and record event types for create/update/delete.
  • WorkspaceMigrationValidateBuildAndRunService
    Calls the metadata event emitter after running migrations so all metadata changes (from any module) emit events from a single place.
  • Create events
    Sourced from the create action payload (flatEntity / flatFieldMetadatas) because fromToAllFlatEntityMaps does not provide a before/after diff for creates. Update/delete events still use the fromToAllFlatEntityMaps comparison.

2. Actor context (second commit)

  • MetadataEventEmitter
    Accepts optional actorContext (userId, workspaceMemberId) and includes it on emitted batch events.
  • WorkspaceMigrationValidateBuildAndRunService
    Passes actorContext from the request into the metadata event emitter.
  • Metadata resolvers & services
    All metadata modules resolve @AuthUser({ allowUndefined: true }) and @AuthUserWorkspaceId() and pass userId and workspaceMemberId through to the migration/event pipeline. Both are optional so API-key–authenticated requests (no user) still emit events without a user identity.

Shared some questions on Discord about the PR.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 2, 2026

Warnings
⚠️

Changes were made to package.json, but not to yarn.lock - Perhaps you need to run yarn install?

Generated by 🚫 dangerJS against e019a94

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 53 files

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 2, 2026

Greptile Overview

Greptile Summary

This PR adds metadata eventing to track schema changes (objectMetadata, fieldMetadata, views, etc.) and emit events for webhooks and future audit logs. It also threads actor context (userId, workspaceMemberId) through all metadata operations to attribute changes.

Key changes:

  • Implemented MetadataEventEmitter service that groups workspace migration actions into batch events and emits them via NestJS EventEmitter2
  • Created MetadataEventsToDbListener that enqueues webhook jobs when metadata events are triggered
  • Added CallWebhookJobsForMetadataJob to match webhooks by operation patterns and dispatch events
  • Extended all metadata resolvers (field, object, view, webhook, etc.) with @AuthUser({ allowUndefined: true }) and @AuthUserWorkspaceId() decorators to capture actor context
  • Threaded actor context through service layers down to WorkspaceMigrationValidateBuildAndRunService
  • Defined event types in twenty-shared package for create/update/delete operations

Implementation approach:

  • Events are sourced from workspace migrations after successful execution
  • Create events use flatEntity/flatFieldMetadatas from action payload (since fromToAllFlatEntityMaps lacks before/after for creates)
  • Update/delete events use fromToAllFlatEntityMaps comparison to generate diff and identify changed fields
  • Actor context is optional to support API-key authentication without user identity

Confidence Score: 4/5

  • Safe to merge with minor style inconsistencies that should be addressed
  • The implementation is solid with good separation of concerns and proper event handling. Score reflects some minor code style issues (import ordering, enum usage) and lack of tests, but the core logic is sound and follows established patterns
  • Pay attention to packages/twenty-server/src/engine/metadata-event-emitter/enums/metadata-event-action.enum.ts (uses enum instead of string literals per guidelines) and resolver files with inconsistent import ordering

Important Files Changed

Filename Overview
packages/twenty-server/src/engine/metadata-event-emitter/metadata-event-emitter.ts New service that emits batch metadata events from workspace migrations, groups events by entity and action type
packages/twenty-server/src/engine/metadata-event-emitter/listeners/metadata-events-to-db.listener.ts Event listener that enqueues webhook jobs when metadata events are emitted
packages/twenty-server/src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service.ts Updated to accept and pass actor context (userId, workspaceMemberId) to metadata event emitter
packages/twenty-server/src/engine/metadata-modules/webhook/jobs/call-webhook-jobs-for-metadata.job.ts New job that transforms metadata events into webhook payloads and dispatches them to matching webhooks

Sequence Diagram

sequenceDiagram
    participant Client
    participant Resolver
    participant Service
    participant MigrationService as WorkspaceMigrationValidateBuildAndRunService
    participant Runner as WorkspaceMigrationRunnerService
    participant EventEmitter as MetadataEventEmitter
    participant EventEmitter2 as NestJS EventEmitter2
    participant Listener as MetadataEventsToDbListener
    participant WebhookQueue as Webhook Message Queue
    participant WebhookJob as CallWebhookJobsForMetadataJob
    participant WebhookRepo as WebhookRepository
    participant CallWebhook as CallWebhookJob

    Client->>Resolver: GraphQL Mutation (create/update/delete metadata)
    Note over Resolver: @AuthUser() user<br/>@AuthUserWorkspaceId() workspaceMemberId
    Resolver->>Service: Pass userId & workspaceMemberId
    Service->>MigrationService: validateBuildAndRunWorkspaceMigration({ actorContext })
    MigrationService->>MigrationService: Build workspace migration
    MigrationService->>Runner: run(workspaceMigration)
    Runner-->>MigrationService: Migration complete
    MigrationService->>EventEmitter: emitMetadataEventsFromMigration({ actions, actorContext })
    EventEmitter->>EventEmitter: groupActionsByMetadataNameAndAction()
    loop For each metadata entity & action
        EventEmitter->>EventEmitter: emitMetadataBatchEvent({ userId, workspaceMemberId })
        EventEmitter->>EventEmitter2: emit('metadata.{entity}.{action}', MetadataEventBatch)
    end
    EventEmitter2->>Listener: @OnEvent('metadata.*.created/updated/deleted')
    Listener->>WebhookQueue: Add CallWebhookJobsForMetadataJob
    WebhookQueue->>WebhookJob: Process MetadataEventBatch
    WebhookJob->>WebhookRepo: Find matching webhooks
    WebhookRepo-->>WebhookJob: Webhook entities
    WebhookJob->>WebhookJob: transformMetadataEventBatchToWebhookEvents()
    Note over WebhookJob: Includes userId & workspaceMemberId<br/>in webhook payload
    loop For each webhook event chunk
        WebhookJob->>WebhookQueue: Add CallWebhookJob batch
    end
    WebhookQueue->>CallWebhook: Process webhook delivery
    CallWebhook->>Client: HTTP POST to webhook URL

Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 files reviewed, 12 comments

Edit Code Review Agent Settings | Greptile

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 2, 2026

Additional Comments (3)

packages/twenty-server/src/engine/metadata-modules/page-layout-tab/resolvers/page-layout-tab.resolver.ts
imports not in alphabetical order

AuthUser should come before AuthUserWorkspaceId alphabetically

import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthUserWorkspaceId } from 'src/engine/decorators/auth/auth-user-workspace-id.decorator';

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/twenty-server/src/engine/metadata-modules/page-layout-tab/resolvers/page-layout-tab.resolver.ts
Line: 7:8

Comment:
imports not in alphabetical order

`AuthUser` should come before `AuthUserWorkspaceId` alphabetically

```suggestion
import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthUserWorkspaceId } from 'src/engine/decorators/auth/auth-user-workspace-id.decorator';
```

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

packages/twenty-server/src/engine/metadata-modules/page-layout-widget/resolvers/page-layout-widget.resolver.ts
imports not in alphabetical order

import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthUserWorkspaceId } from 'src/engine/decorators/auth/auth-user-workspace-id.decorator';

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/twenty-server/src/engine/metadata-modules/page-layout-widget/resolvers/page-layout-widget.resolver.ts
Line: 7:8

Comment:
imports not in alphabetical order

```suggestion
import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthUserWorkspaceId } from 'src/engine/decorators/auth/auth-user-workspace-id.decorator';
```

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

packages/twenty-server/src/engine/metadata-modules/view-group/resolvers/view-group.resolver.ts
imports not in alphabetical order

import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthUserWorkspaceId } from 'src/engine/decorators/auth/auth-user-workspace-id.decorator';

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/twenty-server/src/engine/metadata-modules/view-group/resolvers/view-group.resolver.ts
Line: 11:12

Comment:
imports not in alphabetical order

```suggestion
import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthUserWorkspaceId } from 'src/engine/decorators/auth/auth-user-workspace-id.decorator';
```

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 2, 2026

🚀 Preview Environment Ready!

Your preview environment is available at: http://bore.pub:43941

This environment will automatically shut down when the PR is closed or after 5 hours.

@@ -0,0 +1 @@
export type MetadataEventAction = 'created' | 'updated' | 'deleted';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this metadata-event-emitter be in core modules next to event-emitter module?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had the same thoughts, but workspace-event-emitter was in the engine too directly while being registered inside core modules, so I followed the convention.

@prastoin prastoin self-requested a review February 3, 2026 09:50
@prastoin
Copy link
Contributor

prastoin commented Feb 3, 2026

Hey there, on prod right now will review afterwards

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files (changes from recent commits).

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="packages/twenty-front/src/modules/settings/components/SettingsDatabaseEventsForm.tsx">

<violation number="1" location="packages/twenty-front/src/modules/settings/components/SettingsDatabaseEventsForm.tsx:70">
P2: `WebhookEntitySelect` doesn’t actually disable interaction. Even with `disabled={disabled}`, the dropdown can still open and `onChange` will fire, so read-only forms can still modify the entity selection. Update `WebhookEntitySelect` to block clicks/selection when disabled (e.g., pass `disableClickForClickableComponent` to `Dropdown` and guard `handleSelect`).</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Copy link
Contributor

@prastoin prastoin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there ! NIce job overall !
Main concerns:

  • We might wanna rethink the introduced type context. We might wanna use the AuthContext ( for apiKey support too )
  • Using discriminated union in the twenty-shared type system

import type { AllUniversalWorkspaceMigrationAction } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/workspace-migration-action-common';

type MetadataEventActorContext = {
userId?: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Could both properties be optional at the same time ? same question with both defined ? ( if not please use one or the other pattern )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

workspaceMemberId is the identifier for workspace‑scoped things like RLS, permissions, and audit trails, while userId is the stable, global identity that helps consumers correlate activity across workspaces or even if a membership is removed/recreated. So for user‑initiated actions that impact metadata, we should set both in my opinion.

Removing workspaceMemberId for now since we can add it later if users request, but I have a feeling that for org-level consumers with multiple workspaces, both can be relevant when integrating with internal systems.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really @mabdullahabaid, for RLS, permissions, etc, we would rather use userWorkspaceId (a third contender!). You can always go from WorkspaceMemberId to userWorkspaceId or to userId, or from userWorkspaceMemberId to WorkspaceMemberId or userId, but you need the workspaceId to go from userId to WorkspaceMemberId or UserWorpsaceId.

In any case, as discussed, I think you should use the existing context object as much as possible rather than re-inventing your own

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FelixMalfait workspaceId is always emitted in the webhook payload. So a combination of workspaceId and userId should be sufficient to reach userWorkspaceId or workspaceMemberId in that case.

As mentioned, I removed the workspaceMemberId for this reason since a combination of userId and workspaceId helps locate workspaceMemberId and userWorkspaceId when needed.

That works?

'*.*',
];

const webhooks = await this.webhookRepository.find({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: cc @charlesBochet do we aim having a dynamic custom webhook flat maps cache entries per operations that could ligthen database impact ?

private getUpdatedFieldsFromEvent(
event: MetadataRecordEvent,
): string[] | undefined {
if ('updatedFields' in event.properties) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Not mandatory but using Object.prototype.hasOwnProperty.call is safer and more performant ( especially for the expected volume here )
Same for aboves


const WEBHOOK_JOBS_CHUNK_SIZE = 20;

@Processor(MessageQueue.webhookQueue)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Praise: Overall very clean !

metadataEventBatch: MetadataEventBatch<MetadataRecordEvent>;
webhooks: WebhookEntity[];
}): CallWebhookJobData[] {
const result: CallWebhookJobData[] = [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of scope: We could improve CallWebhookJobData typing by using a discriminated union.
Even the end user as to inspect response integrity in order to determine available fields.
And so our own devxp would be enhanced not having to infer context by properties shape

Comment on lines 345 to 346
const universalIdentifier =
'universalIdentifier' in action ? action.universalIdentifier : undefined;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    const universalIdentifier =
      action.universalIdentifier

action: AllUniversalWorkspaceMigrationAction,
metadataName: AllMetadataName,
fromToAllFlatEntityMaps: FromToAllFlatEntityMaps,
result: GroupedEvents,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remark: not a fan of mutations here

fromToAllFlatEntityMaps: FromToAllFlatEntityMaps,
result: GroupedEvents,
): void {
const fieldMetadatas =
Copy link
Contributor

@prastoin prastoin Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remark: using in should always be as last resort. 99% of the time this could be prevented by improving args typing

>;

@Injectable()
export class MetadataEventEmitter {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @mabdullahabaid lets have a peer session on this file if you feel like to

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 39 files (changes from recent commits).

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="packages/twenty-server/src/engine/metadata-event-emitter/metadata-event-emitter.ts">

<violation number="1" location="packages/twenty-server/src/engine/metadata-event-emitter/metadata-event-emitter.ts:77">
P2: User-initiated metadata events no longer include `workspaceMemberId`. The initiator context contains this field, but the batch event only forwards `userId` and `apiKeyId`, which breaks the stated actor context requirement and removes member attribution for user-authenticated changes.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

@prastoin
Copy link
Contributor

prastoin commented Feb 4, 2026

Hey @mabdullahabaid I've started to refactor the main emitter service as discussed on discord
It still need some tsc-errors fixes here and there.
Your PR is blocked by #17687
Also we will need to transpile from UniversalFlatEntity to FlatEntity the metadata that has been migrated to be fully workspace agnostic within workspace migration
In my opinion, we will have to recompute the cache post run in order to avoid having a painful redundant transpilation step.
This way we will only keep the from out of the current fromToAllFlatEntityMaps

@prastoin
Copy link
Contributor

prastoin commented Feb 4, 2026

How to get this PR merged

I will try to split the dependency PR in order to unlock this one
As we migrate metadata to the agnostic workspace migration pattern, this will introduce a "data loss regression" ( sending less data especially ids ) in the current created event emition

TL;DR => Lets refactor the fromToFlatEntityMaps.to computation not to be passed from the api metadata directly but from a post runner cache recomputation

Comment on lines +219 to +223
this.metadataEventEmitter.emitMetadataEventsFromMigration({
actions: validateAndBuildResult.workspaceMigration.actions,
fromToAllFlatEntityMaps: args.fromToAllFlatEntityMaps,
workspaceId: args.workspaceId,
});
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: System-level operations like workspace creation trigger metadata events without actor context (userId, apiKeyId) because getWorkspaceAuthContext() fails outside of an HTTP request context.
Severity: MEDIUM

Suggested Fix

Modify the signature of validateBuildAndRunWorkspaceMigrationFromTo to accept an optional initiatorContext. Propagate this context from its callers, such as synchronizeTwentyStandardApplicationOrThrow, and pass it to emitMetadataEventsFromMigration. This ensures system-initiated events can be properly attributed.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location:
packages/twenty-server/src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service.ts#L219-L223

Potential issue: During system-level operations such as workspace initialization or dev
seeding, the `validateBuildAndRunWorkspaceMigrationFromTo` function is called. This
triggers `emitMetadataEventsFromMigration` without an `initiatorContext`. The fallback
mechanism, `getWorkspaceAuthContext()`, fails because these operations run outside of an
HTTP request's async context. A `try-catch` block silently handles this failure,
resulting in metadata events and webhooks being sent without `userId` or `apiKeyId`.
This creates inconsistent audit trails, where user-initiated actions have actor context
but system-initiated ones do not.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="packages/twenty-server/src/engine/metadata-event-emitter/metadata-event-emitter.ts">

<violation number="1" location="packages/twenty-server/src/engine/metadata-event-emitter/metadata-event-emitter.ts:400">
P2: Index update actions are explicitly ignored, so metadata.index.updated events are never emitted. This creates a gap where index updates won’t trigger webhooks/audit events even though other metadata updates do. Consider emitting an update event using the same entityId-to-universalIdentifier lookup used for other entityId-based metadata actions (view, role, etc.) or provide a concrete custom mapping instead of returning undefined.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

fromToAllFlatEntityMaps: FromToAllFlatEntityMaps;
}): MetadataUpdateEventWithMetadataName | undefined {
switch (action.metadataName) {
case 'index': {
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Index update actions are explicitly ignored, so metadata.index.updated events are never emitted. This creates a gap where index updates won’t trigger webhooks/audit events even though other metadata updates do. Consider emitting an update event using the same entityId-to-universalIdentifier lookup used for other entityId-based metadata actions (view, role, etc.) or provide a concrete custom mapping instead of returning undefined.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/twenty-server/src/engine/metadata-event-emitter/metadata-event-emitter.ts, line 400:

<comment>Index update actions are explicitly ignored, so metadata.index.updated events are never emitted. This creates a gap where index updates won’t trigger webhooks/audit events even though other metadata updates do. Consider emitting an update event using the same entityId-to-universalIdentifier lookup used for other entityId-based metadata actions (view, role, etc.) or provide a concrete custom mapping instead of returning undefined.</comment>

<file context>
@@ -423,6 +397,10 @@ export class MetadataEventEmitter {
     fromToAllFlatEntityMaps: FromToAllFlatEntityMaps;
   }): MetadataUpdateEventWithMetadataName | undefined {
     switch (action.metadataName) {
+      case 'index': {
+        // TODO implement custom index update action transpiler as it's not like the others
+        return undefined
</file context>
Fix with Cubic

@mabdullahabaid
Copy link
Member Author

@prastoin thank you so much for giving a hand with this. I shall wait while taking up some other tasks in the meantime. Please let me know when you're done with the refactor on your end. I shall follow the lead and update my work accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants