Skip to content

feat: Handle analytics#289

Open
gewenyu99 wants to merge 3 commits intomainfrom
analytics
Open

feat: Handle analytics#289
gewenyu99 wants to merge 3 commits intomainfrom
analytics

Conversation

@gewenyu99
Copy link
Contributor

We have no good way of sending feature flag and custom properties to LLM Gateway, now we do. This should also unlock experiments for things that depend on LLMA.

Feature flags

The Wizard will send all feature flags with wizard prefix. Adds a getAllFlagsForWizard that fetches and caches all flags. In the future, we can reused this for everything to make sure we only evaluate flags once.

The LLM gateway will capture $feature/feature_flag_name: value in properties, this can be used for experiments.

Custom properties

We can only set custom properties once at the beginning of a run. What I propose is to have the concepts of variants controlled by a wizard-variant multivariate feature flag

Then, we can set what ever custom properties like this in constants.ts

/** Variant key -> metadata for wizard run (VARIANT flag selects which entry to use). */
export const WIZARD_VARIANTS: Record<string, Record<string, string>> = {
  base: { VARIANT: 'base' },
  subagents: { 
		VARIANT: 'subagents',
		FOO: 'BAR'
		POTATO: 'tomato'
 },
};

For example, if subagents is selected, all LLM gateway captures will include the custom properties variant: subagents, foo: bar, potato: tomato as custom properties.

See LLM gateway PR for more:
PostHog/posthog#49120

@github-actions
Copy link

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci android
  • /wizard-ci angular
  • /wizard-ci astro
  • /wizard-ci django
  • /wizard-ci fastapi
  • /wizard-ci flask
  • /wizard-ci javascript-node
  • /wizard-ci javascript-web
  • /wizard-ci laravel
  • /wizard-ci next-js
  • /wizard-ci nuxt
  • /wizard-ci python
  • /wizard-ci rails
  • /wizard-ci react-native
  • /wizard-ci react-router
  • /wizard-ci sveltekit
  • /wizard-ci swift
  • /wizard-ci tanstack-router
  • /wizard-ci tanstack-start
  • /wizard-ci vue

Test an individual app:

  • /wizard-ci android/Jetchat
  • /wizard-ci angular/angular-saas
  • /wizard-ci astro/astro-hybrid-marketing
Show more apps
  • /wizard-ci astro/astro-ssr-docs
  • /wizard-ci astro/astro-static-marketing
  • /wizard-ci astro/astro-view-transitions-marketing
  • /wizard-ci django/django3-saas
  • /wizard-ci fastapi/fastapi3-ai-saas
  • /wizard-ci flask/flask3-social-media
  • /wizard-ci javascript-node/express-todo
  • /wizard-ci javascript-node/fastify-blog
  • /wizard-ci javascript-node/hono-links
  • /wizard-ci javascript-node/koa-notes
  • /wizard-ci javascript-node/native-http-contacts
  • /wizard-ci javascript-web/saas-dashboard
  • /wizard-ci laravel/laravel12-saas
  • /wizard-ci next-js/15-app-router-saas
  • /wizard-ci next-js/15-app-router-todo
  • /wizard-ci next-js/15-pages-router-saas
  • /wizard-ci next-js/15-pages-router-todo
  • /wizard-ci nuxt/movies-nuxt-3-6
  • /wizard-ci nuxt/movies-nuxt-4
  • /wizard-ci python/meeting-summarizer
  • /wizard-ci rails/fizzy
  • /wizard-ci react-native/expo-react-native-hacker-news
  • /wizard-ci react-native/react-native-saas
  • /wizard-ci react-router/react-router-v7-project
  • /wizard-ci react-router/rrv7-starter
  • /wizard-ci react-router/saas-template
  • /wizard-ci react-router/shopper
  • /wizard-ci sveltekit/CMSaasStarter
  • /wizard-ci swift/hackers-ios
  • /wizard-ci tanstack-router/tanstack-router-code-based-saas
  • /wizard-ci tanstack-router/tanstack-router-file-based-saas
  • /wizard-ci tanstack-start/tanstack-start-saas
  • /wizard-ci vue/movies

Results will be posted here when complete.

@gewenyu99 gewenyu99 requested a review from a team February 25, 2026 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant