Skip to content

Schemas#480

Draft
maoberlehner wants to merge 56 commits intomainfrom
feature/WDX-264-schemas
Draft

Schemas#480
maoberlehner wants to merge 56 commits intomainfrom
feature/WDX-264-schemas

Conversation

@maoberlehner
Copy link
Contributor

This is a WIP PR containing that should get split up into multiple PRs (MAPI v1 and schemas) before merging.

Introduce a new `@storyblok/api-client` package that provides a modern,
typed TS client for the Content Delivery (CAPI) stories endpoints,
generated from our OpenAPI spec. This improves DX and consistency when
consuming the Stories API.

- Add OpenAPI CAPI stories spec and shared schemas for story field types
- Generate typed CAPI client (fetch-based) and SDK with `get` and `getAll` helpers
- Implement `createApiClient` wrapper with region-aware base URL and access token handling
- Add basic tests using MSW + OpenAPI mocks to validate `stories.get` and `stories.getAll`
- Configure build (tsdown), linting, testing, and npm packaging for the new package

Fixes WDX-281
Switch to the ky-based client plugin and configure automatic retries
(including 429s) with backoff to make the CAPI client more resilient to
transient failures. Extend tests to cover retry behavior and error
handling with and without `throwOnError`, and ignore generated sources
in git.
The API accepts both numeric IDs and string identifiers; update the
OpenAPI schema so clients generate correct types and validation matches
actual behavior
Use fileURLToPath with import.meta.url instead of __dirname so tests can
resolve the OpenAPI spec path correctly in an ES module environment
Align GitHub branding across package READMEs to improve documentation
consistency and professionalism
Fix typo and standardize "GitHub repo" phrasing to improve clarity for
users creating minimal reproducible examples when they can't share
company code
Use local CONTRIBUTING.md and correct package naming so contributors see
repo-specific guidelines instead of central .github docs
Align CAPI client and OpenAPI spec with personal access token security
by moving authentication from query parameters to the client auth
configuration, avoiding token leakage in URLs and keeping docs
consistent.
- Remove unused input type schemas that were never referenced by MAPI or CAPI
- Delete story-content-input and all field-type-input variants (231 lines)
- Remove duplicate components.yaml (BearerAuth already in security-schemes)
- Fix China region server URL (storyblok.cn → storyblokchina.cn)
- Normalize indentation in shared responses.yaml
Ensure story schema required fields and properties reflect actual
Storyblok responses and metadata, and document pagination headers on the
stories list endpoint to keep API docs accurate and complete
Introduce reusable cache strategy handlers (cache-first, network-first,
swr) so cache behavior is explicit, easier to reason about, and
override. Simplify SWR semantics to always return cached data when
present and revalidate in the background with per-key deduplication, and
improve in-memory cache defaults (implicit storedAt, documented
eviction).
Add optional relation inlining for stories.get and stories.getAll.

Also fetch missing rel_uuids automatically and resolve relation cycles safely.
Remove --compact from preview publishing so CI does not require packages to be pre-published on npm.
Align tests and utilities with the new stories client generation path
after directory restructuring.
The documented behavior for handling relation UUIDs no longer matches
the current client implementation, so removing it prevents misleading
usage assumptions and keeps the README accurate.
Prepare @storyblok/api-client for a 0.1.0 release so it can be published
and consumed as a publicly usable package
… auto-detection

Introduces a token-bucket throttle manager that limits concurrent CDN
requests by default, mirroring the server-side rate limit tiers:

- SINGLE_OR_SMALL (50 req/s): single story fetches or per_page ≤ 25
- MEDIUM (15 req/s): per_page 26–50
- LARGE (10 req/s): per_page 51–75
- VERY_LARGE (6 req/s): per_page 76–100

The throttle is on by default (auto-detect mode). Users can pass
`rateLimit: number` for a fixed limit, `rateLimit: { maxConcurrent,
adaptToServerHeaders }` for full control, or `rateLimit: false` to
disable entirely. Server-returned X-RateLimit-Policy headers are
respected and used to dynamically tighten limits at runtime.

Relation fetches (fetchMissingRelations) now go through the same
throttle manager, replacing the previous fixed worker-pool approach.
cv can be 0, which is a valid value used to bypass the CDN cache.
The previous truthy check would skip applyCvToQuery for this value,
causing cv=0 to never be forwarded in requests.
SWR background revalidation captures the query at request time, so a
stale response may carry an older cv. Without a guard, this would
overwrite a newer currentCv and potentially flush valid cached entries.
Now updateCv returns early if nextCv is lower than currentCv.
…thod

At-scale scenarios (distributed Redis, webhook-triggered invalidation,
SSG builds) require external control over when the cache is cleared.

Add cache.flush: 'auto' | 'manual' config option (default 'auto') and
expose client.flushCache() that clears the provider and resets currentCv.
With 'manual', cv changes are still tracked but no auto-flush occurs.
…runcation

fetchMissingRelations splits UUIDs into chunks of 50 (UUID_CHUNK_SIZE)
but did not pass per_page to the API. The Storyblok API defaults to
per_page=25, silently dropping roughly half of the requested stories.
Now per_page is explicitly set to UUID_CHUNK_SIZE on every chunk request,
matching the existing js-client behavior.
pickQueryContext iterated every key in QUERY_CONTEXT_KEYS and assigned
unconditionally, creating explicit undefined entries for keys absent from
baseQuery. These could serialize as query params like cv=undefined.
Now only defined values are copied into the returned query object.
Replace the verbose README with the standard template used across
packages in this repo (@storyblok/richtext). The detailed docs will be
hosted on the Storyblok docs platform separately.
The cv (cache version) parameter was already defined on datasource_entries
but was missing from both GET /v2/cdn/datasources and
GET /v2/cdn/datasources/{id}, even though the API accepts and returns it.
Storyblok _uid values are opaque strings, not RFC 4122 UUIDs. Using
format: uuid generates overly strict types and can reject valid API data
from tools that validate the format. Removed from all three _uid fields
in table-field.yaml (thead items, tbody items, body cell items).
Remove the packageManager field to avoid tightly coupling the package to
a specific pnpm version and update @types/node to align with the current
Node.js type definitions for better tooling support
Limit resolveFieldValue to internal use to reduce public API surface and
prevent unintended external dependencies
…chitecture

Working on MAPI client improvements revealed room for improvement in
the CAPI client's architecture and public API surface.

- Extract each API resource (stories, links, tags, datasources,
  datasource-entries, spaces) into dedicated modules under
  src/resources/ with factory functions and dependency injection
- Unify method signatures to accept an options object
  ({ query, signal, throwOnError }) instead of bare query parameters,
  enabling per-call throwOnError overrides and abort signal support
- Introduce ClientError class for structured HTTP error handling,
  replacing raw unknown errors via an error interceptor
- Switch rate-limit throttle from time-based token bucket to
  concurrency limiter that releases slots on request completion
- Deduplicate generated client/core boilerplate into
  src/generated/shared/ (~5x reduction in generated code)
- Standardize OpenAPI CAPI error responses via shared $ref
  definitions and mark story parent_id as nullable
- Switch build to unbundled output to preserve module structure
Add optional query parameter to datasources.get() for consistency with
all other resource methods in the client. While cv is auto-forwarded by
requestWithCache, the missing query option prevented users from passing
any endpoint-specific query parameters.
- Pass query params from cached requests through to `getSpaceApi` so
  the access token is included in the spaces/me URL
- Add test to assert token is present in the request URL for
  spaces.get()
- Update OpenAPI specs to model nullable fields with union types and
  add missing link date fields, keeping generated clients accurate
- Document preview tokens in QA env template/docs and clarify linting
  workflow with `--fix` to reduce setup and review friction
Set `per_page` on the relations query to align API pagination with the
UUID chunk size and avoid missing or duplicated relations, and simplify
`ClientError` to expose a single `response` object instead of redundant
fields
Support injecting a custom Fetch API-compatible implementation into the
client config so callers can integrate framework-specific fetch (e.g.
SSR caching) or add instrumentation/logging around requests.
…tions

Move throttle execution from requestWithCache to individual call sites so
throttle slots guard only HTTP round-trips, not post-processing. This
prevents nested throttle acquisition when fetchMissingRelations calls
throttleManager.execute while the outer requestWithCache slot is held,
which caused deadlocks in fixed rateLimit mode.
…document draft cache bypass

- stories.get() now automatically adds find_by=uuid when the identifier
  matches a UUID pattern, preventing silent 404s for callers who pass a
  UUID without the required query param
- Add JSDoc to CacheConfig and shouldUseCache documenting that version:draft
  requests always bypass the cache
…duce duplication

The relation-resolution logic (parse relations, build relation map, fetch
missing rel_uuids, populate map) was copy-pasted between get and getAll.
Extract it into a resolveRelationMap helper in inline-relations.ts to
reduce duplication and make future changes less error-prone.
Add the new `@storyblok/schema` package to centralize Storyblok
Management and Content API types and Zod schemas, generated from the
existing OpenAPI specs. This enables consumers (and internal packages)
to share a single, spec-driven schema layer and reuse consistent type
and validation logic instead of duplicating shapes.

Also fix and refine a few spec and generator edge cases so schema
generation is stable and accurate:
- Correct `force_update` default to match its string enum and avoid type
mismatches.
- Model `multilink.target` as a `oneOf` string/`null` schema for clearer
validation.
- Patch `openapi-zod-client` schema resolver normalization so spec
schema names resolve reliably across MAPI/CAPI.
- Wire patched dependency and workspace config so generation is
reproducible.
- Update task docs to reflect the new package, its responsibilities, and
revised part ordering.
… type-safe story content

- Rename generated types from snake_case to PascalCase and split MAPI types into a dedicated mapi-types.ts output file
- Refactor generate.ts to produce separate CAPI/MAPI type and zod-schema files
- Add const generic to defineField to preserve literal component_whitelist values
- Export FieldValue type and CAPI generated types from schema index
- Add Schema generic to createApiClient and createStoriesResource so callers can pass a component union for narrowed story content types
- Add type-level tests covering Schema generic narrowing, bloks whitelist resolution, and fallback behaviour
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant