wip

Author: pcattori

packages/route-pattern/README.md

@@ -14,8 +14,8 @@ It does not discuss the algorithms nor data structures used by the matching engi
 ## Non-goals
 
 - Matching URL fragments (`#section`)
-- Matching the URL port (`:8080`)
-- Matching the URL credentials (`user:pass@`)
+- Matching URL port (`:8080`)
+- Matching URL credentials (`user:pass@`)
 - Caching
 - Request/Response handling
 
@@ -64,7 +64,7 @@ You can use any combination of these to create a route pattern, for example:
 // ...and so on...
 ```
 
-**Delimiters:** Route patterns use the first occurrences of `://`, `/`, and `?` as delimiters to split a route pattern into its parts.
+**Part delimiters:** Route patterns use the first occurrences of `://`, `/`, and `?` as delimiters to split a route pattern into its parts.
 Pathname-only route patterns are the most common, so route patterns are assumed to be pathname-only unless `://` or `?` are present.
 As a result, hostnames must begin with `://` and searches must begin with `?` to distinguish both from pathnames.
 
@@ -90,107 +90,35 @@ However, omitting a pathname means "match the 'empty' pathname" (namely `""` and
 // ✗ doesn't match: https://api.example.com/users
 ```
 
-
 ## Pattern modifiers
 
-Before describing [wildcards](#wildcards), [params](#params), and [optionals](#optionals),
-its important to note that each pattern modifier applies only in the same part of the URL where it appears.
+Each pattern modifier — [param](#params), [glob](#globs), or [optional](#optionals) — applies only in the same part of the URL where it appears.
 As a result:
 
-- Wildcards and params do not match characters that appear outside of their part of the route pattern
+- Params and globs do not match characters that appear outside of their part of the route pattern
 - Optionals must begin and end within the same part of the route pattern
 
-### Wildcards
-
-|            | protocol | hostname | pathname | search |
-| ---------- | -------- | -------- | -------- | ------ |
-| Supported? | ❌       | ✅       | ✅       | ❌     |
-
-Wildcards match dynamic parts of a URL.
-
-Route patterns support two types of wildcards:
-
-- `*` ("star") for matching anything _within_ a segment
-- `**` ("star star") for matching anything, even across multiple segments
-
-As a result, wildcards correspond to these regular expressions:
-
-|          | `*`       | `**`   |
-| -------- | --------- | ------ |
-| hostname | `/[^.]*/` | `/.*/` |
-| pathname | `/[^/]*/` | `/.*/` |
-
-```ts
-'/files/*';
-// ✓ matches: /files/photo.jpg
-// ✗ doesn't match: /files/2023/photo.jpg
-
-'/docs/**';
-// ✓ matches: /docs/api/v1/intro.html
-// ✗ doesn't match: /docs (no trailing content)
-
-'://*.example.com';
-// ✓ matches: ://cdn.example.com
-// ✗ doesn't match: ://api.staging.example.com
-
-'://**.api.com';
-// ✓ matches: ://tenant.v1.api.com
-// ✗ doesn't match: ://api.com (no prefix)
-```
-
-Route patterns can have multiple wildcards, even within the same segment.
-
-```ts
-'/assets/**/static/**/*.css';
-// ✓ matches: /assets/v2/themes/static/dark/main.css
-// ✗ doesn't match: /assets/v2/themes/static/main.js
-
-'://us-**.cdn.com/cars/*-*';
-// ✓ matches: ://us-east.staging.cdn.com/cars/audi-a4.jpg
-// ✗ doesn't match: ://us-east.staging.cdn.com/cars/toyota.jpg
-```
-
-Wildcards only match characters within the same part of the URL:
-
-```ts
-'://api.**/users';
-// ✓ matches: ://api.example.com/users
-// ✗ doesn't match: ://api.example.com/123/users
-```
-
 ### Params
 
 |            | protocol | hostname | pathname | search |
 | ---------- | -------- | -------- | -------- | ------ |
 | Supported? | ❌       | ✅       | ✅       | ❌     |
 
-Params, like wildcards, match dynamic parts of the URL but they also give you access to the matched values.
-
-A param is written as:
-
-- `:` followed by a name for capturing anything within a segment (similar to `*`)
-- `::` followed by a name for capturing anything, even across multiple segments (similar to `**`)
-
-**Note:** Param names must be [JavaScript identifiers](#javascript-identifier).
+Params match dynamic parts of a URL within a segment.
 
-As a result, params correspond to these regular expressions:
-
-|          | `:<name>`   | `::<name>` |
-| -------- | ----------- | ---------- |
-| hostname | `/([^.]*)/` | `/(.*)/`   |
-| pathname | `/([^/]*)/` | `/(.*)/`   |
+They are written as a `:` optionally followed by a [JavaScript identifier](#javascript-identifier) that acts as its name:
 
 ```ts
 'products/:id';
 // /products/wireless-headphones → { id: 'wireless-headphones' }
 // /products/123 → { id: '123' }
+```
 
-// ❌ Error - missing or invalid param name
-'products/:123';
+When a param name is not given, the matched value won't be returned:
 
-'docs/::path';
-// /docs/api/v1/intro.html → { path: 'api/v1/intro.html' }
-// /docs/guide → { path: 'guide' }
+```ts
+'products/:-shoes';
+// /products/tennis-shoes -> {}
 ```
 
 Param names must be unique:
@@ -204,13 +132,9 @@ Param names must be unique:
 
 // ❌ Bad - duplicate param name across hostname and pathname
 '://:region.api.example.com/users/:region';
-
-// ✅ Good - `::` param captures across segments
-'files/::path/download';
-// /files/2023/photos/vacation.jpg/download → { path: '2023/photos/vacation.jpg' }
 ```
 
-Params can be mixed with static text, wildcards, and even other params:
+Params can be mixed with static text and even other params:
 
 ```ts
 'users/@:id';
@@ -222,22 +146,34 @@ Params can be mixed with static text, wildcards, and even other params:
 'api/v:major.:minor-:channel';
 // /api/v2.1-beta → { major: '2', minor: '1', channel: 'beta' }
 
-'://:region.:env.api.example.com';
-// us-east.staging.api.example.com → { region: 'us-east', env: 'staging' }
+'://us-:region.:env.api.example.com';
+// us-east.staging.api.example.com → { region: 'east', env: 'staging' }
+```
+
+### Globs
+
+|            | protocol | hostname | pathname | search |
+| ---------- | -------- | -------- | -------- | ------ |
+| Supported? | ❌       | ✅       | ✅       | ❌     |
+
+Globs match dynamic parts of a URL, but — unlike [params](#params) — they are not limited to a single segment.
 
-'cdn/::path/*.jpg';
-// /cdn/images/2023/vacation.jpg → { path: 'images/2023' }
+They are written as a `*` optionally followed by a [JavaScript identifier](#javascript-identifier) that acts as its name:
 
-'://::subdomain.api.com/:version/*';
-// tenant.v1.api.com/v2/users → { subdomain: 'tenant.v1', version: 'v2' }
+```ts
+// todo
+```
+
+When a glob name is not given, the matched value won't be returned:
+
+```ts
+// todo
 ```
 
-Params only match characters within the same part of the URL:
+Globs share a namespace with params:
 
 ```ts
-'://api.::domain/users';
-// ✓ matches: ://api.example.com/users → { domain: 'example.com' }
-// ✗ doesn't match: ://api.example.com/123/users
+// todo
 ```
 
 ### Optionals
@@ -274,23 +210,23 @@ Optionals can span any characters and contain static text, params, or wildcards:
 // /users/sarah/settings/profile → { id: 'sarah', section: 'profile' }
 // /users/sarah/settings/profile/edit → { id: 'sarah', section: 'profile' }
 
-'users/:userId(/files/*)';
+'users/:userId(/files/:)';
 // /users/sarah → { userId: 'sarah' }
 // /users/sarah/files/document.pdf → { userId: 'sarah' }
 
-'users/:userId(/docs/**)';
+'users/:userId(/docs/*)';
 // /users/sarah → { userId: 'sarah' }
 // /users/sarah/docs/projects/readme.md → { userId: 'sarah' }
 
-'users/:userId(/files/::path)';
+'users/:userId(/files/*path)';
 // /users/sarah → { userId: 'sarah' }
 // /users/sarah/files/projects/docs/readme.md → { userId: 'sarah', path: 'projects/docs/readme.md' }
 
 '://(www.)shop.example.com';
 // shop.example.com → {}
 // www.shop.example.com → {}
 
-'://(*.)api.example.com(/v*)';
+'://(:.)api.example.com(/v:)';
 // api.example.com → {}
 // cdn.api.example.com/v2 → {}
 ```

packages/route-pattern/src/lib/parse.test.ts

@@ -0,0 +1,13 @@
+import * as assert from 'node:assert/strict';
+import { describe, it } from 'node:test';
+
+import { parseProtocol } from './parse.ts';
+
+describe('parse', () => {
+  it('parses protocol', () => {
+    assert.deepStrictEqual(parseProtocol('http(s)'), [
+      { span: [0, 4], type: 'text', value: 'http' },
+      { span: [4, 3], type: 'optional', nodes: [{ span: [5, 1], type: 'text', value: 's' }] },
+    ]);
+  });
+});

packages/route-pattern/src/lib/parse.ts

@@ -0,0 +1,42 @@
+import { lexHostname, lexPathname, lexProtocol, type Token } from './tokenize.ts';
+
+type Optional = { type: 'optional'; nodes: Array<Token>; span: [number, number] };
+type Node = Token | Optional;
+
+function parseOptionals(tokens: Iterable<Token>) {
+  const nodes: Array<Node> = [];
+
+  let optional: Optional | null = null;
+  for (const token of tokens) {
+    if (token.type === '(') {
+      if (optional) {
+        throw new Error(`Nested paren: ${optional.span[0]} ${token.span[0]}`);
+      }
+      optional = { type: 'optional', nodes: [], span: token.span };
+      continue;
+    }
+    if (token.type === ')') {
+      if (!optional) {
+        throw new Error(`Unbalanced paren: ${token.span[0]}`);
+      }
+      optional.span[1] = optional.nodes.reduce((acc, node) => acc + node.span[1], 0) + 2;
+      nodes.push(optional);
+      optional = null;
+      continue;
+    }
+    (optional?.nodes ?? nodes).push(token);
+  }
+  return nodes;
+}
+
+export function parseProtocol(source: string) {
+  return parseOptionals(lexProtocol(source));
+}
+
+export function parseHostname(source: string) {
+  return parseOptionals(lexHostname(source));
+}
+
+export function parsePathname(source: string) {
+  return parseOptionals(lexPathname(source));
+}

packages/route-pattern/src/lib/tokenize.test.ts

@@ -0,0 +1,38 @@
+import * as assert from 'node:assert/strict';
+import { describe, it } from 'node:test';
+
+import { lexProtocol, lexHostname, lexPathname } from './tokenize.ts';
+
+describe('lex', () => {
+  it('lexes protocol', () => {
+    assert.deepStrictEqual(Array.from(lexProtocol('http(s)')), [
+      { span: [0, 4], type: 'text', value: 'http' },
+      { span: [4, 1], type: '(' },
+      { span: [5, 1], type: 'text', value: 's' },
+      { span: [6, 1], type: ')' },
+    ]);
+  });
+  it('lexes hostname', () => {
+    assert.deepStrictEqual(Array.from(lexHostname('(*tenant.:sub.)remix.run')), [
+      { span: [0, 1], type: '(' },
+      { span: [1, 7], type: 'glob', name: 'tenant' },
+      { span: [8, 1], type: 'text', value: '.' },
+      { span: [9, 4], type: 'param', name: 'sub' },
+      { span: [13, 1], type: 'text', value: '.' },
+      { span: [14, 1], type: ')' },
+      { span: [15, 9], type: 'text', value: 'remix.run' },
+    ]);
+  });
+  it('lexes pathname', () => {
+    assert.deepStrictEqual(Array.from(lexPathname('/products/:id(/v:version/*path)')), [
+      { span: [0, 10], type: 'text', value: '/products/' },
+      { span: [10, 3], type: 'param', name: 'id' },
+      { span: [13, 1], type: '(' },
+      { span: [14, 2], type: 'text', value: '/v' },
+      { span: [16, 8], type: 'param', name: 'version' },
+      { span: [24, 1], type: 'text', value: '/' },
+      { span: [25, 5], type: 'glob', name: 'path' },
+      { span: [30, 1], type: ')' },
+    ]);
+  });
+});

packages/route-pattern/src/lib/tokenize.ts

@@ -0,0 +1,71 @@
+export type Token = { span: [number, number] } & (
+  | { type: 'text'; value: string }
+  | { type: 'param'; name: string }
+  | { type: 'glob'; name: string }
+  | { type: '(' | ')' }
+);
+
+type Lexer = (source: string, index: number) => Token | null;
+
+function* tokenize(source: string, lexers: Array<Lexer>): Generator<Token> {
+  let buffer = '';
+
+  let index = 0;
+  while (index < source.length) {
+    let token: Token | null = null;
+    for (const lex of lexers) {
+      token = lex(source, index);
+      if (token) break;
+    }
+
+    if (token) {
+      if (buffer) {
+        yield { type: 'text', value: buffer, span: [index - buffer.length, buffer.length] };
+        buffer = '';
+      }
+      yield token;
+      index += token.span[1];
+    } else {
+      buffer += source[index];
+      index += 1;
+    }
+  }
+
+  if (buffer) {
+    yield { type: 'text', value: buffer, span: [source.length - buffer.length, buffer.length] };
+  }
+}
+
+const parensLexer: Lexer = (source, index) => {
+  const char = source[index];
+  if (char === '(' || char === ')') {
+    return { type: char, span: [index, 1] };
+  }
+  return null;
+};
+
+const identifierRE = /[a-zA-Z_$][a-zA-Z_$0-9]*/;
+
+const paramRE = new RegExp('^:(' + identifierRE.source + ')?');
+const paramLexer: Lexer = (source, index) => {
+  const match = paramRE.exec(source.slice(index));
+  if (!match) return null;
+  const name = match[1];
+  if (name === undefined) throw new Error('todo: missing param name');
+  return { type: 'param', name, span: [index, match[0].length] };
+};
+
+const globRE = new RegExp('^\\*(' + identifierRE.source + ')?');
+const globLexer: Lexer = (source, index) => {
+  const match = globRE.exec(source.slice(index));
+  if (!match) return null;
+  const name = match[1];
+  if (name === undefined) throw new Error('todo: missing glob name');
+  return { type: 'glob', name, span: [index, match[0].length] };
+};
+
+export const lexProtocol = (source: string) => tokenize(source, [parensLexer]);
+export const lexHostname = (source: string) =>
+  tokenize(source, [paramLexer, globLexer, parensLexer]);
+export const lexPathname = (source: string) =>
+  tokenize(source, [paramLexer, globLexer, parensLexer]);