Add setup for template migrations (#14502)

This PR adds the initial setup and a first codemod for the template
migrations. These are a new set of migrations that operate on files
defined in the Tailwind v3 config as part of the `content` option (so
your HTML, JavaScript, TSX files etc.).

The migration for this is integrated in the new `@tailwindcss/upgrade`
package and will require pointing the migration to an input JavaScript
config file, like this:

```
npx @tailwindcss/upgrade --config tailwind.config.js
```

The idea of template migrations is to apply breaking changes from the v3
to v4 migration within your template files.

## Migrating !important syntax

The first migration that I’m adding with this PR is to ensure we use the
v4 important syntax that has the exclamation mark at the end of the
utility.

For example, this:

```html
<div class="!flex sm:!block"></div>
```

Will now turn into:

```html
<div class="flex! sm:block!"></div>
```

## Architecture considerations

Implementation wise, we make use of Oxide to scan the content files fast
and efficiently. By relying on the same scanner als Tailwind v4, we
guarantee that all candidates that are part of the v4 output will have
gone through a migration.

Migrations itself operate on the abstract `Candidate` type, similar to
the type we use in the v4 codebase. It will parse the candidate into its
parts so they can easily be introspected/modified. Migrations are typed
as:

```ts
type TemplateMigration = (candidate: Candidate) => Candidate | null
``` 

`null` should be returned if the `Candidate` does not need a migration. 

We currently use the v4 `parseCandidate` function to get an abstract
definition of the candidate rule that we can operate on. _This will
likely need to change in the future as we need to fork `parseCandidate`
for v3 specific syntax_.

Additionally, we're inlining a `printCandidate` function that can
stringify the abstract `Candidate` type. It is not guaranteed that this
is an identity function since some information can be lost during the
parse step. This is not a problem though, because migrations will only
run selectively and if none of the selectors trigger, the candidates are
not updated. h/t to @RobinMalfait for providing the printer.

So the overall flow of a migration looks like this:

- Scan the config file for `content` files
- Use Oxide to extract a list of candidate and their positions from
these `content` files
- Run a few migrations that operate on the `Candidate` abstract type.
- Print the updated `Candidate` back into the original `content` file.

---------

Co-authored-by: Robin Malfait <[email protected]>
Co-authored-by: Jordan Pittman <[email protected]>

Author: 3 people

CHANGELOG.md

@@ -32,6 +32,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - _Experimental_: Add CSS codemods for `@apply` ([#14411](https://github.com/tailwindlabs/tailwindcss/pull/14434))
 - _Experimental_: Add CSS codemods for migrating `@tailwind` directives ([#14411](https://github.com/tailwindlabs/tailwindcss/pull/14411), [#14504](https://github.com/tailwindlabs/tailwindcss/pull/14504))
 - _Experimental_: Add CSS codemods for migrating `@layer utilities` and `@layer components` ([#14455](https://github.com/tailwindlabs/tailwindcss/pull/14455))
+- _Experimental_: Add template codemods for migrating important utilities (e.g. `!flex` to `flex!`) ([#14502](https://github.com/tailwindlabs/tailwindcss/pull/14502))
 
 ### Fixed
 

crates/node/src/lib.rs

@@ -82,6 +82,16 @@ pub struct Scanner {
   scanner: tailwindcss_oxide::Scanner,
 }
 
+#[derive(Debug, Clone)]
+#[napi(object)]
+pub struct CandidateWithPosition {
+  /// The candidate string
+  pub candidate: String,
+
+  /// The position of the candidate inside the content file
+  pub position: i64,
+}
+
 #[napi]
 impl Scanner {
   #[napi(constructor)]
@@ -108,6 +118,22 @@ impl Scanner {
       .scan_content(input.into_iter().map(Into::into).collect())
   }
 
+  #[napi]
+  pub fn get_candidates_with_positions(
+    &mut self,
+    input: ChangedContent,
+  ) -> Vec<CandidateWithPosition> {
+    self
+      .scanner
+      .get_candidates_with_positions(input.into())
+      .into_iter()
+      .map(|(candidate, position)| CandidateWithPosition {
+        candidate,
+        position: position as i64,
+      })
+      .collect()
+  }
+
   #[napi(getter)]
   pub fn files(&mut self) -> Vec<String> {
     self.scanner.get_files()

crates/oxide/src/lib.rs

@@ -124,6 +124,28 @@ impl Scanner {
         new_candidates
     }
 
+    #[tracing::instrument(skip_all)]
+    pub fn get_candidates_with_positions(
+        &mut self,
+        changed_content: ChangedContent,
+    ) -> Vec<(String, usize)> {
+        self.prepare();
+
+        let content = read_changed_content(changed_content).unwrap_or_default();
+        let extractor = Extractor::with_positions(&content[..], Default::default());
+
+        let candidates: Vec<(String, usize)> = extractor
+            .into_iter()
+            .map(|(s, i)| {
+                // SAFETY: When we parsed the candidates, we already guaranteed that the byte slices
+                // are valid, therefore we don't have to re-check here when we want to convert it back
+                // to a string.
+                unsafe { (String::from_utf8_unchecked(s.to_vec()), i) }
+            })
+            .collect();
+        candidates
+    }
+
     #[tracing::instrument(skip_all)]
     pub fn get_files(&mut self) -> Vec<String> {
         self.prepare();

crates/oxide/src/parser.rs

@@ -82,6 +82,18 @@ impl<'a> Extractor<'a> {
 
         candidates
     }
+
+    pub fn with_positions(input: &'a [u8], opts: ExtractorOptions) -> Vec<(&'a [u8], usize)> {
+        let mut result = Vec::new();
+        let extractor = Self::new(input, opts).flatten();
+        for item in extractor {
+            // Since the items are slices of the input buffer, we can calculate the start index
+            // by doing some pointer arithmetics.
+            let start_index = item.as_ptr() as usize - input.as_ptr() as usize;
+            result.push((item, start_index));
+        }
+        result
+    }
 }
 
 impl<'a> Extractor<'a> {

integrations/cli/upgrade.test.ts renamed to integrations/upgrade/index.test.ts

@@ -1,4 +1,47 @@
-import { css, json, test } from '../utils'
+import { css, html, js, json, test } from '../utils'
+
+test(
+  `upgrades a v3 project to v4`,
+  {
+    fs: {
+      'package.json': json`
+        {
+          "dependencies": {
+            "@tailwindcss/upgrade": "workspace:^"
+          }
+        }
+      `,
+      'tailwind.config.js': js`
+        /** @type {import('tailwindcss').Config} */
+        module.exports = {
+          content: ['./src/**/*.{html,js}'],
+        }
+      `,
+      'src/index.html': html`
+        <h1>🤠👋</h1>
+        <div class="!flex sm:!block"></div>
+      `,
+      'src/input.css': css`
+        @tailwind base;
+        @tailwind components;
+        @tailwind utilities;
+      `,
+    },
+  },
+  async ({ exec, fs }) => {
+    await exec('npx @tailwindcss/upgrade -c tailwind.config.js')
+
+    await fs.expectFileToContain(
+      'src/index.html',
+      html`
+        <h1>🤠👋</h1>
+        <div class="flex! sm:block!"></div>
+      `,
+    )
+
+    await fs.expectFileToContain('src/input.css', css`@import 'tailwindcss';`)
+  },
+)
 
 test(
   'migrate @apply',

packages/@tailwindcss-node/src/compile.ts

@@ -4,7 +4,10 @@ import fs from 'node:fs'
 import fsPromises from 'node:fs/promises'
 import path, { dirname, extname } from 'node:path'
 import { pathToFileURL } from 'node:url'
-import { compile as _compile } from 'tailwindcss'
+import {
+  __unstable__loadDesignSystem as ___unstable__loadDesignSystem,
+  compile as _compile,
+} from 'tailwindcss'
 import { getModuleDependencies } from './get-module-dependencies'
 
 export async function compile(
@@ -14,59 +17,78 @@ export async function compile(
   return await _compile(css, {
     base,
     async loadModule(id, base) {
-      if (id[0] !== '.') {
-        let resolvedPath = await resolveJsId(id, base)
-        if (!resolvedPath) {
-          throw new Error(`Could not resolve '${id}' from '${base}'`)
-        }
-
-        let module = await importModule(pathToFileURL(resolvedPath).href)
-        return {
-          base: dirname(resolvedPath),
-          module: module.default ?? module,
-        }
-      }
+      return loadModule(id, base, onDependency)
+    },
+    async loadStylesheet(id, base) {
+      return loadStylesheet(id, base, onDependency)
+    },
+  })
+}
 
-      let resolvedPath = await resolveJsId(id, base)
-      if (!resolvedPath) {
-        throw new Error(`Could not resolve '${id}' from '${base}'`)
-      }
-      let [module, moduleDependencies] = await Promise.all([
-        importModule(pathToFileURL(resolvedPath).href + '?id=' + Date.now()),
-        getModuleDependencies(resolvedPath),
-      ])
-
-      onDependency(resolvedPath)
-      for (let file of moduleDependencies) {
-        onDependency(file)
-      }
-      return {
-        base: dirname(resolvedPath),
-        module: module.default ?? module,
-      }
+export async function __unstable__loadDesignSystem(css: string, { base }: { base: string }) {
+  return ___unstable__loadDesignSystem(css, {
+    base,
+    async loadModule(id, base) {
+      return loadModule(id, base, () => {})
+    },
+    async loadStylesheet(id, base) {
+      return loadStylesheet(id, base, () => {})
     },
+  })
+}
 
-    async loadStylesheet(id, basedir) {
-      let resolvedPath = await resolveCssId(id, basedir)
-      if (!resolvedPath) throw new Error(`Could not resolve '${id}' from '${basedir}'`)
-
-      if (typeof globalThis.__tw_readFile === 'function') {
-        let file = await globalThis.__tw_readFile(resolvedPath, 'utf-8')
-        if (file) {
-          return {
-            base: path.dirname(resolvedPath),
-            content: file,
-          }
-        }
-      }
+async function loadModule(id: string, base: string, onDependency: (path: string) => void) {
+  if (id[0] !== '.') {
+    let resolvedPath = await resolveJsId(id, base)
+    if (!resolvedPath) {
+      throw new Error(`Could not resolve '${id}' from '${base}'`)
+    }
 
-      let file = await fsPromises.readFile(resolvedPath, 'utf-8')
+    let module = await importModule(pathToFileURL(resolvedPath).href)
+    return {
+      base: dirname(resolvedPath),
+      module: module.default ?? module,
+    }
+  }
+
+  let resolvedPath = await resolveJsId(id, base)
+  if (!resolvedPath) {
+    throw new Error(`Could not resolve '${id}' from '${base}'`)
+  }
+  let [module, moduleDependencies] = await Promise.all([
+    importModule(pathToFileURL(resolvedPath).href + '?id=' + Date.now()),
+    getModuleDependencies(resolvedPath),
+  ])
+
+  onDependency(resolvedPath)
+  for (let file of moduleDependencies) {
+    onDependency(file)
+  }
+  return {
+    base: dirname(resolvedPath),
+    module: module.default ?? module,
+  }
+}
+
+async function loadStylesheet(id: string, base: string, onDependency: (path: string) => void) {
+  let resolvedPath = await resolveCssId(id, base)
+  if (!resolvedPath) throw new Error(`Could not resolve '${id}' from '${base}'`)
+
+  if (typeof globalThis.__tw_readFile === 'function') {
+    let file = await globalThis.__tw_readFile(resolvedPath, 'utf-8')
+    if (file) {
       return {
         base: path.dirname(resolvedPath),
         content: file,
       }
-    },
-  })
+    }
+  }
+
+  let file = await fsPromises.readFile(resolvedPath, 'utf-8')
+  return {
+    base: path.dirname(resolvedPath),
+    content: file,
+  }
 }
 
 // Attempts to import the module using the native `import()` function. If this

packages/@tailwindcss-upgrade/package.json

@@ -27,6 +27,8 @@
     "access": "public"
   },
   "dependencies": {
+    "@tailwindcss/node": "workspace:^",
+    "@tailwindcss/oxide": "workspace:^",
     "enhanced-resolve": "^5.17.1",
     "globby": "^14.0.2",
     "mri": "^1.2.0",
@@ -35,6 +37,7 @@
     "postcss-import": "^16.1.0",
     "postcss-selector-parser": "^6.1.2",
     "prettier": "^3.3.3",
+    "string-byte-slice": "^3.0.0",
     "tailwindcss": "workspace:^"
   },
   "devDependencies": {