Merge branch 'main' into bump

Author: extremeheat

.github/AGENTS.md

@@ -0,0 +1,80 @@
+minecraft-data-generator extracts and generates Minecraft server-side data for multiple Minecraft versions via a Fabric mod that hooks into the vanilla Minecraft server.
+This generated data is then stored inside the https://github.com/PrismarineJS/minecraft-data repo, a cross language repo holding minecraft data on blocks, items and more.
+
+There is a different mod for each Minecraft version this repo supports, under the `mc/<version>` folder.
+Each version has a Gradle project at `:mc:<version>` and generating the data is done by running the Gradle task:
+```
+./gradlew :mc:<version>:runServer
+```
+Generated data ends up under `mc/<version>/run/server/minecraft-data` in the repo when run locally.
+
+## Repo Layout
+- mc/                       — per-Minecraft-version directories of OUR extractor code (e.g. `mc/1.21.8`)
+- versions.json             — canonical JSON array of versions that are supported (i.e. there are folders in mc/ for it)
+- mc-source/<version>       — (External) minecraft java edition code for relevant version with Mojang mappings that you can reference for changes.
+    * we store the last 2 most recent versions in the folder to save space, but if you need more versions you can do `cd mc-source/1.21.3 && git fetch origin client1.21.3 && git switch client1.21.3` (as these folders are a repo cloned with specific branches)
+    * there is a .diff file in mc-source/ like `1.21.7_to_1.21.8.diff` that contains result of `git diff --no-index old new` that you can reference for changes (note it's typically large)
+    * Since our generator is using Mojang mappings, the API naming is the same.
+- README.md — info how to set up
+
+## Typical Flow
+
+minecraft-data-generator typically needs updates every time there is a new Minecraft version, particularly if there was code changes
+around one of the areas that our extractors touches (e.g. an API changes, gets renamed, etc.).
+
+As explained in the README.md, there is an auto update workflow that typically creates scaffolding PRs (under the `bump` branch) whenever there is a new minecraft update.
+These PRs are simply copying old version code into new, so there may be changes and other code fixes needed to support the new version. It's your
+job to help plug the gaps by making these changes until the build passes for the given version.
+It's also possible that there maybe missing or broken generated data even if the build is passing: in these cases, listen to the user's request
+and figure out how to replicate, debug and fix the logical issues in data generation.
+
+## Troubleshooting
+
+Simply iterating over errors one by one is a good way to fix most issues. Inside mc-source/ you are provided the latest
+code for the latest minecraft versions you can reference to investigate any code changes that we may need to accomodate in our
+generator code.
+
+Sometimes, some APIs might change inside the mc code and make the data that the current code extracts no longer
+valid. In these cases, do your best to conform the new data to the old structure even if it feels wrong. If you
+can't, for example, let's say something about effects are removed from the game--but our data gen still expects it.
+What to do? You should set the data to null, and inform the user about the issue -- perhaps a schema change is needed.
+You can propose a new schema to the user (so that the relevant changes can be made to minecraft-data) and if the user confirms, you'd update the generator to output in that new schema.
+
+### Steps
+- Reproduce the problem locally inside the container and run the generator task.
+  - Example run command: `./gradlew :mc:1.21.6:runServer --stacktrace`
+  - See if the problem exists on an older version -- if so inform the user, otherwise compare logs and code
+- Inspect diffs in `mc-source/` to correlate Mojang API changes.
+  - See mc-source/
+- Validate generated output at `mc/<version>/run/server/minecraft-data` to make sure all files are there
+
+---
+
+## Examples
+
+Session 1 — Compile error after bump PR
+- User prompt: "The bump PR for mc/1.21.8 fails to compile. Build error: cannot find symbol method RegistryKey.of(...)."
+- Agent actions:
+  1. Reproduce: `./gradlew :mc:1.21.8:runServer --stacktrace`
+  2. Inspect compile error and mc-source/1.21.7_to_1.21.8.diff for API renames.
+  3. Update generator code: edit mc/1.21.8/src/main/java/.../SomeRegistryAdapter.java to use the new Registry API.
+  4. Build again and run generation.
+
+Session 2 — Runtime NPE during generation
+- User prompt: "Generation starts but crashes with NullPointerException in EnchantmentsDataGenerator."
+- Agent actions:
+  1. Reproduce with stacktrace: `./gradlew :mc:1.20.4:runServer --stacktrace`
+  2. Add temporary logging to EnchantmentsDataGenerator to identify the null object, or run in debugger.
+  3. Fix by guarding against null Registries or empty lists and add defensive checks.
+  4. Re-run generation and verify produced files.
+- If problem persists:
+  * Add logging to old version too
+  * Check relevant code diff to see if anything changed related to this
+
+Session 3 — Semantically incorrect generated data
+- User prompt: "Output JSON is valid but item tags are incorrect after the bump."
+- Agent actions:
+  1. Reproduce generation and open the produced JSON at mc/<version>/run/server/minecraft-data/tags/items.json
+  2. Compare against previous version output (`git diff --no-index`) to identify mismatches.
+  3. Inspect generator assumptions vs. mc-source diffs (names, tag keys, or logic changes).
+  4. Correct mapping logic in generator class and run generation.

.github/commands/send.js

@@ -9,7 +9,7 @@ module.exports = async ([mergedArtifactURL], helpers) => {
     branch: 'master',
     inputs: {
       versions: JSON.stringify(require('../../versions.json')),
-      mergedArtifactURL: mergedArtifactURL,
+      mergedArtifactURL,
       prNumber: process.env.PR_NUMBER
     }
   }

.github/commands/setupCopilot.js

@@ -0,0 +1,97 @@
+/**
+ * setupCopilot.js
+ *
+ * Reads versions from:
+ *  - versions.json at repo root (if present), else
+ *  - directories under mc/
+ *
+ * For each version it runs:
+ *   ./gradlew :mc:<version>:runServer --stacktrace
+ *
+ * Uses child_process.execSync(cmd, { stdio: 'inherit' }) so logs stream to the action console.
+ * Errors are caught, logged, and the loop continues. At the end the script writes
+ * success_versions and failed_versions to $GITHUB_OUTPUT (if available) for downstream steps.
+ *
+ * To force the action to fail if any version errors, set env FAIL_ON_ERROR=true in the workflow.
+ */
+
+const cp = require('child_process')
+const fs = require('fs')
+const { join } = require('path')
+
+const failOnError = process.env.FAIL_ON_ERROR === 'true'
+
+// Save some time by skipping old versions as we are not parallelizing builds
+const versions = require('../../versions.json')
+const SKIP_VERSIONS = versions.slice(0, -2) // all but last 2 versions
+
+async function main () {
+
+  if (!Array.isArray(versions) || versions.length === 0) {
+    console.error('No versions found (no VERSION env, no versions.json, no mc/* directories).')
+    process.exit(1)
+  }
+
+  console.log('Versions to build:', versions.join(', '))
+
+  process.chdir(join(__dirname, '../../'))
+
+  const successes = []
+  const failures = []
+
+  for (const v of versions) {
+    if (SKIP_VERSIONS.includes(v)) {
+      console.log(`Skipping old version ${v}...`)
+      continue
+    }
+    console.log('')
+    console.log('=== Building version:', v, '===')
+    const cmd = `./gradlew :mc:${v}:runServer --stacktrace`
+    try {
+      // stream output to the runner logs
+      cp.execSync(cmd, { stdio: 'inherit' })
+      successes.push(v)
+      console.log(`✅ ${v} succeeded`)
+    } catch (err) {
+      failures.push(v)
+      console.error(`❌ ${v} failed (continuing to next version)`)
+      // continue to next version
+    }
+  }
+
+  // Write outputs for downstream jobs, if GITHUB_OUTPUT available
+  const ghOut = process.env.GITHUB_OUTPUT
+  const latestVersion = versions[versions.length - 1]
+  const secondLatestVersion = versions[versions.length - 2]
+  if (ghOut) {
+    try {
+      fs.appendFileSync(ghOut, `success_versions=${JSON.stringify(successes)}\n`)
+      fs.appendFileSync(ghOut, `failed_versions=${JSON.stringify(failures)}\n`)
+      fs.appendFileSync(ghOut, `latest_version=${latestVersion}\n`)
+      fs.appendFileSync(ghOut, `second_latest_version=${secondLatestVersion}\n`)
+    } catch (e) {
+      console.warn('Could not write to GITHUB_OUTPUT:', e && e.message)
+    }
+  } else {
+    console.log('GITHUB_OUTPUT not found; printing summaries to stdout instead.')
+    console.log('success_versions=', JSON.stringify(successes))
+    console.log('failed_versions=', JSON.stringify(failures))
+    console.log('latest_version=', JSON.stringify(latestVersion))
+    console.log('second_latest_version=', JSON.stringify(secondLatestVersion))
+  }
+
+  console.log('')
+  console.log('Summary:')
+  console.log('  succeeded:', JSON.stringify(successes))
+  console.log('  failed:   ', JSON.stringify(failures))
+
+  if (failOnError && failures.length > 0) {
+    console.error('FAIL_ON_ERROR is true and some builds failed — exiting with non-zero code.')
+    process.exit(1)
+  }
+
+  // otherwise exit 0 so the workflow continues (Copilot can triage failures)
+  process.exit(0)
+}
+
+main()

.github/workflows/build.yml

@@ -2,11 +2,10 @@ name: Java CI with Gradle
 on:
   push:
     branches:
-      - main
+      - '**'
   pull_request:
     branches:
-      - main
-      - bump
+      - '**'
 
 permissions:
   pull-requests: write

.github/workflows/copilot-setup-steps.yml

@@ -0,0 +1,76 @@
+# https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment
+on:
+  workflow_dispatch:
+  push:
+    paths:
+      - .github/workflows/copilot-setup-steps.yml
+  pull_request:
+    paths:
+      - .github/workflows/copilot-setup-steps.yml
+
+jobs:
+  copilot-setup-steps:
+    name: Build and generate data (single-step loop via Node)
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+
+    # Set the permissions to the lowest permissions possible needed for your steps.
+    # Copilot will be given its own token for its operations.
+    permissions:
+      # If you want to clone the repository as part of your setup steps, for example to install dependencies, you'll need the `contents: read` permission. If you don't clone the repository in your setup steps, Copilot will do this for you automatically after the steps complete.
+      contents: write
+      actions: write
+      pull-requests: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Validate Gradle wrapper
+        uses: gradle/actions/wrapper-validation@v4
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Setup Gradle
+        uses: gradle/actions/setup-gradle@v4
+        with:
+          add-job-summary-as-pr-comment: on-failure
+
+      - name: Run Copilot setup script
+        id: sb
+        run: node .github/commands/setupCopilot.js
+        shell: bash
+
+      - name: Checkout latest mc code (external repo)
+        # If the external repo is public, the default token is fine.
+        # If it's private, create a personal access token with repo access and add it
+        # to this repo's secrets as EXTRA_REPO_TOKEN.
+        uses: actions/checkout@v4
+        with:
+          repository: extremeheat/extracted_minecraft_data
+          path: mc-source/${{ steps.sb.outputs.latest_version }}
+          ref: client${{ steps.sb.outputs.latest_version }}
+          fetch-depth: 1
+
+      - name: Checkout second latest mc code (external repo)
+        # If the external repo is public, the default token is fine.
+        # If it's private, create a personal access token with repo access and add it
+        # to this repo's secrets as EXTRA_REPO_TOKEN.
+        uses: actions/checkout@v4
+        with:
+          repository: extremeheat/extracted_minecraft_data
+          path: mc-source/${{ steps.sb.outputs.second_latest_version }}
+          ref: client${{ steps.sb.outputs.second_latest_version }}
+          fetch-depth: 1
+
+      - run: ls mc-source
+
+      - name: Write diff
+        # NOTE: this returns exit code 1 if there are differences
+        run: git diff --no-index ${{ steps.sb.outputs.second_latest_version }} ${{ steps.sb.outputs.latest_version }} > ${{ steps.sb.outputs.second_latest_version }}_to_${{ steps.sb.outputs.latest_version }}.diff
+        working-directory: mc-source
+        continue-on-error: true

.gitignore

@@ -118,6 +118,7 @@ gradle-app.setting
 !gradle-wrapper.jar
 
 bin/
+mc-source/
 
 node_modules
 package-lock.json

package.json

@@ -4,7 +4,7 @@
   "description": "This tool generates minecraft-data files based on a running a server on client classpath using a fabric mod. The supported versions can be read in the directory structure. Every version has its own directory.",
   "main": "index.js",
   "scripts": {
-    "fix": "standard --fix",
+    "fix": "standard --fix .github tools",
     "bump": "node tools/newVersion.js"
   },
   "author": "",