Add vendorization support

[candleindark]

This PR includes all the relevant commits (first 4) in #2430. Checkout #2430 for changes regarding those commits.

Additionally, this PR closes #2382 with the following changes.

Additionally, it replaces the hardcoded DAND:, the hardcoded RRID, and the supported licenses with the corresponding attributes in schema instance config defined in dandischema.conf.

Particularly, the hardcoded DAND: that are handled are

• git grep DANDI: -- dandiapi | grep -v -e test_ -e 'subject=' -e 'verbose_name'

  • dandiapi/api/models/version.py: 'identifier': f'DANDI:{self.dandiset.identifier}',
  • dandiapi/api/models/version.py: 'id': f'DANDI:{self.dandiset.identifier}/{self.version}',
  • dandiapi/api/services/metadata/__init__.py: f'DANDI:{publishable_version.dandiset.identifier}/{publishable_version.version}'
  • dandiapi/api/tests/fuzzy.py:DANDISET_SCHEMA_ID_RE = Re(r'DANDI:\d{6}')
    • This is used in tests, so it is not replaced.
  • dandiapi/api/views/dandiset.py: identifier = identifier.removeprefix('DANDI:')

• git grep DANDI: -- web | grep -v -e test_ -e 'subject=' -e 'verbose_name'

  • web/src/components/DandisetList.vue: DANDI:<b>{{ item.dandiset.identifier }}</b>
  • web/src/utils/schema.ts: deepCopySchema['properties']['identifier']['pattern'] = '^DANDI:\\d{6}$'
    • This line may not e needed in the first place. (See Add vendorization support #2584 (comment))
  • web/src/views/DandisetLandingView/DownloadDialog.vue: // Use the special 'DANDI:' url prefix if appropriate.
    • This is a comment. replacement not needed.
  • web/src/views/DandisetLandingView/DownloadDialog.vue: const dandiUrl = `DANDI:${identifier}`;
    • This is the shorten url for https://www.dandiarchive.org/. The hardcode doesn't need to be replaced. Make sure the containing function downloadCommand works for other instances of DANDI though. (See Add vendorization support #2584 (comment) for details)

TODO:

• drop temporary commit "tmp: point dandi-cli integration test CI to a temporary branch of dandi-cli" used for testing
• Answer for Add vendorization support #2584 (comment)
• Handle the case where settings.DANDI_SCHEMA_INSTANCE_CONFIG.instance_identifier is None in metadata.py
• Revert or drop temporary commit "tmp: temporarily set the dandischema dependency to the devendorize branch".
• Have the dandischema dependency point to its release version of 0.12.0 once Vendor-Configurable Metadata Models dandi-schema#294 is merged and released as version of 0.12.0

Note

• The failure in the CI workflow named "Test Integration with dandi-cli / test (3.13 release)" should be ignore since the released version of dandi-cli can't be adjusted to run tests with the correct vendor specific information.
• This PR is based on Have the api/info/ endpoint provide instance config info #2430 and is targeted to be merged to the corresponding branch of Have the api/info/ endpoint provide instance config info #2430.

Incidentally, this PR also makes the following minor changes.

• Improve metadata for PublishActivity by using the version of the dandi-archive instead of a hardcoded version number.

Release Notes

This PR makes the instance config as defined in dandischema.conf, per dandi/dandi-schema#294, available through the api/info/ endpoint and replaces the hardcoded DAND:, the hardcoded RRID, and the supported licenses with the corresponding attributes in schema instance config defined in dandischema.conf.

[candleindark]

@mvandenburgh Why do we need the following line?

dandi-archive/web/src/utils/schema.ts

Line 13 in 2ff1bc7

deepCopySchema['properties']['identifier']['pattern'] = '^DANDI:\\d{6}$'

The JSON schema of Dandiset already specifies the exact same pattern.

https://github.com/dandi/schema/blob/c90cd41cb18a29e75abaa13728c13125f38d8236/releases/0.6.10/dandiset.json#L1389

[candleindark]

@mvandenburgh I don't think the DANDI: hardcoded at

dandi-archive/web/src/views/DandisetLandingView/DownloadDialog.vue

Line 141 in 9aa8b1f

const dandiUrl = `DANDI:${identifier}`;

needs to be replaced because it is a shortened URL for the https://dandiarchive.org instance. All instances of DANDI at other domain will just use the long URL.

[candleindark]

@yarikoptic Should be bother to replace the hardcoded 'spdx:CC0-1.0' in the following?

dandi-archive/dandiapi/api/management/commands/create_dev_dandiset.py

Line 35 in 9aa8b1f

'license': ['spdx:CC0-1.0'],

It is one of the default supported licenses, and create_dev_dandiset is just for development.

[candleindark]

@yarikoptic Should be bother to replace the hardcoded 'spdx:CC0-1.0' in the following?

dandi-archive/dandiapi/api/management/commands/create_dev_dandiset.py

Line 35 in 9aa8b1f

'license': ['spdx:CC0-1.0'],

It is one of the default supported licenses, and create_dev_dandiset is just for development.

@yarikoptic Never mind, I replaced the hardcoded license.

[yarikoptic]

following up to the question to @mvandenburgh . So that TODO was added in

• http://github.com/dandi/dandi-archive/commit/433e351f4c29a7d9f26067553e6e75ec5f8204cc

which added (2nd line unrelated, included for completness)

      // TODO: Fix this upstream in the schema
      schema['properties']['identifier']['pattern'] = '^DANDI:\\d{6}$'
      schema['$defs']['Project']['properties']['wasAssociatedWith']['items']['oneOf'][2]['properties']['identifier']['pattern'] = '^RRID:.*'

and it didn't reference any particular PR or even dandi-schema version. Looking at that commit state of dandi-schema it was

setup.py:        'dandischema==0.11.0',  # schema version 0.6.9

so used 0.6.9 schema. And the diff between the two is

❯ diff -Naur releases/0.6.{9,10}/dandiset.json
--- releases/0.6.9/dandiset.json	2025-01-17 15:27:21.826885797 -0500
+++ releases/0.6.10/dandiset.json	2025-05-16 16:02:57.225766934 -0400
@@ -1070,7 +1070,7 @@
           "default": null,
           "description": "RRID of the software from scicrunch.org.",
           "nskey": "schema",
-          "pattern": "^RRID\\:.*",
+          "pattern": "^RRID:.*",
           "title": "Research resource identifier",
           "type": "string"
         },
@@ -1203,7 +1203,7 @@
       "type": "string"
     },
     "schemaVersion": {
-      "default": "0.6.9",
+      "default": "0.6.10",
       "nskey": "schema",
       "readOnly": true,
       "title": "Schema Version",
@@ -1218,7 +1218,7 @@
     },
     "description": {
       "description": "A description of the Dandiset",
-      "maxLength": 3000,
+      "maxLength": 10000,
       "nskey": "schema",
       "title": "Description",
       "type": "string"
@@ -1386,7 +1386,7 @@
     "identifier": {
       "description": "A Dandiset identifier that can be resolved by identifiers.org.",
       "nskey": "schema",
-      "pattern": "^DANDI\\:\\d{6}$",
+      "pattern": "^DANDI:\\d{6}$",
       "readOnly": true,
       "title": "Dandiset identifier",
       "type": "string"

so it seems that the 0.6.10 had fixed the pattern and we can just remove that ad-hoc fixup of the schema. @mvandenburgh please confirm but meanwhile we proceed under such an assumption

[candleindark]

@mvandenburgh I don't think the DANDI: hardcoded at

dandi-archive/web/src/views/DandisetLandingView/DownloadDialog.vue

Line 141 in 9aa8b1f

const dandiUrl = `DANDI:${identifier}`;

needs to be replaced because it is a shortened URL for the https://dandiarchive.org instance. All instances of DANDI at other domain will just use the long URL.

In our discussion about this, @yarikoptic concurred, so let's proceed with my initial judgement.

[Copilot]

Pull Request Overview

This PR updates the DANDI archive codebase to utilize vendor-specific information from schema instance configuration instead of hardcoded values. The changes replace hardcoded DANDI: prefixes, RRID identifiers, and license lists with corresponding attributes from dandischema.conf.

• Replaces hardcoded instance-specific strings with configuration-driven values
• Updates metadata generation to use schema instance configuration
• Adds frontend support for dynamic instance naming

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
web/src/types/index.ts	Adds interface definition for instance configuration
web/src/components/DandisetList.vue	Updates to display dynamic archive name instead of hardcoded "DANDI"
dev/.env.docker-compose-native	Adds environment variable for instance name
dandiapi/settings/base.py	Imports and configures schema instance configuration
dandiapi/api/views/dandiset.py	Updates identifier prefix removal to use dynamic instance name
dandiapi/api/tests/test_version.py	Updates test assertions to use schema configuration values
dandiapi/api/tests/test_tasks.py	Updates test assertions for publish task metadata
dandiapi/api/tests/test_asset.py	Updates test assertions for asset publishing
dandiapi/api/services/metadata/init.py	Updates metadata ID generation to use dynamic instance name
dandiapi/api/models/version.py	Updates version metadata to use configuration-driven identifiers
dandiapi/api/models/metadata.py	Updates publish metadata to use dynamic instance configuration
dandiapi/api/management/commands/create_dev_dandiset.py	Updates license selection to use configured licenses
.github/workflows/frontend-ci.yml	Adds instance name environment variable
.github/workflows/cli-integration.yml	Updates to use temporary test branch
.github/workflows/backend-ci.yml	Adds instance configuration environment variables

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[mvandenburgh]

LGTM

[jjnesbitt]

@candleindark I believe from our end this is all set. The failing CLI tests will are fixed by dandi/dandi-cli#1739 (confirmed by this test run aimed at the CLI branch). At this point, we're just waiting for the dandi-schema PR to be merged and released. Then, we can proceed with this PR (dropping the temporary commits and merging).

[jjnesbitt]

We'll just need to make sure to squash when we merge.

[yarikoptic]

FWIW - https://pypi.org/project/dandischema/0.12.0/ is now out. Test against it?!

[jjnesbitt]

The CLI integration tests are failing, which is expected and okay, since all of the tests see a schema version of 0.7.0, while expecting 0.6.10.

Per this comment, we will merge this PR first, and then the CLI PR.

[dandibot]

🚀 PR was released in v0.20.0 🚀