Skip to content

Add support for all known instances in short URL format, removed dandi-staging and renamed EMBER* ones into EMBER-DANDI* #1750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yarikoptic merged 7 commits into from
Dec 4, 2025
Merged

Add support for all known instances in short URL format, removed dandi-staging and renamed EMBER* ones into EMBER-DANDI* #1750

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ae19957
Support all known instances in short URL format
Copilot Nov 21, 2025
1bc12f8
Build instance name pattern from known_instances
Copilot Nov 21, 2025
bee2655
Update documentation to reflect all known instances support in short …
Copilot Nov 27, 2025
6bd8500
Fix test_helptext to expect <INSTANCE>: instead of DANDI:
Copilot Dec 1, 2025
6d1bc06
rf: make "INSTANCE-NAME:ID" URI pattern more stringent - no arbitrary…
yarikoptic Dec 3, 2025
c463d67
Make naming of instances consistent with deployed and remove dandi-st…
yarikoptic Dec 3, 2025
5f60b9b
Fix instance name in the doc
yarikoptic Dec 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DEVELOPMENT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ development command line options.
variable providing the key for a specific known DANDI instance corresponds to the name
of the instance. For example, the environment variable `DANDI_API_KEY` provides the key
for the known instance named `dandi` and the environment variable
`EMBER_SANDBOX_API_KEY` provides the key for the known instance named `ember-sandbox`.
`EMBER_DANDI_SANDBOX_API_KEY` provides the key for the known instance named `ember-dandi-sandbox`.
I.e., the environment variable name is the capitalized version of the instance's name
with "-" replaced by "_" suffixed by "_API_KEY". Providing API keys through environment
variables avoids using keyrings, thus making it possible to "temporarily" use another
Expand Down
7 changes: 2 additions & 5 deletions dandi/cli/tests/test_instances.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,10 @@ def test_cmd_instances(monkeypatch):
"dandi-sandbox:\n"
" api: https://api.sandbox.dandiarchive.org/api\n"
" gui: https://sandbox.dandiarchive.org\n"
"dandi-staging:\n"
" api: https://api.sandbox.dandiarchive.org/api\n"
" gui: https://sandbox.dandiarchive.org\n"
"ember:\n"
"ember-dandi:\n"
" api: https://api-dandi.emberarchive.org/api\n"
" gui: https://dandi.emberarchive.org\n"
"ember-sandbox:\n"
"ember-dandi-sandbox:\n"
Copy link
Member

@yarikoptic yarikoptic Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dandi/ember would it be ok to change or they are in wide use and we should provide deprecation cycle?

We are just trying to match the names of the instances as in the INSTANCE_ID provided in infrastructure deployment and would be visible in their IDs on the instances (ATM they are still all DANDI:)

Copy link

@sandyhider sandyhider Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yarikoptic we don't foresee any issues. Lets go ahead with the changes.

" api: https://api-dandi-sandbox.emberarchive.org/api\n"
" gui: https://apl-setup--ember-dandi-archive.netlify.app/\n"
"linc:\n"
Expand Down
15 changes: 4 additions & 11 deletions dandi/consts.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,13 +123,6 @@ def urls(self) -> Iterator[str]:
"https://dandiarchive.org",
"https://api.dandiarchive.org/api",
),
# Deprecated. Remove early 2026.
# Should come before dandi-sandbox so _rev map does map to sandbox
"dandi-staging": DandiInstance(
"dandi-staging",
"https://sandbox.dandiarchive.org",
"https://api.sandbox.dandiarchive.org/api",
),
"dandi-sandbox": DandiInstance(
"dandi-sandbox",
"https://sandbox.dandiarchive.org",
Expand All @@ -150,13 +143,13 @@ def urls(self) -> Iterator[str]:
"https://staging.lincbrain.org",
"https://staging-api.lincbrain.org/api",
),
"ember": DandiInstance(
"ember",
"ember-dandi": DandiInstance(
"ember-dandi",
"https://dandi.emberarchive.org",
"https://api-dandi.emberarchive.org/api",
),
"ember-sandbox": DandiInstance(
"ember-sandbox",
"ember-dandi-sandbox": DandiInstance(
"ember-dandi-sandbox",
"https://apl-setup--ember-dandi-archive.netlify.app/",
"https://api-dandi-sandbox.emberarchive.org/api",
),
Expand Down
9 changes: 6 additions & 3 deletions dandi/dandiarchive.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,10 @@ class _dandi_url_parser:
dandiset_id_grp = f"(?P<dandiset_id>{DANDISET_ID_REGEX})"
# Should absorb port and "api/":
server_grp = "(?P<server>(?P<protocol>https?)://(?P<hostname>[^/]+)/(api/)?)"
# Build instance name pattern from known instances to avoid matching unknown patterns
instance_name_pattern = "|".join(
re.escape(name.upper()) for name in known_instances
)
known_urls: list[tuple[re.Pattern[str], dict[str, Any], str]] = [
# List of (regex, settings, display string) triples
#
Expand All @@ -589,13 +593,12 @@ class _dandi_url_parser:
# for not only "dandiarchive.org" URLs
(
re.compile(
rf"(?P<instance_name>DANDI):"
rf"(?P<instance_name>{instance_name_pattern}):"
rf"{dandiset_id_grp}"
rf"(/(?P<version>{VERSION_REGEX}))?",
flags=re.I,
),
{},
"DANDI:<dandiset id>[/<version>]",
"<INSTANCE>:<dandiset id>[/<version>]",
),
(
re.compile(r"https?://gui\.dandiarchive\.org/.*"),
Expand Down
2 changes: 1 addition & 1 deletion dandi/tests/test_dandiapi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -869,7 +869,7 @@ def test_asset_as_readable_open(new_dandiset: SampleDandiset, tmp_path: Path) ->
("dandi", "DANDI_API_KEY"),
("dandi-api-local-docker-tests", "DANDI_API_LOCAL_DOCKER_TESTS_API_KEY"),
("dandi-sandbox", "DANDI_SANDBOX_API_KEY"),
("ember-sandbox", "EMBER_SANDBOX_API_KEY"),
("ember-dandi-sandbox", "EMBER_DANDI_SANDBOX_API_KEY"),
],
)
def test_get_api_key_env_var(instance_name: str, expected_env_var_name: str) -> None:
Expand Down
45 changes: 43 additions & 2 deletions dandi/tests/test_dandiarchive.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,15 +108,56 @@
version_id="draft",
),
),
# lower cased
# numeric version
(
"dandi:000027/0.210831.2033",
"DANDI:000027/0.210831.2033",
DandisetURL(
instance=known_instances["dandi"],
dandiset_id="000027",
version_id="0.210831.2033",
),
),
# Test other instances with short format
(
"DANDI-SANDBOX:000029",
DandisetURL(
instance=known_instances["dandi-sandbox"],
dandiset_id="000029",
version_id=None,
),
),
(
"LINC:000029",
DandisetURL(
instance=known_instances["linc"],
dandiset_id="000029",
version_id=None,
),
),
(
"LINC:000029/0.210831.2033",
DandisetURL(
instance=known_instances["linc"],
dandiset_id="000029",
version_id="0.210831.2033",
),
),
(
"EMBER-DANDI:000029",
DandisetURL(
instance=known_instances["ember-dandi"],
dandiset_id="000029",
version_id=None,
),
),
(
"EMBER-DANDI-SANDBOX:000029/draft",
DandisetURL(
instance=known_instances["ember-dandi-sandbox"],
dandiset_id="000029",
version_id="draft",
),
),
(
"http://localhost:8000/api/dandisets/000002/",
DandisetURL(
Expand Down
2 changes: 1 addition & 1 deletion dandi/tests/test_helptext.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ def get_helptext(command):

def test_resource_identifier_helptext():
# The \n chars must be included for correct rendering
correct = "Accepted resource identifier patterns:\n - DANDI:<dandiset id>[/<version>]\n"
correct = "Accepted resource identifier patterns:\n - <INSTANCE>:<dandiset id>[/<version>]\n"

ls_helptext = get_helptext(['dandi', 'ls'])
assert correct in ls_helptext
Expand Down
4 changes: 2 additions & 2 deletions dandi/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -577,8 +577,8 @@ def get_instance(dandi_instance_id: str | DandiInstance) -> DandiInstance:
dandi_id = dandi_instance_id
instance = known_instances[dandi_id]
if dandi_id == "dandi-staging":
lgr.warning(
"'dandi-staging' DANDI instance identifier is deprecated. "
raise ValueError(
"'dandi-staging' DANDI instance identifier was removed. "
"The instance was renamed into 'dandi-sandbox', please use that identifier instead."
)
if redirector_url is None:
Expand Down
7 changes: 2 additions & 5 deletions docs/source/cmdline/instances.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,18 +22,15 @@ Example output:
dandi-sandbox:
api: https://api.sandbox.dandiarchive.org/api
gui: https://sandbox.dandiarchive.org
dandi-staging:
api: https://api.sandbox.dandiarchive.org/api
gui: https://sandbox.dandiarchive.org
linc-staging:
api: https://staging-api.lincbrain.org/api
gui: https://staging.lincbrain.org
linc:
api: https://api.lincbrain.org/api
gui: https://lincbrain.org
ember-sandbox:
ember-dandi-sandbox:
api: https://api-dandi-sandbox.emberarchive.org/api
gui: https://apl-setup--ember-dandi-archive.netlify.app/
ember:
ember-dandi:
api: https://api-dandi.emberarchive.org/api
gui: https://dandi.emberarchive.org
6 changes: 4 additions & 2 deletions docs/source/ref/urls.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,10 @@ has one, and its draft version will be used otherwise.
(case insensitive; ``version`` cannot be "draft") when it redirects
to one of the other URL formats

- :samp:`DANDI:{dandiset-id}[/{version}]` (case insensitive)
— Refers to a Dandiset on the main DANDI Archive instance named "dandi".
- :samp:`{instance-name}:{dandiset-id}[/{version}]` (case insensitive,
where ``instance-name`` is a known DANDI instance such as ``DANDI``,
``DANDI-SANDBOX``, ``LINC``, ``EMBER``, etc.)
— Refers to a Dandiset on the specified DANDI Archive instance.
`parse_dandi_url()` converts this format to a `DandisetURL`.

- Any ``https://gui.dandiarchive.org/`` or
Expand Down
Loading