Consider allowing alphanumeric `sub` and `ses` labels following BIDS specification

[glopesdev]

Is there a strong reason why subject and session labels must be numerical values in the neuro blueprint?

In the BIDS specification it looks like these two attributes are defined as Entities, and specifically the Value type of an entity is defined as being either "A non-negative integer, potentially zero-padded for consistent width." or "An alphanumeric string. Note that labels MUST not collide when casing is ignored".

In the example for Case collision intolerance they specifically show an example of a valid string sub-S1 which is non-numeric.

Allowing for alphanumeric subject names would avoid the need for an additional id attribute, since in many cases it could perhaps be plugged in directly into sub.

Allowing alphanumeric session names would potentially avoid the need for additional datetime fields, since we could plug the ISO8601 alphanumeric format as a unique session identifier directly into ses.

Is the concern that chronological order might be lost? ISO8601 format ensures that lexicographic sorting matches chronological time, so this shouldn't be a problem. For independent subjects, conceptually the order is in principle not important most of the times, and of course if it is then the experimenter would be able to use the format to indicate this anyway without requiring change.

[niksirbi]

You make a good case @glopesdev. From the perspective of the spec, I see no reason to disallow alphanumeric strings as values. This way people could use the spec much more flexibly, as in the examples you've highlighted.

I'll let @JoeZiminski comment on what that would mean on the datashuttle side. I can see that we'd have to abandon some nice perks like auto-suggesting the next logical sub or ses values, or specifying a range of subjects/sessions, but I don't know how deep the ramifications would go beyond that. Perhaps there is a world in which:

• alphanumeric values are allowed and valid, but
• ordered numerical values get you some extra functionality perks in datashuttle

[JoeZiminski]

Hi @glopesdev thanks a lot for this! these are some good points. While there are some benefits to ordered numerical values (e.g. like @niksirbi mentioned for datashuttle) I agree these are not sufficient to mandate rather than just recommend their use in NeuroBlueprint.

However, I'm not sure that date, time or datetime should be allowed as a label because of the potential ambiguity. For example sub-20230101, it is not clear if this is a date or a user-supplied ID. Similarly, although those familiar with ISO8601 datetimes will recognize them from the T, for those unfamiliar it may not be clear what this code means. In general it is nice to force all user-supplied information to be in a separate key-value pairs as it is self documenting.

I would probably stick to enforcing the date, time and datetime are placed under specific date-, time-, datetime- keys. Things do then get a bit convoluted ("We recommend sub- labels are numeric, consecutive codes with identical number of characters. However, these may be alphanumeric, unless they are date, time or datetime in which case they must be placed under a date, time or datetime key.").

Similarly if any alphanumeric character is allowed, should<label> always refer to some kind of animal ID, or it is permissible to put any non-colliding information into this field? Quite often subject IDs are nonstandard, e.g. I've seen projects include a mix of computer-generated IDs ("34234") and animal names ("john"). Should labels be required to all take the same 'form'? We could say "We recommend sub- labels are consecutive integers of identical length , though these may be alphanumeric but in this case must represent a unique animal ID that take the same 'form'" (and then define ID and 'form') (and the ID cannot be date/time/datetime even if they are unique in the project). Or, we could just directly inherit the entities concept from BIDS.

However, with NeuroBlueprint designed as a quick-to-read-and-get-started-with subset of BIDS, I wonder if it is simpler to restrict to numeric labels rather than introduce these caveats / added complexity.

That being said, it may be worth adding some additional complexity to facilitate adoption. I guess for Bonsai / Aeon this format is a huge pain because you essentially have a lot of unnecessary numeric characters in the names and many session ids? e.g. 20250722T122100 becomes ses-1231_datetime-20250722T122100 instead of just ses-20250722T122100?

[glopesdev]

However, I'm not sure that date, time or datetime should be allowed as a label because of the potential ambiguity. For example sub-20230101, it is not clear if this is a date or a user-supplied ID.

@JoeZiminski I feel it will be very hard and confusing to mandate and validate the semantics rather than the syntax of identifiers. I feel the purpose and meaning of identifiers should be the purview of the researcher.

By definition, even the numerical format can already be used to represent whatever a researcher might want, e.g. an index to an array or pointer to a table somewhere, it just gets more verbose if we exclude alphanumeric codes.

What would be the motivation to exclude specific forms of identifiers? As long as they are unique, isn't the goal of session and subject identifiers to be used as keys in a table (pandas or otherwise)?

with NeuroBlueprint designed as a quick-to-read-and-get-started-with subset of BIDS

BIDS already feels pretty easy to get started, especially the simple constraints around sub and ses.

If we make it more complicated to use the ses identifiers, and require that to use date identifiers we have both to use a different attribute and still generate a new arbitrary key for ses, it seems like we are deviating from this goal.

I guess for Bonsai / Aeon this format is a huge pain because you essentially have a lot of unnecessary numeric characters in the names and many session ids?

I don't feel this is just for Aeon, but rather a general problem. As stated in the Common principles of the BIDS specification: "(...) it is important to keep in mind that files also have to be readable. They have to be readable by machines, and this implies that filenames cannot be longer than 255 characters."

We already had previous cases (admittedly rare, but they did happen) where we were hitting the historical 260 max path length limit on Windows due to excessively long names, so I'm not sure it is wise to make the amount of metadata stored in the file name very long. If possible to condense keys into fewer (and standard) attributes, it feels like this would be an advantage for wider adoption.

[niksirbi]

I think I'm with @glopesdev on this. We can recommend best practices, but we can't prevent users from (ab)using the entities as they wish, as long as the format is compatible.

So in this case, we should require sub and ses labels to be unique and alphanumeric (basically, completely align with BIDS on this), but don't set any further hard requirement on them.

We can recommend that they use ordered numerical values for sub and ses, and that they store datetimes in a separate field (we can reinforce that "habit" with some examples), but should not (and probably cannot) preclude them from doing otherwise.

If I were a NeuroBlueprint user, I would also be tempted to use ISO8601 datetime as the session value 🙈

[JoeZiminski]

Thanks @glopesdev @niksirbi on balance I agree it makes sense to allow this flexibility and not much is achieved enforcing vs. recommending numeric values, and there are many cases where having an unnecessary numeric value is in fact problematic.

I guess the solution to any potential ambiguity in the folder name is the convention that metadata #30 should be used to fix information used for analysis (e.g. acquisition date) with information in the folder names only serving to be convenient identifiers. Analysis tools we write could read pre-defined folder name key-value pairs alongside metadata for convenience, but that is outside the scope of NeuroBlueprint.

On the datashuttle side, we could throw an error if 'suggest next sub or ses' is used when non-numeric values are found in the project, and add an option to turn off validation of numeric sub or ses values.

I'll write a PR and tag you for review, thanks for the suggestion!