Skip to content

Remote module inclusion without @#6898

Merged
bentsherman merged 3 commits into251117-module-system-implementation-with-remote-inclusionfrom
251117-module-system-implementation-v2-with-remote-inclusion
Mar 6, 2026
Merged

Remote module inclusion without @#6898
bentsherman merged 3 commits into251117-module-system-implementation-with-remote-inclusionfrom
251117-module-system-implementation-v2-with-remote-inclusion

Conversation

@jorgee
Copy link
Contributor

@jorgee jorgee commented Mar 6, 2026

Summary

This PR removes the @ prefix from remote module references throughout the codebase. DSL include statements now use scope/name format instead of @scope/name. Local/built-in includes still require /, ./, ../, or plugin/ prefixes to be unambiguous.

Changes

Module reference format (scope/name)

  • ModuleReference.groovy: Removed optional @? from MODULE_NAME_PATTERN; fullName no longer includes @
  • IncludeDef.groovy: Remote module detection changed from startsWith('@') to absence of local prefixes + valid scope/name pattern validation via ModuleReference.parse()
  • ModuleResolver.java (nf-lang): Replaced startsWith("@") with isRemoteModule(source) static helper that validates the scope/name regex pattern
  • ResolveIncludeVisitor.java (nf-lang): isRemoteModule now delegates to ModuleResolver.isRemoteModule() for consistent pattern validation
  • PipelineSpec.groovy: Backward-compat: strips leading @ when reading old nextflow_spec.json entries
  • All CLI commands (CmdModuleInstall, CmdModuleRemove, CmdModuleInfo, CmdModuleRun): removed '@' + args[0] prepending; args[0] passed directly to ModuleReference.parse()

.checksum.module-info properties file

  • ModuleChecksum.groovy: Renamed CHECKSUM_FILE constant to MODULE_INFO_FILE = ".module-info". The file format changed from a raw checksum string to a Java properties file with a checksum=<sha256> entry. save() merges into existing properties; load() reads the checksum property
  • InstalledModule.groovy: Renamed field checksumFilemoduleInfoFile; enum value MISSING_CHECKSUMNO_REMOTE_MODULE
  • ModuleStorage.groovy: Uses .module-info as the marker file when scanning modules/ for installed modules; skips .module-info when creating bundles
  • ModuleRegistryClient.groovy: Simplified encodeName (no longer strips @)

Key Design Decisions

1. Folder modules/scope/name without .module-info → treated as "not a remote module"

When the resolver encounters a directory that exists at modules/scope/name but has no .module-info file, it means the folder was not installed via nextflow module install (e.g. a manually created or pre-existing local directory):

  • Resolution (nextflow run): warns and continues (same behaviour as MODIFIED)
  • Install (nextflow module install): fails with an error pointing to the existing folder and suggesting --force to override

2. nextflow module list does not show folders without .module-info

ModuleStorage.listInstalled() scans the modules/ tree and only recognises directories that contain a .module-info file. Folders without this marker are silently ignored. This means manually-created modules/scope/name directories are invisible to the module list command.

3. nextflow module publish scope/name creates/updates .module-info with the current checksum

When publishing a module using a module reference (scope/name), the publish command creates or updates the .module-info file with the current directory checksum. This marks the directory as a registered remote module and sets the integrity baseline.

Note for cherry-pick

Commit 3119155 contains only CLI changes (renaming subcommands) and is safe to cherry-pick to PR #6768.

Test plan

  • nextflow module install nf-core/fastqc installs to modules/nf-core/fastqc/ with .module-info containing checksum=<sha256>
  • nextflow module list shows only modules with .module-info; manual folders are not shown
  • DSL include include { FASTQC } from 'nf-core/fastqc' resolves correctly
  • DSL include include { FASTQC } from '@nf-core/fastqc' fails with a clear error
  • nextflow module install on a folder without .module-info fails without --force
  • Unit tests: make test module=nextflow passes

🤖 Generated with Claude Code

jorgee added 2 commits March 6, 2026 13:09
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
@jorgee jorgee requested a review from a team as a code owner March 6, 2026 12:23
@jorgee jorgee requested review from bentsherman and pditommaso March 6, 2026 12:30
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
@bentsherman bentsherman merged commit 2fa01e8 into 251117-module-system-implementation-with-remote-inclusion Mar 6, 2026
18 of 19 checks passed
@bentsherman bentsherman deleted the 251117-module-system-implementation-v2-with-remote-inclusion branch March 6, 2026 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants