Skip to content

Module metadata: Fix stale module detection and add per-type metadata index#21231

Open
bcoles wants to merge 1 commit intorapid7:masterfrom
bcoles:msf-module-cache
Open

Module metadata: Fix stale module detection and add per-type metadata index#21231
bcoles wants to merge 1 commit intorapid7:masterfrom
bcoles:msf-module-cache

Conversation

@bcoles
Copy link
Copy Markdown
Contributor

@bcoles bcoles commented Apr 4, 2026

Description

Fixes two bugs in the metadata cache and adds a per-type index to eliminate an O(n) scan that runs on every module_metadata(type) call.

Bug fixes in cache.rb

1. Stale module detection was a no-op

In refresh_metadata_instance_internal, the delete_if block compared module_metadata.type != module_metadata.type (same object, both sides), which is always false. This meant modules that changed type (e.g. Auxiliary reclassified as Exploit) would leave stale entries in the cache. Fixed to compare against metadata_obj.type.

2. delete_if block relied on implicit return of &&

The delete_if block returned the result of path.eql?(…) && type != type. When both conditions are true, && returns the right-hand operand — in this case a truthy string comparison result, which happened to work, but was fragile and unclear. Replaced with an explicit is_stale boolean variable.

Performance: per-type metadata index

module_metadata(type) previously ran filter_map over all ~6,600 entries on every call (~14ms each). This is called once per module type during startup and tab-completion. Replaced with a pre-built @module_metadata_by_type hash:

  • Built once after store initialization and after bulk refresh_metadata operations
  • Incrementally updated on single-module changes (common case)
  • Full rebuild only when a type mismatch deletion occurs (rare)

Measured improvement: ~14ms → ~0.001ms per call.

Minor cleanup in cache.rb

Replaced allowed_paths.select{|x| path.index(x) == 0}.empty? with allowed_paths.any? { |x| path.start_with?(x) } to avoid allocating a temporary array on every loop iteration.

Simplified get_cache_key

Replaced multi-line string concatenation with string interpolation.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes stale module detection in Msf::Modules::Metadata::Cache and introduces a per-type metadata index to avoid repeated full-cache scans when retrieving metadata by module type, plus a small load-path filtering optimization in the module manager cache.

Changes:

  • Fix stale module detection when a module’s type changes but its path stays the same.
  • Add @module_metadata_by_type index and rebuild/update logic to speed up module_metadata(type) lookups.
  • Optimize allowed path checks by switching from select(...).empty? to any? + start_with?.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
lib/msf/core/modules/metadata/cache.rb Fixes stale-type deletion bug and adds per-type metadata index used by module_metadata(type).
lib/msf/core/module_manager/cache.rb Avoids temporary array allocation when filtering metadata entries by allowed paths.

@bcoles bcoles force-pushed the msf-module-cache branch 2 times, most recently from 41e43d5 to 780789c Compare April 4, 2026 03:25
@bcoles bcoles requested a review from Copilot April 4, 2026 03:25
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

@bcoles bcoles force-pushed the msf-module-cache branch from 3c8adfb to c108ea4 Compare April 4, 2026 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

3 participants