harness-pi: opt-in setting to load Pi extensions from the filesystem (`extensions: boolean`, default false)

[hteek]

Description

@ai-sdk/harness-pi 1.0.101. createPiSession hardcodes filesystem extension discovery off:

const resourceLoader = new DefaultResourceLoader({
  cwd: sessionWorkDir,
  agentDir: hostAgentDir,          // per-session tmp dir, not `input.agentDir`
  ...
  // Pi runs in the host process, so its default resource discovery reaches
  // the host developer's personal config ...
  noExtensions: true,

The reasoning is right for the default case: pi runs in the host process, so loading a developer's ~/.pi/agent/extensions or a checked-out repo's .pi/extensions would execute untrusted code in the server. But it leaves no opt-in for deployments where the harness process is the sandbox.

Use case

We run harness-pi inside a Firecracker microVM (one VM per session, disposable HOME, the VM is the trust boundary). We want the guest image to ship pi extensions as files, the way pi itself is configured:

/opt/pi/agent/extensions/rtk.ts            # from `rtk init --agent pi`
/opt/pi/agent/extensions/context-mode -> node_modules/context-mode   # pi manifest package

Today the only route is extensionFactories, which means hand-porting every extension into TypeScript inside our package: an inline copy of rtk's rtk.ts, a loader that resolves context-mode's adapter path and imports it, plus tests for both. Each new extension is another port. Tools that already publish a pi extension (rtk, context-mode, anything with a pi manifest in package.json) cannot be used as published.

Also, agentDir: hostAgentDir means that even if noExtensions were lifted, <agentDir>/extensions would resolve to an empty tmp dir; the caller's agentDir setting is only used for auth.json/models.json/settings, not discovery.

Proposal

A PiHarnessSettings.extensions?: boolean, default false, that:

1. sets noExtensions: !extensions on the DefaultResourceLoader;
2. when enabled, passes the caller's agentDir (falling back to hostAgentDir) to the loader so <agentDir>/extensions is discovered;
3. takes the noTools: "builtin" path so extension-registered tools reach the model (same gap as harness-pi: tools registered by extensionFactories extensions never reach the model (dropped by tools allowlist) #20402).

Default behavior is unchanged. The docstring should state the trust implication plainly: enabling this runs whatever is under <agentDir>/extensions and the working directory's .pi/extensions, so only enable it where the harness process is already isolated.

Patch we run

--- a/dist/index.d.ts
+++ b/dist/index.d.ts
     readonly extensionFactories?: ReadonlyArray<ExtensionFactory>;
+    /**
+     * Load Pi extensions from the filesystem: `<agentDir>/extensions` and the
+     * session working directory's `.pi/extensions`. Defaults to false; only
+     * enable where the harness process itself is the sandbox.
+     */
+    readonly extensions?: boolean;
--- a/dist/index.js
+++ b/dist/index.js
-  const hasExtensionFactories = extensionFactories.length > 0;
+  const fsExtensions = input.settings.extensions === true;
+  const hasExtensionFactories = extensionFactories.length > 0 || fsExtensions;
   ...
-    agentDir: hostAgentDir,
+    agentDir: fsExtensions ? agentDir : hostAgentDir,
   ...
-    noExtensions: true,
+    noExtensions: !fsExtensions,
   ...
-      ...hasMcpServers ? { noTools: "builtin" } : { tools: toolNames },
+      ...hasMcpServers || hasExtensionFactories ? { noTools: "builtin" } : { tools: toolNames },
   ...
-          ...settings.extensionFactories ? { extensionFactories: settings.extensionFactories } : {}
+          ...settings.extensionFactories ? { extensionFactories: settings.extensionFactories } : {},
+          ...settings.extensions ? { extensions: settings.extensions } : {}

Verified: with extensions: true and agentDir set, a plain .ts under <agentDir>/extensions and a symlinked package directory with a pi manifest both load at session start, and their registered tools are listed by the model. With the setting unset nothing changes.

Happy to open a PR against packages/harness-pi if the shape is acceptable.

[ai-sdk-factory]

Identify issue type

Identified as Feature
Confidence: High

Rationale

This is an API and behavior enhancement for an intentionally disabled capability, not a failure of existing documented behavior.

Evidence

• The reporter requests a new opt-in PiHarnessSettings.extensions?: boolean setting for filesystem extension discovery.
• Current behavior intentionally disables filesystem extensions by default; the proposal preserves that default while adding capability for sandboxed deployments.
• The requested changes include using the caller's agentDir and exposing extension-registered tools when the new option is enabled.

[ai-sdk-factory]

Feature Analysis

Feature is not available.

Feature Description

• Add an optional PiHarnessSettings.extensions boolean, defaulting to false, that enables Pi filesystem extension discovery from the configured agent directory and the session working directory while making extension-registered tools available to the model.
• Current main identifies @ai-sdk/harness-pi as version 1.0.104, but its source and built declarations expose only extensionFactories; the resource loader receives agentDir: hostAgentDir and hardcodes noExtensions: true.
• The Pi Harness documentation explicitly says user, project, and settings-based filesystem extensions remain disabled, and the passing harness-pi test suite includes an assertion for this disabled default.

AI SDK Fit

• The capability fits as a Pi-specific createPi adapter setting over the upstream DefaultResourceLoader, alongside the existing agentDir, extensionFactories, and mcpServers settings.
• An additive opt-in with a secure false default preserves compatibility and the existing isolation posture while supporting deployments where the entire harness process is already inside the trust boundary.
• Filesystem discovery, selecting the discovery directories, preserving the extension runtime, and exposing registered tools are acceptance dimensions of one usable extension-loading capability rather than independently valuable public features.

Implications / Considerations

• Implementation is limited primarily to packages/harness-pi: add the public boolean to PiHarnessSettings and PiSessionSettings, forward it through createPi, and update the generated package declarations through the normal build.
• When enabled, DefaultResourceLoader should receive the caller's resolved agentDir or fall back to hostAgentDir, and noExtensions should become false; the default path must remain hostAgentDir with discovery disabled.
• Disabling noExtensions enables Pi's complete normal discovery behavior, which includes <agentDir>/extensions, <cwd>/.pi/extensions, and extension or package sources configured through settings, so documentation must not imply that only the two conventional directories are enabled.
• The existing extension-runtime preservation and session-rebuild logic should treat filesystem discovery as an active extension source, while continuing to avoid reinitializing extensions during resource-only reloads and rebuilding them when the Pi session is replaced.
• Current main passes a restrictive tools allowlist unless MCP servers are configured; filesystem extensions therefore also require the noTools: "builtin" path whenever any extension source is active so extension-registered tools reach the model.
• An end-to-end extension-tool test should verify both invocation and stream semantics: the current translator marks only built-in and MCP tools as provider-executed and only MCP tools as dynamic, so filesystem extension tools may require adjacent providerExecuted or dynamic-tool classification work beyond merely appearing in the provider request.
• Tests should cover the false default, setting propagation, custom and fallback agent directories, a direct TypeScript extension, a directory or symlinked package with a pi manifest, settings-based sources, model-visible and executable registered tools, coexistence with inline extensions and MCP, resource reloads, session rebuilds, and extension load errors.
• Documentation should update the Pi provider page and package README with an isolated-process example, the full discovery scope, reload behavior, and an explicit warning that the option executes arbitrary code; a focused Pi harness example would demonstrate the intended sandboxed deployment.
• No provider protocol, @ai-sdk/provider specification, model-provider integration, or public ai package API change is required; this is an adapter-local use of an API already provided by Pi.
• No AI SDK UI type or framework integration change is expected because extension tools can use existing harness tool stream parts, but UI-facing behavior should be tested if such tools are classified as dynamic native tools.
• Enabling discovery executes extension code in the Node.js process with access to its filesystem, environment, credentials, and network; symlinks and settings-configured package sources can broaden the executed code beyond visibly colocated files, so extensions: true must not be presented as a sandbox mechanism.
• The harness calls Pi resource reload without an interactive project-trust resolver, and Pi settings default to a trusted project state in this embedding path; consequently the opt-in effectively authorizes project-local .pi code and should document that trust decision explicitly.
• The implementation depends on the installed Pi package retaining DefaultResourceLoader discovery, agentDir, noExtensions, package-manifest, and tool-selection semantics; these should be pinned by focused tests because the harness package is experimental and its Pi dependency can evolve.

Relevant Documentation

• Pi Harness
• Pi SDK
• Extensions
• Pi Packages

[ai-sdk-factory]

Feature implementation

Created feature implementation and opened PR: #20409

Summary

Added PiHarnessSettings.extensions with a secure false default, configured Pi filesystem discovery to use the caller's agentDir when enabled, preserved extension runtimes across resource reloads, and enabled extension-registered tools.

Testing

Added runtime coverage for default isolation, custom and fallback agent directories, and extension tool availability, plus compile-time coverage for the new setting.

End-to-end Validation

• Added and successfully ran a filesystem-extensions example that creates a TypeScript Pi extension, registers a tool, and verifies that the harness loads it.

Documentation

Updated the Pi provider documentation and package README with configuration, discovery scope, runtime behavior, defaults, and explicit arbitrary-code execution warnings.

[hteek]

One more piece the setting needs, found while running the patch above in production: skillsOverride in createPiSession keeps only skills whose file path is inside the workspace (plus the harness-provided ones):

skillsOverride: base => ({
  ...base,
  skills: [
    ...base.skills.filter(skill => isWithinWorkspace(skill.filePath, sessionWorkDir, hostWorkDir)),
    ...harnessSkills,
  ],
}),

With extensions: true and the caller's agentDir, pi discovers <agentDir>/skills and this filter then drops every entry. Pi packages that ship both an extension and skills ("pi": { "extensions": [...], "skills": ["./skills"] }, e.g. ponytail, context-mode) end up half-loaded: the extension runs, the skills never reach <available_skills>.

#20409 leaves the filter unchanged. Suggest, when filesystem extensions are enabled, also keeping skills under the configured agent directory:

skill =>
  isWithinWorkspace(skill.filePath, sessionWorkDir, hostWorkDir) ||
  (filesystemExtensions && isWithinDirectory(path.join(agentDir, 'skills'), skill.filePath))

Same trust argument as for extensions: if the process is allowed to execute <agentDir>/extensions, prompting from <agentDir>/skills is not a new exposure. Default (extensions unset) unchanged.

We run this as part of our patch; verified in a deployed guest that the six skills now appear in the system prompt.

[hteek]

Two follow-ups after running #20409's shape (plus the skill filter above) against real pi packages. Both apply to pi-session.ts as of 1250135.

1. bindExtensions() is gated on MCP servers

piSession = session;
if (hasMcpServers) {
  await piSession.bindExtensions({ mode: 'print' });
}

AgentSession.bindExtensions() is what emits session_start and then runs extendResourcesFromExtensions() (resources_discover). With extensions: true and no mcpServers, filesystem extensions load and their tools reach the model, but neither event ever fires: extensions that hook session_start never see it, and skills an extension contributes via resources_discover ("pi": { "skills": ["./skills"] } packages register them this way, e.g. obra/superpowers) never enter the resource loader at all. The e2e example in #20409 checks a registered tool only, so it passes. Suggest:

if (hasMcpServers || hasExtensions) {
  await piSession.bindExtensions({ mode: 'print' });
}

2. <agentDir>/skills is not enough for the skill filter

My suggestion above only covers skills discovered from <agentDir>/skills. Skills registered through resources_discover carry the path the extension hands over (for a symlinked package under <agentDir>/extensions/<pkg>/skills/..., or wherever the package resolves them), so they are still dropped. Once filesystem extensions are enabled the process already executes code from those locations, so keeping their skills adds no new exposure; the simplest rule that works for both cases:

skill =>
  filesystemExtensions ||
  isWithinWorkspace(skill.filePath, sessionWorkDir, hostWorkDir),

Default (extensions unset) unchanged in both cases.

Repro we use as a regression test: a probe extension under <agentDir>/extensions that returns { skillPaths: [dir] } from resources_discover and dumps event.systemPrompt in before_agent_start; with a second skill under <agentDir>/skills, both must show up in <available_skills> on the first turn. Without (1) the extension skill is missing; without (2) it is missing even when (1) is applied. Verified in our deployed guest: 14 superpowers skills plus the agentDir ones are listed.

[hteek]

Third follow-up, same setup (extensions: true, real pi packages), against pi-session.ts as of 1250135. This one is a bug in the shape rather than a gap: skills registered through resources_discover show up on the first prompt and then vanish.

reloadResourcesOnly() forgets resources_discover resources

async function reloadResourcesOnly(): Promise<void> {
  ...
  await resourceLoader.reload();
  ...
}

DefaultResourceLoader.reload() rebuilds skills/prompts/themes from settings and the filesystem only. It resets lastSkillPaths and extensionSkillSourceInfos, i.e. everything a previous extendResources() (= resources_discover) added is gone. Pi's own AgentSession.reload() accounts for this: after resourceLoader.reload() it re-emits session_start and calls extendResourcesFromExtensions('reload'). harness-pi calls the loader directly and never re-runs discovery.

Sequence per turn (runTurn):

1. rebuildPiSession() → bindExtensions() → resources_discover → extension skills in the loader and in the system prompt. ✅
2. Same turn, immediately after: if (!resourcesReloaded) await reloadResourcesOnly() → extension skills dropped from the loader. The prompt is not rebuilt here, so nothing visible yet.
3. Next _rebuildSystemPrompt() regenerates from resourceLoader.getSkills(): extension skills gone. In practice that happens within seconds, because pi-mcp-adapter calls pi.setActiveTools() once a server connects (and applySessionInstructions() does the same on instruction changes). <agentDir>/skills and workspace skills survive because reload() re-reads them from disk.

Observed: the guest listed all superpowers skills on the very first prompt and only the agentDir ones afterwards ("no brainstorming skill installed here").

Fix we run

async function reloadResourcesOnly(): Promise<void> {
  ...
  // resourceLoader.reload() forgets every skill/prompt/theme path an
  // extension registered through resources_discover. Re-run discovery the
  // way AgentSession.reload() does.
  if (filesystemExtensions && piSession) {
    await piSession.extendResourcesFromExtensions('reload');
  }
}

Re-emits only resources_discover (not session_start, so no MCP reconnect or extension state reset) and rebuilds the base system prompt itself. extendResourcesFromExtensions is private in the .d.ts; a public entry point on AgentSession (or making resourceLoader.reload() preserve extension-contributed paths) would be the clean upstream shape.

Regression test (extends the one from my previous comment): run two turns and, in the probe extension, call pi.setActiveTools(pi.getActiveTools()) on agent_end to force the prompt rebuild. The second turn's event.systemPrompt must still list the resources_discover skill. Fails without the fix (AssertionError: turn 2), passes with it. Ours: stormreply/agenthub-ai-sdk#157.