PLDG Dashboard & Storacha Integration - 2025-05-22 (PLDG Cohort-3) - Real-Time Metrics, AI Agents & MongoDB Migration Toolkit

[seetadev]

PLDG Dashboard & Storacha Integration - 2025-05-22 - Real-Time Metrics, AI Agents & MongoDB Migration Toolkit

Contributors

Cohort-3 PLDG Contributors: @HarshS1611, @kaf-lamed-beyt, @pranavkonde, @sumanjeet0012, @Aldorax, @Pfed-prog, @Devanshgoel-123, @CeylonT, @Nkovaturient, @seetadev

Accomplished Tasks

• Deployed v1 of the PLDG real-time dashboard with MongoDB and GitHub data pipelines
• Built MongoDB → Storacha migration CLI with support for nested and relational data

In Progress

• Created a prototype agent to query, summarize, and annotate PLDG activity stored in Storacha
• Developed ElizaOS plugin to embed dashboard AI agents inside contributor workflows (e.g., PRs, weekly standups)
• Integrated Storacha MCP to replace transient JSON storage with content-addressed historical data

Update

PLDG Dashboard Core Enhancements:

• Added builder insights panel based on GitHub metadata and Airtable feedback forms

• Dashboard now supports ElizaOS plugin interface, enabling contributors to:

  • Ask natural language questions about PRs, issues, and technical metrics
  • Automatically generate summaries for weekly async updates
  • Surface AI-suggested contributions or overdue tasks

• Storacha MCP-backed historical data layer replaces previous in-memory storage:

  • Weekly snapshots of engagement, PR velocity, and module progress now stored immutably
  • Enables trend comparisons and metric evolution across months
  • IPLD-powered chunking allows storage of nested Airtable/GitHub merged datasets

Migration Toolkit Progress:

• Finished MVP of the MongoDB-to-Storacha CLI tool:

  • Supports field mapping, nested structure flattening, IPLD encoding
  • CLI logs include CID mapping and error diagnostics

• Agent layer built on top of migrated datasets:

  • Performs tagging, summarization, and anomaly detection using hosted LLM
  • Extends into ElizaOS workflows for context-aware interactions

What Went Well

• Storacha MCP greatly simplified long-term storage and versioning
• ElizaOS integration made it easy to surface AI insights in contributor workflows
• Builder feedback led to streamlined UI and modular dashboard components

What Was Difficult

• Schema inference for MongoDB migration required special handling for embedded arrays and polymorphic fields
• Matching GitHub metrics to contributor intentions needed additional Airtable cross-referencing
• Some latency in large dataset summaries; optimizing for parallel LLM calls

Demo Videos

Coming soon — Storacha-backed dashboard walkthrough + ElizaOS agent querying GitHub engagement history

Please visit our vercel deployment at :

Cohort-1 (main branch): https://pldg-dashboard.vercel.app/

Cohort-2 and Cohort-1 (dev branch with Storacha integration and MongoDB to Storacha bridge): https://pldg-dashboard-theta.vercel.app/

Screencast for MongoDB to Storacha Migration Tooling Demo (by Harsh) shared via Discord:

Save from PLDG Dashboard Database (MongoDB) to Storacha: https://drive.google.com/file/d/1GmefF9QFyJH9NtD84z6g2T03SPH78TIM/view?usp=sharing

Retrieve from Storacha to PLDG Dashboard Database (MongoDB): https://drive.google.com/file/d/1yPcldFgsfJob2IVAjEZ-gXsem7is_oxp/view?usp=sharing

Github: https://github.com/seetadev/pldg-dashboard

Related Pull Requests:

• feat: mongodDB integration HarshS1611/storacha-migration-tool#11

• Feat: update docs and handle process file in memory HarshS1611/storacha-migration-tool#12

• Feat: retrieve file HarshS1611/storacha-migration-tool#14

• Feat: Add retrieveJsonInSpace method to StorachaMigrator HarshS1611/storacha-migration-tool#15

Questions

• What are best practices for sharing Storacha CIDs across agents for cross-program collaboration?
• How do we track ElizaOS plugin usage to refine agent prompts and UX?
• Are there performance benchmarks for using Storacha MCP with high-frequency updates (e.g., GitHub hooks)?

Next Tasks

• Add historical comparison charts to the PLDG dashboard using Storacha CID logs
• Expand ElizaOS plugin to support fine-tuned prompts for specific roles (e.g., reviewers vs. maintainers)
• Improve MongoDB migration CLI UX with GUI wrapper
• Run user testing sessions with builders for dashboard feedback and onboarding ease

[Pfed-prog]

What are best practices for sharing Storacha CIDs across agents for cross-program collaboration?

I suppose it would be possible to provide a cid with storacha as a gateway.

Currently available process, however, looks disorganized.
For example, saturn dCDN docs process is not documented and docs page is not available online storacha/w3up#1644.

For instance, retrieval process looks very different from upload process.

In upload storacha tutorial @web3-storage/w3up-client npm package used with js client.

Meanwhile retrieval tutorial covers only http retrieval.

In my experience http gateways have low throughput.

I am very curios about Performance Optimization: Addressing latency issues in summarizing large datasets through asynchronous processing or efficient batching.

[seetadev]

@Pfed-prog : Thanks for the thoughtful question and sharp observations — you’ve touched on several key aspects of cross-agent collaboration with Storacha, and you’re absolutely right to highlight both the opportunity and the current friction points.

Sharing Storacha CIDs across agents is core to enabling decentralized, modular workflows.

Wish to share that I am currently reviewing:

1. Use UCAN-Backed Access Control : please visit https://github.com/storacha/upload-service and https://docs.storacha.network/concepts/ucans-and-storacha/ : For secure and permissioned sharing, leverage UCAN tokens issued via the w3up-client. These enable scoped access to CIDs, useful when multiple programs need controlled access.

2. Some interesting pointers at https://github.com/storacha/w3infra

As a next step, I would encourage you to try:

1. Use a lightweight schema (e.g., JSON with CID, timestamp, content type, program ID) when sharing CIDs across agents.

2. To improve coordination, consider building a lightweight index or pubsub layer that helps agents find relevant CIDs without centralized orchestration.

In reference to: On Upload vs Retrieval Inconsistency

We'll discuss with Alan and Jamie about it — the current upload vs retrieval paths. The upload flow (using w3up-client) is clean and developer-friendly, while retrieval is mostly documented via HTTP GETs.

To bridge this, we will also discuss some ideas:

Introduce IPFS-native retrieval in agents — ideally using JS clients like ipfs-http-client, or lightweight Rust/Go IPFS clients depending on the agent runtime.

Document a uniform “CID lifecycle” across upload, share, retrieve, and revoke workflows — we’ll be working to unify these in Storacha examples.

In reference to Performance Optimization Ideas: Thanks for sharing feedback on latency — especially when summarizing or analyzing large datasets.

To reduce latency and improve performance, we could try:

Asynchronous Chunk Processing: Break datasets into smaller blocks that can be pre-processed in parallel before upload.

Batch Retrieval with DAG Traversals: Instead of retrieving flat files, use IPLD DAGs with batched traversal and prefetching.

CID Preloading and Warm Caches: Where possible, preload or pin relevant CIDs across nodes/agents before triggering processing tasks.

I will ask your question to Jamie and will include in the weekly update "questions" section.

[Pfed-prog]

Thank you so much for more details.
I have also noticed that there is @storacha/client package which seems to be 1-to-1 to @web3-storage/w3up-client.
https://www.npmjs.com/package/@storacha/client