Context Retrieval Persona #13
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
srdas
reviewed
Aug 8, 2025
There was a problem hiding this comment.
Tested the PR and the code works well.
- With a notebook in context, the persona collects the relevant context from the Python Data Science Handbook - the context retrieved appears to be relevant, however need to check if the persona misses anything that is relevant.
- Markdown file created explaining what is added to the RAG db extracted from the Python Data Science Handbook repo.
- RAG db created as well.
Items to check:
- If additional notebooks are used for context retrieval, does it overwrite or add to the existing vector store?
- The markdown file is overwritten at the moment but we may want to retain it. Better to create a new markdown file for each notebook that is processed with the title of the notebook included.
Will review the code and leave comments as well.
srdas
requested changes
Aug 8, 2025
| Modify parameters in `rag_core.py`: | ||
| ```python | ||
| rag = PythonDSHandbookRAG( | ||
| embedding_model="sentence-transformers/all-MiniLM-L6-v2", |
There was a problem hiding this comment.
Can this be updated to take in the chosen embedding model from Jupyter-AI. The embedding model would then need to be called using the functions in Jupyter AI.
Comment on lines
+25
to
+30
| if not os.path.exists(notebook_path): | ||
| return f"Error: Notebook file not found at {notebook_path}" | ||
|
|
||
| if not notebook_path.endswith('.ipynb'): | ||
| return f"Error: File must be a .ipynb notebook file, got {notebook_path}" | ||
|
|
There was a problem hiding this comment.
- This should be sent to the chat panel not just printed in the logs.
- When I tried this with a
.pyfile instead of a notebook.ipynbfile, it still processed the context retrieval? Not sure why. - When I gave it a non-existent file, it still processed the RAG, pulling up various pandas notebook from the PDSH.
srdas
requested changes
Aug 8, 2025
| repo_url: str = "https://github.com/jakevdp/PythonDataScienceHandbook.git", | ||
| local_repo_path: str = None, | ||
| vector_store_path: str = None, | ||
| embedding_model: str = "sentence-transformers/all-MiniLM-L6-v2", |
There was a problem hiding this comment.
Do we have to use this model? Can we take the chosen embedding model from Jupyter AI's config file?
jupyter_ai_personas/context_retrieval_persona/rag_integration_tool.py
Outdated
Show resolved
Hide resolved
jupyter_ai_personas/context_retrieval_persona/rag_integration_tool.py
Outdated
Show resolved
Hide resolved
jupyter_ai_personas/context_retrieval_persona/rag_integration_tool.py
Outdated
Show resolved
Hide resolved
jupyter_ai_personas/context_retrieval_persona/rag_integration_tool.py
Outdated
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Professionals working in Jupyter Notebooks often need to find relevant examples, documentation, and best practices while coding, but existing solutions require manually searching through documentation or switching between multiple tools. Users need a context-aware assistant that can analyze their current notebook work and understand their technical context, search through comprehensive data science resources using semantic search, and generate structured reports with relevant code examples and next steps.
Solution
This is a persona that combines multi-agent intelligence with RAG to provide intelligent context-aware assistance. The solution uses a 3-agent team architecture: NotebookAnalyzer extracts libraries and analysis stage from notebooks, KnowledgeSearcher performs semantic search through the Python Data Science Handbook using RAG, and MarkdownGenerator creates comprehensive actionable reports.
Changes
Testing Instructions
Test Notebook: Open test_context_retrieval.ipynb and follow the test cases. RAG Integration: Run test_rag_integration() function to verify RAG system which auto-clones handbook and builds vector store. Jupyter-AI Chat: Test with @ContextRetriever help me with pandas operations or @ContextRetriever notebook: /path/to/test_context_retrieval.ipynb.
Future Work
I may potentially explore opportunities to implement Pocketflow into this persona to implement a more simple and efficient core graph abstraction.