Adding magics functionality to the litellm implementation
#1437
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dlqqq
reviewed
Aug 11, 2025
This was referenced Aug 12, 2025
dlqqq
approved these changes
Aug 12, 2025
There was a problem hiding this comment.
@srdas This is a great start, thank you! Just one extra comment below.
In addition, I've opened some follow-up issues (titled [magics] ...) for you to explore. They can be addressed in separate PRs.
This is already a fantastic start, and I'm excited to merge this once the last comment is addressed.
Comment on lines
+369
to
+370
| self.transcript.append({"role": "user", "content": prompt}) | ||
| self.transcript.append({"role": "assistant", "content": output}) |
There was a problem hiding this comment.
Can you bound this list to be of length self.max_history * 2? Right now, the ai() method is manually truncating self.transcript every time it appends it to the chat history. It would be more reliable to add some logic to bound the length of self.transcript in the _append_exchange() method itself.
There was a problem hiding this comment.
Yes, this is now done. Tested everything to make sure it is working. Here is a notebook if you want to test all the magics features (unzip to use).
magics_litellm.ipynb.zip
jonahjung22
pushed a commit
to jonahjung22/jupyter-ai
that referenced
this pull request
Aug 15, 2025
…rlab#1437) * Adding `magics` functionality to the `litellm` implementation * updates for initializing aliases and resolving types, mypy * Update magics.py * init alias * Update test_magics.py * Update magics.py * Changed `error` to `fix` in magics * Update index.md * Update test_magics.py * Update magics.py * streamlining magics * removed update command * initial_chat_model * magics pre-commit * Update magics.py
6 tasks
|
Is there any example about how to use litellm with %%ai magic？ |
|
@cszhbo The refactoring of Jupyter AI to use To your question about usage examples: This video shows the new workflow with chat and magics. The vast range of models offered by Screen.Recording.2025-08-21.at.7.45.49.AM.movHope this helps! |
|
It is very helpful. Appreciate your detailed demonstration. |
dlqqq
added a commit
that referenced
this pull request
Aug 21, 2025
* [litellm] Initial migration to LiteLLM (#1425) * add new /api/ai/models endpoint * add litellm dependency * add new model ID input component * fix custom model ID input * update ConfigManager to accept litellm model IDs * update Jupyternaut to use litellm * remove LangChain partner packages from dependencies * remove LangChain provider entry points * pre-commit * rename /api/ai/models => /api/ai/models/chat * simplify model settings UI * [litellm] Remove LangChain from source code (#1435) * remove LangChain from jupyter_ai_magics * remove unused useServerInfo() hook in frontend * remove LangChain references from jupyter_ai * remove langchain from test & cookiecutter packages * move all completions code to jupyter_ai.completions * remove LangChain provider code from ConfigManager * minor fixes to allow JAI to start * Implementing model parameters input * Implement new Secrets UI and REST API (#1439) * add EnvSecretsManager and secrets REST API * make ListSecrets endpoint more generic * fix UpdateSecretsRequest type * fix bug where last secret cannot be deleted * remove error log used in dev * add new <SecretsSection /> component * tweak UI appearance * make settings page scrollable, fix #1409 * show no static secrets by default * show something when no secrets are present * adjust language * fix success & error alerts on updating secrets * fix lint * fix bug when .gitignore and .env do not exist * bump to 2s poll interval in EnvSecretsManager * disable model parameters input for now, fix merge conflicts * Adding `magics` functionality to the `litellm` implementation (#1437) * Adding `magics` functionality to the `litellm` implementation * updates for initializing aliases and resolving types, mypy * Update magics.py * init alias * Update test_magics.py * Update magics.py * Changed `error` to `fix` in magics * Update index.md * Update test_magics.py * Update magics.py * streamlining magics * removed update command * initial_chat_model * magics pre-commit * Update magics.py * missing model id in magics (#1457) Thanks David for approving this. * Added sublists for providers using %ai (#1458) * Fix `litellm` branch CI, add `EnvSecretsManager` tests (#1459) * update prettier ignore file * fix prettier errors & warnings * fix mypy errors in jupyter_ai * skip ConfigManager and inline completion tests until fixed in v3 * add tests for secrets manager * fix mypy error * run minimum dependency tests on Python 3.10 * comment out precommit job to be added in future * updated test_magics.py --------- Co-authored-by: Sanjiv Das <[email protected]> * [litellm] Display model IDs to match search string (#1445) * Enable the use of API Keys in magics from the `.env` file (#1465) * magics to use dotenv * Update magics.py * Update magics.py * Add model parameters input (#1456) * start of backend implementation * fixes api code * implemented type mapping * integrating api with ui * Working prototype of the model parameter input component with correct model parameters and dropdown feature! * added api to save parameters to config and removed old model-fields * code debugging, changing UI, and adding success error message * latest changes * fix type safety issues in model-parameters-input * fix frontend lint checks, update comment * fix mypy errors, add comments * small type fix * Updated coercion typing, improved code functionality, removed unnecessary code * pass chat model arguments to Jupyternaut * delete/update existing parameter * hide params controlling tool usage from frontend --------- Co-authored-by: David L. Qiu <[email protected]> * Clean up AI settings page (#1468) * Clean up AI settings page * remove debug line added by accident * Move LiteLLM model list to `jupyter_ai_magics` (#1469) * move model list to jupyter_ai_magics * fix mypy --------- Co-authored-by: Jonah Jung <[email protected]> Co-authored-by: Sanjiv Das <[email protected]> Co-authored-by: Jonah Jung <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactored various functionality for the
%aiand%%aimagics to use the newlitellmbackbone added in PR #1426 . Fixes Issue #1432run_ai_cellto invokelitellminstead oflangchain.registercommand toaliaswhich is more natural when registering an alias for amodel_id.deletecommand todealiaswhich is more natural.updatecommand as it is working well and is convenient.helpis working and shows the updated commands.listcommand to show model IDs and aliases.resetcommand is working.default_language_modeltoinitial_language_model.aliasestoinitial_aliases.errorcommand tofix, updated it's handling..envapproach.See the demo below showing all of it working:
Screen.Recording.2025-07-30.at.2.30.01.PM.mov
Nice to have (among other items):
formatoptions to be much more stable using agentic capabilities with better prompts.widgetsas another output format option.TODOcommented items inmagics.pyfor consideration.