A global override to disable API calls to mermaid.ink for graph visualization

[sundaraa-deshaw]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langgraph.prebuilt import create_react_agent

@tool
def search(query):
    """Perform web search for the given question"""
    return f"Search results for {query}"

tools = [search]
llm = ...

agent = create_react_agent(
    tools=tools,
    model=llm,
)

agent # Renders in notebook as mermaid image

Description

CompiledGraph.get_graph().draw_mermaid_png() - also set as the default while rendering Langgraph graph in Jupyter, sends the generated mermaid syntax code to mermaid.ink server.

1. Is there a way to avoid sending code to an external server like here, so that we avoid sending sensitive data to an external server.
2. While we can pass the draw_method, it is not the honored in CompiledGraph::_repr_mimebundle_ which Jupyter uses to render the graph.
3. Is it possible to set a global override for the draw_method or the mermaid server URL?
4. Is it possible to set a global default to set the renderer of the graph object to use say draw_png and not draw_mermaid_png in _repr_mimebundle_?

Are there any other ways to avoid sending the data to an external server?

5. Also noting that, I tried to set the draw_method to PYPPETEER per instructions in https://langchain-ai.github.io/langgraph/how-tos/visualization/#using-mermaid-pyppeteer but I get this trace:

ElementHandleError                        Traceback (most recent call last)
Cell In[11], line 59
     54 import nest_asyncio
     55 nest_asyncio.apply()
     57 display(
     58     Image(
---> 59         agent.get_graph().draw_mermaid_png(
     60             curve_style=CurveStyle.LINEAR,
     61             node_colors=NodeStyles(first="#ffdfba", last="#baffc9", default="#fad7de"),
     62             wrap_label_n_words=9,
     63             output_file_path=None,
     64             draw_method=MermaidDrawMethod.PYPPETEER,
     65             background_color="white",
     66             padding=10,
     67         )
     68     )
     69 )

File .env/lib64/python3.11/site-packages/langchain_core/runnables/graph.py:667, in Graph.draw_mermaid_png(self, curve_style, node_colors, wrap_label_n_words, output_file_path, draw_method, background_color, padding, frontmatter_config)
    659 from langchain_core.runnables.graph_mermaid import draw_mermaid_png
    661 mermaid_syntax = self.draw_mermaid(
    662     curve_style=curve_style,
    663     node_colors=node_colors,
    664     wrap_label_n_words=wrap_label_n_words,
    665     frontmatter_config=frontmatter_config,
    666 )
--> 667 return draw_mermaid_png(
    668     mermaid_syntax=mermaid_syntax,
    669     output_file_path=output_file_path,
    670     draw_method=draw_method,
    671     background_color=background_color,
    672     padding=padding,
    673 )

File .env/lib64/python3.11/site-packages/langchain_core/runnables/graph_mermaid.py:276, in draw_mermaid_png(mermaid_syntax, output_file_path, draw_method, background_color, padding)
    273 if draw_method == MermaidDrawMethod.PYPPETEER:
    274     import asyncio
--> 276     img_bytes = asyncio.run(
    277         _render_mermaid_using_pyppeteer(
    278             mermaid_syntax, output_file_path, background_color, padding
    279         )
    280     )
    281 elif draw_method == MermaidDrawMethod.API:
    282     img_bytes = _render_mermaid_using_api(
    283         mermaid_syntax, output_file_path, background_color
    284     )

File python-3.11/std/lib64/python3.11/site-packages/nest_asyncio.py:31, in _patch_asyncio.<locals>.run(main, debug)
     29 task = asyncio.ensure_future(main)
     30 try:
---> 31     return loop.run_until_complete(task)
     32 finally:
     33     if not task.done():

File std/lib64/python3.11/site-packages/nest_asyncio.py:99, in _patch_loop.<locals>.run_until_complete(self, future)
     96 if not f.done():
     97     raise RuntimeError(
     98         'Event loop stopped before Future completed.')
---> 99 return f.result()

File asyncio/futures.py:203, in Future.result(self)
    201 self.__log_traceback = False
    202 if self._exception is not None:
--> 203     raise self._exception.with_traceback(self._exception_tb)
    204 return self._result

File asyncio/tasks.py:277, in Task.__step(***failed resolving arguments***)
    273 try:
    274     if exc is None:
    275         # We use the `send` method directly, because coroutines
    276         # don't have `__iter__` and `__next__` methods.
--> 277         result = coro.send(None)
    278     else:
    279         result = coro.throw(exc)

File .env/lib64/python3.11/site-packages/langchain_core/runnables/graph_mermaid.py:318, in _render_mermaid_using_pyppeteer(mermaid_syntax, output_file_path, background_color, padding, device_scale_factor)
    314 await page.goto("about:blank")
    315 await page.addScriptTag(
    316     {"url": "https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"}
    317 )
--> 318 await page.evaluate(
    319     """() => {
    320             mermaid.initialize({startOnLoad:true});
    321         }"""
    322 )
    324 # Render SVG
    325 svg_code = await page.evaluate(
    326     """(mermaidGraph) => {
    327             return mermaid.mermaidAPI.render('mermaid', mermaidGraph);
    328         }""",
    329     mermaid_syntax,
    330 )

File pyppeteer/page.py:1116, in Page.evaluate(self, pageFunction, force_expr, *args)
   1114 if frame is None:
   1115     raise PageError('No main frame.')
-> 1116 return await frame.evaluate(pageFunction, *args, force_expr=force_expr)

File pyppeteer/frame_manager.py:308, in Frame.evaluate(self, pageFunction, force_expr, *args)
    306 if context is None:
    307     raise ElementHandleError('ExecutionContext is None.')
--> 308 return await context.evaluate(
    309     pageFunction, *args, force_expr=force_expr)

File pyppeteer/execution_context.py:53, in ExecutionContext.evaluate(self, pageFunction, force_expr, *args)
     47 async def evaluate(self, pageFunction: str, *args: Any,
     48                    force_expr: bool = False) -> Any:
     49     """Execute ``pageFunction`` on this context.
     50 
     51     Details see :meth:`pyppeteer.page.Page.evaluate`.
     52     """
---> 53     handle = await self.evaluateHandle(
     54         pageFunction, *args, force_expr=force_expr)
     55     try:
     56         result = await handle.jsonValue()

File pyppeteer/execution_context.py:112, in ExecutionContext.evaluateHandle(self, pageFunction, force_expr, *args)
    110 exceptionDetails = _obj.get('exceptionDetails')
    111 if exceptionDetails:
--> 112     raise ElementHandleError('Evaluation failed: {}'.format(
    113         helper.getExceptionMessage(exceptionDetails)))
    114 remoteObject = _obj.get('result')
    115 return self._objectHandleFactory(remoteObject)

ElementHandleError: Evaluation failed: ReferenceError: mermaid is not defined
    at __pyppeteer_evaluation_script__:2:17

System Info

pip freeze | egrep -e 'langchain|pyppeteer|nest-asyncio'
langchain==0.3.14
pyppeteer==1.0.2
nest-asyncio==1.5.8

python -m langchain_core.sys_info

System Information
------------------
> OS:  Linux
> OS Version:  #1 SMP Thu Feb 6 21:20:51 EST 2025
> Python Version:  3.11.8 (main, Sep 17 2024, 14:28:20) [GCC 10.3.1 20210422 (Red Hat 10.3.1-1)]

Package Information
-------------------
> langchain_core: 0.3.48
> langchain: 0.3.14
> langchain_community: 0.3.14
> langsmith: 0.2.10
> langchain_anthropic: 0.2.4
> langchain_cli: 0.0.31
> langchain_experimental: 0.3.4
> langchain_openai: 0.2.14
> langchain_text_splitters: 0.3.5
> langchainplus_sdk: 0.0.21
> langgraph_gen: 0.0.6
> langgraph_reflection: 0.0.1
> langgraph_sdk: 0.1.60
> langgraph_supervisor: 0.0.15
> langgraph_swarm: 0.0.8

Optional packages not installed
-------------------------------
> langserve

Other Dependencies
------------------
> aiohttp: 3.8.4
> anthropic: 0.30.0
> async-timeout: 4.0.3
> dataclasses-json: 0.6.3
> defusedxml: 0.7.1
> gitpython: 3.1.40
> gritql: Installed. No version info available.
> httpx: 0.27.0
> httpx-sse: 0.4.0
> jinja2>=3.1.5: Installed. No version info available.
> jsonpatch<2.0,>=1.33: Installed. No version info available.
> langchain-core<0.4.0,>=0.3.40: Installed. No version info available.
> langchain>=0.1.0: Installed. No version info available.
> langgraph: 0.3.21
> langgraph-prebuilt<0.2.0,>=0.1.7: Installed. No version info available.
> langgraph<0.4.0,>=0.3.5: Installed. No version info available.
> langgraph>=0.2.74: Installed. No version info available.
> langserve[all]: Installed. No version info available.
> langsmith-pyo3: Installed. No version info available.
> langsmith<0.4,>=0.1.125: Installed. No version info available.
> mypy>=1.8.0: Installed. No version info available.
> numpy: 1.26.4+deshaw5
> openai: 1.59.7
> orjson: 3.10.16+deshaw1
> packaging<25,>=23.2: Installed. No version info available.
> pydantic: 2.9.2
> pydantic-settings: 2.6.0
> pydantic<3.0.0,>=2.5.2;: Installed. No version info available.
> pydantic<3.0.0,>=2.7.4;: Installed. No version info available.
> PyYAML: 6.0.2
> PyYAML>=5.3: Installed. No version info available.
> pyyaml>=6.0.2: Installed. No version info available.
> requests: 2.31.0
> requests-toolbelt: 1.0.0
> SQLAlchemy: 1.4.50
> tenacity: 8.2.3
> tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
> tiktoken: 0.7.0
> tomlkit: 0.12.3
> typer[all]: Installed. No version info available.
> typing-extensions>=4.7: Installed. No version info available.
> uvicorn: 0.26.0
> zstandard: 0.22.0

[PxPerfectMike]

Hi sundaraa-deshaw,

You're absolutely right to be concerned about sending potentially sensitive data to an external server through the mermaid.ink API. I can suggest several approaches to address this issue:

1. Modify the _render_mermaid_using_api function

You can modify the LangGraph source code directly by navigating to the langchain_core/runnables/graph_mermaid.py file and locating the _render_mermaid_using_api function. There, you can replace the mermaid.ink URL with a local server URL.

This approach involves:

1. Finding the _render_mermaid_using_api function in the source code
2. Replacing https://mermaid.ink with a local URL like http://localhost:3000
3. Setting up a local mermaid renderer service

2. Set up a local Mermaid renderer

You can use the official mermaid-cli tool to render Mermaid diagrams locally without sending data to an external service. This is available via Docker or can be installed via npm.

# Install mermaid-cli
npm install -g @mermaid-js/mermaid-cli

# Or use Docker
docker run --rm -v "$(pwd):/data" mermaid-js/mermaid-cli -i input.mmd -o output.png

3. Create a custom rendering function

Since the issue is happening in the CompiledGraph._repr_mimebundle_ method which doesn't honor the draw_method parameter, you could create a subclass to override this behavior:

from langgraph.prebuilt.agent import CompiledGraph
from langchain_core.runnables.graph import MermaidDrawMethod

# Create a subclass of CompiledGraph with modified behavior
class SafeCompiledGraph(CompiledGraph):
    def _repr_mimebundle_(self, **kwargs):
        # Override to use PYPPETEER instead of API for rendering
        bundle = super()._repr_mimebundle_(**kwargs)
        if "image/png" in bundle:
            # Replace with your custom rendering logic
            graph = self.get_graph()
            mermaid_syntax = graph.draw_mermaid()
            # Use your own rendering method here
            # ...
        return bundle

4. Fix the PYPPETEER method issues

For the PYPPETEER rendering method, you're encountering issues with event loops, which is a known issue. The error occurs because you're trying to call an async function from an existing event loop.

To fix this:

1. Make sure you have the latest versions of pyppeteer and nest_asyncio
2. Ensure nest_asyncio.apply() is called before using pyppeteer
3. Consider creating a synchronous wrapper for the async pyppeteer functions

5. Monkey patch the repr_mimebundle method

Since the issue is specifically with the Jupyter integration, you could monkey patch the method:

from langgraph.prebuilt import CompiledGraph
from langchain_core.runnables.graph import MermaidDrawMethod
import types

# Store the original method
original_repr = CompiledGraph._repr_mimebundle_

# Create a new method that uses PYPPETEER
def new_repr_mimebundle(self, **kwargs):
    # Call original but modify the result
    bundle = original_repr(self, **kwargs)
    if "image/png" in bundle:
        # Replace with locally rendered version
        png_bytes = self.get_graph().draw_mermaid_png(
            draw_method=MermaidDrawMethod.PYPPETEER
        )
        bundle["image/png"] = png_bytes
    return bundle

# Replace the method
CompiledGraph._repr_mimebundle_ = new_repr_mimebundle

This would ensure that all graph renderings use the PYPPETEER method rather than the API, even in Jupyter notebooks.

[sundaraa-deshaw]

Thanks for the response.

1. Modify the _render_mermaid_using_api function
2. Set up a local Mermaid renderer

This would mean, we need to be able to override the mermaid URL, and that requires a code change in langchain-core source. Though this works, the idea behind this discussion, was to make the URL configurable.

3. Create a custom rendering function

This works, but we need to ensure that all references in langgraph* point to the new version of the graph SafeCompiledGraph. Still doable, but requires changes in source.

4. Fix the PYPPETEER method issues

We are using this version of nest-asycio
nest-asyncio 1.5.8

and as mentioned in the trace, nest_asyncio.apply() is called.

5. Monkey patch the repr_mimebundle method

This is do-able.

net-net: we are looking for ways to globally override the URLs or the default rendering method.