How to correctly identify or test configurable ids when using RunnableLambda and ConfigurableField

[codekiln]

UPDATE - see please see detailed pre-bug report below

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from dotenv import load_dotenv

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import PromptTemplate
from langchain_core.runnables import RunnableLambda, ConfigurableField

load_dotenv()

"""
This module tests a variation of 
[How to route between sub-chains](https://python.langchain.com/v0.2/docs/how_to/routing/#using-a-custom-function-recommended)
that uses ConfigurableFields; see also [https://python.langchain.com/v0.2/docs/how_to/configure/](https://python.langchain.com/v0.2/docs/how_to/configure/)

While the original version without configuration works, it fails with configuration
"""


def get_llm():
    # substitute the LLM you have creds for
    from langchain_aws import ChatBedrock

    return ChatBedrock(model_id="anthropic.claude-3-haiku-20240307-v1:0")


chain = (
    PromptTemplate.from_template(
        """Given the user question below, classify it as either being about `LangChain`, `Anthropic`, or `Other`.
                                
                                Do not respond with more than one word.
                                
                                <question>
                                {question}
                                </question>
                                
                                Classification:"""
    )
    | get_llm()
    | StrOutputParser()
)

LANGCHAIN_DEFAULT_PROMPT = """You are an expert in langchain. \
                        Always answer questions starting with "As Harrison Chase told me". \
                        Respond to the following question:
                        
                        Question: {question}
                        Answer:"""

langchain_chain = (
    PromptTemplate.from_template(LANGCHAIN_DEFAULT_PROMPT).configurable_fields(
        # THIS IS NEW relative to [How to route between sub-chains | 🦜️🔗 LangChain](https://python.langchain.com/v0.2/docs/how_to/routing/#using-a-custom-function-recommended)
        # See also [How to configure runtime chain internals | 🦜️🔗 LangChain](https://python.langchain.com/v0.2/docs/how_to/configure/) for more details
        template=ConfigurableField(id="langchain_prompt", name="LangChain", description="LangChain")
    )
    | get_llm()
)
anthropic_chain = (
    PromptTemplate.from_template(
        """You are an expert in anthropic. \
                                Always answer questions starting with "As Dario Amodei told me". \
                                Respond to the following question:
                                
                                Question: {question}
                                Answer:"""
    )
    | get_llm()
)
general_chain = (
    PromptTemplate.from_template(
        """Respond to the following question:
                                
                                Question: {question}
                                Answer:"""
    )
    | get_llm()
)


def route(info):
    if "anthropic" in info["topic"].lower():
        return anthropic_chain
    elif "langchain" in info["topic"].lower():
        return langchain_chain
    else:
        return general_chain


full_chain = {"topic": chain, "question": lambda x: x["question"]} | RunnableLambda(route)


def test_invoke():
    result = full_chain.invoke({"question": "how do I use Anthropic?"})
    assert "Dario" in result.content
    result = full_chain.invoke({"question": "how do I use LangChain?"})
    assert "Harrison" in result.content

    # okay, so far, so good. Now let's try configuring.
    configurable_id_fields_langchain = [c.id for c in langchain_chain.config_specs]
    assert "langchain_prompt" in configurable_id_fields_langchain

    substitute_string = "Harrison Chase"
    substitute_with = "Nuno Campos"
    substituted_prompt = LANGCHAIN_DEFAULT_PROMPT.replace(substitute_string, substitute_with)
    configured_langchain_chain = langchain_chain.with_config(configurable={"langchain_prompt": substituted_prompt})
    langchain_result = configured_langchain_chain.invoke({"question": "how do I use LangChain?"})
    assert substitute_with in langchain_result.content

    configured_full_chain = full_chain.with_config(configurable={"langchain_prompt": substituted_prompt})
    configured_full_chain_result = configured_full_chain.invoke({"question": "how do I use LangChain?"})
    assert substitute_with in configured_full_chain_result.content

    configurable_id_fields_full_chain = [c.id for c in full_chain.config_specs]
    assert "langchain_prompt" in configurable_id_fields_full_chain
    # E       AssertionError: assert 'langchain_prompt' in []

Description

• See source code above in github at create failing test of routing and configuring · codekiln/langchain-playground-1@0f5ad1e
• I'm trying to combine How to route between sub-chains | 🦜️🔗 LangChain with How to configure runtime chain internals | 🦜️🔗 LangChain.
• I'm seeking a way to create a RunnableLambda which routes between different chains, each of which may be an instance of a chain with a ConfigurableField declared on a call to .configurable_fields(...), e.g. a RunnableConfigurableFields.
• up until now, I've been using <chain>.config_specs as a way to test in my unit tests whether a chain has the expected configurables. Now that I've introduced routing with a RunnableLambda, I've discovered that it breaks these tests, because I can no longer detect from the chain reference which fields are configurable inside of the routed chains.
• At first, I assumed that this implied I would not be able to obtain the expected configured results when using routing. To my surprise, it does end up working; in the test script, the very last call to invoke() does return Nuno Campos, even though the chain itself does not contain the key for the configurable.

So as a result, I have these questions:

1. Why or how doesn't the chain disclose its configurables in all cases?
2. How do you unit test configurable chains without turning them into integration tests? Just a bunch of mocking?

System Info

(langchainy) ~/dev/langchain-playground git:[main]
pip freeze | grep langchain
langchain==0.2.6
langchain-aws==0.1.11
langchain-community==0.2.6
langchain-core==0.2.22
langchain-openai==0.1.14
langchain-text-splitters==0.2.2
-e git+ssh://[email protected]/langchain-ai/langgraph.git@5232ea260606b0a5cdba2cf31bacdd4ab15d5a2c#egg=langgraph

[dosubot[bot]]

@codekiln Hello! I'm here to assist you with your issue.

To correctly identify or test configurable IDs when using RunnableLambda and ConfigurableField in LangChain, you can utilize the ConfigurableField class and its variants (ConfigurableFieldSingleOption, ConfigurableFieldMultiOption). These classes are designed to handle fields that can be configured by the user, each with a unique id.

Here's an example of how you can define and test configurable IDs:

from langchain_core.runnables.utils import ConfigurableField, ConfigurableFieldSingleOption, ConfigurableFieldMultiOption

# Define some configurable fields
field1 = ConfigurableField(id="field1", name="Field 1", description="This is field 1")
field2 = ConfigurableFieldSingleOption(id="field2", options={"opt1": 1, "opt2": 2}, default="opt1", name="Field 2")
field3 = ConfigurableFieldMultiOption(id="field3", options={"optA": "A", "optB": "B"}, default=["optA"], name="Field 3")

# Function to test if a given ID is in the list of configurable fields
def is_configurable_id(configurable_fields, test_id):
    return any(field.id == test_id for field in configurable_fields)

# List of configurable fields
configurable_fields = [field1, field2, field3]

# Test the function
test_id = "field2"
print(is_configurable_id(configurable_fields, test_id))  # Output: True

test_id = "field4"
print(is_configurable_id(configurable_fields, test_id))  # Output: False

In addition, you can refer to the unit tests provided in LangChain's test suite to see how configurable fields are tested in practice. For example, the test_configurable_fields function demonstrates how to define and test configurable fields for a fake LLM and a prompt template:

def test_configurable_fields() -> None:
    fake_llm = FakeListLLM(responses=["a"])  # str -> List[List[str]]

    assert fake_llm.invoke("...") == "a"

    fake_llm_configurable = fake_llm.configurable_fields(
        responses=ConfigurableField(
            id="llm_responses",
            name="LLM Responses",
            description="A list of fake responses for this LLM",
        )
    )

    assert fake_llm_configurable.invoke("...") == "a"

    assert fake_llm_configurable.config_schema().schema() == {
        "title": "RunnableConfigurableFieldsConfig",
        "type": "object",
        "properties": {"configurable": {"$ref": "#/definitions/Configurable"}},
        "definitions": {
            "Configurable": {
                "title": "Configurable",
                "type": "object",
                "properties": {
                    "llm_responses": {
                        "title": "LLM Responses",
                        "description": "A list of fake responses for this LLM",
                        "default": ["a"],
                        "type": "array",
                        "items": {"type": "string"},
                    }
                },
            }
        },
    }

    fake_llm_configured = fake_llm_configurable.with_config(
        configurable={"llm_responses": ["b"]}
    )

    assert fake_llm_configured.invoke("...") == "b"

    prompt = PromptTemplate.from_template("Hello, {name}!")

    assert prompt.invoke({"name": "John"}) == StringPromptValue(text="Hello, John!")

    prompt_configurable = prompt.configurable_fields(
        template=ConfigurableField(
            id="prompt_template",
            name="Prompt Template",
            description="The prompt template for this chain",
        )
    )

    assert prompt_configurable.invoke({"name": "John"}) == StringPromptValue(
        text="Hello, John!"
    )

This example shows how to create configurable fields for a fake LLM and a prompt template, and how to test their configurations. This approach ensures that you can correctly identify and test configurable IDs ^[1][2].

Additionally, the RunnableConfigurableFields class in LangChain provides a structured way to dynamically configure runnables. Here is an example of using RunnableConfigurableFields with LLMs:

from langchain_core.prompts import PromptTemplate
from langchain_core.runnables import ConfigurableField
from langchain_openai import ChatOpenAI

model = ChatOpenAI(temperature=0).configurable_fields(
    temperature=ConfigurableField(
        id="temperature",
        name="LLM Temperature",
        description="The temperature of the LLM",
    )
)
# This creates a RunnableConfigurableFields for a chat model.

# When invoking the created RunnableSequence, you can pass in the
# value for your ConfigurableField's id which in this case
# will be change in temperature

prompt = PromptTemplate.from_template("Pick a random number above {x}")
chain = prompt | model

chain.invoke({"x": 0})
chain.invoke({"x": 0}, config={"configurable": {"temperature": 0.9}})

This example demonstrates how to dynamically configure a chat model's temperature using RunnableConfigurableFields ^[1].

The Runnable class in LangChain provides methods like configurable_fields and config_schema to handle dynamic configurations. The config_schema method returns a pydantic model that can be used to validate the configuration, ensuring that the configurable fields are correctly identified and tested ^[1].

To continue talking to Dosu, mention @dosu.

[dosubot[bot]]

The issue you're encountering is due to the fact that the RunnableLambda class does not inherently disclose the configurable fields of the chains it routes to. This is because RunnableLambda itself does not have direct knowledge of the internal structure or configuration of the chains it routes to.

To address this, you can manually aggregate the configurable fields from the sub-chains and expose them through the RunnableLambda. Here's an approach to achieve this:

1. Aggregate Configurable Fields: Collect the configurable fields from each sub-chain and combine them.
2. Expose Configurable Fields: Modify the RunnableLambda to expose these aggregated configurable fields.

Here's an example of how you can implement this:

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import PromptTemplate
from langchain_core.runnables import RunnableLambda, ConfigurableField, RunnableConfigurableFields
from typing import Any, Optional, List

def get_llm():
    from langchain_aws import ChatBedrock
    return ChatBedrock(model_id="anthropic.claude-3-haiku-20240307-v1:0")

# Define the chains
LANGCHAIN_DEFAULT_PROMPT = """You are an expert in langchain. \
                        Always answer questions starting with "As Harrison Chase told me". \
                        Respond to the following question:
                        
                        Question: {question}
                        Answer:"""

langchain_chain = (
    PromptTemplate.from_template(LANGCHAIN_DEFAULT_PROMPT).configurable_fields(
        template=ConfigurableField(id="langchain_prompt", name="LangChain", description="LangChain")
    )
    | get_llm()
)

anthropic_chain = (
    PromptTemplate.from_template(
        """You are an expert in anthropic. \
                                Always answer questions starting with "As Dario Amodei told me". \
                                Respond to the following question:
                                
                                Question: {question}
                                Answer:"""
    )
    | get_llm()
)

general_chain = (
    PromptTemplate.from_template(
        """Respond to the following question:
                                
                                Question: {question}
                                Answer:"""
    )
    | get_llm()
)

# Define the routing function
def route(info):
    if "anthropic" in info["topic"].lower():
        return anthropic_chain
    elif "langchain" in info["topic"].lower():
        return langchain_chain
    else:
        return general_chain

# Create a custom RunnableLambda that aggregates configurable fields
class CustomRunnableLambda(RunnableLambda):
    def __init__(self, func, chains: List[RunnableConfigurableFields]):
        super().__init__(func)
        self.chains = chains

    @property
    def config_specs(self):
        # Aggregate config specs from all chains
        specs = []
        for chain in self.chains:
            specs.extend(chain.config_specs)
        return specs

# Instantiate the custom RunnableLambda
full_chain = {"topic": chain, "question": lambda x: x["question"]} | CustomRunnableLambda(route, [langchain_chain, anthropic_chain, general_chain])

# Test the configuration
def test_invoke():
    result = full_chain.invoke({"question": "how do I use Anthropic?"})
    assert "Dario" in result.content
    result = full_chain.invoke({"question": "how do I use LangChain?"})
    assert "Harrison" in result.content

    # Check configurable fields
    configurable_id_fields_langchain = [c.id for c in langchain_chain.config_specs]
    assert "langchain_prompt" in configurable_id_fields_langchain

    substitute_string = "Harrison Chase"
    substitute_with = "Nuno Campos"
    substituted_prompt = LANGCHAIN_DEFAULT_PROMPT.replace(substitute_string, substitute_with)
    configured_langchain_chain = langchain_chain.with_config(configurable={"langchain_prompt": substituted_prompt})
    langchain_result = configured_langchain_chain.invoke({"question": "how do I use LangChain?"})
    assert substitute_with in langchain_result.content

    configured_full_chain = full_chain.with_config(configurable={"langchain_prompt": substituted_prompt})
    configured_full_chain_result = configured_full_chain.invoke({"question": "how do I use LangChain?"})
    assert substitute_with in configured_full_chain_result.content

    # Check aggregated configurable fields
    configurable_id_fields_full_chain = [c.id for c in full_chain.config_specs]
    assert "langchain_prompt" in configurable_id_fields_full_chain

test_invoke()

In this example, the CustomRunnableLambda class aggregates the configurable fields from the sub-chains and exposes them through its config_specs property. This allows you to detect and test the configurable fields in the full chain, even when routing through RunnableLambda ^[1].

To continue talking to Dosu, mention @dosu.

[codekiln]

@dosu, thank you for this suggestion to create class CustomRunnableLambda(RunnableLambda) which is capable of proxying the config_specs of the component chains.

I have tried this approach and can confirm that it does work, as long as any chains that contain ConfigurableField ids that you want to detect are known at design time, that is, at the time of instantiation of the CustomRunnableLambda; you can see the updated version at langchain-playground-1/test_routing_and_configuring.py at main · codekiln/langchain-playground-1 in commit 1b73e70 - test_routing_and_configuring.py.

So now I've got a proof of concept, thanks to you.

But how can this design be improved?
It seems like there are at least two classes of solutions to this problem.

1. The first class of solutions use the constraint that the developer needs to know ahead of time all of the chains that contain configurables, and pass them to the RunnableLambda, much like your CustomRunnableLambda solution does. Currently the CustomRunnableLambda implies that all chains that are passed to it to also utilize CustomRunnableLambda for any runnable lambdas in its chains. In this class of solution, there are other potential solutions; for example, is there a way that if the developer knows certain prompt ids are available in the chains of a RunnableLambda, they could use type hints to mark a runnable lambda as intending to be able to be configured with certain configurable ids?
2. The second class of solutions would work recursively by redesigning something about RunnableLambda itself. Perhaps one could make a specialized RunnableLambda which is designed for routing. But we already have this in RunnableBranch; indeed, it seems that while RunnableBranch is now deprecated in favor of RunnableLambda, it might have been introduced to allow configuration to be exposed. Can you create a pytest example for me that shows whether RunnableBranch exposes the ids of its ConfigurableFields?

[dosubot[bot]]

Here's a pytest example that demonstrates whether RunnableBranch exposes the IDs of its ConfigurableFields. This will help determine if RunnableBranch can be used as a solution for exposing configurable IDs in a routed chain:

import pytest
from langchain_core.runnables.branch import RunnableBranch
from langchain_core.runnables.base import ConfigurableField

def test_runnable_branch_exposes_configurable_ids():
    # Create a RunnableBranch instance with ConfigurableFields
    field1 = ConfigurableField(id="field1_id", name="Field 1")
    field2 = ConfigurableField(id="field2_id", name="Field 2")
    
    runnable_branch = RunnableBranch(
        (lambda x: isinstance(x, str), lambda x: x.upper()),
        (lambda x: isinstance(x, int), lambda x: x + 1),
        lambda x: "default"
    )
    
    # Manually set the config specs for testing purposes
    runnable_branch.default.config_specs = [field1, field2]
    
    # Check if the IDs of the ConfigurableFields are exposed
    config_specs = runnable_branch.config_specs
    assert any(spec.id == "field1_id" for spec in config_specs)
    assert any(spec.id == "field2_id" for spec in config_specs)

    # Optionally, print the specs for manual inspection
    print(config_specs)

# Run the test
pytest.main([__file__])

This pytest example checks if RunnableBranch exposes the IDs of its ConfigurableFields by asserting that the IDs are present in the configuration specs ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[codekiln]

@dosu as written, the example test fails.

runnable_branch.default.config_specs = [field1, field2]

This line returns AttributeError: property 'config_specs' of 'RunnableLambda' object has no setter.

Maybe you can adapt my example above to using RunnableBranch, rather than trying to manufacture the config_specs attribute? That way, there would already be a real chain with a ConfigurableField declared.

Also, did you happen to have any insight about the first class of solutions I mentioned earlier, where if the developer knows certain prompt ids are to be available in the chains of a RunnableLambda, they could use type hints to mark a runnable lambda as intending to be able to be configured with certain configurable ids?

[dosubot[bot]]

Let's adapt your example to use RunnableBranch instead of RunnableLambda and see if it exposes the IDs of its ConfigurableFields. Additionally, I'll provide insight into using type hints to mark a RunnableLambda as intending to be configured with certain configurable IDs.

Adapting Example to Use RunnableBranch

Here's how you can adapt your example to use RunnableBranch:

import pytest
from dotenv import load_dotenv
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import PromptTemplate
from langchain_core.runnables import ConfigurableField
from langchain_core.runnables.branch import RunnableBranch

load_dotenv()

def get_llm():
    from langchain_aws import ChatBedrock
    return ChatBedrock(model_id="anthropic.claude-3-haiku-20240307-v1:0")

LANGCHAIN_DEFAULT_PROMPT = """You are an expert in langchain. \
                        Always answer questions starting with "As Harrison Chase told me". \
                        Respond to the following question:
                        
                        Question: {question}
                        Answer:"""

langchain_chain = (
    PromptTemplate.from_template(LANGCHAIN_DEFAULT_PROMPT).configurable_fields(
        template=ConfigurableField(id="langchain_prompt", name="LangChain", description="LangChain")
    )
    | get_llm()
)

anthropic_chain = (
    PromptTemplate.from_template(
        """You are an expert in anthropic. \
                                Always answer questions starting with "As Dario Amodei told me". \
                                Respond to the following question:
                                
                                Question: {question}
                                Answer:"""
    )
    | get_llm()
)

general_chain = (
    PromptTemplate.from_template(
        """Respond to the following question:
                                
                                Question: {question}
                                Answer:"""
    )
    | get_llm()
)

def route(info):
    if "anthropic" in info["topic"].lower():
        return anthropic_chain
    elif "langchain" in info["topic"].lower():
        return langchain_chain
    else:
        return general_chain

# Create a RunnableBranch instance
full_chain = RunnableBranch(
    (lambda x: "anthropic" in x["topic"].lower(), anthropic_chain),
    (lambda x: "langchain" in x["topic"].lower(), langchain_chain),
    default=general_chain
)

def test_invoke():
    result = full_chain.invoke({"question": "how do I use Anthropic?", "topic": "anthropic"})
    assert "Dario" in result.content
    result = full_chain.invoke({"question": "how do I use LangChain?", "topic": "langchain"})
    assert "Harrison" in result.content

    # Check configurable fields
    configurable_id_fields_langchain = [c.id for c in langchain_chain.config_specs]
    assert "langchain_prompt" in configurable_id_fields_langchain

    substitute_string = "Harrison Chase"
    substitute_with = "Nuno Campos"
    substituted_prompt = LANGCHAIN_DEFAULT_PROMPT.replace(substitute_string, substitute_with)
    configured_langchain_chain = langchain_chain.with_config(configurable={"langchain_prompt": substituted_prompt})
    langchain_result = configured_langchain_chain.invoke({"question": "how do I use LangChain?", "topic": "langchain"})
    assert substitute_with in langchain_result.content

    configured_full_chain = full_chain.with_config(configurable={"langchain_prompt": substituted_prompt})
    configured_full_chain_result = configured_full_chain.invoke({"question": "how do I use LangChain?", "topic": "langchain"})
    assert substitute_with in configured_full_chain_result.content

    # Check aggregated configurable fields
    configurable_id_fields_full_chain = [c.id for c in full_chain.config_specs]
    assert "langchain_prompt" in configurable_id_fields_full_chain

test_invoke()

Using Type Hints to Mark Configurable IDs

To use type hints or other mechanisms to mark a RunnableLambda as intending to be configured with certain configurable IDs, you can use the ConfigurableField class from the langchain_core.runnables.utils module. This class allows you to specify which fields are configurable and provide type hints for them.

Here's an example:

from typing import Any, Optional
from pydantic import BaseModel
from langchain_core.runnables import RunnableSerializable
from langchain_core.runnables.utils import ConfigurableField

class CustomRunnableLambda(RunnableSerializable[dict, dict]):
    output_key: str

    def invoke(
        self, input: Any, config: Optional[RunnableConfig] = None, **kwargs: Any
    ) -> dict:
        return self._call_with_config(self.process_input, input, config, **kwargs)

    def process_input(self, input: dict) -> dict:
        return {self.output_key: input["foo"] * 2}

# Marking the `output_key` as a configurable field
runnable = CustomRunnableLambda(output_key="result")
configurable_runnable = runnable.configurable_fields(
    output_key=ConfigurableField(
        id="output_key",
        name="Output Key",
        description="The key to use for the output dictionary"
    )
)

# Example invocation with a configurable ID
configurable_runnable.invoke(
    {"foo": 10}, config={"configurable": {"output_key": "custom_result"}}
)

In this example:

• The CustomRunnableLambda class is defined with an output_key attribute.
• The invoke method processes the input and uses the output_key to determine the key in the output dictionary.
• The configurable_fields method is used to mark output_key as a configurable field, allowing it to be set dynamically at runtime.

This approach ensures that your RunnableLambda can be configured with certain IDs, even if the exact chains are not known at design time ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[codekiln]

I haven't had time to create a detailed bug report yet, and I don't quite have time for that now, but I now need to write everything out before I forget; it's been about two weeks.

I stumbled on a behavior that was for me an astonishing behavior if not also a genuine bug: RunnableLambda.deps only looks for non-locals, so any module-level chains that are referenced for routing in the function passed to the RunnableLambda will not surface their config schemas past the RunnableLambda barrier.

My team only detected this difference because a configurable chain was working with a particular configuration in our chain integration tests, but wouldn't work with the same configuration in our langserve endpoint integration tests. The reason why ended up being that APIHandler.config_keys parameter ends up getting used in __init__ farther down when assembling self._ConfigPayload by calling runnable.config_schema, and runnable.config_schema() for the RunnableLambda returned {}, so as a result, _get_config_and_input didn't get the configurables when calling _unpack_request_config. Specifically, on this line, the config_dicts ended up as [{}] instead of a list with the object representing all the configurable elements. This is sanitized by line 203 farther down projected_config = {k: config[k] for k in config_keys if k in config} - here the k in config requires that something be in that config schema in order for it to be considered configurable.

The workaround we adopted was to use RunnableBranch for routing. While RunnableBranch is mentioned as "legacy" in favor of "recommended" RunnableLambda in the 0.2 routing how-to docs, it correctly surfaces module-level chain dependency config schemas, while RunnableLambda does not. This inconsistency to me is either a bug or the reason why RunnableLambda should not be recommended over RunnableBranch in How to route between sub-chains | 🦜️🔗 LangChain.

• TODO post detailed bug report with reproduction criteria

[tomassommareqt]

@codekiln Thanks for reporting this behavior! I've run into the similar issues where configs are lost due to using RunnableLambda. It would be nice if a proper bug report could be submitted, or if the guidance could be provided by the maintainers.