Unable to Execute Third-Party Unit Test Package with VSCode Debugger

[TeomanEgeSelcuk]

VS Code Version: 1.89.1
OS Version: 23H2 (Windows 11)

Steps to Reproduce:

1. Create a new Python project in VSCode with a virtual environment (myenv) and install DeepEval using pip.
2. Create a wrapper script (run_deepeval.py) to handle the DeepEval command.
3. Configure the launch.json file with the following settings:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Deepeval",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/langchain_env/run_deepeval.py",
            "args": [
                "test",
                "run",
                "${workspaceFolder}/tests/test_${fileBasename}"
            ],
            "debugStdLib": true,
            "justMyCode": true
        }
    ]
}

4. Try to debug the project using the VSCode debugger.

Expected Behavior:
The VSCode debugger should execute the DeepEval package and run the unit tests successfully.

Actual Behavior:
The VSCode debugger throws an error:

C:\Users\myuser\anaconda3\envs\myenv\python.exe: No module named deepeval.__main__; 'deepeval' is a package and cannot be directly executed

Additional Information:

• I have ensured that DeepEval is installed in the virtual environment (myenv).
• I have verified that the Python interpreter is correctly set to the virtual environment in VSCode.
• I have tried upgrading DeepEval to the latest version (0.21.45) using pip.

Question:
How can I configure VSCode to properly execute the DeepEval package within the debugger? Is there something wrong with my launch.json configuration, or is there a better way to set this up?

[paulacamargo25]

@TeomanEgeSelcuk thanks for your issue report, it looks that there is an issue with your script, could you manage it to work just in the terminal? could you send me how you execute it?

[vs-code-engineering]

Hey @karthiknadig, this issue might need further attention.

@TeomanEgeSelcuk, you can help us out by closing this issue if the problem no longer exists, or adding more information.

[eleanorjboyd]

closing, please comment with a response to the above question and we can reopen if this is still an issue! Thanks

[shinkenuu]

To make it work be sure to have program pointing to your virtualenv bin folder where deepeval lives.

You can check that running which deepeval

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Deepeval",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/.venv/bin/deepeval",
            "args": [
                "test",
                "run",
                "${workspaceFolder}/tests/test_${fileBasename}"
            ],
        }
    ]
}

[eleanorjboyd]

Hi! Thanks for getting back to me. If you were to run this in your terminal, what command would you use to do so? Trying to see how to mirror that command in the launch.json will be our best bet. Also are you still using the wrapper script or are you trying to call deepeval, the package, directly?

[shinkenuu]

Hello o/

In the terminal use deepeval test run tests/plantai/agents/test_carie.py -vv while my python venv with deepeval installed is active.

I'd use "module": "deepeval" as this is a python package (similar to pytest), but that didnt work - thats how I ended up here.

This is the relevant configuration in my .vscode/launch.json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "DeepEval",
            "type": "debugpy",
            "request": "launch",
            "program": ".venv/bin/deepeval",
            "envFile": "${workspaceFolder}/.env",
            "args": [
                "test",
                "run",
                "-vv",
                "tests/plantai/agents/test_carie.py"
            ]
        }
    ]
}

[vs-code-engineering]

Hey @karthiknadig, this issue might need further attention.

@TeomanEgeSelcuk, you can help us out by closing this issue if the problem no longer exists, or adding more information.