Adopt Proposed API for Test Coverage

[eleanorjboyd]

A new API has been created to allow for test coverage to be reported to the user. Implement this for python and also provide feedback on the API as I go.

[MathisFederico]

Hello, I would love help out to have access to coverage data.

In fact I just forked the repo this week and started to dig in the extension testing structure to see if it would be possible to have automated running of tests on modifications similarly to wallaby.js but in python for a better TDD experience.

A requirement for a smooth experience would be to only re-run affected tests by real-time edits and thus to know for each modified statement what is the list of tests that pass it, and thus coverage data.

I would be happy to contribute if I can be of any use.

[eleanorjboyd]

Hi @MathisFederico, thank you for your offer! Automatic run on modification isn't something we have planned so this is always something you could contribute once coverage is finished.

I am planning on outlining the design for coverage when I work on this next in 2 weeks so your input on that would be helpful. Additionally I am always looking for test cases to use especially those that cover common scenarios and edge cases. Ill keep this thread in the loop, thanks

[MathisFederico]

Hi @MathisFederico, thank you for your offer! Automatic run on modification isn't something we have planned so this is always something you could contribute once coverage is finished.

Being a heavy test user, I'd be happy to contribute ! @eleanorjboyd

That was the plan anyway but coverage data is a requirement, so I might as well help on this first. (It will get me started on the codebase practice anyway)

I am planning on outlining the design for coverage when I work on this next in 2 weeks so your input on that would be helpful.

Anytime ! Feel free to ping me whenever. I did start to look into it myself and it was difficult to get all the interesting data (each test context for each line covered) except by parsing the html output from coverage.py

I'm curious to see what you have in mind for coverage anyway and how you picture displaying it to user.

Additionally I am always looking for test cases to use especially those that cover common scenarios and edge cases. Ill keep this thread in the loop, thanks

I have done quite a lot of tests through pytest on my different open-source projects, dummy versions of those could cover a diverse set of pytest cases.

You'll tell me if you want me to dig into those. I'd be happy to directly add them myself into a PR if I'm shown a basic example.

[MathisFederico]

Hi @eleanorjboyd, any news on the coverage data API ?

[aaronsteers]

I'm extremely excited about seeing Python language support for integrated code coverage visibility directly in the IDE.

@kieferrm and @eleanorjboyd - Is this the correct issue to watch for updates on adding Python-language support for the new VS Code Code Coverage API, and is there any specific timeline you have in mind now that this API has launched?

If not immediately on the horizon within this extension, does anyone know if there are plans (in the community perhaps) for something like a bespoke Python Codecoverage extension?