Build up a test and support matrix

[detiber]

• What Operating Systems/versions do we expect to support?
• What Python versions?
• What does manual testing look like?
• Do we need "mock" organizations/repos for manual/automated e2e type testing?
• What does automated testing look like?

[annegentle]

For manual testing, locally I'm using Python 3.9.6 when running the gh_search.py script to get my inventory listings. I am running against the CiscoDevNet org which has about 1,100 repos, 750 public.

[detiber]

@annegentle @justaugustus any objections to skipping Python 2.x support and just targeting Python 3.x versions that are currently supported by the Python community?

If for no other reason, it would completely eliminate the potential for a whole category of bugs surrounding string/unicode handling in Python for testing purposes.

It might cause a pain for folks running really old LTS Linux distros, but we could probably simplify supporting them by shipping a container image containing the tools rather than trying to support out of date Python versions.

[annegentle]

No objections to skipping Python 2.x support from here.

[justaugustus]

No objections from me either!

[annegentle]

The last piece to determine a way forward is how we could "mock" organizations/repos for manual/automated e2e type testing.

Since the GitHub documentation uses octocat (for user) and github (for org), does using those for testing purposes make sense?

Another approach would be to use the cisco-open org for automated testing. Thoughts?

[lelia]

We actually have a separate cisco-ospo org that I've recently been using to test out organization management tools and reusable GitHub workflows. That would definitely be my preference over using cisco-open for testing. Do you know if there's limitations to using octocat/github in a testing context? I'd love to see the relevant docs for more info!