RSTT

[Ematrion]

Submitting Author: Bucher David (@Ematrion)
Package Name: RSTT
One-Line Description of Package: Competition simulation tool to evaluate ranking
Repository Link (if existing): https://github.com/Ematrion/rstt
EiC: @eliotwrobson

---

Code of Conduct & Commitment to Maintain Package

• I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package after should it be accepted.
• I have read and will commit to package maintenance after the review as per the pyOpenSci Policies Guidelines.

Description

• Include a brief paragraph describing what your package does:

Package enable sport/games simulation providing different tournament format, state of the art rating system and probabilistic model. It is build around the interaction of a ranking, used to seed competition, and the resulting games, used to update the ranking. It allows users to compared trained ranking with the simulation model, and answer question such has 'Does the ranking have a unique stationary distribution'.

It aims at enabling simulation based research in the context of competition.

Community Partnerships

We partner with communities to support peer review with an additional layer of
checks that satisfy community requirements. If your package fits into an
existing community please check below:

• Astropy: My package adheres to Astropy community standards
• Pangeo: My package adheres to the Pangeo standards listed in the pyOpenSci peer review guidebook

Scope

• Please indicate which category or categories this package falls under:

  • Data retrieval
  • Data extraction
  • Data processing/munging
  • Data deposition
  • Data validation and testing
  • Data visualization
  • Workflow automation
  • Citation management and bibliometrics
  • Scientific software wrappers
  • Database interoperability

Domain Specific

• Geospatial
• Education

---

• Explain how and why the package falls under these categories (briefly, 1-2 sentences). For community partnerships, check also their specific guidelines as documented in the links above. Please note any areas you are unsure of:

It helps producing well defined, yet complex synthetic dataset with a precise and short syntax, 20 lines of code or less for millions of games. It is possible to reproduce others research (check tutorial_3) in the field.

• Who is the target audience and what are the scientific applications of this package?
  Include but not limited to:

  • data scientist testing rating system for large video game title or sport leagues.
  • game developer who wants to make simple test.
  • teaching tool to illustrate ranking behaviour in an intuitive context.
  • mathematician studying Markov process and stationery distribution.

• Are there other Python packages that accomplish similar things? If so, how does yours differ?
  I know there is a paper on a tool call MMBench. I could not access it. It seems limited to its own component while RSTT allows user to define and integrate custom model. It does not seems to address the interplay between ranking and game generation project. Does not seems to tune the initial ranking state.

• Any other questions or issues we should be aware of:

• I developed this tool to perform my master thesis. I improved the code to make it a public tool, and I intend to make several publication based on it.

• Technically, the abstraction around which the tool are build, are general enough to be extended to recommandation system in a broad sense.

• It is labelled as Alpha version. With the review process I want to reach a 1.0.0 first official stable release.

• There are many features and functionalities I want to incorporate later.

P.S. Have feedback/comments about our review process? Leave a comment here

[Ematrion]

Link to MMBench: https://dl.acm.org/doi/10.1145/3539597.3573023

About the scope, I feel like it matches the section:
"Scope for packages that support analytics, statistics and modeling"

As for a statement of need: https://escholarship.org/content/qt2w32w4b5/qt2w32w4b5.pdf?t=p18038
D.Aldous, 2017 describes a neglected field. My tool covers this field and generalise the scope to arbitrary system, not just the Elo.

It Supports reproductible research:
The tutorial 3 perfectly reproduces original diagrams using RSTT simulation only.

If there is a consensual tools for simulation based research in the field, it would make life of both research and reviewer significantly easier and increase productivity.

[eliotwrobson]

Editor in Chief checks

Hi there! Thank you for submitting your package for pyOpenSci
review. Below are the basic checks that your package needs to pass
to begin our review. If some of these are missing, we will ask you
to work on them before the review process begins.

Please check our Python packaging guide for more information on the elements
below.

• Installation The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g. conda-forge, bioconda).
  • The package imports properly into a standard Python environment import package.
• Fit The package meets criteria for fit and overlap.
• Documentation The package has sufficient online documentation to allow us to evaluate package function and scope without installing the package. This includes:
  • User-facing documentation that overviews how to install and start using the package.
  • Short tutorials that help a user understand how to use the package and what it can do for them.
  • API documentation (documentation for your code's functions, classes, methods and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
• Core GitHub repository Files
  • README The package has a README.md file with clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
  • Contributing File The package has a CONTRIBUTING.md file that details how to install and contribute to the package.
  • Code of Conduct The package has a CODE_OF_CONDUCT.md file.
  • License The package has an OSI approved license.
    NOTE: We prefer that you have development instructions in your documentation too.
• Issue Submission Documentation All of the information is filled out in the YAML header of the issue (located at the top of the issue template).
• Automated tests Package has a testing suite and is tested via a Continuous Integration service.
• Repository The repository link resolves correctly.
• Package overlap The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly.
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

• Initial onboarding survey was filled out
  We appreciate each maintainer of the package filling out this survey individually. 🙌
  Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. 🙌

---

---

Editor comments

• Regarding the documentation, it looks like the documentation site only contains the API reference, while the checklist asks for more (including tutorials and installation instructions). I think the jupyter notebooks you have with worked examples satisfy this requirement however, so it would be great if these could be integrated into the docs site.
• At a glance, the readme looks good, but please be sure to go through the README checklist, as this is what will be used in the official review.
• It looks like your code is annotated with typeguards (which provide runtime type checks), but I don't see a workflow that does static typechecking with mypy. You should consider adding this, along with a lint workflow (using ruff or a similar tool). You might also want to consider switching to uv over poetry, since uv is faster.
• On the pypi page, the emojis don't resolve correctly.
• It seems like this package could be a very useful tool for research (as you used it in your masters thesis already). If the goal is to use this as a research tool that people can cite, I would strongly encourage you to complete the archiving steps needed for the JOSS submission, then people can cite the JOSS publication once that's out (https://www.pyopensci.org/software-peer-review/how-to/author-guide.html#journal-of-open-source-software-joss-submission)
• You mentioned wanting to incorporate additional features, but you don't have any open issues on the GitHub repo. Adding planned features as open issues makes it easier for external reviewers to understand the full planned scope of the project.

Overall, the package looks really good and you've clearly put a lot of work into it. With the items above resolved, this will make a strong submission. Once the above are all taken care of (at least to the degree the checklist requires), feel free to close this issue and we can get the review started. In the meantime, I'll start looking for an editor 👍🏽

[Ematrion]

Hi,

Thank you very much for your comments.
I've been working on your points - one being very challenging.

• regarding static typing, I played around mypy, pyrefly and pyright. In short. It is going to be very hard, or impossible to have meaningful static checker for the package. (Linting with ruff will be added as workflow).

I designed rstt to test externally implemented system - i.e without knowing what the system requiers as update input nor what rating it uses. I designed the runtime-checker to provide understandable error feedback for the user. It uses a specific version of typeguard that define Protocols with members existence, not method signature.

I understand that a code which can not be verified staticaly can be challenging to review (or even use).

In the contribution.md, I do require that example code and notebook do not break. There is a workflow that runs the examples to ensure it.

[eliotwrobson]

@Ematrion that makes sense. The use of static typing isn't a hard requirement, but was just a suggestion if it wasn't too much effort on top of the type annotations you already had. No worries if it causes issues and isn't worth setting up.