Skip to content

DB update and Request processor#53

Merged
timflutre merged 65 commits intomasterfrom
fix_ui_test
Feb 3, 2026
Merged

DB update and Request processor#53
timflutre merged 65 commits intomasterfrom
fix_ui_test

Conversation

@juliendiot42
Copy link
Collaborator

Hello, @timflutre (cc. @chabrault ) I've made some big updates to the game:

  1. Better usage of the database

Now the app fully use the power of the data-base rather than storing information in
files. The genetic values calculation have been optimized too. Rather than calculating
those before each phenotyping, it is calculated after the individuals creation and stored
in the DB. This doesn't affect much the plant material creation (the genotypes of the new
individuals is already in memory, and only 3 values are stored per individuals in the DB)
but it make the phenotyping request much faster as it can grab the genetic values directly
rather than loading the genotypes (rather slow) and calculating it.

  1. Enhanced UI

Additional UI improvements have been made:

  • Phenotype visualization: Interactive data tables with advanced filtering (for all) and
    graphic plots (currently for tester and admin only, but I planned to implement a better system to select which player can have access to this).
  • Request tracking: Real-time progress indicators of the requests (cf. point 3)
  • Plant material informations: All players can now see: parent names, cross types,
    phenotypic values, and selection status
  • Genotyping interface: Redesigned download page with better request separation
  1. Asynchronous Request Processing

A background worker process (./src/request_worker.R) now execute the pheno/geno/pltmat
requests asynchronously. So the app is not frozen for all the users when one someone make a request.

When a player send a request, the Shiny app only store the requests information in the database
(which is fast). The worker (which automatically start with the application) regularly fetch the database
for unprocessed requests execute them one by one. The players can see the progress of their request with progress bar in the app.

To monitor the worker, the admin page have a new tab with:

  • the worker status (with a button to restart it if it unexpectedly crashed)
  • the list of all the requests (ie. the requests queue, currently processing request, and
    processed requests)
  • the worker's log displaying information about what the worker is doing

To test, I have deployed the docker image with this changes with the development tag eg.:

docker run -d --rm --name plantbreedgame -p 80:3838 \
    -v /host/path/to/plantBreedGameData:/var/lib/plantBreedGame \
    juliendiot/plantbreedgame:development

This commit refact the DB to save the game's data (rather than in
files). The DB schema have been heavilly modify to integrate the
relations between all the data.
to update status or hashed password of the breeders.
Mark controls individuals for final evaluation
Instead of using DATA_DB as default for the db function,
now it uses an R option. This was preventing the DB to be initialised
when the setup script was launched from the game.
Note: Evaluation report have not been updated
A "worker" is started at the app initialisation.
This worker constantly read the DB for unprocessed requests and process
them.
I've saw some problems where the DB is sometime locked because either
the app, or the worker is using it.
Those settings should allow better concurrent usage of the db.
If the worker die durring the processing of a request, this request
would be marked as "processing" undefinitly. This commit make the
request processor check for unfinished request and mark them as failed
if their "processing time" is unexpectedly long.
fix bug when the input reset only worked 1 time (other reset didn't
actually reset the input)
Before, the individuals submission requests was saved in the DB with a progress
of 0, which made the worker try to execute those requests and locked it.

This have been fixed by:
- Marking the individual submission related request's progress to 1
- Explicitly listing the type of request the worker fetch for processing
Also refact to test in separated files.
Should not be needed anymore with the request processor
@timflutre timflutre merged commit 44d4acd into master Feb 3, 2026
6 checks passed
@timflutre
Copy link
Owner

Looks great Julien! I will notify Cédric Goby from Institut Agro Montpellier who has been deploying it recently on INRAE's SK8 service, so that he can test it.

@juliendiot42 juliendiot42 deleted the fix_ui_test branch February 9, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants