Skip to content

add pheval command to readme or runner #21

@iQuxLE

Description

@iQuxLE

Issue:

Currently the runners do not also execute pheval-utils generate-benchmark-stats -r benchmark.yaml

Current-Workflow

The runners are putting results into output/pheval-disease-results/subdir1/subdir2.
subdir1 describes:

  • model
  • strategy
  • nr_of_ppkts
  • nr_of_results_per_tsv
  • (e.g. ada_wgt_5084ppkts_top10)

subdir2 describes:

  • model
  • strategy
  • collection_name
  • nr_of_ppkts,
  • nr_of_results_per_tsv
  • (e.g. ada_wgt_lrd_hpo_embeddings_5084ppkts_top10)

The reason for the subdir2 is simply if people want to check the performance using different fields to be indexed from the ontology.

Updated Workflow

We want to have the results in output/subdir1/subdir2/pheval-disease-results to be perfectly read by pheval.
So that a benchmark.yaml can be configured and this needs a directory pheval-disease-results where all tsv results are inside.

However, the benchmark.yaml will always need to be structured manually.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions