Validation function and data

[akansh12]

Dear organizers,
Thank you for organizing this challenge, I have a question regarding the validation data.
Suppose I train my model on collaborators: [1,2,3,4,5], the score data frame provides scores. On which data does it provide the score. Is it something like the model gets to train on collaborators: [1,2,3,4] and tested on collaborators [5]. Or there is some other logic.

Thanks.
Akansh

[sarthakpati]

Hi,

Thanks for your interest in the FeTS Challenge!

For each collaborator, the data is split 80/20 for training/validation. For every round of federated training, the training data is used and the corresponding metrics and generated using the validation data.

Cheers,
Sarthak

[akansh12]

What if more than one collaborator is selected? I believe what ideal situation should be that if the model is trained on collaborators 1,2,3 then it should be validated on data from collaborator 4. No part of collaborators 1,2,3 should be present in validation. Is this the case?

Regards
Akansh

[sarthakpati]

Hi,

Data from each collaborator (80% - training and 20% for validation is split before training begins) is used in the validation stage to ensure the model's performance is calculated in a fair manner.