@@ -107,17 +107,20 @@ The aggregation happens separately for all positive predictions (i.e., $p_c^{m_i
107107($p_c^{m_i} < 0.5$). If the aggregated value is larger for the positive predictions than for the negative predictions,
108108the ensemble makes a positive prediction for class $c$:
109109
110+ <img width="2297" height="114" alt="image" src="https://github.com/user-attachments/assets/2f0263ae-83ac-41ea-938a-c71b46082c22" />
111+ <!-- For some reason, this formula does not render in GitHub markdown. Therefore, I rendered it locally and added it as an image. The rendered formula is:
110112$$
111113\t ext{ensemble}(c) = \b egin{cases}
1121141 & \t ext{if } \s um_{i: p_c^{m_i} \g eq 0.5} [\t ext{confidence}_c^{m_i} \c dot \t ext{model_weight}_{m_i} \c dot \t ext{trust}_c^{m_i}] > \s um_{i: p_c^{m_i} < 0.5} [\t ext{confidence}_c^{m_i} \c dot \t ext{model_weight}_{m_i} \c dot \t ext{trust}_c^{m_i}] \\
1131150 & \t ext{otherwise}
114116\e nd{cases}
115117$$
118+ -->
116119
117120Here, confidence is the model's (self-reported) confidence in its prediction, calculated as
118- $$
121+ $
119122\t ext{confidence}_c^{m_i} = 2|p_c^{m_i} - 0.5|
120- $$
123+ $
121124For example, if a model makes a positive prediction with $p_c^{m_i} = 0.55$, the confidence is $2|0.55 - 0.5| = 0.1$.
122125One could say that the model is not very confident in its prediction and very close to switching to a negative prediction.
123126If another model is very sure about its negative prediction with $p_c^{m_j} = 0.1$, the confidence is $2|0.1 - 0.5| = 0.8$.
@@ -131,7 +134,8 @@ Trust is based on the model's performance on a validation set. After training, w
131134on a validation set for each class. If the `ensemble_type` is set to `wmv-f1`, the trust is calculated as 1 + the F1 score.
132135If the `ensemble_type` is set to `mv` (the default), the trust is set to 1 for all models.
133136
134- 3. After a decision has been made for each class independently, the consistency of the predictions with regard to the ChEBI hierarchy
137+ # ## Inconsistency correction
138+ After a decision has been made for each class independently, the consistency of the predictions with regard to the ChEBI hierarchy
135139and disjointness axioms is checked. This is
136140done in 3 steps :
137141- (1) First, the hierarchy is corrected. For each pair of classes $A$ and $B$ where $A$ is a subclass of $B$ (following
@@ -145,4 +149,4 @@ both, we select one with the higher class score and set the other to 0.
145149- (3) Since the second step might have introduced new inconsistencies into the hierarchy, we repeat the first step, but
146150with a small change. For a pair of classes $A \subseteq B$ with predictions $1$ and $0$, instead of setting $B$ to $1$,
147151we now set $A$ to $0$. This has the advantage that we cannot introduce new disjointness-inconsistencies and don't have
148- to repeat step 2.
152+ to repeat step 2.
0 commit comments