Skip to content

Commit dcd0915

Browse files
authored
Merge branch 'dev' into fix/error-propagation
2 parents 9769e23 + 5d1305e commit dcd0915

File tree

1 file changed

+8
-4
lines changed

1 file changed

+8
-4
lines changed

README.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -107,17 +107,20 @@ The aggregation happens separately for all positive predictions (i.e., $p_c^{m_i
107107
($p_c^{m_i} < 0.5$). If the aggregated value is larger for the positive predictions than for the negative predictions,
108108
the ensemble makes a positive prediction for class $c$:
109109
110+
<img width="2297" height="114" alt="image" src="https://github.com/user-attachments/assets/2f0263ae-83ac-41ea-938a-c71b46082c22" />
111+
<!-- For some reason, this formula does not render in GitHub markdown. Therefore, I rendered it locally and added it as an image. The rendered formula is:
110112
$$
111113
\text{ensemble}(c) = \begin{cases}
112114
1 & \text{if } \sum_{i: p_c^{m_i} \geq 0.5} [\text{confidence}_c^{m_i} \cdot \text{model_weight}_{m_i} \cdot \text{trust}_c^{m_i}] > \sum_{i: p_c^{m_i} < 0.5} [\text{confidence}_c^{m_i} \cdot \text{model_weight}_{m_i} \cdot \text{trust}_c^{m_i}] \\
113115
0 & \text{otherwise}
114116
\end{cases}
115117
$$
118+
-->
116119
117120
Here, confidence is the model's (self-reported) confidence in its prediction, calculated as
118-
$$
121+
$
119122
\text{confidence}_c^{m_i} = 2|p_c^{m_i} - 0.5|
120-
$$
123+
$
121124
For example, if a model makes a positive prediction with $p_c^{m_i} = 0.55$, the confidence is $2|0.55 - 0.5| = 0.1$.
122125
One could say that the model is not very confident in its prediction and very close to switching to a negative prediction.
123126
If another model is very sure about its negative prediction with $p_c^{m_j} = 0.1$, the confidence is $2|0.1 - 0.5| = 0.8$.
@@ -131,7 +134,8 @@ Trust is based on the model's performance on a validation set. After training, w
131134
on a validation set for each class. If the `ensemble_type` is set to `wmv-f1`, the trust is calculated as 1 + the F1 score.
132135
If the `ensemble_type` is set to `mv` (the default), the trust is set to 1 for all models.
133136

134-
3. After a decision has been made for each class independently, the consistency of the predictions with regard to the ChEBI hierarchy
137+
### Inconsistency correction
138+
After a decision has been made for each class independently, the consistency of the predictions with regard to the ChEBI hierarchy
135139
and disjointness axioms is checked. This is
136140
done in 3 steps:
137141
- (1) First, the hierarchy is corrected. For each pair of classes $A$ and $B$ where $A$ is a subclass of $B$ (following
@@ -145,4 +149,4 @@ both, we select one with the higher class score and set the other to 0.
145149
- (3) Since the second step might have introduced new inconsistencies into the hierarchy, we repeat the first step, but
146150
with a small change. For a pair of classes $A \subseteq B$ with predictions $1$ and $0$, instead of setting $B$ to $1$,
147151
we now set $A$ to $0$. This has the advantage that we cannot introduce new disjointness-inconsistencies and don't have
148-
to repeat step 2.
152+
to repeat step 2.

0 commit comments

Comments
 (0)