You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/concepts/metrics/context_utilization.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ Context utilization is like a reference free version of [context_precision](cont
4
4
5
5
6
6
```{math}
7
-
\text{Context Precision@K} = \frac{\sum_{k=1}^{K} \left( \text{Precision@k} \times v_k \right)}{\text{Total number of relevant items in the top } K \text{ results}}
7
+
\text{Context Utilization@K} = \frac{\sum_{k=1}^{K} \left( \text{Precision@k} \times v_k \right)}{\text{Total number of relevant items in the top } K \text{ results}}
8
8
````
9
9
10
10
```{math}
@@ -18,9 +18,9 @@ Where $K$ is the total number of chunks in `contexts` and $v_k \in \{0, 1\}$ is
18
18
Question: Where is France and what is it's capital?
19
19
Answer: France is in Western Europe and its capital is Paris.
20
20
21
-
High context precision: ["France, in Western Europe, encompasses medieval cities, alpine villages and Mediterranean beaches. Paris, its capital, is famed for its fashion houses, classical art museums including the Louvre and monuments like the Eiffel Tower", "The country is also renowned for its wines and sophisticated cuisine. Lascaux’s ancient cave drawings, Lyon’s Roman theater and the vast Palace of Versailles attest to its rich history."]
21
+
High context utilization: ["France, in Western Europe, encompasses medieval cities, alpine villages and Mediterranean beaches. Paris, its capital, is famed for its fashion houses, classical art museums including the Louvre and monuments like the Eiffel Tower", "The country is also renowned for its wines and sophisticated cuisine. Lascaux’s ancient cave drawings, Lyon’s Roman theater and the vast Palace of Versailles attest to its rich history."]
22
22
23
-
Low context precision: ["The country is also renowned for its wines and sophisticated cuisine. Lascaux’s ancient cave drawings, Lyon’s Roman theater and", "France, in Western Europe, encompasses medieval cities, alpine villages and Mediterranean beaches. Paris, its capital, is famed for its fashion houses, classical art museums including the Louvre and monuments like the Eiffel Tower",]
23
+
Low context utilization: ["The country is also renowned for its wines and sophisticated cuisine. Lascaux’s ancient cave drawings, Lyon’s Roman theater and", "France, in Western Europe, encompasses medieval cities, alpine villages and Mediterranean beaches. Paris, its capital, is famed for its fashion houses, classical art museums including the Louvre and monuments like the Eiffel Tower",]
24
24
```
25
25
26
26
## Example
@@ -44,7 +44,7 @@ score.to_pandas()
44
44
45
45
## Calculation
46
46
47
-
Let's examine how context utilization was calculated using the low context precision example:
47
+
Let's examine how context utilization was calculated using the low context utilization example:
48
48
49
49
**Step 1**: For each chunk in retrieved context, check if it is relevant or not relevant to arrive at the answer for the given question.
50
50
@@ -58,8 +58,8 @@ Let's examine how context utilization was calculated using the low context preci
0 commit comments