Skip to content

Commit ef080e7

Browse files
Update summaries.md
1 parent a8593a9 commit ef080e7

File tree

1 file changed

+15
-4
lines changed

1 file changed

+15
-4
lines changed

notes/summaries.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,13 @@ date: "November 07 2025"
55
previewImg: "/images/blog/sci3.jpg"
66
---
77
<p align="center">
8-
<img src="/images/blog/sci5.png" alt="LLM-as-a-Judge scores chart" width="800">
8+
<img src="/images/blog/sci5.png"
9+
alt="LLM-as-a-Judge scores chart"
10+
style="width:90%; height:auto;">
911
</p>
1012

1113

14+
1215
## Abstract
1316

1417
We present a comprehensive approach to democratizing access to scientific knowledge through large-scale, **structured summarization** of academic literature. We retrieved and processed ~**100 million** research papers from the public internet, leveraging existing datasets from **bethgelab**, **PeS2o**, **Hugging Face**, and **Common Pile**. We designed a standardized **JSON schema** for scientific paper summaries and **post-trained two models****Qwen 3 14B** and **Nemotron 12B**—to produce summaries in this format. Our evaluation combines **LLM-as-a-Judge** and a **QA dataset**. Fine-tuned models achieve performance on our evals comparable to leading closed models (e.g., GPT-5, Claude 4.5). **Nemotron 12B** offers ~**2.25×** higher throughput than Qwen 3 14B, making it attractive for large-scale processing.
@@ -17,10 +20,13 @@ With this preliminary blog post, we **release a fine-tuned models, 100k paper su
1720
A live **visualization tool** at [https://laion.inference.net/](https://laion.inference.net/) demonstrates the utility of structured summaries. We plan to release structured summaries for the full **100M** paper corpus.
1821

1922
<p align="center">
20-
<img src="/images/blog/sci4.png" alt="LLM-as-a-Judge scores chart" width="800">
23+
<img src="/images/blog/sci4.png"
24+
alt="LLM-as-a-Judge scores chart"
25+
style="width:90%; height:auto;">
2126
</p>
2227

2328

29+
2430
## Introduction
2531

2632
Access to scientific knowledge remains constrained by paywalls, licensing, and copyright, slowing research and education. Our **Project Alexandria** ([arXiv:2502.19413](https://arxiv.org/abs/2502.19413)) showed that it is legally and technically feasible to **extract factual knowledge** while respecting copyright via **Knowledge Units**—structured, style-agnostic representations of content. However, research-paper corpora vary in format and structure, making it hard to compare similar claims or retrieve knowledge efficiently. Building on Alexandria, we introduce a **pipeline** to collect, process, and summarize papers into **structured outputs** consumable by humans and AI systems alike. Our aims: * **Create** a massive, openly accessible, well-structured summary dataset of scientific literature * **Develop** models capable of generating **structured, factual** summaries * **Demonstrate** the utility of these summaries for scientific tasks * **Explore** decentralized computing to process at global scale This brief outlines **methodology**, **results**, and **implications** for the scientific community—and humanity.
@@ -65,7 +71,10 @@ We used **two complementary approaches**: 1. **LLM-as-a-Judge** — Ensemble of
6571
### 3.1 LLM-as-a-Judge
6672

6773
<p align="center">
68-
<img src="/images/blog/sci2.jpg" alt="LLM-as-a-Judge scores chart" width="600">
74+
<img src="/images/blog/sci2.jpg"
75+
alt="LLM-as-a-Judge scores chart"
76+
style="width:90%; height:auto;">
77+
6978

7079

7180
| Model | Score (1–5) |
@@ -86,7 +95,9 @@ We used **two complementary approaches**: 1. **LLM-as-a-Judge** — Ensemble of
8695
### 3.2 QA Accuracy
8796

8897
<p align="center">
89-
<img src="/images/blog/sci.jpg" alt="QA evaluation accuracy chart" width="600">
98+
<img src="/images/blog/sci.jpg"
99+
alt="LLM-as-a-Judge scores chart"
100+
style="width:90%; height:auto;">
90101

91102

92103
| Model | Accuracy (%) |

0 commit comments

Comments
 (0)