📑Article Link | 🤗 Download Models | 🤗 Download Pre-extracted Embeddings | 📑 Cite
The official Repo for Paper accepted in NPJ Digital Medicine, 2025 PathOrchestra: A Comprehensive Foundation Model for Computational Pathology with Over 100 Diverse Clinical-Grade Tasks
- 7-07-2025: Features Released (ing)
- 3-31-2025: Article Online
- 3-02-2025: Model Weights (V1.0.0) Released
- 7-14-2024: Initial Release
| Model Name | Release Date | Model Architecture | Download Link |
|---|---|---|---|
| PathOrchestra_V1.0.0 | 03-2025 | ViT-l/16 | 🤗 Hugging Face |
To support downstream applications, we provide pre-extracted embeddings from PathOrchestra_V1.0.0, which are available for download on 🤗 Hugging Face.
First, clone the repository and navigate into the project directory:
git clone https://github.com/yanfang-research/PathOrchestra.git
cd PathOrchestraNext, create a Conda environment and install the required dependencies:
conda env create -f environment.yml
conda activate PathOrchestraTo access the model weights, please request permission via the Hugging Face model page using the links provided in the Model Weights. Note that you must be logged into your Hugging Face account to download the weights.
Following authentication (using huggingface_hub), the pretrained checkpoints and image transforms for PathOrchestra can be directly loaded using the timm library. This method automatically downloads the model weights to the huggingface_hub cache in your home directory, which timm will automatically find when using the commands below:
import timm
from huggingface_hub import login
# Authenticate with your User Access Token (https://huggingface.co/settings/tokens)
login(token=your_hf_token)
model = timm.create_model(
"hf-hub:AI4Pathology/PathOrchestra",
pretrained=True,
init_values=1e-5,
dynamic_img_size=True,
)
model.eval()You can use the pretrained encoder to extract features from pathology patches, as follows:
import torch
from PIL import Image
from torchvision import transforms
from huggingface_hub import hf_hub_download
# Define preprocessing transform
transform = transforms.Compose([
transforms.Resize(224),
transforms.ToTensor(),
transforms.Normalize(mean=(0.485, 0.456, 0.406),
std=(0.229, 0.224, 0.225))
])
image_path = hf_hub_download(repo_id="AI4Pathology/PathOrchestra", filename="example.png")
image = Image.open(image_path).convert("RGB")
image = transform(image).unsqueeze(0) # Add batch dimension
with torch.inference_mode():
features = model(image) # Extract patch-level embedding
print(feature_emb.shape)These pre-extracted features can be used for ROI classification (e.g., via linear probing), slide-level classification (e.g., using multiple instance learning), and various other machine learning applications.
You can also use PIANO (Pathology Image ANalysis Orchestrator), a user-friendly PyTorch library, to load and use PathOrchestra along with comprehensive tools for pathology image processing and analysis.
We are pleased to assist researchers in evaluating their models using our private datasets. For more details, please feel free to contact us at the provided email (yanfang@pjlab.org.cn).
The project was built on top of amazing repositories such as DINOv2, UNI, and Timm (ViT model implementation). We thank the authors and developers for their contribution.
If you find our work useful in your research or if you use parts of this code please consider citing our paper:
Yan, F., Wu, J., Li, J., Wang, W., Lu, J., Chen, W., ... & Wang, Z. (2025). Pathorchestra: A comprehensive foundation model for computational pathology with over 100 diverse clinical-grade tasks. arXiv preprint arXiv:2503.24345
@article{yan2025pathorchestra,
title={Pathorchestra: A comprehensive foundation model for computational pathology with over 100 diverse clinical-grade tasks},
author={Yan, Fang and Wu, Jianfeng and Li, Jiawen and Wang, Wei and Lu, Jiaxuan and Chen, Wen and Gao, Zizhao and Li, Jianan and Yan, Hong and Ma, Jiabo and others},
journal={arXiv preprint arXiv:2503.24345},
year={2025}
}