The Bio-OS TES (Task Execution Service) is a cloud-native, GA4GH-compliant task execution framework designed to support large-scale biomedical data analysis and workflow execution. It is implemented based on TESK, serving as a TES Server implementation that operates seamlessly across multiple Chinese cloud service providers (CSPs) such as Volcano Engine and other Kubernetes-based infrastructures.
The system is composed of four interoperable microservices, each is followed by its GitHub repository:
| Service | Repository | Description |
|---|---|---|
| API Server | GBA-BI/tes-api | The entry point of the TES ecosystem. It exposes the GA4GH TES RESTful API, receives user task submissions, manages lifecycle states (QUEUED, RUNNING, COMPLETE, etc.), and coordinates scheduling and execution across clusters. |
| Scheduler | GBA-BI/tes-scheduler | The global scheduler that allocates queued tasks to available Kubernetes clusters. It evaluates task requirements, quotas, and priorities to achieve balanced and efficient resource usage. |
| Kubernetes Agent | GBA-BI/tes-k8s-agent | A lightweight agent deployed in each Kubernetes cluster. It receives scheduled tasks from the API server, launches them as Kubernetes Jobs/Pods, monitors execution, and reports task status and logs back. |
| Filer | GBA-BI/tes-filer | The data I/O service that manages input and output files for TES tasks. It supports multiple storage backends (local, NFS, S3, etc.) and ensures efficient, secure, and standardized data movement before and after task execution. |
Users interact only with the TES API endpoint, while task scheduling, execution, and data movement are transparently handled by the underlying system.
Bio-OS TES is designed as a modular, scalable, and GA4GH-compliant execution framework, enabling interoperability across institutions, infrastructures, and cloud vendors.
Standardization – All APIs aim to confrom the GA4GH TES specification, allowing seamless integration with workflow engines such as Cromwell, Nextflow, and Toil.
Cloud-native architecture – Each service runs as an independent containerized microservice within Kubernetes, supporting elastic scaling and fault isolation.
Separation of concerns – The four repositories correspond to distinct functional layers: API orchestration, scheduling, execution, and file I/O, ensuring clarity and maintainability.
Interoperability – Through the unified TES interface, Bio-OS TES can integrate with GA4GH components such as WES (Workflow Execution Service) and DRS (Data Repository Service).
All TES components are Helm-installable microservices designed to run on Kubernetes clusters (version ≥ 1.20), a typical institutional deployment includes:
One unique TES API Server and TES Scheduler for orchestration.
One or more TES Agents distributed across computing clusters for task execution.
TES Filer is invoked on demand during task execution to handle data staging and result archiving.
For deployment, you can refer to the manifests directory to build and install these services in your own Kubernetes cluster using Helm.
This project is licensed under the Apache 2.0 License - see the LICENSE file for details.