Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
2c3d809
created framework for new docs
TheAssembler1 Jun 26, 2025
3aa3d34
worked on getting started example
TheAssembler1 Jun 26, 2025
3e3b90c
section numbers
TheAssembler1 Jun 26, 2025
a6b6675
update compiliation and running PDC
TheAssembler1 Jun 26, 2025
0bfe61f
started working on hdf5 vol and pdcpy docs
TheAssembler1 Jun 28, 2025
84575b6
updated introdution with index for installation types
TheAssembler1 Jul 9, 2025
eda4f06
worked on first program examples and installation
TheAssembler1 Jul 16, 2025
492b1f3
worked on first program examples and installation
TheAssembler1 Jul 16, 2025
36640a4
worked on first program examples and installation
TheAssembler1 Jul 16, 2025
3c55cac
message
avimaloo1 Jul 25, 2025
e495a65
Merge branch 'documentation_rework' of https://github.com/hpc-io/pdc …
avimaloo1 Jul 30, 2025
8bed151
checkpoint
TheAssembler1 Jul 30, 2025
2de5262
Merge branch 'documentation_rework' of https://github.com/hpc-io/pdc …
avimaloo1 Jul 30, 2025
4bcba8b
message
avimaloo1 Jul 30, 2025
a39197c
rename services -> using PDC
TheAssembler1 Aug 5, 2025
f2d1249
using PDC update
TheAssembler1 Aug 6, 2025
edd7541
started working on client API
TheAssembler1 Aug 13, 2025
1abb4b2
update C documentation so breathe can extract more docs
TheAssembler1 Aug 13, 2025
caeaf2a
Committing clang-format changes
github-actions[bot] Aug 13, 2025
2a2ab4b
more type documentation
TheAssembler1 Aug 13, 2025
ea29869
Merge branch 'documentation_rework' of https://github.com/hpc-io/pdc …
TheAssembler1 Aug 13, 2025
76b1d82
Add more API functions
TheAssembler1 Aug 13, 2025
be61033
update tools
TheAssembler1 Aug 13, 2025
42c3718
merge with develop
TheAssembler1 Sep 16, 2025
1748938
checkpoint
TheAssembler1 Sep 18, 2025
7f361f1
checkpoint
TheAssembler1 Oct 22, 2025
27009a1
checkpoint
TheAssembler1 Nov 13, 2025
c52fb80
checkpoint
TheAssembler1 Nov 13, 2025
fe73a9b
test
TheAssembler1 Nov 13, 2025
c850894
Committing clang-format changes
github-actions[bot] Nov 13, 2025
514e751
remove border
TheAssembler1 Nov 13, 2025
adebf87
update section numbers
TheAssembler1 Nov 13, 2025
f1f79aa
Merge branch 'develop' into documentation_rework
jeanbez Nov 14, 2025
f5972e4
checkpoint
TheAssembler1 Nov 14, 2025
64635a7
Merge branch 'documentation_rework' of https://github.com/hpc-io/pdc …
TheAssembler1 Nov 14, 2025
0ec74df
Switch core concepts 2.2 with 2.1
TheAssembler1 Nov 14, 2025
c9b5088
Merge branch 'develop' into documentation_rework
TheAssembler1 Nov 19, 2025
25071f5
description of installation targets
TheAssembler1 Nov 19, 2025
11ec149
add versions for dependencies
TheAssembler1 Nov 19, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 36 additions & 10 deletions docs/source/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,10 @@ Suren Byna, Bin Dong, Houjun Tang, Quincey Koziol, Jingqing Mu, Jerome Soumagne,
**1.2.** Installation
---------------------

PDC offers the following methods for installing core dependencies:

1. :ref:`Spack <link_spack>`
2. :ref:`PDC Source <pdc-source>`

PDC offers the following installation targets:
PDC offers several installation targets,
including C and Python APIs, as well as support
for use as an HDF5 VOL connector. These options are listed below,
and the accompanying figure illustrates the installation path for each:

1. `C API`
2. :ref:`Python API (PDCpy) <python-api-pdc-py>`
Expand All @@ -57,7 +55,13 @@ PDC offers the following installation targets:
:alt: PDC Installation Diagram
:align: center

Installation workflow to install the client targets offered by PDC.
Installation workflow to install the client targets offered by PDC.

As seen in the figure above,
PDC offers the following methods for installing core dependencies:

1. :ref:`Spack <link_spack>`
2. :ref:`PDC Source <pdc-source>`

.. note::

Expand Down Expand Up @@ -102,12 +106,35 @@ PDC and its dependencies can be installed with spack:
PDC Source
~~~~~~~~~~

We recommend using GCC version 7 or later. Intel and Cray compilers also work.

When building PDC from source, either MPICH or OpenMPI can be used as the MPI library, if your system
doesn't have one installed, follow `MPICH Installers Guide <https://www.mpich.org/documentation/guides>`_
or `Installing Open MPI <https://docs.open-mpi.org/en/v5.0.x/installing-open-mpi/quickstart.html>`_

We list the required compilers, MPI implementations, supporting libraries,
and optional dependencies along with their recommended versions below:

* **Compilers** (one of the following):

- GCC version 7 or later.
- Intel compiler (version stable)
- Cray compiler (version stable)

* **MPI implementations**:

- OpenMPI (version stable)
- MPICH (version stable)
- Intel MPI or Cray MPI (version stable)

* **Supporting libraries**:

- libfabric (version 1.18.0)
- Mercury (version 2.2.0)

* **Optional dependencies**:

- Python (required if using the PDC Python API) — version 3.x
- HDF5 (required if using the PDC HDF5 API) — version 1.12.1

We provide detailed instructions for installing libfabric, Mercury, and the PDC library below.

.. attention::
Expand Down Expand Up @@ -596,7 +623,6 @@ This section offers the following examples for different PDC target installation
.. note::

All examples omit detailed error checking for clarity. In practice, always check the return values of PDC API calls.
See the section TODO_FIX_REFERENCE for more information on detecting and handling PDC errors.

.. _c-api-first-program:

Expand Down
22 changes: 8 additions & 14 deletions docs/source/roadmap.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,32 +22,26 @@ vision for PDC.
- Expand documentation and example workflows for common HPC use cases.

- **Interoperability**
- Strengthen integration with HDF5, ADIOS, and other I/O frameworks.
- Improve support for various network transports through Mercury (``ofi``, ``tcp``, ``verbs``, etc.).
- Strengthen integration with HDF5 and other I/O frameworks.

**7.3.** Short-Term Goals (Next 6-12 Months)
--------------------------------------------

- Implement enhanced **data caching and eviction policies** to reduce I/O latency.
- Improve **burst buffer management** for hybrid memory/storage architectures.
- Extend **metadata indexing** and search capabilities.
- Add more comprehensive **unit and integration tests** in CI pipelines.
- Expand **PDC client APIs** in C and Python with better documentation and examples.
- Implement enhanced client-side data caching and eviction
policies to automatically exchange data from other MPI ranks and from multiple nodes.

**7.4.** Medium-Term Goals (1-2 Years)
--------------------------------------

- Introduce **multi-tier data management** (memory, SSD, disk, object storage).
- Develop **asynchronous data movement and prefetching** mechanisms.
- Enhance **seurity and authentication** (e.g., Cray DRC, token-based access).
- Support **federated PDC deployments** across distributed sites.
- Integrate data movement between multiple HPC systems and between HPC and Cloud object storage systems.
- Compound data type support for regions.

**7.5.** Long-Term Vision (Beyond 2 Years)
------------------------------------------

- Enable **self-optimizing data placement** based on access patterns and system telemetry.
- Integrate with **workflow and data provenance frameworks** for end-to-end data lifecycle management.
- Expand **AI-driven data management features**, such as automated cache tuning and prediction-based prefetching.
- Integrate with workflow and data provenance frameworks for end-to-end data lifecycle management.
- Expand AI-driven data management features, such as automated cache tuning and prediction-based prefetching.
- Achieve production-grade stability and adoption across DOE and large HPC facilities.
- Integrate PDC in DOE applications and workflows.

*Last updated: November 2025*
Loading