Skip to content

Commit 57fcb71

Browse files
committed
Refactor policy package doc into separate docs for clarity
1 parent 608056e commit 57fcb71

File tree

6 files changed

+314
-306
lines changed

6 files changed

+314
-306
lines changed

docs/operator/policy_packages.md

Lines changed: 0 additions & 306 deletions
This file was deleted.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"label": "Policy Packages",
3+
"collapsible": true
4+
}
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
id: policy-package-algorithms
3+
title: Developing Policy Package algorithms
4+
---
5+
6+
### Adding a new algorithm class
7+
8+
The system for registering algorithms within policy packages is
9+
intended to be extensible so that new algorithm classes can be added
10+
relatively easily. The basic workflow is as follows:
11+
12+
- The `get_algorithms` function within the policy package (see above)
13+
should return a dictionary of functions of the new class, indexed
14+
by name
15+
- The core Rucio code should maintain a dictionary of functions of the
16+
new class, ready to be called when required. The details of this
17+
will differ depending on what the new class actually does and how it
18+
integrates with the Rucio code, but typically the algorithm name to
19+
be used will be selected by a value in the config file, as for the
20+
current `lfn2pfn` and `non_deterministic_pfn` algorithm types.
21+
22+
### lfn2pfn vs. non_deterministic_pfn algorithms
23+
24+
`lfn2pfn` algorithms and `non_deterministic_pfn` algorithms are
25+
conceptually similar, but there are important differences between
26+
them. Both produce a physical filename for use on an RSE, however
27+
`lfn2pfn` algorithms can only be used on deterministic RSEs - for
28+
example, disk systems where the appropriate physical filename can be
29+
derived from the file's scope and name alone (as well as
30+
protocol-specific information for the RSE in question).
31+
`non_deterministic_pfn` algorithms are used on non-deterministic
32+
RSEs (most often tape systems), and may use additional information
33+
about the file (such as its metadata, any datasets that it is a part
34+
of, etc.) to construct the physical filename. Because files cannot
35+
be uploaded directly to non-deterministic storage,
36+
`non_deterministic_pfn` algorithms are only ever called for
37+
replications, but `lfn2pfn` algorithms can also be called for
38+
initial uploads.
39+
40+
The `lfn2pfn` algorithm to be used is determined by the
41+
`lfn2pfn_algorithm` attribute of the relevant RSE. If this is not set,
42+
the `lfn2pfn_algorithm_default` value from the `[policy]` section of
43+
the config file is used instead. The `non_deterministic_pfn` algorithm
44+
to be used is determined by the `naming_convention` attribute of the
45+
relevant RSE.

0 commit comments

Comments
 (0)