Skip to content

Commit 39587ee

Browse files
tmadlenerm-filaandresailer
authored
Introduce a new links category for the YAML definitions (#691)
* Make the reader and validator understand links Add the new category so that it can be read * Make code generation create necessary link code Effectively a header per link with a typedef and a common .cc file with registration code for all links * Make sure that SIO backend also works * Separate link registration macros * Remove special casing from roundtrip tests again * Make sure to make link collections available * Make sure that all user facing types exist Otherwise things start to break in cases where users explicitly use these types at the moment. * Add documentation for new capabilities and links * Update available templates documentation * Only generate SIOBlocks if necessary * Use walrus operator to improve readability --------- Co-authored-by: Mateusz Jakub Fila <37295697+m-fila@users.noreply.github.com> Co-authored-by: Andre Sailer <andre.philippe.sailer@cern.ch>
1 parent 497b1be commit 39587ee

26 files changed

+351
-92
lines changed

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ version and build that automatically. This behavior is controlled via the
3434
`USE_EXTERNAL_CATCH2` cmake variable. It defaults to `AUTO` but can also be set
3535
to `ON` or `OFF` to fully control the desired behavior.
3636

37-
### Python > 3.6
37+
### Python >= 3.8
3838

3939
Check your Python version by doing:
4040

‎doc/datamodel_syntax.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,38 @@ define which `Types` can be used with this interface class, in this case the
155155
not allow for mutable access to their data.** They can be used in relations
156156
between objects, just like normal `datatypes`.
157157

158+
## Definition of links
159+
Podio offers a templated `Link` class ([see here for more details](links.md))
160+
that allows one to link two arbitrary datatypes without having to introduce a
161+
`OneToOneRelation` or `OneToManyRelation` inside the corresponding datatypes. In
162+
order to keep the full definition of a datamodel in the YAML file it is possible
163+
to declare `links` in the YAML file:
164+
165+
```yaml
166+
links:
167+
ExampleLink:
168+
Description: "A link between two (podio generated) objects"
169+
Author: "It could be you"
170+
From: ExampleHit
171+
To: TypeWithEnergy
172+
```
173+
174+
This definition will yield the following typedefs
175+
```cpp
176+
using ExampleLinkCollection = podio::LinkCollection<ExampleHit, TypeWithEnergy>;
177+
178+
using ExampleLink = typename ExampleLinkCollection::value_type;
179+
// this is equivalent to
180+
// using ExampleLink = podio::Link<ExampleHit, TypeWithEnergy>;
181+
182+
using MutableExampleLink = typename ExampleLinkCollection::mutable_type;
183+
// this is equivalent to
184+
// using MutableExampleLink = podio::MutableLink<ExampleHit, TypeWithEnergy>;
185+
```
186+
187+
additionally, this will generate the necessary code to enable I/O for this link
188+
type.
189+
158190
### Assigning to interface types
159191

160192
Interface types support the same functionality as normal (immutable) datatypes.

‎doc/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Welcome to PODIO's documentation!
1515
examples.md
1616
frame.md
1717
userdata.md
18+
links.md
1819
storage_details.md
1920
cmake.md
2021
advanced_topics.md

‎doc/links.md‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -45,12 +45,16 @@ For a more detailed explanation of the internals and the actual implementation
4545
see [the implementation details](#implementation-details).
4646

4747
## How to use `Link`s
48-
Using `Link`s is quite simple. In line with other datatypes that are
49-
generated by podio all the functionality can be gained by including the
50-
corresponding `Collection` header. After that it is generally recommended to
51-
introduce a type alias for easier usage. **As a general rule `Links` need
52-
to be declared with the default (immutable) types.** Trying to instantiate them
53-
with `Mutable` types will result in a compilation error.
48+
Using `Link`s is quite simple. The most straight forward way is to simply
49+
declare them as part of the datamodel, [as described
50+
here](datamodel_syntax.md#definition-of-links). That will result in code
51+
generation that effectively does what is described below here. However, it's not
52+
strictly necessary to do that in case non-generated code is preferred. In line
53+
with other datatypes that are generated by podio all the functionality can be
54+
gained by including the corresponding `Collection` header. After that it is
55+
generally recommended to introduce a type alias for easier usage. **As a general
56+
rule `Links` need to be declared with the default (immutable) types.** Trying to
57+
instantiate them with `Mutable` types will result in a compilation error.
5458

5559
```cpp
5660
#include "podio/LinkCollection.h"

‎doc/templates.md‎

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -26,19 +26,22 @@ Note that some of the information below will only apply to either of these gener
2626
Currently PODIO loads templates that are placed in [`<prefix>/python/templates`](/python/templates).
2727
They are broadly split along the classes that are generated for each datatype or component from the EDM definition:
2828

29-
| template file(s) | content | generated file(s) |
30-
|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|
31-
| `Component.h.jinja2` | Definition for each component | `[<package>/]<component-name>.h` |
32-
| `Data.h.jinja2` | POD struct of each datatype (living in the POD layer) | `[<package>/]<datatype-name>Data.h` |
33-
| `Obj.{h,cc}.jinja2` | `Obj` class for each datatype (living in the object layer) and managing resources | `[<package>/]<datatype-name>Obj.h`, `src/<datatype-name>Obj.cc` |
34-
| `[Mutable]Object.{h,cc}.jinja2` | The user facing interfaces for each datatype (living in the user layer) | `[<package>/][Mutable]<datatype-name>.h`, `src/[Mutable]<datatype-name>.cc` |
35-
| `Collection.{h,cc}.jinja2` | The user facing collection interface (living in the user layer) | `[<package>/]<datatype-name>Collection.h`, `src/<datatype-name>Collection.cc` |
36-
| `CollectionData.{h,cc}.jinja2` | The classes managing the collection storage (not user facing!) | `[<package>/]<datatype-name>CollectionData.h`, `src/<datatype-name>CollectionData.cc` |
37-
| `datamodel.h.jinja2` | The *full datamodel header* that includes everything of a generated EDM (via including all generated `Collections`). | `[<package>]/<package>.h` |
38-
| `selection.xml.jinja2` | The `selection.xml` file that is necessary for generating a root dictionary for the generated datamodel | `src/selection.xml` |
39-
| `SIOBlock.{h,cc}.jinja2` | The SIO blocks that are necessary for the SIO backend | `[<package>/]<datatype-name>SIOBlock.h`, `src/<datatype-name>SIOBlock.cc` |
40-
| `MutableStruct.jl.jinja2` | The mutable struct definitions of components and datatypes for julia | `[<package>/]<datatype-name>Struct.jl`, `[<package>/]<component-name>Struct.jl` |
41-
| `ParentModule.jl.jinja2` | The constructor and collection definitions of components and datatypes in the data model are contained within a single module named after the package-name | `[<package>/]<package-name>.jl` |
29+
| template file(s) | content | generated file(s) |
30+
|------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|
31+
| `Component.h.jinja2` | Definition for each component | `[<package>/]<component-name>.h` |
32+
| `Data.h.jinja2` | POD struct of each datatype (living in the POD layer) | `[<package>/]<datatype-name>Data.h` |
33+
| `Obj.{h,cc}.jinja2` | `Obj` class for each datatype (living in the object layer) and managing resources | `[<package>/]<datatype-name>Obj.h`, `src/<datatype-name>Obj.cc` |
34+
| `[Mutable]Object.{h,cc}.jinja2` | The user facing interfaces for each datatype (living in the user layer) | `[<package>/][Mutable]<datatype-name>.h`, `src/[Mutable]<datatype-name>.cc` |
35+
| `Collection.{h,cc}.jinja2` | The user facing collection interface (living in the user layer) | `[<package>/]<datatype-name>Collection.h`, `src/<datatype-name>Collection.cc` |
36+
| `CollectionData.{h,cc}.jinja2` | The classes managing the collection storage (not user facing!) | `[<package>/]<datatype-name>CollectionData.h`, `src/<datatype-name>CollectionData.cc` |
37+
| `datamodel.h.jinja2` | The *full datamodel header* that includes everything of a generated EDM (via including all generated `Collections`). | `[<package>]/<package>.h` |
38+
| `selection.xml.jinja2` | The `selection.xml` file that is necessary for generating a root dictionary for the generated datamodel | `src/selection.xml` |
39+
| `SIOBlock.{h,cc}.jinja2` | The SIO blocks that are necessary for the SIO backend | `[<package>/]<datatype-name>SIOBlock.h`, `src/<datatype-name>SIOBlock.cc` |
40+
| `LinkCollection.h.jinja2` | The header that is generated for each *Link* containing effectively typedefs only | |
41+
| `DatamodelLinksSIOBlock.cc.jinja2` | The .cc file that is necessary for enabling SIO based I/O for *Link*s | |
42+
| `DatamodelLinks.cc.jinja2` | The global .cc file that is necessary to enable I/O for all *Link*s | |
43+
| `MutableStruct.jl.jinja2` | The mutable struct definitions of components and datatypes for julia | `[<package>/]<datatype-name>Struct.jl`, `[<package>/]<component-name>Struct.jl` |
44+
| `ParentModule.jl.jinja2` | The constructor and collection definitions of components and datatypes in the data model are contained within a single module named after the package-name | `[<package>/]<package-name>.jl` |
4245

4346

4447
The presence of a `[<package>]` subdirectory for the header files is controlled by the `includeSubfolder` option in the yaml definition file.

‎include/podio/LinkCollection.h‎

Lines changed: 7 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -8,30 +8,17 @@
88
#define PODIO_ENABLE_SIO 0
99
#endif
1010

11-
/// Macro for registering links at the CollectionBufferFactory by injecting the
12-
/// corresponding buffer creation function.
13-
#define PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT) \
11+
/// Main macro for declaring links. Takes care of registering the necessary
12+
/// buffer creation functionality with the CollectionBufferFactory.
13+
#define PODIO_DECLARE_LINK(FromT, ToT) \
1414
const static auto PODIO_PP_CONCAT(REGISTERED_LINK_, __COUNTER__) = \
1515
podio::detail::registerLinkCollection<FromT, ToT>(podio::detail::linkCollTypeName<FromT, ToT>());
1616

17-
/// Macro for registering the necessary SIOBlock for a Link with the SIOBlock factory
18-
#define PODIO_REGISTER_LINK_SIOFACTORY(FromT, ToT) \
19-
const static auto PODIO_PP_CONCAT(LINK_SIO_BLOCK_, __COUNTER__) = podio::LinkSIOBlock<FromT, ToT>{};
20-
2117
#if PODIO_ENABLE_SIO && __has_include("podio/detail/LinkSIOBlock.h")
22-
#include "podio/detail/LinkSIOBlock.h"
23-
/// Main macro for declaring links. Takes care of the following things:
24-
/// - Registering the necessary buffer creation functionality with the
25-
/// CollectionBufferFactory.
26-
/// - Registering the necessary SIOBlock with the SIOBlock factory
27-
#define PODIO_DECLARE_LINK(FromT, ToT) \
28-
PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT) \
29-
PODIO_REGISTER_LINK_SIOFACTORY(FromT, ToT)
30-
#else
31-
/// Main macro for declaring links. Takes care of the following things:
32-
/// - Registering the necessary buffer creation functionality with the
33-
/// CollectionBufferFactory.
34-
#define PODIO_DECLARE_LINK(FromT, ToT) PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT)
18+
#include <podio/detail/LinkSIOBlock.h>
19+
/// Macro for registering the necessary SIOBlock for a Link with the SIOBlock factory
20+
#define PODIO_DECLARE_LINK_SIO(FromT, ToT) \
21+
const static auto PODIO_PP_CONCAT(LINK_SIO_BLOCK_, __COUNTER__) = podio::LinkSIOBlock<FromT, ToT>{};
3522
#endif
3623

3724
#endif // PODIO_LINKCOLLECTION_H

‎python/podio_gen/cpp_generator.py‎

Lines changed: 38 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,14 +91,16 @@ def pre_process(self):
9191

9292
return {}
9393

94-
def post_process(self, _):
94+
def post_process(self, datamodel):
9595
"""Do the cpp specific post processing"""
9696
self._write_edm_def_file()
9797

9898
if "ROOT" in self.io_handlers:
9999
self._prepare_iorules()
100100
self._create_selection_xml()
101101

102+
if the_links := datamodel["links"]:
103+
self._write_links_registration_file(the_links)
102104
self._write_all_collections_header()
103105
self._write_cmake_lists_file()
104106

@@ -207,6 +209,23 @@ def do_process_interface(self, _, interface):
207209
self._fill_templates("Interface", interface)
208210
return interface
209211

212+
def do_process_link(self, _, link):
213+
"""Process a link definition and generate the necessary code"""
214+
link["include_types"] = []
215+
for rel in ("From", "To"):
216+
rel_type = link[rel]
217+
include_header = f"{rel_type.bare_type}Collection"
218+
if self._is_interface(rel_type.full_type):
219+
# Interfaces do not have a Collection header
220+
include_header = rel_type.bare_type
221+
link["include_types"].append(
222+
self._build_include_for_class(
223+
include_header, self._needs_include(rel_type.full_type)
224+
)
225+
)
226+
self._fill_templates("LinkCollection", link)
227+
return link
228+
210229
def print_report(self):
211230
"""Print a summary report about the generated code"""
212231
if not self.verbose:
@@ -506,8 +525,10 @@ def _write_list(name, target_folder, files, comment):
506525

507526
def _write_all_collections_header(self):
508527
"""Write a header file that includes all collection headers"""
509-
510-
collection_files = (x.split("::")[-1] + "Collection.h" for x in self.datamodel.datatypes)
528+
collection_files = (
529+
x.split("::")[-1] + "Collection.h"
530+
for x in list(self.datamodel.datatypes.keys()) + list(self.datamodel.links.keys())
531+
)
511532
self._write_file(
512533
os.path.join(self.install_dir, self.package_name, f"{self.package_name}.h"),
513534
self._eval_template(
@@ -520,6 +541,20 @@ def _write_all_collections_header(self):
520541
),
521542
)
522543

544+
def _write_links_registration_file(self, links):
545+
"""Write a .cc file that registers all the link collections that were
546+
defined with this datamodel"""
547+
link_data = {"links": links, "incfolder": self.incfolder}
548+
self._write_file(
549+
"DatamodelLinks.cc",
550+
self._eval_template("DatamodelLinks.cc.jinja2", link_data),
551+
)
552+
if "SIO" in self.io_handlers:
553+
self._write_file(
554+
"DatamodelLinkSIOBlock.cc",
555+
self._eval_template("DatamodelLinksSIOBlock.cc.jinja2", link_data),
556+
)
557+
523558
def _write_edm_def_file(self):
524559
"""Write the edm definition to a compile time string"""
525560
model_encoder = DataModelJSONEncoder()

0 commit comments

Comments
 (0)