Pydantic by ax3l · Pull Request #130 · picmi-standard/picmi

ax3l · 2025-11-01T04:34:41Z

Use a declarative approach to PICMI. Ensure construction & assignment to parameters both trigger validations. Ensure easier extensibility in downstream codes. Enable serialization, so people can easily query their PICMI options, e.g., in post-processing. Close #94

vibe code a draft, iterate and amend
ensure docs build well: old & new
confirm it works with WarpX, doc extensions work
confirm it works with FBPIC
confirm it works with UCLA team (QuickPIC, QPAD?)
confirm it works with PIConGPU team

Prompt + 5-10 follow-ups: > PICMI, which is used as a standard interface in WarpX, is very hard to extend, maintain and develop. > > I would like you to use modern Python 3.10+ and Pydantic 2+ to redesign it, focusing on typing support for checks and very easy extensibility for extra attributes in downstream codes like WarpX. > > First, read all of PICMI in terms of docs and implementation. > > Then, working one class at a time refactor it for using pydantic. > > Make sure the user-facing API stays exactly the same in terms of named classes, attributes per class, and docstrings. > > Godspeed.

ax3l · 2025-11-01T06:29:27Z

Docs/source/standard/interactions.rst

@@ -0,0 +1,13 @@
+Interactions


Moved a bit around, see #131

ax3l · 2025-11-01T06:29:33Z

Docs/source/standard/standard.rst

   constants
   applied_field
   diagnostics
+   interactions


Moved a bit around, see #131

ax3l · 2025-11-01T06:36:37Z

PICMI_Python/base.py



-class _DocumentedMetaClass(type):
+class _DocumentedMetaClass(ModelMetaclass):


I think we should be able to remove all the logic we needed to extend the documentation strings of classes:
Now, all our docs are in the individual fields of a class and the general string of a PICMI class should be able to stay the same in downstream code.

ax3l · 2025-11-01T06:37:41Z

PICMI_Python/base.py

+    Base class for all PICMI classes using Pydantic for validation and extensibility.
+
+    This class allows code-specific extensions (e.g., warpx_* kwargs) via Pydantic's
+    extra fields mechanism while maintaining type safety for standard attributes.
+    """


Not sure if all this mechanism is still needed. I would simply derive in downstream code from a PICMI class and extend it with more fields...

ax3l · 2025-11-01T06:38:56Z

PICMI_Python/diagnostics.py

+from .particles import PICMI_Species, PICMI_MultiSpecies
+
+# Type aliases using Python 3.10+ union syntax
+PICMI_Grid = (


Unused imports from refactoring.

We need to install pre-commits & ruff in PICMI to clean this repo up.

These type aliases are actually very useful to have. We have them in our implementation as well. But why would you have PICMI_Grid in diagnostics.py?

ax3l · 2025-11-01T06:40:41Z

PICMI_Python/particles.py

+        # Extract user-defined keywords from extra fields
+        if self.model_extra:
+            self.user_defined_kw = {}
+            keys_to_remove = []
+
+            # Check density expression
+            for k, v in self.model_extra.items():
+                if re.search(rf'\b{k}\b', self.density_expression):
+                    self.user_defined_kw[k] = v
+                    keys_to_remove.append(k)
+                # Check momentum expressions
+                elif self.momentum_expressions[0] is not None and re.search(rf'\b{k}\b', self.momentum_expressions[0]):
+                    self.user_defined_kw[k] = v


See above. Don´t think this is still needed, we can derive in downstream codes and add new fields with their own validators/types.

ax3l · 2025-11-01T06:41:04Z

PICMI_Python/particles.py

+        # Validate lengths
+        if not (lenx == maxlen or lenx == 1):
+            raise ValueError("Length of x doesn't match len of others")
+        if not (leny == maxlen or leny == 1):
+            raise ValueError("Length of y doesn't match len of others")
+        if not (lenz == maxlen or lenz == 1):
+            raise ValueError("Length of z doesn't match len of others")
+        if not (lenux == maxlen or lenux == 1):
+            raise ValueError("Length of ux doesn't match len of others")


Remove this -- can be checked by type.

chillenzer

Hi,

Hope you're all having a splendid start into 2026!

I'm the new lead developer on PIConGPU's PICMI support. PIConGPU's currently investing quite a bit into its PICMI support and the general direction of this PR suits us very well! We'd be interested in collaborating more closely on the evolution of PICMI, particularly as we have some capacity to do so at the moment. Would you care for a more detailed chat on this via VC?

My two pennies worth of comments you'll find below (just skimmed through, not a full review by far!). On the matter of tearing down infrastructure like _DocumentedMetaClass and _ClassWithInit, I'd agree that this is likely something that Pydantic can easily do for us and that removing them would make implementing PICMI for a particular code more "pythonic" and less based on custom conventions.

Hoping to hear from you soon!

Best,
Julian

chillenzer · 2026-01-09T12:19:50Z

PICMI_Python/base.py


-codename = None
+from pydantic import BaseModel, ConfigDict, Field, PrivateAttr, model_validator
+from pydantic._internal._model_construction import ModelMetaclass


This looks dangerous!

chillenzer · 2026-01-09T12:25:52Z

PICMI_Python/diagnostics.py

+from .particles import PICMI_Species, PICMI_MultiSpecies
+
+# Type aliases using Python 3.10+ union syntax
+PICMI_Grid = (


These type aliases are actually very useful to have. We have them in our implementation as well. But why would you have PICMI_Grid in diagnostics.py?

chillenzer · 2026-01-09T12:26:57Z

PICMI_Python/diagnostics.py

-        self.name = name
-
-        self.handle_init(kw)
+    grid: "PICMI_Cartesian1DGrid | PICMI_Cartesian2DGrid | PICMI_Cartesian3DGrid | PICMI_CylindricalGrid" = Field(description="Grid object for the diagnostic")


This is probably where PICMI_Grid was supposed to be used. More instances below.

chillenzer · 2026-01-09T12:31:45Z

PICMI_Python/fields.py

+PICMI_Grid = (
+    "PICMI_Cartesian1DGrid | PICMI_Cartesian2DGrid | "
+    "PICMI_Cartesian3DGrid | PICMI_CylindricalGrid"
+)


I'm typically inclined to arrange things to avoid forward references. What's the argument for not starting with the grids and defining this alias after all grids are defined?

chillenzer · 2026-01-09T12:33:03Z

PICMI_Python/fields.py

-        self.handle_init(kw)
+    methods_list: list[str] = ['FFT', 'Multigrid']
+
+    grid: "PICMI_Cartesian1DGrid | PICMI_Cartesian2DGrid | PICMI_Cartesian3DGrid | PICMI_CylindricalGrid" = Field(description="Grid instance. Grid object for the diagnostic")


We'd probably want to use the alias defined above.

chillenzer · 2026-01-09T12:44:09Z

PICMI_Python/simulation.py

+from .interactions import PICMI_FieldIonization
+
+# Type unions for PICMI objects using Python 3.10+ union syntax
+# (Note: Not used in type hints anymore, but kept for backwards compatibility)


ax3l · 2026-01-09T17:58:34Z

Hi @chillenzer ,

Happy New Year!

I am out of office today, but happy to meet in the coming week(s)!

I only marked it in the PR commit so far, but this draft is just an initial vibe code without a lot of manual improvements yet. So don't spend too much time reviewing it just yet pls :)

Yes, let us coordinate and improve the in implementation to be more modern and pedantic!

ax3l · 2026-01-28T19:20:04Z

@chillenzer do you like to take this branch, pull it in your fork, open a new PR from it?
That way we both can push to it.

ax3l · 2026-02-18T07:05:47Z

Continued in #133

ax3l · 2026-02-18T07:07:21Z

@chillenzer oh wait, are you sure #133 was built on this branch? #133 has way less lines changed so far.

chillenzer · 2026-02-18T13:13:56Z

Your observation is correct: #133 starts from scratch and copies and adjusts snippets from #130 that I can verify with the PIConGPU test suite. So, eventually #133 will take precedence but we're far from there yet.

ax3l added the enhancement New feature or request label Nov 1, 2025

ax3l force-pushed the topic-pydantic branch 10 times, most recently from b045351 to 7be5b0e Compare November 1, 2025 06:06

ax3l force-pushed the topic-pydantic branch from 7be5b0e to 1c1281c Compare November 1, 2025 06:17

ax3l commented Nov 1, 2025

View reviewed changes

Docs/source/standard/interactions.rst

@@ -0,0 +1,13 @@

Interactions

Copy link

Member Author

ax3l Nov 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved a bit around, see #131

ax3l commented Nov 1, 2025

View reviewed changes

Docs/source/standard/standard.rst

constants

applied_field

diagnostics

interactions

Copy link

Member Author

ax3l Nov 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved a bit around, see #131

ax3l commented Nov 1, 2025

View reviewed changes

chillenzer reviewed Jan 9, 2026

View reviewed changes

ax3l mentioned this pull request Jan 28, 2026

Roadmap #132

Open

6 tasks

chillenzer mentioned this pull request Feb 17, 2026

WIP: Pydantic refactoring #133

Draft

ax3l closed this Feb 18, 2026

ax3l reopened this Feb 18, 2026



		class _DocumentedMetaClass(type):
		class _DocumentedMetaClass(ModelMetaclass):

Conversation

ax3l commented Nov 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ax3l Nov 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chillenzer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chillenzer Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ax3l commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ax3l commented Jan 28, 2026

Uh oh!

ax3l commented Feb 18, 2026

Uh oh!

ax3l commented Feb 18, 2026

Uh oh!

chillenzer commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

ax3l commented Nov 1, 2025 •

edited

Loading

ax3l Nov 1, 2025 •

edited

Loading

chillenzer Jan 9, 2026 •

edited

Loading

ax3l commented Jan 9, 2026 •

edited

Loading

chillenzer commented Feb 18, 2026 •

edited

Loading