Skip to content

Commit 9b769b8

Browse files
gchalumpfacebook-github-bot
authored andcommitted
Add robust field filtering in TBEDataConfig.from_json (#5164)
Summary: X-link: facebookresearch/FBGEMM#2163 Add field validation and filtering to `TBEDataConfig.from_json` to handle JSON inputs, used in benchmark, that may contain unknown or extra fields. The current `tbe_reporter` has adhoc properties added to the output that mismatch with the current structure. This change makes JSON deserialization more robust by filtering out unrecognized fields and logging warnings for missing expected fields and info messages for ignored extra fields. This allows the code to gracefully handle evolving schema or data from external sources without breaking deserialization. Differential Revision: D87658217
1 parent 1bb480f commit 9b769b8

File tree

1 file changed

+15
-1
lines changed

1 file changed

+15
-1
lines changed

fbgemm_gpu/fbgemm_gpu/tbe/bench/tbe_data_config.py

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99

1010
import dataclasses
1111
import json
12+
import logging
1213
from typing import Any, Optional
1314

1415
import torch
@@ -78,7 +79,20 @@ def from_dict(cls, data: dict[str, Any]):
7879
@classmethod
7980
# pyre-ignore [3]
8081
def from_json(cls, data: str):
81-
return cls.from_dict(json.loads(data))
82+
raw = json.loads(data)
83+
allowed = {f.name for f in dataclasses.fields(cls)}
84+
filtered = {k: v for k, v in raw.items() if k in allowed}
85+
missing = allowed - set(filtered.keys())
86+
extra = set(raw.keys()) - allowed
87+
if missing:
88+
logging.warning(
89+
f"TBEDataConfig.from_json: Missing expected fields not loaded: {sorted(missing)}"
90+
)
91+
if extra:
92+
logging.info(
93+
f"TBEDataConfig.from_json: Ignored unknown fields from input: {sorted(extra)}"
94+
)
95+
return cls.from_dict(filtered)
8296

8397
def dict(self) -> dict[str, Any]:
8498
tmp = dataclasses.asdict(self)

0 commit comments

Comments
 (0)