Bug Report - TypeError in fmt_numeric when cell is missing #1723
Description
Current Behaviour
💥 TypeError in fmt_numeric: Unsupported format string passed to NoneType.format when value is None
:
I encountered a TypeError when running ydata-profiling on a dataset that contains missing (NULL) values in numeric columns. It seems that fmt_numeric() attempts to format None as a number, which leads to an error.
This issue occurs when ydata-profiling tries to process datasets with missing numerical values.
It would be great if the library could handle None values more gracefully.
Expected Behaviour
If value is None, the function should skip formatting and return None or "N/A" instead of attempting to format None.
Proposed Fix:
Modify fmt_numeric() in formatters.py to handle None values safely.
Or, the branch logic before that may need to be checked.
def fmt_numeric(value, precision=3):
if value is None:
return None # Or return "N/A"
return f"{{:.{precision}g}}".format(value)
This ensures that missing values are not formatted as numbers, preventing the TypeError.
Data Description
when a numeric/timestamp column has any missing cell in Spark DataFrame (pd.timedelta X)
Code that reproduces the bug
# Sample 10%
df = spark.sql(
"select * from @@@@.@@@@ where rand() < 0.1"
).cache()
# type casting 1
df_casted = df.select(
[
(
col(field.name).cast("string").alias(field.name)
if isinstance(field.dataType, (DateType, TimestampType))
else col(field.name)
)
for field in df.schema
]
)
# type casting 2
complex_columns = [
field.name
for field in df.schema.fields
if isinstance(field.dataType, (ArrayType, MapType, StructType))
]
for col_name in complex_columns:
df_casted = df_casted.withColumn(col_name, to_json(col(col_name)))
profile = ProfileReport(df_casted, title=app_name, explorative=True)
profile.to_file(f"/tmp/ydata.html")pandas-profiling version
v2.2.3
Dependencies
dependencies:
- bzip2=1.0.8
- ca-certificates=2025.1.31
- conda-pack=0.8.1
- libffi=3.4.2
- liblzma=5.6.4
- libsqlite=3.49.1
- libzlib=1.3.1
- ncurses=6.5
- openssl=3.4.1
- pip=25.0.1
- pyspark=3.5.3
- python=3.9.21
- readline=8.2
- setuptools=75.8.2
- tk=8.6.13
- wheel=0.45.1
- pip:
- executing==2.2.0
- fastjsonschema==2.21.1
- great-expectations==0.18.22
- jupyter-events==0.12.0
- notebook-shim==0.2.4
- pandocfilters==1.5.1
- phik==0.12.4
- pydantic-core==2.27.2
- python-json-logger==3.2.1
- ruamel-yaml-clib==0.2.12
- soupsieve==2.6
- stack-data==0.6.3
- tzdata==2025.1
- ydata-profiling==4.12.2
OS
macos
Checklist
- There is not yet another bug report for this issue in the issue tracker
- The problem is reproducible from this bug report. This guide can help to craft a minimal bug report.
- The issue has not been resolved by the entries listed under Common Issues.