Feedback and Questions Regarding Parsynthesizer

[Abreu6]

Hello,

As part of my master thesis on synthetic data generation for healthcare, we tested and evaluated SDV’s ParSynthesizer on different segments of a clinical dataset.

Our main conclusions and findings were:
-> The model is easy to configure and performs efficiently on smaller data samples.
-> However, the generated data showed some consistency and quality issues, particularly in preserving correlations between fields and capturing more subtle patterns from the original dataset
-> In particular, it struggled to preserve correlations between fields and missed subtle but important patterns present in the original data. One example we observed was the occurrence of diagnostic exams dated before the corresponding medical consultations
or the presence of medical consultations dated after the recorded date of death.
-> Additionally, due to its full in-memory processing design, we faced scalability issues when attempting to synthesize larger datasets, even after filtering and simplifying the input.

Could you please share with us any suggestions for improving results, such as advanced configurations, alternative preprocessing steps, or known limitations to consider, we would be happy to incorporate them into our testing.

Best regards.

[npatki]

Hi @Abreu6, nice to meet you. We appreciate you filing the feedback so that we improve our product. Are you able to share a bit more about which dataset(s) you tested for your thesis, and which quality metrics you were looking at? Without it, it's hard to know if we have the same undersatnding.

See our suggestions and comments below.

it struggled to preserve correlations between fields and missed subtle but important patterns present in the original data. One example we observed was the occurrence of diagnostic exams dated before the corresponding medical consultations

If I'm understanding correctly, this is a hard-and-fast rule that must be true of all sequences in the dataset: The diagnostic exam date must always occur before the medical consultation date (no exceptions).

All SDV synthesizers generate data probabilistically, meaning that they are not set up to learn deterministic (hard-and-fast) rules out-of-the-box. (Here is a blog article that dives into this concept.) So if you'd like a synthesizer to learn a rule to follow 100% of the time, we recommend using our constraints feature -- maybe something like the InequalityConstraint?

Are there other, probabilistic correlations you tested for (eg. a higher value in column X tends to correlate with a higher value in column Y)? Data pre-processing can definitely help with these.

we faced scalability issues when attempting to synthesize larger datasets, even after filtering and simplifying the input.

Yes, we've heard this feedback and we're tracking the request for optimizing performance in issue #1965. (Unfortunately, the team has not had a chance to prioritize this yet -- but every bit of feedback helps!)

I'm curious how large your datasets were? And how did you go about filtering/simplifying the inputs? Our recommended approach (for now, until we optimize the performance) would be to run on a sub-sample of sequences in your data; you can use the get_random_sequence_subset utility function that we provide.

[Abreu6]

Hi,
Thank you for the detailed response and clarification.
To provide further context on our setup:

• We tested ParSynthesizer on a longitudinal clinical dataset containing ~29.000 records and 26 attributes. The data represents patient episodes across various event types (consultations, exams, analyses, diagnoses, etc.), with each patient potentially having multiple temporal records.

Evaluation Metrics:

• We used SDMetrics’ suite, focusing on: Column Shapes Score, Column Pair Trends Score, Synthetic Quality Score (SQS), Correlation Similarity, Field Correlation and Distribution Stability.

Constraints:

• Initially, I encountered issues when trying to apply constraints involving context columns, as they are not supported. To work around this, I duplicated the relevant column outside the context configuration, but the generated results still failed to respect the defined constraint.

Figure: Configuration of a custom inequality constraint in the PARSynthesizer. The constraint enforces that `DTA_EPISODIO` (episode start date) must occur before or on `DTA_OBITO_DUP` (duplicated death date). Context columns such as `DTA_NASCIMENTO`, `GENERO`, `CONCELHO`, and `DTA_OBITO` guide the conditional generation process but cannot be used directly in constraints.

Scalability:

• We hit performance issues when exceeding past 30k rows, even after reducing dimensionality and sequence length. We’ll experiment with the get_random_sequence_subset() utility next, as suggested.

Best regards

[npatki]

Hi @Abreu6, thanks for the detailed explanation. Do let us know how your experiments work when using the get_random_sequence_subset utility.

As for adding the constraints:

Initially, I encountered issues when trying to apply constraints involving context columns, as they are not supported. To work around this, I duplicated the relevant column outside the context configuration, but the generated results still failed to respect the defined constraint.

Constraints should work on the PARSynthesizer as long as all the columns are either context columns or non-context columns. (See these docs.) In your case, it seems that you ultimately want to add an Inequality constraint for DTA_EPISODIO <= DTA_OBITO . However, DTA_OBITO is a context column and DTA_EPISODIO is a non-context column, so adding the Inequality constraint is not supported.

To resolve this issue, you could write a custom constraint, but merely duplicating the DTA_OBITO column (and making it non-context) will not solve the underlying issue -- as you would then just encounter issues with trying to reconcile the DTA_OBITO with its duplicated version.

The recommended approach would be transform the DTA_EPISODIO column instead. Instead of the date, the column would contain the difference between the death date and the episode date. Later after sampling synthetic data, you can recreate the DTA_EPISODIO date by adding up the DTA_OBITO and the difference column. Does that make sense?

This is a bit complicated for now due to the way the PAR algorithm and transformations work. Though our team does have a solution in-mind for this -- it just hasn't been prioritized yet since we haven't received any requests for it. Based on your feedback, I will file a new feature request for us to track. Thanks!

[Abreu6]

Hello, thanks for the clarification.

I went ahead and implemented the suggested workaround — transforming DTA_EPISODIO into the difference relative to DTA_OBITO and then reconstructing the episode dates after sampling. Unfortunately, I’m still observing records where episodes occur after death.

Is there anything else I could try to ensure that episodes after death are not generated?

Thanks again for the support!

[npatki]

Hi @Abreu6, it's a little hard to discuss since I don't know the code you're using. I have a completely working prototype that creates the correct data, 100% of the time using the suggested workaround.

Details

Just to make sure we're on the same page: The problem is that you have a context column (patient_death) that must always occur after a non-context column (visit_date).

In my workaround, I re-wrote the visit_date column to instead represent the number of days before the death date that the visit happened.

Now this is the data that I gave to PAR Synthesizer. PAR Synthesizer is designed to only create data within the range that it observes. This is what the synthetic data looked like:

Now, I could re-create the synthetic visit date by subtracting from the death date, thus ensuring that the visit date is always before the death date.

Does that make sense?

Next Steps

If you are still observing rows that do not satisfy the constraint, one of three things must be happening:

1. There could be an bug somewhere in your pre- or post-processing code
2. In your original data, it could be the case that the visit date is occurring after the death date for some reason (eg. the data was input into the system incorrectly). PARSynthesizer will only be able to generate the patterns that it observes, so this may be a good thing to sanity check.
3. There is some issue in the PARSynthesizer such that it's synthesizing negative values for the num_days_before_death column that is created.

Perhaps it would be good to sanity check that that the num_days_before_death column is always >0 in the real data, and >0 in the synthetic data.

Does that make sense?

Long Term

In the long term, I have filed feature #2637, which would allow you to just add the Inequality constraint for these columns.

[Abreu6]

Hello, it makes sense yes.

I re-implemented the workaround and re-checked both pre- and post-processing.
I transform visit_date into EPISODE_DIFF_DAYS = (death_or_last_episode − visit_date).days, train PAR on that (with context columns), and then reconstruct visit_date = anchor − EPISODE_DIFF_DAYS.

I also parse all DTA_* columns as day-first datetimes to avoid metadata mismatches (e.g., DTA_ANALISE), and I rebuilt the patient-level anchor in the synthetic data (death if present, else last visit_date).

This is the code I am running:

import pandas as pd
import numpy as np
from sdv.metadata import SingleTableMetadata
from sdv.sequential import PARSynthesizer

# ============== CONFIG ==============
INPUT_CSV = "input_dataset.csv"   # enter our real dataset
OUTPUT_CSV = "synthetic_data.csv"	# the output of the synthetic dataset

ID_COL    = "DOENTE_ID"
SEQ_DATE  = "DTA_EPISODIO"       # original episode date (will be transformed away)
ANCHOR_COL= "DTA_ANCHOR"         # patient-level anchor (death or last episode)
DIFF_COL  = "EPISODE_DIFF_DAYS"  # integer offset >= 0
SEQ_IDX   = "SEQ_IDX"            # numeric sequence index for ordering

# Context columns (patient-level)
CONTEXT_COLS = ["DTA_NASCIMENTO", "GENERO", "CONCELHO", "DTA_OBITO", ANCHOR_COL]

# Sequential columns (episode-level) to keep for modeling
SEQUENTIAL_KEEP = [DIFF_COL]  # we reconstruct DTA_EPISODIO later
# ====================================

# ---------- help function-------------.----
def parse_all_dta_columns(df: pd.DataFrame) -> list[str]:
    """Parse every column whose name starts with 'DTA_' as datetime (day-first).
    Returns the list of parsed columns."""
    dta_cols = [c for c in df.columns if str(c).upper().startswith("DTA_")]
    # some datasets have HR/only-time columns; keep them out if desired
    dta_cols = [c for c in dta_cols if c.upper() != "HR_ALTA"]
    for c in dta_cols:
        s = df[c].astype(str).str.strip()
        s = s.replace({"": np.nan, "nan": np.nan, "NaT": np.nan})
        df[c] = pd.to_datetime(s, errors="coerce", dayfirst=True)
    return dta_cols
# -----------------------------

# 1) Load
df = pd.read_csv(INPUT_CSV, sep=None, engine="python")

# 2) Parse ALL DTA_* dates (includes DTA_ANALISE, DTA_EXAME, etc.)
DATE_COLS = parse_all_dta_columns(df)

# Ensure required columns exist
missing = [col for col in [ID_COL, SEQ_DATE] if col not in df.columns]
if missing:
    raise ValueError(f"Missing required columns: {missing}")

# 3) Sort by patient + episode date
df = df.sort_values([ID_COL, SEQ_DATE]).reset_index(drop=True)

# 4) Patient-level anchor (death if present, else last episode)
if "DTA_OBITO" not in df.columns:
    df["DTA_OBITO"] = pd.NaT

death_per_patient        = df.groupby(ID_COL)["DTA_OBITO"].transform("max")
last_episode_per_patient = df.groupby(ID_COL)[SEQ_DATE].transform("max")

df[ANCHOR_COL] = np.where(
    death_per_patient.notna(),
    pd.to_datetime(death_per_patient, errors="coerce"),
    pd.to_datetime(last_episode_per_patient, errors="coerce")
)

# 5) Episode diff (days from anchor back to episode) -> non-negative int
diff_days   = (df[ANCHOR_COL] - df[SEQ_DATE]).dt.days
df[DIFF_COL]= diff_days.clip(lower=0).astype("Int64")  # NA-aware int

# 6) Sequence index
df[SEQ_IDX] = df.groupby(ID_COL).cumcount()

# --- Sanity checks on REAL data -----
has_death              = df["DTA_OBITO"].notna()
viol_real_after_death  = (df.loc[has_death, SEQ_DATE] > df.loc[has_death, "DTA_OBITO"]).sum()
viol_real_nonpos       = (df[DIFF_COL] <= 0).sum()     # strict > 0 check
print(f"[REAL] rows with episode AFTER death: {int(viol_real_after_death)}")
print(f"[REAL] rows with {DIFF_COL} <= 0: {int(viol_real_nonpos)}")

# 7) Training dataframe 
keep_cols = [ID_COL, SEQ_IDX] + [c for c in CONTEXT_COLS if c in df.columns] + SEQUENTIAL_KEEP
train_df  = df[keep_cols].copy()

# 8) Metadata (auto-detect, then override key types)
metadata = SingleTableMetadata()
metadata.detect_from_dataframe(train_df)

# Force known types
metadata.update_column(ID_COL,  sdtype="id")
metadata.update_column(SEQ_IDX, sdtype="numerical")
metadata.update_column(DIFF_COL, sdtype="numerical")

# Mark present datetime columns in train_df (all parsed DTA_* that survived keep_cols)
for col in DATE_COLS:
    if col != SEQ_DATE and col in train_df.columns:
        metadata.update_column(col, sdtype="datetime", datetime_format="%Y-%m-%d")

# Define sequence
metadata.set_sequence_key(ID_COL)
metadata.set_sequence_index(SEQ_IDX)


# 9) Fit PAR
synthesizer = PARSynthesizer(
    metadata=metadata,
    enforce_min_max_values=True,  # helps keep DIFF >= learned min (0)
    enforce_rounding=False,
    context_columns=[c for c in CONTEXT_COLS if c in train_df.columns],
)

synthesizer.fit(train_df)

# 10) Sample (one sequence per real patient)
num_sequences = train_df[ID_COL].nunique()
syn = synthesizer.sample(num_sequences=num_sequences)

# 11) Reconstruct episode date & consistency
if ANCHOR_COL in syn.columns:
    syn[ANCHOR_COL] = pd.to_datetime(syn[ANCHOR_COL], errors="coerce")
if "DTA_OBITO" in syn.columns:
    syn["DTA_OBITO"] = pd.to_datetime(syn["DTA_OBITO"], errors="coerce")

# sanitize DIFF
syn[DIFF_COL] = pd.to_numeric(syn[DIFF_COL], errors="coerce").fillna(0).clip(lower=0).round().astype(int)

# reconstruct
syn[SEQ_DATE] = syn[ANCHOR_COL] - pd.to_timedelta(syn[DIFF_COL], unit="D")

# rebuild anchor from SYN (death if present, else last episode)
death_per_patient_syn        = syn.groupby(ID_COL)["DTA_OBITO"].transform("max") if "DTA_OBITO" in syn.columns else pd.Series(pd.NaT, index=syn.index)
last_episode_per_patient_syn = syn.groupby(ID_COL)[SEQ_DATE].transform("max")
syn[ANCHOR_COL] = np.where(
    death_per_patient_syn.notna(),
    pd.to_datetime(death_per_patient_syn, errors="coerce"),
    pd.to_datetime(last_episode_per_patient_syn, errors="coerce")
)

# keep DIFF consistent
syn[DIFF_COL] = (syn[ANCHOR_COL] - syn[SEQ_DATE]).dt.days
syn[DIFF_COL] = syn[DIFF_COL].fillna(0).clip(lower=0).astype(int)

# order and ssave
syn = syn.sort_values([ID_COL, SEQ_DATE]).reset_index(drop=True)

# --- Sanity checks on SYN data ---
viol_syn_after_death = 0
if "DTA_OBITO" in syn.columns:
    has_death_syn = syn["DTA_OBITO"].notna()
    viol_syn_after_death = (syn.loc[has_death_syn, SEQ_DATE] > syn.loc[has_death_syn, "DTA_OBITO"]).sum()
viol_syn_nonpos = (syn[DIFF_COL] <= 0).sum()
print(f"[SYN ] rows with episode AFTER death: {int(viol_syn_after_death)}")
print(f"[SYN ] rows with {DIFF_COL} <= 0: {int(viol_syn_nonpos)}")

syn.to_csv(OUTPUT_CSV, index=False)
print(f"Saved to {OUTPUT_CSV}")

Thank you.