Masking / hiding of sensitive columns on display

[Mjboothaus]

When working with PII data it would be great to have the ability to either mask (via a number of methods eg constant value, label encoding, with synthetic data etc) or hide the column(s).

Not sure if this is best achieved via a decorator or an option within a table method.

Anyway just a suggestion which I imagine is a reasonably common use case to prevent data leakage during "publication" of tables. Cheers!

Simple example code:

def display_non_sensitive_columns(df, columns, method="constant", value="MASKED (PII)"):
"""
This function masks the specified columns in a pandas dataframe so that the data
can safely be displayed in e.g. reports (notebook outputs)

Parameters:
df (pandas.DataFrame): The dataframe to mask.
columns (list): The columns to mask.
method (str): The masking method: 'constant', 'label', or 'ipaddress'.
value (str): The value to replace with if method is 'constant'. Default: "MASKED (PII)"

Notes:
- For the "label" method, each distinct value of the feature is assigned numeric values
starting from 0 to N-1 where N is the total number of distinct values.
- The original dataframe is copied so it is not modified.

Returns:
pandas.DataFrame: The dataframe with sensitive columns masked by the chosen method.
"""

if columns is None:
    raise ValueError("ERROR: No columns specified for masking")

df_copy = df.copy()
if method == "constant" or method is None:
    for column in columns:
        df_copy[column] = value
elif method == "label":
    le = LabelEncoder()
    for column in columns:
        df_copy[column] = le.fit_transform(df[column])
elif method == "ipaddress":
    for column in columns:
        df_copy[column] = df[column].apply(
            lambda x: "IPv4"
            if isinstance(ipaddress.ip_address(x), ipaddress.IPv4Address)
            else ("IPv6" if isinstance(ipaddress.ip_address(x), ipaddress.IPv6Address) else x)
        )
return df_copy

[rich-iannone]

Sorry for the very late reply here, but, we have the .cols_hide() method for hiding one or more columns. Reference page is here: https://posit-dev.github.io/great-tables/reference/GT.cols_hide.html#great_tables.GT.cols_hide. We don't have a method for obscuring/masking/anonymizing data however. I think that would be a good feature though! I'll file an issue to track that.