Multidimensional/composite target encoding

[sunishchal-recentive]

I would like to do target encoding on the composite of multiple columns, but the current functionality only allows a single column to be encoded.

For example: I have a column names product and another named color and I'd like a unique target encoding value for each product+color combination. Currently, I can only have an encoding for each unique product and each unique color separately.

The workaround would be to concatenate the column values together and then target encode, but that is a bit clunky and leads to some unnecessary categorical features in my dataframe. Let me know if this is something worth raising a PR for.

The implementation I'm thinking is optionally allowing a new argument called something like composite_cols (open to better naming suggestions). This arg can be a list of lists, where each inner list indicates the column names to be concatenated together, and each element in the outer list makes up a composite column. If passed, convert the values to string and concatenate them together before passing into the encoder the same way as regular cols. The composite column can be named as the concatenation of all its component column names.

[PaulWestenthanner]

If we'd implement it for TargetEncoder we'd need it also for all other encoders where each column is encoded independently (which are all encoders except hashing).
I think the library should focus on just encoding and not do these kinds of feature engineering. My subjective opinion is that leaving concatenation to the user is the way to go. Is it really that clunky? It's just a line of code is it? What exactly do you mean by leads to some unnecessary categorical features? Do you mean that if you concat product and color to productcolor it will also encode product and color (if you do not explicitely specify columns)? On this topic I agree it's annoying.
Maybe we could not change all the encoders but offer some preprocessing functions? There could be a module preprocessing with a function create_composite_columns(input_df: pd.DataFrame, composite_cols: List[List[str]]) -> pd.DataFrame that will concatenate the columns and drop the individual cols for encoding.

[sunishchal-recentive]

Great! I like the preprocessing idea. I will scope it out and work on a PR for this in the next couple weeks.

[Sunishchal]

@PaulWestenthanner

One point of clarification on what you wrote:

that will concatenate the columns and drop the individual cols for encoding.

The use case I have in mind is to get an encoding of the joint product & color fields, but not return a string column of those two (as it will internally handle the encoding and then throw away the concatenated field). I may or may not want to also encode product & color separately. Here is a pseudocode example for how I would do it now.

df['product_color'] = df.product + df.color
encoder = ce.TargetEncoder(cols=['product_color', 'product', 'color'])
encoder.fit(X, y)
X_enc = encoder.transform(X)
X_enc = X_enc.drop(columns=['product_color'])

So it is a minor nuisance (2 extra lines of code). The cleanest solution I can think of is to allow tuples to be passed into the cols arg which indicates those columns should be concatenated before encoding. So I may have spoken too soon about the preprocessing idea. Let me know if this seems like enough of a quality of life improvement to warrant a modification.

I'm also curious about this point:

Do you mean that if you concat product and color to productcolor it will also encode product and color (if you do not explicitly specify columns)? On this topic I agree it's annoying.

If I were in this position and didn't want to specify columns (maybe I have a long list of cat columns to encode), then I think it would be simple enough to drop product & color before encoding? Let me know if I'm missing something and if you'd like me to build a solution for it.

[PaulWestenthanner]

I like the idea that you can choose if you want to encode product and color also as separate columns or not.
I also like the tuple solution. This does not interfere with the current API and can be understood if documented well.
If you want to create this PR please go ahead. I think it's a useful feature and it's sufficiently clean and backwards compatible. The only thing I'd ask for is to implement it in all encoders not just target encoding in order to have a uniform API across all encoders