How to compute ROAS with time-lagged inputs (following Jin et al. 2017)?

[mcb00]

Hey yall,

I’m trying to replicate the ROAS computation in equation 10 from the Jin et al. (2017) paper using pymc-marketing. The challenge I’m facing is how to correctly compute ROAS when media inputs are time-lagged via adstock. The example notebook uses the channel contributions, but I don't think that will work to get ROAS over a finite time interval as in the paper.

In the paper, the media contribution from channel $m$ at time $t$, denoted as $\hat{Y}^m_t(x_{t-L+1,m},\dots,x_{t,m})$, depends on the adstocked media from $t-L+1$ up to $t$.
To compute ROAS over a finite interval $(t_0,t_1)$, we need to compare predicted outcomes under two scenarios:
1. Actual observed media spend.
2. A counterfactual where spend is adjusted (e.g. set to zero), but only during the interval $(t_0, t_1)$. The pre-change lags $(t_0-L+1, t_0)$ should remain as observed. That means we need to recompute the adstocked spend for this counterfactual media spend from the beginning of the pre-change period $t_0-L+1$ up to the end of the post-change period $t_1+L-1$.

My question is: Is there a way to query a trained pymc-marketing model for in-sample predictions using a custom media spend time series? This would need to allow setting the raw input spend to arbitrary values during the historical period and having the model recompute the adstocked spend so we can isolate incremental effects correctly.

Is there a recommended way to accomplish this?

Thanks!

[williambdean]

Hi @mcb00,

The forward_pass method handles the media transformation but doesn't save off the adstocked spend. Adstock and saturated spend is saved off:

pymc-marketing/pymc_marketing/mmm/mmm.py

Lines 406 to 412 in 0b27a25

	first, second = (
	(self.adstock, self.saturation)
	if self.adstock_first
	else (self.saturation, self.adstock)
	)
	return second.apply(x=first.apply(x=x, dims="channel"), dims="channel")

However, you should be able to use the channel_data data

pymc-marketing/pymc_marketing/mmm/mmm.py

Lines 485 to 489 in 0b27a25

	channel_data_ = pm.Data(
	name="channel_data",
	value=self.preprocessed_data["X"][self.channel_columns],
	dims=("date", "channel"),
	)

and the adstock attribute in a custom model block for the time being and sample using the compute_deterministic / sample_posterior_predictive function in PyMC if you've already trained the model.

Maybe something like:

...
mmm.fit(X, y)

with pm.Model(coords=mmm.model.coords) as new_model:
    # Since this is scaled
    channel_data = mmm.model["channel_data"].eval()
    pm.Deterministic(
        "adstocked_spend",
        mmm.adstock.apply(channel_data, dims="channel"),
        dims=("date", "channel"),
    )
    idata = pm.compute_deterministic(mmm.fit_result, var_names=["adstocked_spend"])

EDIT: The apply method for adstock creates variables as well so a new model would be needed with compute_deterministic / sample_posterior_predictive would be needed.

Note

The channel data can be switched out with custom data as well if needed

# To visualize the adstocked spend
import matplotlib.pyplot as plt
from pymc_marketing.plot import plot_curve

idata.adstocked_spend.isel(date=slice(0, 30)).pipe(plot_curve, "date")
plt.show()

[mcb00]

Thanks @williambdean ! This is super helpful. Eventually I realized I can get the Jin et al. 2017 ROAS by doing something like:

X_modified = X.copy()
X_modified.iloc[t_0:t_1, col_idx] = 0
pps_original = mmm.sample_posterior_predictive(X, extend_idata=False, combined=True, random_seed=np.random.default_rng(seed));
pps_modified = mmm.sample_posterior_predictive(X_modified, extend_idata=False, combined=True, random_seed=np.random.default_rng(seed));
channel_contributions = pps_original.y - pps_modified.y

This gives posterior samples of the numerator of equation 10 in the paper. Then you can just divide by the spend in the time period to get the period ROAS.

[williambdean]

Awesome. Glad it helped!