Contradictory results

[KatatoniaSh]

In a model with non_revenue KPI where I have revenue per kpi, I get negative baseline. Upon my investigation, I use this code to determine the prior probability that the total treatment contribution exceeds 100% of total observed outcome and it is equal to 1. So, it means that I have been optimistic about the media.
When I run the model and check the results, the prior value for all channels for % contribution which will be provided by summary_table() is around 80%.
To me, these are contradictory. I would be thankful if someone can explain.
Also, how can I set the priors based on the dataset that I have. How can I investigate which model is better.

[cpulavarthi]

Hello @KatatoniaSh,

Thank you for contacting us!

The contradictory results you have mentioned is due to the difference in what the code you referenced was trying to compute and what the summary_table outputs. The code is calculating “the prior probability that the total treatment contribution exceeds 100%” as mentioned in the description above the code snippet. This probability is 1 based on your output. The summary_table on the other hand computes the contribution percentages for each of the media channels and the total contribution from all media channels. If each media channel has 80% prior contribution, even having 2 media channels gives an approximate total contribution of 160% which would definitely explain why the probability of the total contribution exceeding 100% is 1.

Regarding guidance on setting priors, I would recommend that you check our documentation on Introduction to Priors and the pages that follow in the Priors section. We recommend using business understanding, past trends, and any relevant experimental data, to guide prior selection. The documentation contains detailed information regarding the choice of prior types (ROI, mROI, Contribution) and guidance relating to choosing the right distribution, prior calibration, etc. If there is a vast difference in the prior and posterior contributions in your summary table, it could be indicating improper prior selection. Do reach out if you have any specific questions regarding prior selection.

The out-of-sample prediction metrics can be used to compare multiple models and choose the best one. In general, various post-modeling checks can help you assess the model’s integrity, stability and plausibility.

I hope this answers your questions. If not, it would be helpful if you shared screenshots of your contradictory results as this would help us further investigate this issue to find the root cause. Please also let us know if you have organic channels and/ or non-media treatments included in your model. Feel free to reach out if you have any further questions or suggestions regarding Meridian.

Google Meridian Support Team