Improvements on multidimensional notebook #1876
Conversation
…book, all the hierarchical parameters were removed in favour of fully pooled parameters over geographies. However the text still discusses hierarchies so I'm restoring them.
…between partial, full and no pooling models. We now have examples of all three so it's a bit more a friendly introduction.
…native prior_predictive method, I think it's better to call pymc's prior predictive method. The advatnage is that y_original_scale is easily recovered so it feels more consistent with the way we fetch y_original_scale later in the notebook. It is also confusing to go through the work of adding y_original_scale a few cells above and then make people fetch the scales all over again.
…example data exactly duplicated channels for each geography which would cause intense posterior correlations. New example data is generated from the multi-dimensional mmm. Also removed the change points - those are mixtures and are inheritly difficult to fit. I think we should keep this notebook more narrowly focused on dimensions and hierarchies.
…on because we'd need a zero sum normal and the prior class isn't design to interact with zero sum normals very well (I would like to be able to add two Prior classes together so we can have a seperate mean.)
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
daniel-saunders-phil
changed the title
|
Alright, I got the model to converge again. The optimizer outputs are bit messed up though. I'll have to figure that out next. |
… to be appropriate to the scale of this dataset.
|
@cetagostini Hey mate, if you have a chance to review, that would be much appreciated. I got a bit cavalier with ripping out features that were making it unduly difficult to fit the model so open to discussing those choices. But hey, it fits in 70 seconds now 😃 Also, I couldn't find a idata compression small enough nor is it synced with the yaml file anymore. Lemme know if that's a blocker and I'll try again. |
|
View / edit / reply to this conversation on ReviewNB juanitorduz commented on 2025-08-09T07:13:37Z Line #12. extend_idata=True, Shall we use nutpie? |
There was a problem hiding this comment.
I like the notebook a lot, thanks for all the help here mate, amazing contribution to make the notebook look better the small details, we can log them as issues and solve soon.
Respect to Nutpie, I'll say optional, and up to you if you want to change it 🙌🏻
Amazing @daniel-saunders-phil 🔥
|
@daniel-saunders-phil @cetagostini I suggest we merge #1832 and then you can rebase and merge this one? @daniel-s-tccc do we need to also update the other notebooks that rely on this example (i.e. budget allocations ?) |
|
@juanitorduz you are right, that should be only to run the notebook as budget allocation, and allocation assessment. |
Nutpie (numba) is broken on the MMM class right now. The error is pretty obscure so haven't had time to figure out why. I would prefer to use the default sampler because I don't want to point people toward nutpie until we resolve that. |
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## main #1876 +/- ##
===========================================
- Coverage 91.86% 52.47% -39.39%
===========================================
Files 64 64
Lines 7546 7546
===========================================
- Hits 6932 3960 -2972
- Misses 614 3586 +2972 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
I'm working through the rebase and it's pretty time-consuming. @carlosagostini @juanitorduz The trouble is that everything is so tightly coupled - the text in the optimizer notebook wants to call out specific behaviours that old might exhibited or relies on features of the old dataset. Plus the diffs on notebooks are challenging to parse so I cannot quite tell what I need to keep track of and what is new in Carlo's work. What do we think decoupling the multi-dimensional mmm notebook from the optimizer notebooks? It would be easy to do - we just don't save the result of the multi-dimensional notebook on each run. Then we can use old idata, old data, old yaml to run the optimizer notebooks and their results would be preserved. It would help make the notebook suite a bit more modular - you can improve one notebook without having to rewrite all the others. |
|
@daniel-s-tccc I definitely see the benefit to separate notebooks, at the same time, I could be scare then if they are two detach we could bring issues. Ideally, our models in production can use all notebook functions. I'll jump in your branch, and take a look to your issues and try to give you a hand! |
|
View / edit / reply to this conversation on ReviewNB cetagostini commented on 2025-08-12T08:53:24Z Respect to previous one, channels estimations look quite similar. From storytelling perspective, I'm not sure about the new model, both channels are massively uncertainty, and we don't have clear winners. Previous model definitely was showing different adstocks, and preference to x1.
Could we use set better params for x1? and probably take signals from x2, so values are only in the lower range bringing uncertainty to it? |
…act. Update notebook with code for parameter recovery.
I agree, that's a good plan to finish it. Parameter recovery is hitting some bumps so I'm gonna keep digging to figure out what is happening. |
|
Alright guys, I think we are good here :) |
|
View / edit / reply to this conversation on ReviewNB juanitorduz commented on 2025-08-15T21:06:17Z Can we remove the output ...
|
|
View / edit / reply to this conversation on ReviewNB juanitorduz commented on 2025-08-15T21:06:18Z are we missing some bar plot on the lower right panel? daniel-saunders-phil commented on 2025-08-15T21:58:27Z The optimizer just didn't put any money there because it's the worst channel. I agree it looks weird so I pushed the budget up a bit. |
|
Thanks @daniel-saunders-phil :) I left some minor comments. I think there are some open comments from @carlosagostini (or are they resolved?), once we close them, let's merge this one! I like the new content a lot! Thanks again for making it much better 🙌 |
…ion. Get rid of the reloader error. Remove stray channel variance diagnostic I left in.
|
The optimizer just didn't put any money there because it's the worst channel. I agree it looks weird so I pushed the budget up a bit. View entire conversation on ReviewNB |
|
@juanitorduz tided up a few things. I think we are all clear on Carlos' suggestions - they mostly pertained to the optimizer notebook except the two items he put in a checklist for me. |
Yes, we are good to go here from my side! |
When you have a second, could you approve? I think it's blocked until you say yes because you put in a request for changes last week. |
|
I approved! I think we need @carlosagostini 's blessing :) |
|
yay! Thanks @daniel-saunders-phil |
The current multi-dimensional notebook has a couple of issues:
This PR address those issues and offers a number of other improvements:
It's not finished yet because I'm still getting some pesky divergences. A bit unclear the cause atm.
📚 Documentation preview 📚: https://pymc-marketing--1876.org.readthedocs.build/en/1876/