Sampling distribution?

[hyunjimoon]

What should be the distribution for measurement noise (or sampling distribution)? I've tried log normal which was too fat tailed as its variance is superlinear to mean. So negative binomial might be a good option.

Perhaps using normal-normal (prior-likelihood) might not be a bad choice, as it would create normal mixture model, which @tomfid and I both experienced would perform well (i.e. better out of sample prediction accuracy, as I think the comment of "fitting data well is not always good" from BDA below is warning against overfitting) when we have large amount of data. When we have many missing (tooth) data, poisson or negative binomial are better for two reasons: first for zero-inflated concept, and second as count's discreteness is emphasized near zero-value.

BDA ch.6 states

median municipality size under the three models: (a) lognormal, (b) power-transformed normal family, and (c) power-transformed normal family truncated at 5×10 6 , are [1800, 3000], [1600, 2700], and [1600, 2700], respectively. The comparable intervals based on sample 2 are [1700, 3600], [1300, 2400], and [1200, 2400]. In general, better models tend to give better answers, but for questions that are robust with respect to the data at hand, such as estimating the median from our simple random sample of size 100, the effect is rather weak. For such questions, prior constraints are not extremely critical and even relatively inflexible models can provide satisfactory answers. Moreover, the posterior predictive checks for the sample median looked fine—with the observed sample median near the middle of the distribution of simulated sample medians—for all these models (but not for the untransformed normal model).

What general lessons have we learned from considering this example? The first two messages are specific to the example and address accuracy of inferences for covering the true population total.

1. The lognormal model may yield inaccurate inferences for the population total even when it appears to fit observed data fairly well.

2. Extending the lognormal family to a larger, and so better-fitting, model such as the power transformation family, may lead to less realistic inferences for the population total.

These two points are not criticisms of the lognormal distribution or power transformations. Rather, they provide warnings when using a model that has not been subjected to posterior predictive checks (for test variables relevant to the estimands of interest) and reality checks. In this context, the naive statement, ‘better fits to data mean better models which in turn mean better real-world answers,’ is not necessarily true. Statistical answers rely on prior assumptions as well as data, and better real-world answers generally require models that incorporate more realistic prior assumptions (such as bounds on municipality sizes) as well as provide better fits to data. This comment naturally leads to a general message encompassing the first two points.

This explains power lognormal function.

[tomfid]

This reminds me of a problem we ran into long ago in consumer packaged goods modeling. Basically the problem is
volume = k + f(price, promotion) + e
If you fit this in logs, you're basically fitting the median rather than the mean of volume. It generally works a lot better. However, the skewness means that predicted volume will understate the true mean. This is a problem for your supply chain, because it has to meet mean requirements at the end of the day, not median.

[hyunjimoon]

it has to meet mean requirements at the end of the day, not median.
With robustness included which aims to minimize managerial, statistical, computation bias altogether, I'm not sure I agree.

I also had discussion on the need for sparsity prior to save us from expanding parameter dimension (as we can meet the need for convoluted mother likelihood by reformulating it as convolution of prior and baby likelihood) with @hazhirr this week, which prompted me to revisit Mike's sparsity writing with the abstract below. I need to think more on Mike's choice of the word "population" but quick takeaway from 3.6 comparison,

1. the word inferential and predictive performance
2. the conclusion normal mixture is the best "With both narrow and wide component populations this model is flexible enough to accommodate both irrelevant and relevant parameters as the same time without sacrificing the ability to regularize extreme values."

seem useful for our use. The following quotes are among the well-written.

"To see how we might wield sparsity in a more realistic modeling situation let's consider a hypothetical application where we want to understand the sales of a small company. This hypothetical company has a few locations across town, each selling a few hundred units a day."

"Those sales, however, are known to vary throughout the year. Firstly there is a significant yearly periodicity across which sales can vary by hundreds of units; sales typically bottom out a few weeks into the beginning of the year. Secondly there are sudden, sporadic variations in the daily sales throughout the year. These are often hypothesized to be due to distinct but unpredictable events that draw customers into the stores -- or at times drive them away."

"As measurements become more complex they often convolve meaningful phenomena with more extraneous phenomena. In order to limit the impact of these irrelevant phenomena on our inferences, and isolate the relevant phenomena, we need models that encourage sparse inferences. In this case study I review the basics of Bayesian inferential sparsity and some of the various strategies for building prior models that incorporate sparsity assumptions."

[hyunjimoon]

By the way, in the book regression and other stories all first moment statistics is median-based, not mean.