Welcome to resampling-with Discussions!

[stefanv]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[ben-herbst]

I have spent some time reading (parts of) the current document. Here are some general impressions, in particular order.

• Chapter 2 The resampling method: We jump right in with python. Should we give some help or point the students to other resources? For example, what is this import? Shouldn't we at least explain the difference between core python and the libraries? When should we point out the difference between a python list and a numpy array? When I was still in the teaching business I found it necessary to introduce even good students to core python concepts.
• Chapter 3 What is probability: Perhaps it is just me but I don't find this chapter very useful. Many ideas are introduced but not really going anywhere. Do we want to do any serious rewrites? I have some ideas what can possibly go into a chapter like this, including practical experience from industry, as well as some philosophical issues. I can first give an outline if you think it worthwhile.
• Should we introduce the idea of a random variable and probability distribution? In the examples I have seen so far, uniform distributions are used without mentioning the fact. For instance, the birthday problem can be solved through a reasonably straightforward counting process, if a uniform distribution is assumed. It can also be used to illustrate the Monte Carlo process. The theoretical problem becomes harder if we are given a probability distribution of the birthdates. In this case I am relieved that MC is available!

[ben-herbst]

Thanks, Matthew!

For random variables I wasn't thinking of anything sophisticated. Just the idea that a random variable can have different outcomes with frequencies determined by their probabilities. In code we have to give the possible outcomes as well as the probabilities (even if we tacitly assume a uniform distribution) in order to choose a specific instantiation.

If I have to attempt an explanation, it will go along the lines of: Let's assume an experiment that can have different outcomes, for example, a coin toss can have two outcomes 'H' and 'T' (or we can give it any other two labels at our convenience). We don't know in advance which one of the outcomes will be realised. For that we have to wait for the conclusion of the experiment. It is only after the coin toss that we know what the outcome is. We do know, however, the probabilities of the outcomes. For a fair coin, the probabilities for both outcomes are 0.5. Assuming mutually exclusive events, collectively exhaustive events, the probabilities are always positive and add up to 1. A loaded coin can, for example, have probabilities of 0.3 and 0.7.

The variable that can take on any of multiple outcomes is referred to as a random variable. The source of the uncertainty is the fact that our experiment, or random variable, can take on any one of a number of outcomes. Outcomes that are only known after the experiment has taken place. Note: The power of resampling that we can replace the experiments with simulations, as we do throughout this book.

The probabilities that determine the frequency with which we observe the outcomes is known as the probability distribution associated with the random variable.

[matthew-brett]

Peter and I are just discussing now - Peter also felt that "random variable" was a tough term - and I recalled some dispute about this - found quickly via Wikipedia: "The name “random variable” is a great source of misunderstanding as it is neither random nor is it a variable. It is a function."

https://mml-book.github.io/book/mml-book.pdf

[ben-herbst]

I read the exact same article! I still find it useful to have a term to refer to when talking about a quantity that can take on different values and there is uncertainty to which instance exactly. At the same time, we have certainty about the expected frequencies.

For me these concepts are important enough to give it a name.

But I'm happy to step away from this.

[matthew-brett]

Oh dear - sorry - I really wasn't trying to make you step away - just trying to negotiate with you and myself about what we should explain and where.

My instinct is that the abstract concept of a "random variable" - or maybe "function generating random values"? - is very useful - essential - for doing symbolic mathematics on random quantities - but - will we need it in the book? And if we do - do we need it here, very early on?

To help - Ben - sorry to ask - but can you think of something that is hard to explain without this idea - and we can see if we can work out how much of the concept we need to persuade our readers of?

[pxr687]

Just some thoughts based on the discussion above and the conversation with Matthew yesterday. I agree it would be good to do re-write of 'Chapter 3 What is probability'. At the moment I like that it deals with the philosophical issues, but I think it flits between a lot of different examples in a way that reduces its clarity, as the reader is being asked to hold a lot of new, abstract concepts and new examples in mind concurrently. However, my normal frustration with statistics textbook intros to probability is that they skirt around the philosophical issues (or ignore them entirely) and just starting throwing the formulas at the reader.

I think this (for me!) often creates a distraction around questions like how objective probabilities are (e.g. are they closer to physical laws, or just useful subjective estimations about how systems behave?). I think maybe it would be good to have an interesting yet easy to think about scenario that lets us identify these issues. I think giving a context to what probability IS (or at least, the different ideas about what it is) might give the reader a pragmatic, modelling-based understanding that may make it easier to internalize the content (and see why a code-based, simulation-based approach is useful!). Creating that mindset/understanding might also give the chapter better purpose and direction?

The discussion above and something Matthew mentioned yesterday gave me an idea for a thought experiment-y situation that I think could work well to raise the issues. E.g. we setup a scenario where the reader is aware that a coin has been tampered with and is no longer fair. Within this scenario, we introduce a couple of characters who are betting on the coin's outcome. One character is poised to flip the coin, unaware of its manipulation, while the other character has manipulated the coin's weight to influence its outcome (although the cheater themselves hasn't tossed it yet, they've just weighted it in a way that they think makes it very likely to land on a specific side).

Asking questions about probability from different perspectives in this scenario gives a nice vehicle for bringing out the philosophical issues, but in a context where there's an aspect of human interest (cheating!) but also a concrete experiment (coin flipping).

To bring out the issues we ask questions about the probabilities from a sort of disembodied objective perspective, and from the subjective perspectives of each of the characters (and question how these relate to an objective perspective) e.g.:

• Is there an inherent "true" probability distribution that characterizes the unfair coin? Is this best approached through a frequentist interpretation – considering the distribution of outcomes in a massive number of coin flips? Or is it best thought of some other way?

• The coin flipper believes the coin is fair due to limited information, whereas the cheater possesses additional knowledge affecting their estimation. Are these different estimations purely subjective? Do they reflect a level of belief, or are they referring to some hypothetical extended series of coin flips? Or do they refer to something else?

I think we could use a scenario like this to cover a lot of the ground the chapter currently covers, but in a clearer way. We could also maybe use it to get into random variables (maybe clarifying the misleading name?). E.g. we can define random variables, and then ask about them as they apply to the scenario above?

What do you think? Sorry if this was off topic or vague, its just what came to mind in the discussion yesterday!