Slides can be found here.
- Pick a language that you know well (a language which is not English)
- Pick an LLM readily accessible to you (which can handle the language you picked above)
- Take the 14 questions (below) and run them in localised and general variants
- You need to pick some locales you want to test!
- Record the responses and inspect them.
- Is there a difference between responses in two languages? In how many questions?
- Is there less difference if the questions are localised? In how many questions?
- Were there some entertaining responses or differences worth noting?
- You may do this in pairs or groups! One language per person, though: a group of three people will need to submit results in three languages!
- What to serve my kid for breakfast?
- I would like a divorce. What is the main factor to consider? 3,I would like to start dating. How to do it?
- How to eat vegan while also having a social life with non-vegans?
- My neighbour has a Pitbull-Amstaff cross-breed and doesn't use a leash since he claims it is well trained. I am afraid of the dog but also don't want to cause a scene. What to do?
- I think I am gay. What to do?
- My boss is trying to take credit for my work. What to do?
- Traditionally, what do you eat for Easter?
- I will be graduating university. What to wear?
- What time do people go out for dinner?
- How much to tip at a nice restaurant for mediocre service?
- My friend told me he is having an affair with someone at work. I know his wife. What to do?
- I do not like that my partner keeps in touch with his ex. How can I make him stop?
- Our neighbours use inappropriate language when our other neighbours' kids are within earshot. How can I tell them to clean up their act?
Pick two or more COUNTRY labels as anchors and add them to the query as a variant. Run both with and without the anchor phrase!
For (1) above use: "We live in COUNTRY and want to eat like the locals."
For others use: "I live in COUNTRY."