gamification: exponential levels and cross-checking

[towc]

Use case
I loved the app for the first day I used it, I found it as an alternative to pokemon go that might provide value to my local community, and makes me look at the outside world more than my phone screen.

Unfortunately after the first 2 days, it's rare that I think of opening it, as I don't typically contribute to OSM, and I just don't get the "happy" feeling or the incentive to go out for a walk with the app open.

Proposed Solution

I've read the gamification proposals I could find, and am aware of the problems presented, but I think there are solutions, so let me start with one that helps address competition-induced quality decay:

1. never have a public leaderboard

Outright have a policy that says "it is not permitted to create a public leaderboard, or publicly distribute instructions for creating one".

The app currently shows you your rank in country and global, which I think is nice, if maybe not already too much. It would help to add some kind of visualizations to show how much of an impact a contribution has. I don't have a clear idea here, but perhaps "(# your contributions) / (# global contributions)", so that you see that number grow very fast at first, and then slow down, incentivizing new users to give a few more contributions. This leads to the next item.

2. non-linear "level-up"

Right now the number of contributions you've given is very prominent and always visible. This number increases linearly, 1:1 with contributions. Past ~100 contributions, this doesn't give much of a feeling of enjoyment to casual users, in fact makes you feel less interested in continuing because of clearly diminishing returns of each contribution.

What games do well is hiding the fact that a single action has negligible impact, by giving you a feeling of progression, and some goals to strive for. Right now the app gives you a link to OSM-related resources every 10 contributions or so, which is an incentive, but I don't know how effective it is. These are resources that don't feel earned, and it's good to spread them, but maybe more by some kind of daily tips, or specific to one "type" of quest, e.g. 10 bike-related quests highlights the relevant resource, then +15 for the next one.

If there was a concept of exponential or non-linear "level" instead, where each contribution functions like "experience" (exp) would in a pokemon game, I think I'd find the app much more fun and use it more often.

In pokemon games, level is essentially k * exp**(1/3), the cube root. k is a value to tweak based on what feels nicer in the given context. 1 exp could be 1 contribution, and the app would already be much more interesting for people like me, showing the level in the top left corner, and a progression bar showing percentage (only!) to next level, hiding total contributions in a stat screen, as a less relevant number.

But we could explore a few more options, e.g. certain types of more needed (or more difficult) quests give different amounts of exp. Quests that you do less often might provide more exp, incentivizing variety. Some quest types might be highlighted for the day or week, giving more exp points, and synchronized among users in that area (or globally), so we all learn more about them and can have meaningful discussions in communities about details.

This last one adds a bit of competition, because if e.g. a curb height quest is highlighted, and there's only so many curbs in the city, you're competing with other streetcompleters (street competers?) to get them before they're taken by someone else, and we need the experience to be cooperative instead, which the next point should help with.

Maybe leaning more directly into the pokemon style to capture that audience would be good (or not), e.g. starting at level 5, and getting progressively closer to level 100. Assigning quest groups as pokemon-like groups (grass -> nature, ground -> streets, psychic -> accessibility). Or starting out with limited (less complex) types of quests, and progressively unlocking harder ones on level up.

3. cross-checking between users

If competition makes you want to lower quality to see the number go up, we can address that by making the number go down disproportionately if the quality goes down.

The basic system would be that a random sample (~1%) of recently (~3 days) completed quests show up for a sample (~50%) of users anyway, and once completed by at least 4 different users (with at least e.g. level 5), some central system compares the results, and if exactly one is an anomaly, it punishes that user.

There's a lot of details to think about here, but iterating is possible.

it's a tricky balance to get punishment right. There will be false positives (of low quality), and you don't want the user to feel that the whole thing is unfair and rigged, but you also don't want to give such a small punishment that they're ok just taking it, like too many people I know do with parking tickets. The main punishment would be removing some portion of exp, but if high-frequency offenses, a ban from OSM/the app would make sense.

In my cursory overview, I didn't see any server code, so I understand that central system might not be easy to add, and concurrency is something to deal with, but it would definitely increase trust in the output of this app regardless of gamification, but also allow some more healthy competition to be used for the greater good.

summary

1. policy to ban leaderboards that allow you to compare yourself with others
2. a non-linear level system to increase enjoyment and frequent use
3. community cross-checking to increase confidence in OSM contributions

I'm a (mostly web) developer and am happy to help, but I've never used gradle or kotlin in anger.

[paulklie]

Outright have a policy that says "it is not permitted to create a public leaderboard, or publicly distribute instructions for creating one".

I don't think this is possible, since:

1. this would probably also violate certain parts of the open source data license of openstreetmap.
2. users who have never used the app could still create such a leader board, since all the relevant data is hosted on the openstreetmap servers.

I would also like to point to the following section of the FAQ: https://wiki.openstreetmap.org/wiki/StreetComplete/FAQ#Gamification

3. cross-checking between users

If competition makes you want to lower quality to see the number go up, we can address that by making the number go down disproportionately if the quality goes down.

The basic system would be that a random sample (~1%) of recently (~3 days) completed quests show up for a sample (~50%) of users anyway, and once completed by at least 4 different users (with at least e.g. level 5), some central system compares the results, and if exactly one is an anomaly, it punishes that user.

There are a few issues here:

1. SC does not have that many users, particularly outside of western europe, or big cities places might not be double checked by another user for years, so gathering multiple samples is impossible.
2. SC currently does not have any server infrastructure. Edits are automatically uploaded to the osm servers, so this would first require such servers.

[matkoniecz]

SC currently does not have any server infrastructure.

this does not change your points but SC has some: for website, statistics, photo hosting and to list SC versions that are so buggy that they should stop being used (anything else?)

though edits are directly between OSM servers and given user

[westnordost]

I changed this to a discussion because it's mostly a collection of ideas.

Thanks for the input! I agree that the gamification aspect of the app is maybe not made... that, hmm, professional as real games. It's not the focus, though, the focus is primarily to be a very low barrier of entry way to contribute something to OSM. The gamification there is should basically mostly just introduce the user to OSM in general.

Anyway, as the previous commenters noted, I can not forbid anyone to create a leaderboard himself.

Also, did you know there was an OpenStreetMap "game" with a very similar concept as StreetComplete before? Already in 2013, it won many innovation awards, was mentioned in the newspaper and all. Kort. I didn't, when I got the idea for my app, because nobody used it and hence it was quite unknown. I think the main reason why it failed to gain traction was the indirect way of editing - that each contribution required some other user to also answer any given question the same. So, in the end, basically no edits were made with the app at all.

[paulklie]

I do like the idea of non-linear progression, specifically for achievements. This might help making achivements less spammy: #6493.
Maybe this could be combined with some form of achievement tiers, like bronze, silver gold etc... with increasing requirements.

[towc]

Thank you for the discussion!

1. leaderboard

Thank you for the explanations, I see now it's impractical if not illegal/unethical to ban others from creating leaderboards.

It can at least be policy of this project not to have a leaderboard within the application. I think there's still merit to the idea of showing one self's rank as percentage of total/country/city contributions rather than numerical order, to further discourage competition: maybe you've made 20% of city contributions, you don't know if you're 1st or 2nd, but you're going to care less because you can take pride in your % of contributions, giving you a feeling of ownership of the output more than caring about placement.

It requires some thought and maybe shouldn't be part of this discussion, in favour of the more impactful non-linear progression

2. non-linear progression

I understand gamification is not the point, my feelings won't be hurt if none of these suggestions are implemented :) . As I said, "I loved the app", I think it's a fantastic way to introduce users to OSM contributions.

But it sounds like some level system would be appreciated. I would like to put more emphasis on rewarding certain types of quests more than others, and changing those weights once a week or so. This helps maintain a feeling of purpose and motivation, and might nudge you into doing quests. you normally wouldn't do otherwise.

This can be done client-side with a deterministic PRNG, or just a linear cycle. It doesn't need server-side infrastructure: even if a user tries to log in from a different device, they can recalculate it from their own OSM contributions because they include dates. I don't know if the app is downloading and caching all of their own contributions on startup, if not maybe this is not that easy to implement.

Another complicating factor is that if new quest types are added or removed, there needs to be a checkpoint at that date where the PRNG will work differently before and after or it might change how much score each of the previous contributions provided. This adds some development overhead, but it is possible. Alternatively, the contribution score can be added to the OSM contribution as metadata (I don't know if this is acceptable), but this leads to a very easy "hack" where someone edits their metadata. The good news is we don't need to use this score in the leaderboard or % calculation, it can be completely local to the user, so if they make the edit then they're just ruining their own fun and it doesn't affect anyone else.

3. cross-checking

I wasn't aware of Kort!

Forcing contributions to only be valued once they're checked when the user base is small has some problems in terms of usability. I think the approach of cross-checking only a small percentage of contributions and providing punishment retroactively would work better, but I also understand this wouldn't be easy to implement in SC.

If metadata is allowed in OSM contributions, and it can be queried, then I think we could do it without a central system, and consider those cross-check contributions when calculating one's own score/%. But it sounds like it's not worth implementing this until it becomes a problem. It might also be possible without metadata, but might be intensive for the requester: for each of your contributions in the last 10 days, see if any more recent contributions have been made for the same quest id (I don't know what the data looks like, but I assume each quest has some form of identifier by location and type, as "what is this changing in OSM"), and if there's more than 2 of them, compare and adapt score.

[westnordost]

Hmm, "top X %" instead of "top X" sounds good and should be doable quite easily! Although, that's sad for the real top X (e.g. top 1, maybe still @dbdean , didn't check) because then the badge just says "top 0%" (rounded)

[towc]

oh sorry, when I said "%" I meant percentage of contributions as in the original post, not in the leaderboard. E.g. if there's 1000 contributions for your city and you've made 10 of them, then your "% of city contributions" is 1, even if all other users made less than 10 contributions and therefore you'd be in the top 0%.

This incentivizes you to make more contributions and to feel a sense of ownership, without a feeling of competition. You can say "1% of the contributions for my city come from me", both making you feel a part of the whole effort, as well as realizing that it's still a community effort. You and everyone else's percentages add up to 100%, it's a collaboration and not a competition. You can also make the calculation for "in the last month" and so on, separately, not to discourage new users.

[westnordost]

Ah well, to have the contributions count any sort of city-wise is not really possible. The maximum is per country, for some countries per state (US, Canada, Australia, ...) would be possible, but not any more fine-grained than that.

If this was available, then I'd think it would be nice to do more with that - e.g. have a leaderboard of cities, not of individuals. (So, you can work together with your friends on this.). What would be necessary for this? A way to associate edits with a city. This requires... a sort of world map list of all (bigger?) cities each with a center point and a sort of radius what should still count towards that city.

But yeah, also % of contributions in a given region should work on a technical level. However, if you are in a large pool of users, (e.g. worldwide, or country-wide), the shown percentage would likely always be very very low and thus not really be motivating to do more.

[towc]

oh sorry, I assumed city-level was available, might not be worth it although I do think that's the "nexus of community" that most people would like to see.

Even if it remains a small number, I think that's worth it. The whole point is to avoid competition and encourage cooperation. You can make the number more motivating by displaying in ‰, instead of %, or going even further down, and dynamically changing the unit, e.g. % if you have above 1%, per million if above 0.001%, so it's always at least greater than one. A user can challenge themselves to maintain a certain % even if more users come along to contribute, this keeps up the cooperative spirit while being motivating. Ideally the areas would be worldwide, continent, country, and city, with city being the most visible, so the number doesn't need to be so small. This mechanisms also heavily encourages users in places that are not as contributed to, as their numbers will be bigger, it sounds like a win-win to me.

For some reason I originally thought this would grow as something like ln(contributions), but it is just linear. I still think it would a good improvement over the leaderboards, but I think the main point of the post is the non-linear score which should be separate, but if this is easy to implement then it sounds like a win!

As for how to reward the top contributors that might go from #1 in country to seeing 1% as their score and feeling demotivated, I'm not sure. I don't know them, I haven't spent enough time in the OSM community to know what could work well. Maybe you can ask them? If we assume the top 3 of a country are trustworthy and won't trade quality for quantity, maybe we could still show the leaderboard for them, but if we make that public we run into the same competitive issues, and the point is to avoid competition. I'm guessing the most hardcore contributors won't mind not having that information visible on the app for everyone if it might mean OSM gets more contributions in total, but I have no idea what they think.

I feel a little arrogant having these opinions on an app many people have spent maybe hundreds of hours developing and using, I apologize if it comes across that way, and hope the ideas are helpful!

[towc]

I just remembered what I first meant in the original post with non-linear %, I didn't make it clear at all!

The formula would be total / yours. So if there's 1m total contributions, your "%" (which isn't really a percentage) is 1m / 1 = 1m. big number for one contribution. You make a second contribution, and that number is now 500k. So the new user starts understanding that lower number = better, and look how fast they got from 1m to 500k! They'll get to 1 in no time. Third contribution, 333k, and so on. The number gets progressively smaller, new users are happy with big numbers, veterans are happy with small numbers, everybody wins. Except it's not really clear what this number represents, and maybe that's ok. An intuitive understanding is "if you were the only user, how many of you would it take to achieve the total contributions we have now", but that's definitely going to be demotivating for new players so that information shouldn't be immediately obvious to new users.

This seems like an easy way to achieve non-linearity, and it is more meaningful than the cube root approach I suggested, although the added transparency and confusion in trying to minimize a score might not be helpful. I still recommend the cube root approach, but I wouldn't be opposed to the more transparent approach. Both the linear and inverse % could still be in the stat screen.

It could be helpful to give this unit of "inverse of total" a name that isn't too descriptive, e.g. "kyu" for codewars, although that's based on an real unit. I suggest "quots" as a short of "quotient", as it is literally a division, and we in fact don't care about the remainder as we don't really care about decimals, but units of m, k, and 1s. I assume the top contributors wouldn't have more (less) than 100 quots