Users Page is Broken

[xehu]

Currently, the db of users is so large that the Users Page does not consistently load (and in my experience, is not really loading at all):

[markwhiting]

@xehu Does this stop batches from running? Or just stop us from easily sending bonuses etc?

As mentioned on basecamp. The main courses of action seem to be:

1. paginate users, which involves updating the users page and the search function on that page, and may or may not involve updating some other functions around the system. The pro is that we wouldn't loose anything, and it would probably make our lives better. The con is that its harder and we might mess up.
2. clean the db so that not so many people are loading. This seems easy but has the con that it may fragment our data and make things more annoying in the future.

Other options? Other analysis on the options we have?

[xehu]

This does not seem to stop batches from running. The only harm is that it stops us from easily bonusing workers who complain (it seems that, on occasion, workers are still incorrectly being bonused only $8.80). It also stops an easy search in case people email about finding their link (I'd have to call the users API directly instead).

Other thought: Is it possible to pre-load the users page somehow so that it doesn't have to do it every time? Or alternatively, it is possible to split it into clean subgroups - for example, only load people with hasBanged = true or hasBanged = false, such that we reduce the number of users that we have to load?

[markwhiting]

Good to hear that its not going to block you from running! We can set up an alternative bonus plan until we have this working. Ping me if you want to chat about that.

I like the idea of loading separately, e.g., /users for willbang and /pastusers for hasbanged. That seems like it would be simple and would require fewer changes, however it is also only a short-term solution — we will still end up with a lot of people on both those lists. I think pagination is still a stronger long-term solution.

Thoughts?