In the future, could we learn from incorrect classifications

[jonfroehlich]

If we start serving CV crops to the validation interface, could we start using the incorrect classifications to further improve the model?

[galenweld]

Absolutely - this approach when done offline (I gather) is called hard-negative mining. The idea is pretty simple, as you train, you keep track of which examples you get wrong, and train on those wrong examples specifically. Hooking it up to the validation interface would be some online variation thereof; and while one could presumably come up with some arbitrarily complex implementation, a straightforward approach would be pretty simple: as labels are marked as incorrect, we simply bundle them together, and occasionally run a few more epochs of training on these incorrect labels (probably also incorporating some amount of the larger dataset as well) to further refine the model's weights.