Conversation
Adapted from paper - https://arxiv.org/pdf/2007.14527.pdf Issues 1. Still not completely sure if values used in struct are to be calculated in the struct or somewhere else 2. Relies on ForwardDiff.compute_jacobian 3. Tests and dimension check 4. Not completely clear on how reweighing takes place
That's fine for now. The literature doesn't sem to say how to optimize this more, even though it's quite obvious it can be improved. Seems like a good paper idea, but for now just do it by the book.
Feel free to ask questions on the Slack! Zoe should be around?
This is essential. See the other adaptive reweighing tests |
|
Haven't been able to get help from Zoe, I'll try Slack again later. I'll go through the tests that are written already for other adaptive losses. Is it worth reproducing results from the paper? |
It would be good to try. While it might be too expensive to be a unit test, it would make a good tutorial. |
|
I'm working on a project deadline for this Tuesday. I'll go through and add more detailed comments after that. |
|
This should get rebased due to changes in #553. That would hopefully make it much cleaner too. |
Adapted from paper - https://arxiv.org/pdf/2007.14527.pdf
Treating Issue #501
Issues