New wandb fixes #152
Open
New wandb fixes #152
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Manager for models
[DRAFT] Competition and dataset manager
working dataset management
* Triggering the auto-validator * commit reveal adjustments
* Triggering the auto-validator * updated bittensor and bt client versions
* data fetching refactor and fixes --------- Co-authored-by: Wojtek Jurkowlaniec <[email protected]> Co-authored-by: Wojtek Jurkowlaniec <[email protected]>
* get fresh packages from evaluting competition datasets * remove old dataset before extracting new one * further fixes --------- Co-authored-by: konrad0960 <[email protected]>
* fixed timestamp issue
* simplified model date checking * add progress log for downloading miner model metadata * test DRY chain miner store * possible speedup in getting chain info * cleanup and fix checking cutoff time
* Add graceful shutdown handling for validator main sync error * fix error handing in onnx runner * NEW GREAT REWARDER * Fix new mechanism for rewards, fixed a lot of bugs ^^ * move update_competition_results to where it belongs * WANDB logs as data structure * cleanup * fix tests * fix melanoma handler, don't crash whole process when it fails * DB better logging + protection against validator crashing * fix model evaluation error handling * Error testing (#145) * error testing * loading state debug * Add detailed diagnostics for state saving/loading to identify corruption issues * MORE * new state saving * save state fix * wandb log all at once, check saving state * remove process debug messages * validator serialization fixes * faster logging to WANDB * testing validator save * fighting with states * save state fixes, don't do it so often * double try * save satte less often
* slashing model copying * added double check with local db
* wip slashing model copiers 2 * adjusting slashing mechanics * add melanoma-3 comp
* move modelinfo and modelinfo conversion to proper places * remove repo type from miner model submission * local evaluation - don't set weight (and don't brag about it * add time submitting in model results + remove unnecesarry stuff from try/except * new model for submitting to wandb * switch to default model type repo when downloading miner's model * adding error messages to evaluation + refactored wandb logging structure * cleanup validation, new way of submitting data , reporting errors to wandb * error handling/refactor for chain metadata * collect errors in model manager * adjusting error results * more granular error msgs --------- Co-authored-by: konrad0960 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.