-
Notifications
You must be signed in to change notification settings - Fork 0
Custom query and preprocess missing columns handling #19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 84 commits
Commits
Show all changes
85 commits
Select commit
Hold shift + click to select a range
ebf561e
Merge remote-tracking branch 'origin/feature/1a-zero-vram' into fix/d…
LTan-101104 532206a
db_connection_custom_condition : intialize code for load_jobs_datafr…
LTan-101104 2f1e04c
db_connection_custom_condition : Resetup mock test in conftest.py, in…
LTan-101104 e3693c5
db_connection_custom_condition : Added new parameters of optional fil…
LTan-101104 3d98f56
db_connection_custom_condition : refactor tests in test_preprocess du…
LTan-101104 67c15ab
Merge remote-tracking branch 'origin/main' into fix/db_connection_ana…
LTan-101104 9febbf2
Merge remote-tracking branch 'origin/main' into fix/db_connection_ana…
LTan-101104 54e89dc
db_connection_custom_condition : fix ruff issue
LTan-101104 80de772
db_connection_custom_condition: initialize keyword arguments handling…
LTan-101104 26632ce
db_connection_custom_condition : reintialize simpler functions and ad…
LTan-101104 b7809d3
REsolve merge conflict from branch origin/feature/1a-zero-vram
LTan-101104 d0a99b5
db_connection_custom_condition : intialize warning and new filtering …
LTan-101104 88e41ff
db_connection_custom_condition : add if and try catch in preprocess t…
LTan-101104 3734154
db_connection_custom_condition : revert changes on vram_usage.py, add…
LTan-101104 12bcf08
db_connection_custom_condition : Add custom query and check for dates…
LTan-101104 74a9ec0
db_connection_custom_condition : refactor code in preprocess
LTan-101104 af28b60
Merge with origin/main
LTan-101104 31e6d04
Resolve some issues from last merge
LTan-101104 5894043
db_connection_custom_condition : debug issues of improper handling fo…
LTan-101104 6ace8ca
db_connection_custom_condition : add warning raised options for load_…
LTan-101104 187d900
db_connection_custom_condition : Readjust warnings, add tests for Key…
LTan-101104 359c8fc
db_connection_custom_condition : add docstring for test_load_jobs_duc…
LTan-101104 db2b898
db_connection_custom_condition : Change constants name, omit TODOs, a…
LTan-101104 093fe7f
Add QOS options to include/ omit custom QOS values, add new tests for…
LTan-101104 0f105bd
Update names of helper functions for loading preprocessed
LTan-101104 56e49ce
Merge remote-tracking branch 'origin/main' into HEAD
LTan-101104 6ce7962
Fix pytest
LTan-101104 33734ce
Make changes to error raised test to match error message
LTan-101104 4700f01
Clean up comments and change test names
LTan-101104 0dfdb0a
Change location of load_process_dataframe function, update notebooks …
LTan-101104 e1067b1
Clean up comment in preprocess
LTan-101104 8bdadcd
Added docstring for one of the new test
LTan-101104 1caf951
Integrate some comments in terms of docstring, inconsistent handling …
LTan-101104 e6a3e88
Update mock data to more recent times, fix test for warning checks
LTan-101104 3f89e90
Change mock data Elapsed time to be more accurate
LTan-101104 2aa287e
Moved the test to the correct position
LTan-101104 8696fff
Add recwarn to all preprocess test
LTan-101104 e9bfea4
Create seperate fixture for dataframe and database path
LTan-101104 f67aa2a
Reachnge based on main's preprocess to avoid conflicts
LTan-101104 4a17627
Change name of fixture to avoid conflicts
LTan-101104 ac32f5b
Merge with origin/main
LTan-101104 492525e
Merged with remain part from origin/main
LTan-101104 063a61f
Clean up helper_filter_irrelevant_records and move to conftest
LTan-101104 c749bb2
Remerge with last changes
LTan-101104 df7b22e
Uncomment and fix tests in test_load_processed_jobs_dataframe
LTan-101104 a0db402
Clean up tests, deleting all notna filtering for GPU and GPUType
LTan-101104 61e8533
Add update_helper_filter and start modifying test_preprocess to use this
LTan-101104 f5e9b62
Updated to use new filter helper functions
LTan-101104 6558c4f
Finalize new helper for tests
LTan-101104 08e19b1
Add notes section back to preprocess
LTan-101104 2e7f674
Update constant name, add new suggested comment
LTan-101104 7729e32
Run clean up script for Efficiency Analysis Notebook
LTan-101104 e0a777b
Update to use original version of Efficiency Analysis
LTan-101104 3f2262d
Update qos variable names
LTan-101104 6ea9e45
Changed required and optional column set to enum
LTan-101104 f5ad403
Add remain columns to optional, update warning statement and test
LTan-101104 4a7a6fd
Break code in preprocess_data into multiple different functions
LTan-101104 89f3160
Add new helper for check_infinity_value (overflow) in preprocess
LTan-101104 ace8434
Update Efficiency Analysis notebookk to use new function
LTan-101104 53e6380
Resolved mypy
LTan-101104 3d127b3
Change folder name and file name
LTan-101104 bcb4035
Make changes to notebook
LTan-101104 fee8186
Reorder arguments in utility function
LTan-101104 9fbfecb
Fixed multiple docstring and comments and test file name following su…
LTan-101104 709183c
Change module name so ruff runs
LTan-101104 4ba2cc8
Change test names as suggested
LTan-101104 cf7ae12
Apply remain suggestions for changing test names
LTan-101104 f2ef0c8
Add warning raised for empty dataframe and test
LTan-101104 46af286
Readjust parameter name
LTan-101104 5321f45
Break load_and_preprocessed into 2 functions , one without custom que…
LTan-101104 9ed207b
Change function call in notebooks
LTan-101104 bf65522
Update documentation for Preempted columne
LTan-101104 0d04dfd
Changed to use @pytest.mark.parametrize and omit some unnecessary rec…
LTan-101104 b2092c2
Complete resolving recwarn problem and add new pytest.mark.parametrize
LTan-101104 ba99adb
Change function name and convert columns which should be omitted to a…
LTan-101104 3e274f1
Add apply_filtering options to avoid duplicate filtering in preproces…
LTan-101104 17888c3
Enforce creating new copy in preprocess
LTan-101104 428f21f
Change method imports in efficiency analysis notebooks
LTan-101104 7ae8ec7
Merge parts of codes, awaiting for merge on test preprocess and handl…
LTan-101104 c28475c
Make changes to test_preprocess_origin to use standard filter helper
LTan-101104 afe9b3f
Complete test_preprocess and preprocess
LTan-101104 921e989
Apprehend test_preprocess back
LTan-101104 b1a853b
Complete fix of test_load_and_preprocess, remove all instances of val…
LTan-101104 5748955
Fix ruff and mypy
LTan-101104 6faf44b
Make changes to del statement in conftest __del__() method of databas…
LTan-101104 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.