-
Notifications
You must be signed in to change notification settings - Fork 16
Getting code ready for the next major release #58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
sarthakpati
wants to merge
1,390
commits into
master
Choose a base branch
from
fets_2.0
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 250 commits
Commits
Show all changes
1390 commits
Select commit
Hold shift + click to select a range
675bb2e
Simplify build copy. Rename stages
aristizabal95 4ea515a
Simplify build copy. Rename stages
aristizabal95 4dcb96e
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 678eee4
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 75b1bed
added example for tumor segmentation models
sarthakpati 54d5159
added example for tumor segmentation models
sarthakpati e6f72c7
updated to use the example models
sarthakpati 8d7ecd9
updated to use the example models
sarthakpati 760c0da
added brats nomenclature
sarthakpati 734dd04
added brats nomenclature
sarthakpati 3243bee
updated nomenclature
sarthakpati 00470e6
updated nomenclature
sarthakpati 99a3a2e
fixed list issue
sarthakpati f672747
fixed list issue
sarthakpati b8eadb1
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
sarthakpati 3c1bcb4
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
sarthakpati 440376a
updated hash
sarthakpati 9fe207b
version update
sarthakpati 60aae62
Merge branch 'master' of https://github.com/FeTS-AI/Front-End into fe…
sarthakpati 57d7a38
Merge branch 'master' of https://github.com/FeTS-AI/Front-End into fe…
sarthakpati b7b918b
Move downloaded models to MLCube
aristizabal95 eb53a4b
Move downloaded models to MLCube
aristizabal95 85cb2bf
Fix prev stage removal from brain_extraction
aristizabal95 94b8588
Fix prev stage removal from brain_extraction
aristizabal95 ee47c95
Fix get row information retrieval
aristizabal95 c421040
Fix get row information retrieval
aristizabal95 b5cd751
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 33aabc7
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 a061984
added comment
sarthakpati 687fa90
added comment
sarthakpati c056e69
dynamic download of models
sarthakpati 4e0edb7
dynamic download of models
sarthakpati 4f4e30b
Remove repeated pip installs. Don't use venv
aristizabal95 828cb5b
Remove repeated pip installs. Don't use venv
aristizabal95 e262d51
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 6f1cfd0
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 7e3b2ef
Implement genera, extract stage.
aristizabal95 4a997ef
Implement genera, extract stage.
aristizabal95 f7ebae9
Fix brain file naming issue
aristizabal95 03216e0
Fix brain file naming issue
aristizabal95 56ef4d0
Add status code to extract code
aristizabal95 408f3b8
Add status code to extract code
aristizabal95 a8b5053
no need for cast
sarthakpati 6447316
no need for cast
sarthakpati cd2032f
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 ca4dfce
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 fd8e417
Download model checkpoints at build time
aristizabal95 eb2a501
Download model checkpoints at build time
aristizabal95 e16cf65
Move downloads to happen before copying contents
aristizabal95 b8e2ba9
Move downloads to happen before copying contents
aristizabal95 709d894
Generalize extract stage. Get Traceback
aristizabal95 056d979
Generalize extract stage. Get Traceback
aristizabal95 b3c5dff
Place constants on a separate folder
aristizabal95 2d2f11c
Place constants on a separate folder
aristizabal95 77ec6fd
Add manual step
aristizabal95 10d8adb
Add manual step
aristizabal95 2ce8fa4
Set backup to read-only
aristizabal95 98f73f4
Set backup to read-only
aristizabal95 b009e89
Remove unused dockerfile
aristizabal95 3b3213d
Remove unused dockerfile
aristizabal95 f152385
Fix manual stage
aristizabal95 d8e9a0c
Fix manual stage
aristizabal95 ad37303
[WIP] Implement match stage
aristizabal95 58f497b
[WIP] Implement match stage
aristizabal95 2880193
Implement match stage should_run function
aristizabal95 5713d3d
Implement match stage should_run function
aristizabal95 419733b
Install recent version of GaNDLF
aristizabal95 15a2b0b
Install recent version of GaNDLF
aristizabal95 f5ad518
copy data_prep_models from the correct folder
hasan7n c52193b
copy data_prep_models from the correct folder
hasan7n ef574ab
fix bugs related to tumor segmentation
hasan7n c32dd95
fix bugs related to tumor segmentation
hasan7n 2aefcdc
modify tumor segmentation model download url
hasan7n 40ed04c
modify tumor segmentation model download url
hasan7n 3f04385
fix the bugfix in prepareDataset
hasan7n a6793a7
fix the bugfix in prepareDataset
hasan7n d122ae6
start fixing matching step (WIP)
hasan7n 7b7c35b
start fixing matching step (WIP)
hasan7n e56bb5a
updated link for tumor segmentation model
sarthakpati 54bb2c5
updated link for tumor segmentation model
sarthakpati ee06c51
updated modality order and models
sarthakpati bec43dd
updated modality order and models
sarthakpati 8bd3dbe
merge branch fets_2.0
hasan7n fe1d353
merge branch fets_2.0
hasan7n d2f5abc
Update Dockerfile
sarthakpati 0ff9b41
Update Dockerfile
sarthakpati 99aefe4
replacing string containment with string non-equality
brandon-edwards d0ed8fc
replacing string containment with string non-equality
brandon-edwards 0939398
use tumor files names from dict
hasan7n f246be3
use tumor files names from dict
hasan7n 436a4c2
fix name conventions for extraction models outputs
hasan7n 24d492d
fix name conventions for extraction models outputs
hasan7n 6afa8ce
revert back to non openvino model
hasan7n 2ba17d5
revert back to non openvino model
hasan7n 07ea0f2
remove file extension from tumor mask ID
hasan7n c0125aa
remove file extension from tumor mask ID
hasan7n 1bd29e2
Merge remote-tracking branch 'hasan7n/separate-stages-copied-branch' …
hasan7n 67452ab
Merge remote-tracking branch 'hasan7n/separate-stages-copied-branch' …
hasan7n 2fca4ca
missing and extra modalities detected by string for subject and timep…
brandon-edwards c2d0cdb
missing and extra modalities detected by string for subject and timep…
brandon-edwards 84f8d98
Merge branch 'separate-stages-copied-branch' of https://github.com/ha…
brandon-edwards fed4063
Merge branch 'separate-stages-copied-branch' of https://github.com/ha…
brandon-edwards 696ce37
Merge pull request #1 from hasan7n/separate-stages-copied-branch
aristizabal95 eb82322
Merge pull request #1 from hasan7n/separate-stages-copied-branch
aristizabal95 8ee2bb5
Implement comparison
aristizabal95 b60d3b9
Implement comparison
aristizabal95 d3cdc7c
Fix manual step
aristizabal95 a62b4cc
Fix manual step
aristizabal95 3cb4b6e
Update itcr_connectivity.md
sarthakpati 8efaf43
Sync data to input data. Implement confirm stage
aristizabal95 0737cb2
Sync data to input data. Implement confirm stage
aristizabal95 a7a02c2
Fix manual step trying to inspect nonexistent path
aristizabal95 8aff52b
Fix manual step trying to inspect nonexistent path
aristizabal95 2bcc6a3
Fix report using incorrect out path
aristizabal95 f1719c0
Fix report using incorrect out path
aristizabal95 a1fa8e2
Fix report being created with wrong voxels column
aristizabal95 134a181
Fix report being created with wrong voxels column
aristizabal95 7608e58
Add comment on exact match
aristizabal95 73420d6
Add comment on exact match
aristizabal95 01f28ce
Implement confirm stage (Auto-confirm for now)
aristizabal95 b7926cc
Implement confirm stage (Auto-confirm for now)
aristizabal95 09728b9
Implement split stage
aristizabal95 e66b65a
Implement split stage
aristizabal95 58ace77
Implement pipeline. Fix bugs
aristizabal95 5968853
Implement pipeline. Fix bugs
aristizabal95 5a00b18
Fix stages not returning success status
aristizabal95 1493942
Fix stages not returning success status
aristizabal95 234e9a5
Separate paths by data and labels
aristizabal95 f4ce377
Separate paths by data and labels
aristizabal95 d1e1ece
Implement folder structure validation
aristizabal95 668f591
Implement folder structure validation
aristizabal95 86a593f
Fix pipeline. Implement file-based input.
aristizabal95 363d976
Fix pipeline. Implement file-based input.
aristizabal95 07a4374
Implement empty statistics task
aristizabal95 8ae2953
Implement empty statistics task
aristizabal95 6884c79
Merge branch 'fets_2.0' into separate-stages
aristizabal95 29215d0
Merge branch 'fets_2.0' into separate-stages
aristizabal95 70e3499
Remove outdated data.csv
aristizabal95 905d43b
Remove outdated data.csv
aristizabal95 b9a2cc8
Generate extra split files
aristizabal95 2f4fee2
Generate extra split files
aristizabal95 4297da4
Fix nift not deleting previous folder
aristizabal95 14f92a1
Fix nift not deleting previous folder
aristizabal95 57f4ccf
Skip processing if input data is already prepared
aristizabal95 fb26d06
Skip processing if input data is already prepared
aristizabal95 6f5cd63
Handle errors when validating data
aristizabal95 cd845ca
Handle errors when validating data
aristizabal95 7bd6488
Only display error message on failed val
aristizabal95 861728e
Only display error message on failed val
aristizabal95 619b2ab
Merge branch 'master' of https://github.com/FeTS-AI/Front-End into se…
aristizabal95 ed15ba0
Merge branch 'master' of https://github.com/FeTS-AI/Front-End into se…
aristizabal95 174ef26
Add metadata path
aristizabal95 0e8e950
Add metadata path
aristizabal95 6d16602
Add metadata to all tasks. Implement basic stats
aristizabal95 f253a8b
Add metadata to all tasks. Implement basic stats
aristizabal95 f1f56fc
added fields from EC
sarthakpati 506e44f
added fields from EC
sarthakpati b4d24dc
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 5f5f1de
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 0a4fbf6
added more fields
sarthakpati 379074a
Start working on nnunet stage
aristizabal95 957ca68
Implement extract nnunet
aristizabal95 c428ff0
Handle failure scenarios
aristizabal95 bf9b086
Modularize tumor extraction
aristizabal95 b7bdd49
Add tumor fusing and screenshots
aristizabal95 600d4d8
ignore models folder
aristizabal95 0f26cff
fix git ignore models
aristizabal95 ed3ffba
Don't skip nnunet models
aristizabal95 7e1693c
Decouple mlcube from report details
aristizabal95 f6f9c7b
Use empty vs blank stage info
aristizabal95 67c527e
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 020b911
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 337a854
Removed unused report assignments
aristizabal95 a4751a2
Update instructions for the manual steps
aristizabal95 eb90f88
Change name from reviewed to finalized
aristizabal95 36588f9
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 c6115b6
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 529667f
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 3772b88
Fix EOL error for string literal
aristizabal95 931efdd
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 d61f979
syntax fix
sarthakpati c43975b
Implement brain mask correction rollback
aristizabal95 1ac185d
Update instructions to use finalized folder
aristizabal95 e254fab
Pass hash of index for anonimization
aristizabal95 ca46628
Remove trash during execution
aristizabal95 a119915
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 c1cdc89
Pass stages to medperf through parameters
aristizabal95 e837c7e
Revert "Pass stages to medperf through parameters"
aristizabal95 e3eb58b
Pass report stages to medperf through parameters
aristizabal95 cf6c217
Implement unhandled exception report
aristizabal95 8bdff96
Fix unhandled exceptions being handled
aristizabal95 a070b9d
Check for segmentation hash
aristizabal95 654656e
Fix unhandled error report
aristizabal95 0531d07
Fix statistics showing non-anonymized data
aristizabal95 1b5d8b9
Fix extract failing to hide already existing files
aristizabal95 24165ea
Fix rollback not being handled correctly
aristizabal95 81534eb
Fix stage not working when no stage is identified
aristizabal95 976615a
Fix comparison only testing hash if file exists
aristizabal95 9f1bd9d
Handle review edge cases
aristizabal95 3e9ffb0
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 7a6366b
Import from extract for nnUNet
aristizabal95 d79d5e3
Remove report dependency. Handle empty stats
aristizabal95 d85b430
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 08a9032
Sync generate_report with parent
aristizabal95 f47a8a5
Remove stages file
aristizabal95 bca838b
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 884a035
fix some permission issues inside the container
hasan7n 644aaf6
Handle alternative folder structures
aristizabal95 401319a
Merge branch 'fix-container-perms' of https://github.com/hasan7n/Fron…
aristizabal95 87fa69a
Support semi-prepared input data
aristizabal95 72e9126
Handle edge cases for semi-prepared data
aristizabal95 5f53fd9
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 c97597d
Pass models through tarball. Fix skip issues
aristizabal95 c6f54af
Merge pull request #3 from aristizabal95/nnunet
aristizabal95 c8a0f45
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 a453733
Remove parameter for testing models
aristizabal95 3294eb9
Add logic for invalidating subjects
aristizabal95 d207eee
Fix recursive structure returning wrong path
aristizabal95 4ec51e9
Fix report not being updated after each stage exec
aristizabal95 7eee09f
Update mlcube name
aristizabal95 a28348f
Fix permission issues
aristizabal95 dad972a
fix the permissions thing
hasan7n 63d6220
remove reportfile from sanity check input
hasan7n edb3261
Don't fail silently for extract stage
aristizabal95 0e0090e
Merge branch 'separate-stages' of https://github.com/aristizabal95/Fe…
aristizabal95 238b297
Revert submodule
aristizabal95 82b3bde
Merge pull request #74 from aristizabal95/separate-stages
sarthakpati 16293bd
Fix issue with hardcoded input paths
aristizabal95 e34bd1e
Fix datatype conversion on csv read
aristizabal95 9f9a7b7
Fix gt not found error. Create mlcube constants
aristizabal95 1114439
update is_done functions
aristizabal95 e4e0d21
Remove resolved TODO
aristizabal95 127d7bd
Fix csv stage using wrong status code
aristizabal95 35b97ea
gpu support
hasan7n fe05957
Merge pull request #77 from aristizabal95/separate-stages
sarthakpati 64818a4
add cuda visible devices env var
hasan7n e64bdd9
Generate dataset statistics instead of DICOM ones
aristizabal95 91358e7
added a check for nifti files
sarthakpati 47218ae
Merge branch 'fets_2.0' of https://github.com/FeTS-AI/Front-End into …
aristizabal95 ebb7edb
Fix non-preprocessed NIfTIs not being detected
aristizabal95 cb90aab
Merge pull request #78 from aristizabal95/separate-stages
sarthakpati 0bc3aa1
Update Dockerfile
sarthakpati 51d4c25
Write documentation for RANO
aristizabal95 405b89d
Remove unused image
aristizabal95 c58b76a
Merge pull request #80 from aristizabal95/rano_docs
sarthakpati 69b1dd6
Merge branch 'master' of https://github.com/FeTS-AI/Front-End into fe…
sarthakpati dd1dab9
end of file updated
sarthakpati 3df09a5
Merge pull request #82 from FeTS-AI/fets_2.0_master_sync
sarthakpati d11351d
Update Dockerfile
sarthakpati 6f374ec
using an environment variable and falling back to temp dir
sarthakpati File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,9 +11,9 @@ SET( ${PROJECT_NAME}_Variant "Full" ) # the particular variant of CaPTk (Full/Ne | |
| # So small changes here to formatting or naming convention *may potentially break the linux packager*! | ||
| # STRING(TIMESTAMP TODAY "%Y%m%d") | ||
|
|
||
| SET( PROJECT_VERSION_MAJOR 1 ) | ||
| SET( PROJECT_VERSION_MAJOR 2 ) | ||
| SET( PROJECT_VERSION_MINOR 0 ) | ||
| SET( PROJECT_VERSION_PATCH 2 ) | ||
| SET( PROJECT_VERSION_PATCH 0 ) | ||
| SET( PROJECT_VERSION_TWEAK ) | ||
|
|
||
| # check for the string "nonRelease" in the PROJECT_VERSION_PATCH variable | ||
|
|
@@ -157,12 +157,12 @@ IF(${PROJECT_NAME}_Variant MATCHES "Full" ) | |
| ELSE() | ||
| SET( PROJECT_NAME_EXTENDED "Federated Tumor Segmentation (${PROJECT_NAME}_${${PROJECT_NAME}_Variant})" ) | ||
| ENDIF() | ||
| SET( PROJECT_VENDOR "CBICA - UPenn" ) | ||
| SET( PROJECT_VENDOR "The FeTS-AI Group" ) | ||
| SET( LICENSE_FILE "${PROJECT_SOURCE_DIR}/LICENSE" ) | ||
| SET( README_FILE "${PROJECT_SOURCE_DIR}/README.txt" ) | ||
| SET( PROJECT_DOMAIN "https://www.med.upenn.edu/cbica/captk/" ) | ||
| SET( PROJECT_CONTACT "[email protected]" ) | ||
| SET( COPYRIGHT "Copyright (c) 2019 CBICA. All rights reserved.") | ||
| SET( PROJECT_DOMAIN "https://www.fets.ai" ) | ||
| SET( PROJECT_CONTACT "admin@fets" ) | ||
| SET( COPYRIGHT "Copyright (c) 2023 FeTS-AI. All rights reserved.") | ||
| SET( IDENTIFIER "") | ||
| ADD_DEFINITIONS(-DPROJECT_NAME="${PROJECT_NAME}" ) | ||
| ADD_DEFINITIONS(-DPROJECT_VERSION="${PROJECT_VERSION}" ) | ||
|
|
@@ -544,95 +544,95 @@ IF( NOT BUILD_DOCUMENTATION_ONLY ) | |
| ${LIBNAME_Applications} | ||
| ) | ||
|
|
||
| # update submodule | ||
| IF( NOT EXISTS "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/setup.py" ) | ||
| EXECUTE_PROCESS(COMMAND git submodule update --init --recursive | ||
| WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} | ||
| RESULT_VARIABLE git_result | ||
| OUTPUT_VARIABLE git_ver) | ||
| ENDIF() | ||
| # # update submodule | ||
| # IF( NOT EXISTS "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/setup.py" ) | ||
| # EXECUTE_PROCESS(COMMAND git submodule update --init --recursive | ||
| # WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} | ||
| # RESULT_VARIABLE git_result | ||
| # OUTPUT_VARIABLE git_ver) | ||
| # ENDIF() | ||
|
|
||
| # update submodule | ||
| IF( NOT EXISTS "${PROJECT_SOURCE_DIR}/Models_Pretrained/README.md" ) | ||
| EXECUTE_PROCESS(COMMAND git submodule update --init --recursive | ||
| WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} | ||
| RESULT_VARIABLE git_result | ||
| OUTPUT_VARIABLE git_ver) | ||
| ENDIF() | ||
| # # update submodule | ||
| # IF( NOT EXISTS "${PROJECT_SOURCE_DIR}/Models_Pretrained/README.md" ) | ||
| # EXECUTE_PROCESS(COMMAND git submodule update --init --recursive | ||
| # WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} | ||
| # RESULT_VARIABLE git_result | ||
| # OUTPUT_VARIABLE git_ver) | ||
| # ENDIF() | ||
|
|
||
| # get lfs to pull | ||
| IF( EXISTS "${PROJECT_SOURCE_DIR}/BrainMaGe" ) | ||
| EXECUTE_PROCESS(COMMAND git lfs pull | ||
| WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/BrainMaGe | ||
| RESULT_VARIABLE git_result | ||
| OUTPUT_VARIABLE git_ver) | ||
| ENDIF() | ||
| # # get lfs to pull | ||
| # IF( EXISTS "${PROJECT_SOURCE_DIR}/BrainMaGe" ) | ||
| # EXECUTE_PROCESS(COMMAND git lfs pull | ||
| # WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/BrainMaGe | ||
| # RESULT_VARIABLE git_result | ||
| # OUTPUT_VARIABLE git_ver) | ||
| # ENDIF() | ||
|
|
||
| # copy plans from algorithms | ||
| IF( EXISTS "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/submodules/fets_ai/Algorithms/fets/bin/federations/plans/pt_3dresunet_brainmagebrats.yaml" ) | ||
| FILE( GLOB FETS_AI_PLANS | ||
| "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/submodules/fets_ai/Algorithms/fets/bin/federations/plans/*.yaml" | ||
| ) | ||
| FOREACH(fetsai_plan ${FETS_AI_PLANS}) | ||
| CONFIGURE_FILE("${fetsai_plan}" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/" COPYONLY) | ||
| ENDFOREACH() | ||
| ENDIF() | ||
| # # copy plans from algorithms | ||
| # IF( EXISTS "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/submodules/fets_ai/Algorithms/fets/bin/federations/plans/pt_3dresunet_brainmagebrats.yaml" ) | ||
| # FILE( GLOB FETS_AI_PLANS | ||
| # "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/submodules/fets_ai/Algorithms/fets/bin/federations/plans/*.yaml" | ||
| # ) | ||
| # FOREACH(fetsai_plan ${FETS_AI_PLANS}) | ||
| # CONFIGURE_FILE("${fetsai_plan}" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/" COPYONLY) | ||
| # ENDFOREACH() | ||
| # ENDIF() | ||
|
|
||
| # copy weights from models | ||
| SET( DOWNLOAD_LINK "ftp://www.nitrc.org/home/groups/captk/downloads/models/torch/pt_3dresunet_ss_brainmagebrats_best.pt" ) | ||
| SET( FILE_TO_EXTRACT "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/weights/pt_3dresunet_ss_brainmagebrats_best.pt") | ||
| IF( NOT EXISTS "${FILE_TO_EXTRACT}" ) | ||
|
|
||
| # download exe from url | ||
| MESSAGE( STATUS "Downloading weights for skull stripping" ) | ||
| FILE(DOWNLOAD "${DOWNLOAD_LINK}" "${FILE_TO_EXTRACT}" TIMEOUT 1000000 STATUS STATUS_CODE SHOW_PROGRESS) | ||
| IF(NOT STATUS_CODE EQUAL 0) | ||
| MESSAGE(FATAL_ERROR "Failed to download skull-stripping weights. Status=${STATUS_CODE}") | ||
| ENDIF() | ||
| ENDIF() | ||
| # # copy weights from models | ||
| # SET( DOWNLOAD_LINK "ftp://www.nitrc.org/home/groups/captk/downloads/models/torch/pt_3dresunet_ss_brainmagebrats_best.pt" ) | ||
| # SET( FILE_TO_EXTRACT "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/weights/pt_3dresunet_ss_brainmagebrats_best.pt") | ||
| # IF( NOT EXISTS "${FILE_TO_EXTRACT}" ) | ||
|
|
||
| # # download exe from url | ||
| # MESSAGE( STATUS "Downloading weights for skull stripping" ) | ||
| # FILE(DOWNLOAD "${DOWNLOAD_LINK}" "${FILE_TO_EXTRACT}" TIMEOUT 1000000 STATUS STATUS_CODE SHOW_PROGRESS) | ||
| # IF(NOT STATUS_CODE EQUAL 0) | ||
| # MESSAGE(FATAL_ERROR "Failed to download skull-stripping weights. Status=${STATUS_CODE}") | ||
| # ENDIF() | ||
| # ENDIF() | ||
|
|
||
| # copy network information | ||
| ## todo: change to a direct installation so that git doesn't get confused that submodule has changed | ||
| CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/network.yaml" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/defaults/network.yaml") | ||
| CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/cert_chain.crt" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/pki/cert_chain.crt") | ||
| #CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/pt_3dresunet_brainmagebrats.yaml" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/pt_3dresunet_brainmagebrats.yaml" COPYONLY) | ||
| CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/deepscan_inference.yaml" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/deepscan_inference.yaml" COPYONLY) | ||
| CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/nnunet_inference.yaml" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/nnunet_inference.yaml" COPYONLY) | ||
| # # copy network information | ||
| # ## todo: change to a direct installation so that git doesn't get confused that submodule has changed | ||
| # CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/network.yaml" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/defaults/network.yaml") | ||
| # CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/cert_chain.crt" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/pki/cert_chain.crt") | ||
| # #CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/pt_3dresunet_brainmagebrats.yaml" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/pt_3dresunet_brainmagebrats.yaml" COPYONLY) | ||
| # CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/deepscan_inference.yaml" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/deepscan_inference.yaml" COPYONLY) | ||
| # CONFIGURE_FILE("${PROJECT_SOURCE_DIR}/data/fets/nnunet_inference.yaml" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/plans/nnunet_inference.yaml" COPYONLY) | ||
|
|
||
| # find stand-alone CMD applications | ||
| FILE( GLOB DeepScan_Weights | ||
| "${PROJECT_SOURCE_DIR}/data/fets/*.tar" | ||
| ) | ||
| file(MAKE_DIRECTORY "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/weights/deepscan/") | ||
| FOREACH(deepScan_weight ${DeepScan_Weights}) | ||
| CONFIGURE_FILE("${deepScan_weight}" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/weights/deepscan/" COPYONLY) | ||
| ENDFOREACH() | ||
|
|
||
| INSTALL( | ||
| DIRECTORY "${PROJECT_SOURCE_DIR}/OpenFederatedLearning" | ||
| DESTINATION bin | ||
| PATTERN ".git" EXCLUDE | ||
| PATTERN "venv" EXCLUDE | ||
| ) | ||
| # # find stand-alone CMD applications | ||
| # FILE( GLOB DeepScan_Weights | ||
| # "${PROJECT_SOURCE_DIR}/data/fets/*.tar" | ||
| # ) | ||
| # file(MAKE_DIRECTORY "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/weights/deepscan/") | ||
| # FOREACH(deepScan_weight ${DeepScan_Weights}) | ||
| # CONFIGURE_FILE("${deepScan_weight}" "${PROJECT_SOURCE_DIR}/OpenFederatedLearning/bin/federations/weights/deepscan/" COPYONLY) | ||
| # ENDFOREACH() | ||
|
|
||
| # INSTALL( | ||
| # DIRECTORY "${PROJECT_SOURCE_DIR}/OpenFederatedLearning" | ||
| # DESTINATION bin | ||
| # PATTERN ".git" EXCLUDE | ||
| # PATTERN "venv" EXCLUDE | ||
| # ) | ||
|
|
||
| INSTALL( | ||
| DIRECTORY "${PROJECT_SOURCE_DIR}/LabelFusion" | ||
| DESTINATION bin | ||
| PATTERN ".git" EXCLUDE | ||
| PATTERN "venv" EXCLUDE | ||
| ) | ||
| # INSTALL( | ||
| # DIRECTORY "${PROJECT_SOURCE_DIR}/LabelFusion" | ||
| # DESTINATION bin | ||
| # PATTERN ".git" EXCLUDE | ||
| # PATTERN "venv" EXCLUDE | ||
| # ) | ||
|
|
||
| INSTALL( | ||
| DIRECTORY "${PROJECT_SOURCE_DIR}/BrainMaGe" | ||
| DESTINATION bin | ||
| PATTERN ".git" EXCLUDE | ||
| PATTERN "venv" EXCLUDE | ||
| ) | ||
| # INSTALL( | ||
| # DIRECTORY "${PROJECT_SOURCE_DIR}/BrainMaGe" | ||
| # DESTINATION bin | ||
| # PATTERN ".git" EXCLUDE | ||
| # PATTERN "venv" EXCLUDE | ||
| # ) | ||
|
|
||
|
|
||
| FILE( GLOB_RECURSE ALL_PY_FILES "${PROJECT_SOURCE_DIR}/src/applications/*.py" ) | ||
| FILE( GLOB ALL_PY_FILES "${PROJECT_SOURCE_DIR}/src/applications/*.py" ) | ||
| FOREACH( pyFile ${ALL_PY_FILES} ) | ||
| INSTALL( FILES "${pyFile}" DESTINATION bin ) | ||
| INSTALL( FILES "${pyFile}" DESTINATION bin PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) | ||
| ENDFOREACH() | ||
|
|
||
| # find stand-alone CMD applications | ||
|
|
@@ -1046,9 +1046,9 @@ IF( NOT BUILD_DOCUMENTATION_ONLY ) | |
| SET(CPACK_NSIS_ENABLE_UNINSTALL_BEFORE_INSTALL ON) | ||
| SET(CPACK_NSIS_MUI_ICON "${DATA_DIR}/icons/application/windows/FeTS.ico") | ||
| SET(CPACK_NSIS_MUI_FINISHPAGE_RUN "${EXE_NAME}.exe") | ||
| SET(CPACK_NSIS_HELP_LINK "https:\\\\\\\\www.med.upenn.edu\\\\cbica\\\\captk\\\\") | ||
| SET(CPACK_NSIS_HELP_LINK "https:\\\\\\\\www.fets.ai\\\\") | ||
| SET(CPACK_NSIS_CONTACT "${PROJECT_CONTACT}") | ||
| SET(CPACK_NSIS_URL_INFO_ABOUT "https:\\\\\\\\www.med.upenn.edu\\\\cbica\\\\captk\\\\") | ||
| SET(CPACK_NSIS_URL_INFO_ABOUT "https:\\\\\\\\www.fets.ai\\\\") | ||
| #SET(CPACK_NSIS_MODIFY_PATH "ON") # desktop shortcut doesn't work, for some reason | ||
| # create extra shortcuts and respective uninstall targets | ||
| SET(CPACK_NSIS_CREATE_ICONS_EXTRA "CreateShortCut '\$SMPROGRAMS\\\\$STARTMENU_FOLDER\\\\${EXE_NAME}.lnk' '\$INSTDIR\\\\bin\\\\${EXE_NAME}.exe'") | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,58 +1,97 @@ | ||
| FROM pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime | ||
| FROM ghcr.io/fets-ai/fetstool_docker_dependencies AS fets_base | ||
|
|
||
| LABEL authors="FeTS_Admin <[email protected]>" | ||
|
|
||
| RUN apt-get update -y | ||
| RUN apt-get update && apt-get update --fix-missing && apt-get install -y libnss3 libnspr4 libxcursor1 libxcursor-dev libasound2 libdbus-1-dev libglfw3-dev libgles2-mesa-dev ffmpeg libsm6 libxext6 python3.8 python3.8-venv python3.8-dev python3-setuptools | ||
|
|
||
| RUN apt-get install wget zip unzip software-properties-common gcc g++ make -y | ||
| ENV PATH=/CaPTk/bin/qt/5.12.1/bin:/CaPTk/bin/qt/5.12.1/libexec:$PATH | ||
| ENV CMAKE_PREFIX_PATH=/CaPTk/bin/ITK-build:/CaPTk/bin/DCMTK-build:/CaPTk/bin/qt/5.12.1/lib/cmake/Qt5:$CMAKE_PREFIX_PATH | ||
|
|
||
| RUN apt-get update -y && add-apt-repository ppa:deadsnakes/ppa && apt update -y && apt install python3.7 python3.7-venv python3.7-dev python3-setuptools ffmpeg libsm6 libxext6 -y | ||
| RUN pwd && ls -l | ||
|
|
||
| # We will do git pull on the FeTS_Front-End master, because that is the repo using which the base image is made | ||
| # We will not do compiles on the PR because the idea is that the Xenial build will check the build status of | ||
| # the PR in any case. | ||
| WORKDIR /Front-End | ||
|
|
||
| ARG VERSION=0.0.9 | ||
| # Download model checkpoints to torch checkpoint location | ||
| # https://pytorch.org/docs/stable/hub.html#where-are-my-downloaded-models-saved | ||
| ENV TORCH_HOME="/.pytorch_cache" | ||
| RUN mkdir -p $TORCH_HOME/hub/checkpoints && \ | ||
| wget -O $TORCH_HOME/hub/checkpoints/dpn98-722954780.pth http://data.lip6.fr/cadene/pretrainedmodels/dpn98-722954780.pth --no-check-certificate && \ | ||
| wget -O $TORCH_HOME/hub/checkpoints/resnet50-19c8e357.pth https://download.pytorch.org/models/resnet50-19c8e357.pth | ||
|
|
||
| # download installer | ||
| RUN wget https://fets.projects.nitrc.org/FeTS_${VERSION}_Installer.bin && chmod +x FeTS_${VERSION}_Installer.bin | ||
| COPY src src | ||
|
|
||
| # install FeTS and remove installer | ||
| RUN yes yes | ./FeTS_${VERSION}_Installer.bin --target ./FeTS_${VERSION} -- --cudaVersion 11 && rm -rf ./FeTS_${VERSION}_Installer.bin | ||
| COPY CMakeLists.txt README.txt LICENSE . | ||
|
|
||
| ENV PATH=/workspace/FeTS_${VERSION}/squashfs-root/usr/bin/:$PATH | ||
| ENV LD_LIBRARY_PATH=/workspace/FeTS_${VERSION}/squashfs-root/usr/lib/:$LD_LIBRARY_PATH | ||
| ENV SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL="True" | ||
| COPY cmake_modules cmake_modules | ||
|
|
||
| # set up environment and install correct version of pytorch | ||
| RUN cd ./FeTS_${VERSION}/squashfs-root/usr/bin/OpenFederatedLearning && \ | ||
| rm -rf ./venv && python3.7 -m venv ./venv && ./venv/bin/pip install Cython && \ | ||
| ./venv/bin/pip install --upgrade pip setuptools wheel setuptools-rust && \ | ||
| ./venv/bin/pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html | ||
| COPY data data | ||
|
|
||
| RUN cd ./FeTS_${VERSION}/squashfs-root/usr/bin/OpenFederatedLearning && \ | ||
| ./venv/bin/pip install wheel && \ | ||
| ./venv/bin/pip install scikit-build scikit-learn && \ | ||
| ./venv/bin/pip install SimpleITK==1.2.4 && \ | ||
| ./venv/bin/pip install protobuf==3.17.3 grpcio==1.30.0 && \ | ||
| ./venv/bin/pip install opencv-python==4.2.0.34 | ||
| # ./venv/bin/pip install python-gdcm | ||
| COPY docs_sources docs_sources | ||
|
|
||
| RUN cd ./FeTS_${VERSION}/squashfs-root/usr/bin/OpenFederatedLearning && \ | ||
| ./venv/bin/pip install setuptools --upgrade && \ | ||
| make install_openfl && \ | ||
| make install_fets && \ | ||
| ./venv/bin/pip install -e ./submodules/fets_ai/Algorithms/GANDLF && \ | ||
| cd ../LabelFusion && \ | ||
| rm -rf venv && python3.7 -m venv ./venv && \ | ||
| ./venv/bin/pip install --upgrade pip setuptools wheel setuptools-rust && \ | ||
| ./venv/bin/pip install -e . | ||
| RUN pwd && ls -l | ||
|
|
||
| ## C++ build | ||
| RUN mkdir bin && cd bin && cmake -DCMAKE_INSTALL_PREFIX="./install/appdir/usr" -DITK_DIR="/CaPTk/bin/ITK-build" -DDCMTK_DIR="/CaPTk/bin/DCMTK-build" -DBUILD_TESTING=OFF .. && make -j$(nproc) && make install/strip | ||
|
|
||
| ## Python package installation | ||
| RUN apt-get install software-properties-common curl -y && \ | ||
| add-apt-repository ppa:deadsnakes/ppa -y && apt-get update && \ | ||
| apt-get install python3.8 python3.8-distutils -y && \ | ||
| apt-get remove --purge python3.6 -y && \ | ||
| apt autoremove -y && \ | ||
| apt-get install python3.8-distutils -y && \ | ||
| rm -fr /usr/bin/python /usr/bin/python3 /usr/bin/pip /usr/bin/pip3 && \ | ||
| ln -s /usr/bin/python3.8 /usr/bin/python && ln -s /usr/bin/python3.8 /usr/bin/python3 && \ | ||
| ln -s /usr/bin/pip3.8 /usr/bin/pip && ln -s /usr/bin/pip3.8 /usr/bin/pip3 | ||
|
|
||
| RUN curl -fSsL -O https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \ | ||
| python3.8 get-pip.py && rm get-pip.py | ||
|
|
||
| RUN cd bin/install/appdir/usr/bin/ && pip install --upgrade pip wheel && pip install torch==1.13.1+cpu torchvision==0.14.1+cpu torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cpu && pip install -e . && pip install setuptools-rust Cython scikit-build scikit-learn openvino==2023.0.1 openvino-dev==2023.0.1 && pip install -e . | ||
|
|
||
| ### put together a data example that is already aligned and ready to invoke the brain extraction and tumor segmentation | ||
|
|
||
| # set up the docker for GUI | ||
| ENV LD_LIBRARY_PATH=/CaPTk/bin/qt/5.12.1/lib:$LD_LIBRARY_PATH | ||
| ENV PATH=/Front-End/bin/install/appdir/usr/bin/:$PATH | ||
| ENV QT_X11_NO_MITSHM=1 | ||
| ENV QT_GRAPHICSSYSTEM="native" | ||
|
|
||
| RUN echo "Env paths\n" && echo $PATH && echo $LD_LIBRARY_PATH | ||
|
|
||
| # define entry point | ||
| ENTRYPOINT ["/workspace/FeTS_0.0.9/squashfs-root/usr/bin/FeTS_CLI_Inference"] | ||
| ENTRYPOINT ["python", "/Front-End/bin/install/appdir/usr/bin/PrepareDataset.py"] | ||
|
|
||
| FROM fets_base AS data_prep | ||
|
|
||
| RUN find /Front-End/bin/install/appdir/usr/bin -type f \( -perm -u=x -o -type l \) -exec cp -P {} /usr/bin \; | ||
|
|
||
| WORKDIR / | ||
|
|
||
| COPY ./mlcubes/data_preparation/project/requirements.txt /project/requirements.txt | ||
|
|
||
| RUN pip install --upgrade pip | ||
|
|
||
| RUN pip install -r /project/requirements.txt | ||
|
|
||
| ENV LANG C.UTF-8 | ||
|
|
||
| RUN mkdir /project/stages | ||
|
|
||
| RUN cp /Front-End/bin/install/appdir/usr/bin/*.py /project/stages/ | ||
|
|
||
| RUN cp -R /Front-End/bin/install/appdir/usr/bin/data_prep_models /project/stages/data_prep_models | ||
|
|
||
| # Hotfix: install more recent version of GaNDLF for metrics generation | ||
| RUN pip install git+https://github.com/mlcommons/GaNDLF@616b37bafad8f89d5c816a88f44fa30470601311 | ||
|
|
||
| RUN pip install torch torchvision | ||
|
|
||
| RUN pip install git+https://github.com/MIC-DKFZ/nnUNet.git@nnunetv1 | ||
|
|
||
| ENV nnUNet_raw_data_base="/tmp/nnUNet_raw_data_base" | ||
| ENV nnUNet_preprocessed="/tmp/nnUNet_preprocessed" | ||
|
|
||
| COPY ./mlcubes/data_preparation/project /project | ||
|
|
||
| ENTRYPOINT ["python", "/project/mlcube.py"] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should there be some other (or additional) check(s) here, @aristizabal95 @hasan7n? Maybe something specific to mlcubes?