Sonic dynamic model loading#25

Draft

trevin-lee wants to merge 26 commits intofastmachinelearning:masterfrom

trevin-lee:SonicDynamicModelLoading

Commits on Oct 13, 2025

RetryAction compiles

Martin
authored and
trevin-lee
committed
Include RetryAction in SonicClientBase

Martin
authored and
trevin-lee
committed
Update PR comments

Martin
authored and
trevin-lee
committed
PR comments, fix fillDescriptions

Martin
authored and
trevin-lee
committed
rebase 15_1_0_pre6

Martin
authored and
trevin-lee
committed
Add update server function for client

Martin
authored and
trevin-lee
committed
Add test for Triton retry action in BuildFile.xml
trevin-lee
committed
Implement retry logic in RetryActionDiffServer and add connectToServer method in TritonClient. Update BuildFile.xml and fix formatting in header files.
trevin-lee
committed
Add RetryActionDiffServer class documentation, implement testing constructor for TritonClient, and update BuildFile.xml to include Catch2 for testing.
trevin-lee
committed
SonicTriton: implement retry action against different server; update tests; remove old cfg
trevin-lee
committed
Refactor RetryActionDiffServer to utilize TritonService for server selection; remove unused parameters and improve documentation.
trevin-lee
committed
rebase 15_1_0_pre6

Martin
authored and
trevin-lee
committed
Fixes to compile

Martin
authored and
trevin-lee
committed
PR comments

Martin
authored and
trevin-lee
committed
Move retry options to customize.py

Martin
authored and
trevin-lee
committed
more clean ups

Martin
authored and
trevin-lee
committed
First draft for server health

Martin
authored and
trevin-lee
committed
remove redundant

Martin
authored and
trevin-lee
committed
Use getBestServer in RetryDiffServerAction

Martin
authored and
trevin-lee
committed
Add dynamic model loading and unloading to TritonService
trevin-lee
committed
Update DynamicModelLoadingProducer to use model input for base class requirements
trevin-lee
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Sonic dynamic model loading#25

Sonic dynamic model loading#25
trevin-lee wants to merge 26 commits intofastmachinelearning:masterfrom
trevin-lee:SonicDynamicModelLoading

Commits on Oct 13, 2025

RetryAction compiles

Include RetryAction in SonicClientBase

Update PR comments

PR comments, fix fillDescriptions

rebase 15_1_0_pre6

Add update server function for client

Add test for Triton retry action in BuildFile.xml

Implement retry logic in RetryActionDiffServer and add connectToServer method in TritonClient. Update BuildFile.xml and fix formatting in header files.

Add RetryActionDiffServer class documentation, implement testing constructor for TritonClient, and update BuildFile.xml to include Catch2 for testing.

SonicTriton: implement retry action against different server; update tests; remove old cfg

Refactor RetryActionDiffServer to utilize TritonService for server selection; remove unused parameters and improve documentation.

rebase 15_1_0_pre6

Fixes to compile

PR comments

Move retry options to customize.py

more clean ups

First draft for server health

remove redundant

Use getBestServer in RetryDiffServerAction

Add dynamic model loading and unloading to TritonService

Update DynamicModelLoadingProducer to use model input for base class requirements

Commits on Nov 2, 2025

Fix formatting and syntax issues

Restrict dynamic model loading to fallback server only

Commits on Nov 3, 2025

Enhance RetryActionDiffServer to utilize fallback server for model loading

Add FallbackModelState structure and enhance model loading/unloading in TritonService

Refactor TritonService to remove unservedModels_ and streamline model handling

Sonic dynamic model loading#25trevin-lee wants to merge 26 commits intofastmachinelearning:masterfastmachinelearning/cmssw:masterfrom trevin-lee:SonicDynamicModelLoadingtrevin-lee/cmssw:SonicDynamicModelLoadingCopy head branch name to clipboard

Commits

Commits on Oct 13, 2025

Commits on Nov 2, 2025

Commits on Nov 3, 2025

Sonic dynamic model loading#25
trevin-lee wants to merge 26 commits intofastmachinelearning:masterfrom
trevin-lee:SonicDynamicModelLoading