Support for adapter-specifc pre/post processing through the adapter management APIs

[Lokiiiiii]

Description

Customers are consolidating their model deployments utilizing Multi-Lora to enhance efficiency through multi-tenancy. However, they have distinct pre/post processing requirements for each adapter. Given that the pre/post processing logic is tied to the adapter, it is necessary to manage them as a cohesive unit, ensuring they are created, updated, and deleted in unison.

What’s missing ? Currently, LMI supports custom pre/post processing scripts that are specified statically during the deployment of base models. However, LMI lacks the capability to dynamically register pre/post processing scripts at the time of adapter registration.

Design

1. Check for model.py & custom formatters when registering adapters
2. During inference adapter specific formatters override the base model formatters if specified

Additional

1. New integ tests added and executed - https://github.com/deepjavalibrary/djl-serving/actions/runs/19585974385
2. Existing documentation updated and new documentation added
3. Examples/demos WIP

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Checklist:

• Please add the link of Integration Tests Executor run with related tests.
• Have you manually built the docker image and verify the change?
• Have you run related tests? Check how to set up the test environment here; One example would be pytest tests.py -k "TestVllm1" -m "vllm"
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

• Test A
  Logs for Test A

• Test B
  Logs for Test B

[Lokiiiiii]

This file is a just a refactor of the adapter code that was in async_utilts before.

[ksuma2109]

Custom Formatter specific code LGTM