feat: generate response length based on a histogram when max_tokens is defined in the request#169

mayabar merged 6 commits intollm-d:mainllm-d/llm-d-inference-sim:mainfrom

mayabar:max_tokensmayabar/llm-d-inference-sim:max_tokensCopy head branch name to clipboard