Optimize main.py for inference efficiency and GPU throughput (torch.compile, memory tuning, warp alignment)#253

abdullatifcodes wants to merge 1 commit intomistralai:mainmistralai/mistral-inference:mainfrom

abdullatifcodes:perf/main-inference-optimizationabdullatifcodes/mistral-inference:perf/main-inference-optimizationCopy head branch name to clipboard