Batch launcher script for, koboldcpp.py and koboldcpp.exe.

[wiseman-timelord]

DESCRIPTION:
Here is the demonstration of my batch launcher menu I knocked up with gpt3.5..if you are having issues getting the correct launch parameters or just plain lazy like me, def give this a go.

START OUTPUT..

Configure Kobold CPP Launch...

Choose an option:

1. Run Python Version
2. Run Binary Version

Enter your choice (1 or 2): 2

Enter the Number of Threads: 20

Calculating Blas Threads...
Number of BlasThreads = 20

Enter the Context Size: 8192

Calculating Blas Batch...
BlasBatchSize = 1024

AMD GPU Present (y/n): y

Configuration Complete...

Running Binary Version with Context 8192, Batch 1024, Threads 20, and ClBlas 0 0...

..END OUTPUT

NOTES:
BlasBatch is 1/4 context, with max of 1024, this is to keep most people happy with 2048 context models having a BlasBatch of 512, and also 1024 is the limit.

INSTRUCTIONS:

1. Ensure both, source and exe, are installed into the koboldcpp directory, for full features (always good to have choice).
2. Copy the script below into a file named "run.bat" saved into koboldcpp folder.

BEGIN "run.bat" SCRIPT..

@echo off
cls
Configure Kobold CPP Launch...
timeout /t 2 >nul
echo.
echo.

:MENU
echo Choose an option:
echo 1. Run Python Version
echo 2. Run Binary Version
echo.

set /P choice="Enter your choice (1 or 2): "

if "%choice%"=="1" (
set version=python
) else if "%choice%"=="2" (
set version=binary
) else (
echo Invalid choice. Please try again.
echo.
goto MENU
)

echo.

set /P numthreads="Enter the Number of Threads: "
echo.
echo Calculating Blas Threads...
echo Number of BlasThreads = %numthreads%
echo.

set /P contextsize="Enter the Context Size: "
echo.
echo Calculating Blas Batch...
set /A "blasbatchsize=contextsize/4"
if %blasbatchsize% gtr 1024 set "blasbatchsize=1024"
echo BlasBatchSize = %blasbatchsize%
echo.

echo Configuration Complete...
timeout /t 2 >nul
echo.
echo.

echo Running Kobold CPP...
timeout /t 2 >nul
echo.
echo.

if "%version%"=="python" (
echo Running Python Version with Context %contextsize%, Batch %blasbatchsize%, Threads %numthreads%, and ClBlas 0 0...
timeout /t 2 >nul
echo.
echo.
@echo on
python koboldcpp.py --contextsize %contextsize% --blasbatchsize %blasbatchsize% --threads %numthreads% --blasthreads %numthreads% --useclblast 0 0
@echo off
echo.
echo Python Version completed.
timeout /t 2 >nul
echo.
echo.
) else (
echo Running Binary Version with Context %contextsize%, Batch %blasbatchsize%, Threads %numthreads%, and ClBlas 0 0...
timeout /t 2 >nul
echo.
echo.
koboldcpp.exe --contextsize %contextsize% --blasbatchsize %blasbatchsize% --threads %numthreads% --blasthreads %numthreads% --useclblast 0 0
timeout /t 2 >nul
echo.
echo.
echo Binary Version completed.
echo.
)

timeout /t 2 >nul
pause

..END "run.bat"

NOTICE:
Developers of Kobold CPP may use this code for their own version of "run.bat", though I would suggest to add a few more features from --help known to be commonly used, relevantly gpt3.5 was quite competent at the task.