GPT-ENGINEER+TEXTGEN 100% Stable LOCAL: script-kiddie FIX , no editing repo files xD

[noxiouscardiumdimidium]

STEPS:

0. (optional) if you haven't tried it, download PYGMALION-AI Metharme 1.3B , a 2.9GB high-context AUTOTASKING ENGINE, add the metharme template to the bottom of your prompt <|system|><|user|>Input<|model|>

1. create prelaunchers

2. launch textgen, confirm api address

3. launch gpt-engineer

4. you're done, have fun

prelauncher script i use for textgen (modify args and gpu settings to meet your individual needs and hardware)

create a fake/dummy/spoofed key, doesn't matter what it is, as long as it matches what you put in gpt-engineer

launch(textgen).sh

#!/bin/bash
export HOST_PORT=7861
export CUDA_VISIBLE_DEVICES=0
export TORCH_USE_CUDA_DSA=1
export TORCH_CUDA_ARCH_LIST=8.6+PTX
export PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.25,max_split_size_mb:128M
export OPENAI_API_KEY="dummy" # export ANY string in this field, including the default textgen key "dummy" it works!

python3 ./server.py --model TheBloke_orca_mini_7B-GPTQ --loader exllama --listen --listen-port 7861 --api --gpu-split 8_20 --trust-remote-code --extensions openai

prelauncher i use for gpt-engineer

launch(gpte).sh

#!/bin/bash
export OPENAI_API_KEY="dummy" # make sure it matches what you export/set in textgen
export OPENAI_API_BASE=http://0.0.0.0:5001/v1 #depending on settings, and ports in use when you launched textgen, this may change slightly if 5001 is in use - confirm base with textgen launch-post, prior to launching gpt-engineer

python3 gpt_engineer/main.py projects/MasterCodeBase/ --temperature 1 --steps tdd+