|
| 1 | +# QA Testing for `kernel create` |
| 2 | + |
| 3 | +This command runs QA testing for all `kernel create` template permutations. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +You will build the CLI, create all template variations, deploy them, and provide invoke commands for manual testing. |
| 8 | + |
| 9 | +## Step 1: Build the CLI |
| 10 | + |
| 11 | +From the cli repository root: |
| 12 | + |
| 13 | +```bash |
| 14 | +make build |
| 15 | +``` |
| 16 | + |
| 17 | +The built binary will be at `./bin/kernel`. |
| 18 | + |
| 19 | +## Step 2: Create QA Directory |
| 20 | + |
| 21 | +Create a timestamped QA directory: |
| 22 | + |
| 23 | +```bash |
| 24 | +QA_DIR="./qa-$(date +%Y%m%d-%H%M%S)" |
| 25 | +mkdir -p "$QA_DIR" |
| 26 | +cd "$QA_DIR" |
| 27 | +``` |
| 28 | + |
| 29 | +## Step 3: Get KERNEL_API_KEY |
| 30 | + |
| 31 | +**STOP and ask the human for their `KERNEL_API_KEY`.** |
| 32 | + |
| 33 | +This is required for all deployments. Store it for use in deploy commands: |
| 34 | + |
| 35 | +```bash |
| 36 | +export KERNEL_API_KEY="<value from human>" |
| 37 | +``` |
| 38 | + |
| 39 | +## Step 4: Create All Templates |
| 40 | + |
| 41 | +Use the built CLI binary with non-interactive flags. The command format is: |
| 42 | + |
| 43 | +```bash |
| 44 | +../bin/kernel create -n <name> -l <language> -t <template> |
| 45 | +``` |
| 46 | + |
| 47 | +### Template Matrix |
| 48 | + |
| 49 | +Here are all valid language + template combinations: |
| 50 | + |
| 51 | +| Language | Template | Folder Name | Needs Env File | Required Env Vars | |
| 52 | +| ---------- | -------------- | ----------------- | -------------- | ------------------------------ | |
| 53 | +| typescript | sample-app | ts-sample-app | No | - | |
| 54 | +| typescript | captcha-solver | ts-captcha-solver | No | - | |
| 55 | +| typescript | stagehand | ts-stagehand | Yes | OPENAI_API_KEY | |
| 56 | +| typescript | computer-use | ts-computer-use | Yes | ANTHROPIC_API_KEY | |
| 57 | +| typescript | magnitude | ts-magnitude | Yes | ANTHROPIC_API_KEY | |
| 58 | +| typescript | cua | ts-cua | Yes | OPENAI_API_KEY | |
| 59 | +| typescript | gemini-cua | ts-gemini-cua | Yes | GOOGLE_API_KEY, OPENAI_API_KEY | |
| 60 | +| python | sample-app | py-sample-app | No | - | |
| 61 | +| python | captcha-solver | py-captcha-solver | No | - | |
| 62 | +| python | browser-use | py-browser-use | Yes | OPENAI_API_KEY | |
| 63 | +| python | computer-use | py-computer-use | Yes | ANTHROPIC_API_KEY | |
| 64 | +| python | cua | py-cua | Yes | OPENAI_API_KEY | |
| 65 | + |
| 66 | +### Create Commands |
| 67 | + |
| 68 | +Run each of these (they are non-interactive when all flags are provided): |
| 69 | + |
| 70 | +```bash |
| 71 | +# TypeScript templates |
| 72 | +../bin/kernel create -n ts-sample-app -l typescript -t sample-app |
| 73 | +../bin/kernel create -n ts-captcha-solver -l typescript -t captcha-solver |
| 74 | +../bin/kernel create -n ts-stagehand -l typescript -t stagehand |
| 75 | +../bin/kernel create -n ts-computer-use -l typescript -t computer-use |
| 76 | +../bin/kernel create -n ts-magnitude -l typescript -t magnitude |
| 77 | +../bin/kernel create -n ts-cua -l typescript -t cua |
| 78 | +../bin/kernel create -n ts-gemini-cua -l typescript -t gemini-cua |
| 79 | + |
| 80 | +# Python templates |
| 81 | +../bin/kernel create -n py-sample-app -l python -t sample-app |
| 82 | +../bin/kernel create -n py-captcha-solver -l python -t captcha-solver |
| 83 | +../bin/kernel create -n py-browser-use -l python -t browser-use |
| 84 | +../bin/kernel create -n py-computer-use -l python -t computer-use |
| 85 | +../bin/kernel create -n py-cua -l python -t cua |
| 86 | +``` |
| 87 | + |
| 88 | +## Step 5: Deploy Each Template |
| 89 | + |
| 90 | +For each template directory, you need to: |
| 91 | + |
| 92 | +1. `cd` into the directory |
| 93 | +2. If `NeedsEnvFile` is true, **STOP and ask the human** for the required API keys |
| 94 | +3. Create a `.env` file with those values |
| 95 | +4. Run the deploy command |
| 96 | + |
| 97 | +### Deploy Commands by Template |
| 98 | + |
| 99 | +#### Templates WITHOUT env files (deploy directly): |
| 100 | + |
| 101 | +```bash |
| 102 | +# ts-sample-app |
| 103 | +cd ts-sample-app && ../bin/kernel deploy index.ts && cd .. |
| 104 | + |
| 105 | +# ts-captcha-solver |
| 106 | +cd ts-captcha-solver && ../bin/kernel deploy index.ts && cd .. |
| 107 | + |
| 108 | +# py-sample-app |
| 109 | +cd py-sample-app && ../bin/kernel deploy main.py && cd .. |
| 110 | + |
| 111 | +# py-captcha-solver |
| 112 | +cd py-captcha-solver && ../bin/kernel deploy main.py && cd .. |
| 113 | +``` |
| 114 | + |
| 115 | +#### Templates WITH env files (prompt human first): |
| 116 | + |
| 117 | +For each of these, **STOP and ask the human** for the required API key(s), then create the `.env` file and deploy: |
| 118 | + |
| 119 | +**ts-stagehand** (needs OPENAI_API_KEY): |
| 120 | + |
| 121 | +```bash |
| 122 | +cd ts-stagehand |
| 123 | +echo "OPENAI_API_KEY=<value from human>" > .env |
| 124 | +../bin/kernel deploy index.ts --env-file .env |
| 125 | +cd .. |
| 126 | +``` |
| 127 | + |
| 128 | +**ts-computer-use** (needs ANTHROPIC_API_KEY): |
| 129 | + |
| 130 | +```bash |
| 131 | +cd ts-computer-use |
| 132 | +echo "ANTHROPIC_API_KEY=<value from human>" > .env |
| 133 | +../bin/kernel deploy index.ts --env-file .env |
| 134 | +cd .. |
| 135 | +``` |
| 136 | + |
| 137 | +**ts-magnitude** (needs ANTHROPIC_API_KEY): |
| 138 | + |
| 139 | +```bash |
| 140 | +cd ts-magnitude |
| 141 | +echo "ANTHROPIC_API_KEY=<value from human>" > .env |
| 142 | +../bin/kernel deploy index.ts --env-file .env |
| 143 | +cd .. |
| 144 | +``` |
| 145 | + |
| 146 | +**ts-cua** (needs OPENAI_API_KEY): |
| 147 | + |
| 148 | +```bash |
| 149 | +cd ts-cua |
| 150 | +echo "OPENAI_API_KEY=<value from human>" > .env |
| 151 | +../bin/kernel deploy index.ts --env-file .env |
| 152 | +cd .. |
| 153 | +``` |
| 154 | + |
| 155 | +**ts-gemini-cua** (needs GOOGLE_API_KEY and OPENAI_API_KEY): |
| 156 | + |
| 157 | +```bash |
| 158 | +cd ts-gemini-cua |
| 159 | +cat > .env << EOF |
| 160 | +GOOGLE_API_KEY=<value from human> |
| 161 | +OPENAI_API_KEY=<value from human> |
| 162 | +EOF |
| 163 | +../bin/kernel deploy index.ts --env-file .env |
| 164 | +cd .. |
| 165 | +``` |
| 166 | + |
| 167 | +**py-browser-use** (needs OPENAI_API_KEY): |
| 168 | + |
| 169 | +```bash |
| 170 | +cd py-browser-use |
| 171 | +echo "OPENAI_API_KEY=<value from human>" > .env |
| 172 | +../bin/kernel deploy main.py --env-file .env |
| 173 | +cd .. |
| 174 | +``` |
| 175 | + |
| 176 | +**py-computer-use** (needs ANTHROPIC_API_KEY): |
| 177 | + |
| 178 | +```bash |
| 179 | +cd py-computer-use |
| 180 | +echo "ANTHROPIC_API_KEY=<value from human>" > .env |
| 181 | +../bin/kernel deploy main.py --env-file .env |
| 182 | +cd .. |
| 183 | +``` |
| 184 | + |
| 185 | +**py-cua** (needs OPENAI_API_KEY): |
| 186 | + |
| 187 | +```bash |
| 188 | +cd py-cua |
| 189 | +echo "OPENAI_API_KEY=<value from human>" > .env |
| 190 | +../bin/kernel deploy main.py --env-file .env |
| 191 | +cd .. |
| 192 | +``` |
| 193 | + |
| 194 | +## Step 6: Provide Invoke Commands |
| 195 | + |
| 196 | +Once all deployments are complete, present the human with these invoke commands to test manually: |
| 197 | + |
| 198 | +```bash |
| 199 | +# TypeScript apps |
| 200 | +kernel invoke ts-basic get-page-title --payload '{"url": "https://www.google.com"}' |
| 201 | +kernel invoke ts-captcha-solver test-captcha-solver |
| 202 | +kernel invoke ts-stagehand teamsize-task --payload '{"company": "Kernel"}' |
| 203 | +kernel invoke ts-cu cu-task --payload '{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}' |
| 204 | +kernel invoke ts-magnitude mag-url-extract --payload '{"url": "https://en.wikipedia.org/wiki/Special:Random"}' |
| 205 | +kernel invoke ts-cua cua-task --payload '{"task": "Go to https://news.ycombinator.com and get the top 5 articles"}' |
| 206 | +kernel invoke ts-gemini-cua gemini-cua-task |
| 207 | + |
| 208 | +# Python apps |
| 209 | +kernel invoke python-basic get-page-title --payload '{"url": "https://www.google.com"}' |
| 210 | +kernel invoke python-captcha-solver test-captcha-solver |
| 211 | +kernel invoke python-bu bu-task --payload '{"task": "Compare the price of gpt-4o and DeepSeek-V3"}' |
| 212 | +kernel invoke python-cu cu-task --payload '{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}' |
| 213 | +kernel invoke python-cua cua-task --payload '{"task": "Go to https://news.ycombinator.com and get the top 5 articles"}' |
| 214 | +``` |
| 215 | + |
| 216 | +## Summary Checklist |
| 217 | + |
| 218 | +- [ ] Built CLI with `make build` |
| 219 | +- [ ] Created QA directory |
| 220 | +- [ ] Got KERNEL_API_KEY from human |
| 221 | +- [ ] Created all 12 template variations |
| 222 | +- [ ] Got required API keys from human (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY) |
| 223 | +- [ ] Deployed all 12 apps |
| 224 | +- [ ] Provided invoke commands to human for manual testing |
| 225 | + |
| 226 | +## Cleanup |
| 227 | + |
| 228 | +After QA is complete, the human can remove the QA directory: |
| 229 | + |
| 230 | +```bash |
| 231 | +cd .. |
| 232 | +rm -rf "$QA_DIR" |
| 233 | +``` |
0 commit comments