Skip to content

Commit fdb9c02

Browse files
mcgrofmcgrof
committed
kconfigs: enable Lambda Labs cloud provider in menus
Enable Lambda Labs cloud provider in kdevops configuration system: - kconfigs/Kconfig.bringup: Add Lambda Labs to bringup provider menu - defconfigs/*: Add comprehensive example configurations - lambdalabs-smart: Smart cheapest instance selection - lambdalabs-gpu-*: GPU-specific configurations - lambdalabs-shared-key: Shared SSH key setup - PROMPTS.md: Document AI prompts used for development Lambda Labs defconfig examples: - Single GPU: A10, A100, H100 optimized configurations - Multi-GPU: 8x A100, 8x H100 for ML workloads - Smart selection: Automatic cheapest instance selection - Shared infrastructure: Common SSH key management This completes Lambda Labs integration, making it available through standard kdevops configuration workflows. Generated-by: Claude AI Signed-off-by: Luis Chamberlain <[email protected]>
1 parent 535726d commit fdb9c02

File tree

10 files changed

+138
-0
lines changed

10 files changed

+138
-0
lines changed

‎PROMPTS.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -280,3 +280,59 @@ The implementation successfully added:
280280
to do is to use a separate fact if you want a true dynamic variable. This
281281
is why we switched to an active ref prefix for the baseline and dev group
282282
ref tags.
283+
284+
## Cloud provider integrations
285+
286+
### Adding Lambda Labs cloud provider support with dynamic Kconfig
287+
288+
**Prompt:**
289+
The Lambda Labs company helps you use GPUs online, kind of like AWS, or OCI. Add
290+
support for the terraform support for Lambda Labs. The best provider docs are at
291+
https://registry.terraform.io/providers/elct9620/lambdalabs/latest/docs . Then
292+
To create the kconfig values you will implement support to use the lambda cloud
293+
API to let us query for what type of instances they have available and so forth.
294+
Therefore the Kconfig stuff for Lambda labs will all be dynamic. So we'll want
295+
to expand this as part of what make dynconfig does. However note that dynconfig
296+
does *all* dynamically generated kconfig. We want to add support for make
297+
cloud-config as a new target which is dynamic which is a subset of make
298+
dynconfig ; OK! good luck
299+
300+
**AI:** Claude Code (Opus 4.1)
301+
**Commit:** [To be determined]
302+
**Result:** Complete Lambda Labs integration with dynamic Kconfig generation.
303+
**Grading:** 75%
304+
305+
**Notes:**
306+
307+
The implementation successfully added:
308+
309+
1. **Terraform Provider Integration**: Created complete Terraform configuration
310+
for Lambda Labs including instance management, persistent storage, and SSH
311+
configuration management following existing cloud provider patterns.
312+
313+
2. **Dynamic Kconfig Generation**: Implemented Python script to query Lambda Labs
314+
API for available instance types, regions, and OS images. Generated dynamic
315+
Kconfig files with fallback defaults when API is unavailable.
316+
317+
3. **Build System Integration**: Added `make cloud-config` as a new target for
318+
cloud-specific dynamic configuration, properly integrated with `make dynconfig`.
319+
Created modular Makefile structure for cloud provider dynamic configuration.
320+
321+
4. **Kconfig Structure**: Properly integrated Lambda Labs into the provider
322+
selection system with modular Kconfig files for location, compute, storage,
323+
and identity management.
324+
325+
Biggest issues:
326+
327+
1. **SSH Management**: For this it failed to realize the provider
328+
didn't suport asking for a custom username, so we had to find out the
329+
hard way.
330+
331+
2. **Environment variables**: For some reason it wanted to define the
332+
credential API as an environment variable. This proved painful as some
333+
environment variables do not carry over for some ansible tasks. The
334+
best solution was to follow the strategy similar to what AWS supports
335+
with ~/.lambdalabs/credentials. This a more secure alternative.
336+
337+
Minor issues:
338+
- Some whitespace formatting was automatically fixed by the linter

‎defconfigs/lambdalabs‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Lambda Labs default configuration with smart cheapest instance selection
2+
# Automatically:
3+
# 1. Detects your location from public IP
4+
# 2. Finds the cheapest available GPU instance
5+
# 3. Selects the closest region where it's available
6+
# 4. Creates unique SSH key per project directory for security
7+
# 5. Auto-uploads SSH key to Lambda Labs on first run
8+
CONFIG_TERRAFORM=y
9+
CONFIG_TERRAFORM_LAMBDALABS=y
10+
CONFIG_TERRAFORM_LAMBDALABS_SMART_CHEAPEST=y
11+
CONFIG_TERRAFORM_LAMBDALABS_SSH_KEY_UNIQUE=y
12+
CONFIG_TERRAFORM_LAMBDALABS_SSH_KEY_AUTO_CREATE=y
13+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY=y
14+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE=y
15+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_EMPTY_PASSPHRASE=y

‎defconfigs/lambdalabs-gpu-1x-a10‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Lambda Labs GPU 1x A10 instance - budget-friendly option ($0.75/hr)
2+
# Automatically selects the best available region
3+
CONFIG_TERRAFORM=y
4+
CONFIG_TERRAFORM_LAMBDALABS=y
5+
CONFIG_TERRAFORM_LAMBDALABS_REGION_SMART_INFER=y
6+
CONFIG_TERRAFORM_LAMBDALABS_INSTANCE_TYPE_GPU_1X_A10=y
7+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY=y
8+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE=y
9+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_EMPTY_PASSPHRASE=y

‎defconfigs/lambdalabs-gpu-1x-a100‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Lambda Labs GPU 1x A100 instance - high performance single GPU
2+
CONFIG_TERRAFORM=y
3+
CONFIG_TERRAFORM_LAMBDALABS=y
4+
CONFIG_TERRAFORM_LAMBDALABS_REGION_SMART_INFER=y
5+
CONFIG_TERRAFORM_LAMBDALABS_INSTANCE_TYPE_GPU_1X_A100_SXM4=y
6+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY=y
7+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE=y
8+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_EMPTY_PASSPHRASE=y

‎defconfigs/lambdalabs-gpu-1x-h100‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Lambda Labs GPU 1x H100 instance - latest generation single GPU
2+
CONFIG_TERRAFORM=y
3+
CONFIG_TERRAFORM_LAMBDALABS=y
4+
CONFIG_TERRAFORM_LAMBDALABS_REGION_SMART_INFER=y
5+
CONFIG_TERRAFORM_LAMBDALABS_INSTANCE_TYPE_GPU_1X_H100_SXM5=y
6+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY=y
7+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE=y
8+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_EMPTY_PASSPHRASE=y

‎defconfigs/lambdalabs-gpu-8x-a100‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Lambda Labs GPU 8x A100 instance - multi-GPU compute cluster
2+
CONFIG_TERRAFORM=y
3+
CONFIG_TERRAFORM_LAMBDALABS=y
4+
CONFIG_TERRAFORM_LAMBDALABS_REGION_SMART_INFER=y
5+
CONFIG_TERRAFORM_LAMBDALABS_INSTANCE_TYPE_GPU_8X_A100=y
6+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY=y
7+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE=y
8+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_EMPTY_PASSPHRASE=y

‎defconfigs/lambdalabs-gpu-8x-h100‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Lambda Labs GPU 8x H100 instance - top-tier multi-GPU cluster
2+
CONFIG_TERRAFORM=y
3+
CONFIG_TERRAFORM_LAMBDALABS=y
4+
CONFIG_TERRAFORM_LAMBDALABS_REGION_SMART_INFER=y
5+
CONFIG_TERRAFORM_LAMBDALABS_INSTANCE_TYPE_GPU_8X_H100_SXM5=y
6+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY=y
7+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE=y
8+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_EMPTY_PASSPHRASE=y

‎defconfigs/lambdalabs-shared-key‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Lambda Labs configuration with shared SSH key (legacy mode)
2+
# Uses a single SSH key name across all projects
3+
# Less secure but simpler for testing
4+
CONFIG_TERRAFORM=y
5+
CONFIG_TERRAFORM_LAMBDALABS=y
6+
CONFIG_TERRAFORM_LAMBDALABS_SMART_CHEAPEST=y
7+
CONFIG_TERRAFORM_LAMBDALABS_SSH_KEY_SHARED=y
8+
# Manual key name can be set via menuconfig
9+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY=y
10+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE=y
11+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_EMPTY_PASSPHRASE=y

‎defconfigs/lambdalabs-smart‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Lambda Labs with smart defaults - cheapest instance and best region
2+
# Automatically selects the cheapest available instance type
3+
# Automatically selects the best available region for that instance
4+
CONFIG_TERRAFORM=y
5+
CONFIG_TERRAFORM_LAMBDALABS=y
6+
CONFIG_TERRAFORM_LAMBDALABS_REGION_SMART_INFER=y
7+
CONFIG_TERRAFORM_LAMBDALABS_INSTANCE_TYPE_GPU_1X_A10=y
8+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY=y
9+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE=y
10+
CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_EMPTY_PASSPHRASE=y

‎kconfigs/Kconfig.bringup‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,13 @@ config KDEVOPS_ENABLE_NIXOS
99
bool
1010
output yaml
1111

12+
config CLOUD_INITIALIZED
13+
bool
14+
default $(shell, test -f .cloud.initialized && echo y || echo n) = "y"
15+
1216
choice
1317
prompt "Node bring up method"
18+
default TERRAFORM if CLOUD_INITIALIZED
1419
default GUESTFS
1520

1621
config GUESTFS

0 commit comments

Comments
 (0)