|
| 1 | +--- |
| 2 | +title: "Infrastructure from Code" |
| 3 | +description: "Deep dive into Shuttle's Infrastructure from Code philosophy and how it differs from traditional approaches" |
| 4 | +icon: "code" |
| 5 | +--- |
| 6 | + |
| 7 | +## What is Infrastructure from Code? |
| 8 | + |
| 9 | +Infrastructure from Code (IfC) is Shuttle's foundational approach where infrastructure requirements are expressed directly in your application code through language-native constructs like type hints, attributes, or decorators, rather than in separate configuration files or management consoles. |
| 10 | + |
| 11 | +## The Philosophy |
| 12 | + |
| 13 | +Traditional cloud development often separates concerns: |
| 14 | + |
| 15 | +- **Application Logic**: Your business code |
| 16 | +- **Infrastructure Configuration**: YAML files, Terraform, CloudFormation |
| 17 | +- **Deployment Scripts**: CI/CD pipelines, Docker configurations |
| 18 | + |
| 19 | +This separation, while architecturally sound, can introduce friction: |
| 20 | + |
| 21 | +- Context switching between code and infrastructure definitions. |
| 22 | +- Synchronization challenges between different environments. |
| 23 | +- Complex dependency management. |
| 24 | +- A higher barrier to entry for new developers. |
| 25 | + |
| 26 | +Shuttle's Infrastructure from Code collapses this complexity by enabling you to express infrastructure needs directly alongside where they are used in your application code. |
| 27 | + |
| 28 | +## How Shuttle's IfC Works |
| 29 | + |
| 30 | +At its core, IfC means that your application code becomes the single source of truth for both your business logic and your infrastructure requirements. |
| 31 | + |
| 32 | +### 1. Declarative Resource Requirements |
| 33 | + |
| 34 | +You declare the resources your application needs directly in your code, typically as function parameters or class members, using language-native features. Shuttle then interprets these declarations to provision the necessary cloud infrastructure. |
| 35 | + |
| 36 | +For example, you might declare a database, a storage bucket, or a secret manager entry. Shuttle understands these declarations and handles the provisioning, configuration, and injection of the actual resources into your running application. |
| 37 | + |
| 38 | +### 2. Automatic Lifecycle Management |
| 39 | + |
| 40 | +When you deploy your application with Shuttle: |
| 41 | + |
| 42 | +- **Provisioning**: Required resources are automatically created on first deployment. |
| 43 | +- **Persistence**: Resources like databases or storage buckets persist across deployments, maintaining their state. |
| 44 | +- **Management**: Shuttle manages updates, backups, and scaling of these resources where applicable. |
| 45 | +- **Cleanup**: Resources are automatically torn down when they are no longer declared in your code, preventing orphaned infrastructure. |
| 46 | + |
| 47 | +## IfC vs. Traditional Infrastructure Provisioning |
| 48 | + |
| 49 | +To illustrate the difference, consider provisioning a PostgreSQL database for your application: |
| 50 | + |
| 51 | +### Traditional Approach Example |
| 52 | + |
| 53 | +With traditional methods, you'd define your infrastructure in separate files and manage environment variables: |
| 54 | + |
| 55 | +```yaml |
| 56 | +# docker-compose.yml or similar IaC tool |
| 57 | +services: |
| 58 | + database: |
| 59 | + image: postgres:13 |
| 60 | + environment: |
| 61 | + POSTGRES_DB: myapp |
| 62 | + POSTGRES_USER: user |
| 63 | + POSTGRES_PASSWORD: password |
| 64 | + |
| 65 | + app: |
| 66 | + build: . |
| 67 | + depends_on: |
| 68 | + - database |
| 69 | + environment: |
| 70 | + DATABASE_URL: postgres://user:password@database:5432/myapp |
| 71 | +``` |
| 72 | +
|
| 73 | +And your application code would retrieve the connection string from environment variables: |
| 74 | +
|
| 75 | +```python |
| 76 | +# Your app code |
| 77 | +import os |
| 78 | +import psycopg |
| 79 | +from psycopg_pool import ConnectionPool |
| 80 | + |
| 81 | +DATABASE_URL = os.environ.get("DATABASE_URL") |
| 82 | +if not DATABASE_URL: |
| 83 | + raise ValueError("DATABASE_URL environment variable is not set") |
| 84 | + |
| 85 | +pool = ConnectionPool(DATABASE_URL) |
| 86 | +# ... use pool to interact with the database ... |
| 87 | +``` |
| 88 | + |
| 89 | +This approach requires manual synchronization between the infrastructure definition and the application's environment variable consumption. |
| 90 | + |
| 91 | +### Infrastructure from Code in Python Example |
| 92 | + |
| 93 | +With Shuttle's IfC in Python, the database requirement is expressed directly in your application code: |
| 94 | + |
| 95 | +```python |
| 96 | +from typing import Annotated |
| 97 | + |
| 98 | +import shuttle_runtime |
| 99 | +import shuttle_task |
| 100 | +from shuttle_aws.s3 import Bucket, BucketOptions, AllowWrite |
| 101 | +from shuttle_aws.rds import RdsPostgres, RdsPostgresOptions |
| 102 | +from shuttle_runtime import Secrets |
| 103 | + |
| 104 | +@shuttle_task.cron(schedule="0 3 * * ? *") |
| 105 | +async def run( |
| 106 | + # Dedicated AWS RDS Postgres with custom options |
| 107 | + production_db: Annotated[ |
| 108 | + RdsPostgres, |
| 109 | + RdsPostgresOptions( |
| 110 | + database_name="prod_metrics", |
| 111 | + allocated_storage_gb=20, |
| 112 | + ), |
| 113 | + ], |
| 114 | + # Dedicated AWS S3 Bucket with write permissions |
| 115 | + data_bucket: Annotated[Bucket, |
| 116 | + BucketOptions( |
| 117 | + bucket_name="my-app-data-bucket-unique", # Must be globally unique |
| 118 | + policies=[ |
| 119 | + AllowWrite( |
| 120 | + account_id="123456789012", # Example AWS account ID |
| 121 | + role_name="MyExternalServiceRole", |
| 122 | + ), |
| 123 | + ], |
| 124 | + ), |
| 125 | + ], |
| 126 | + # A secret automatically managed by Shuttle |
| 127 | + secrets: Secrets, |
| 128 | +): |
| 129 | + # All resources are ready to use as function arguments |
| 130 | + print(f"Connected to production DB host: {production_db.get_connection().dsn}") |
| 131 | + print(f"Data bucket name: {data_bucket.options.bucket_name}") |
| 132 | + print(f"Secrets available: {secrets.get('MY_SECRET')}") |
| 133 | +``` |
| 134 | + |
| 135 | +In this Python example, `RdsPostgres` and `Bucket` are type hints that tell Shuttle what resources are needed. The `Annotated` type allows for additional configuration (e.g., `RdsPostgresOptions`). Shuttle automatically provisions these resources and injects fully configured, ready-to-use instances into your function at runtime. |
| 136 | + |
| 137 | +## Developer Experience & Efficiency |
| 138 | + |
| 139 | +### Unified Codebase for App & Infra |
| 140 | + |
| 141 | +Infrastructure requirements are defined directly within your application code using type hints and decorators. This creates a single source of truth, eliminating the need for separate configuration files (like Terraform or CloudFormation) and reducing context switching. |
| 142 | + |
| 143 | +- **Version Control**: Since infrastructure is code, all changes to your application and its underlying resources are tracked together in version control, simplifying collaboration, code reviews, and rollbacks. |
| 144 | + |
| 145 | +With commands like `shuttle deploy`, `shuttle logs`, and `shuttle destroy`, the platform automatically provisions, updates, and tears down resources. The CLI provides immediate, human-readable feedback (diffs) on planned infrastructure changes, ensuring consistency and reproducibility across environments. |
| 146 | + |
| 147 | +### Enhanced Local Development |
| 148 | + |
| 149 | +## Limitations and Considerations |
| 150 | + |
| 151 | +### When IfC Might Not Be Ideal and Working Within Constraints |
| 152 | + |
| 153 | +Infrastructure from Code may not be ideal in scenarios such as complex multi-service architectures not fully managed by Shuttle, strict compliance requirements that demand external auditing of infrastructure, legacy system integration, or highly diverse multi-language environments where a common IaC tool might be preferred. In these cases, consider: |
| 154 | + |
| 155 | +- **Hybrid Approaches**: Integrating Shuttle-managed parts with external infrastructure definitions managed by traditional IaC tools. |
| 156 | +- **Custom Resources**: For complex infrastructure needs not directly supported by Shuttle's built-in types, you might define them externally and integrate them into your Shuttle project. |
| 157 | +- **Traditional IaC**: Combining Shuttle with traditional Infrastructure as Code tools for legacy components or highly specialized requirements that fall outside Shuttle's immediate scope. |
| 158 | + |
| 159 | +## The Future of Infrastructure from Code |
| 160 | + |
| 161 | +As AI and machine learning technologies advance, they are poised to revolutionize Infrastructure from Code by making it even more approachable and efficient. AI tools combined with succinct syntax will lower the barrier to entry for developers, making it easier to manage infrastructure through code. |
| 162 | + |
| 163 | +- **AI-Enhanced Tooling**: AI will enable more intuitive interaction with infrastructure, allowing developers to focus on application logic rather than complex configurations. |
| 164 | +- **Streamlined Orchestration**: With AI-driven command-line tools, orchestrating infrastructure provisioning will become more streamlined, reducing the need for verbose commands. |
| 165 | +- **Short Context Prompting**: AI will enhance the accuracy of infrastructure management tasks by understanding short context prompts, improving over verbose and error-prone infrastructure tools. |
| 166 | + |
| 167 | +Understanding Infrastructure from Code helps you leverage Shuttle's full potential and design applications that are both powerful and maintainable. |
0 commit comments