Skip to content

Commit 92edf8e

Browse files
docs: add GETTING_STARTED.md and QUICK_REFERENCE.md
Fixes #38 - Created GETTING_STARTED.md with complete setup walkthrough, API keys guide, and contribution guidelines - Created QUICK_REFERENCE.md with command cheatsheet for common tasks 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent 73f46e0 commit 92edf8e

File tree

2 files changed

+471
-0
lines changed

2 files changed

+471
-0
lines changed

GETTING_STARTED.md

Lines changed: 271 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
# Getting Started with Substrate
2+
3+
This guide walks you through setting up and using Substrate, the infrastructure for human knowledge and progress.
4+
5+
---
6+
7+
## Prerequisites
8+
9+
### Required
10+
- **Git** - To clone the repository
11+
- A text editor or IDE
12+
13+
### Optional (for automation)
14+
- **[Bun](https://bun.sh)** - JavaScript/TypeScript runtime for running update scripts
15+
```bash
16+
curl -fsSL https://bun.sh/install | bash
17+
```
18+
19+
---
20+
21+
## Installation
22+
23+
### 1. Clone the Repository
24+
25+
```bash
26+
git clone https://github.com/danielmiessler/Substrate.git
27+
cd Substrate
28+
```
29+
30+
### 2. Explore the Structure
31+
32+
```bash
33+
# Core datasets (GDP, inflation, COVID wastewater, etc.)
34+
ls Data/
35+
36+
# Wellbeing data sources (FRED, CDC, Census, BLS, EPA)
37+
ls Data-Sources/
38+
39+
# Knowledge components
40+
ls Problems/
41+
ls Solutions/
42+
ls Arguments/
43+
ls Claims/
44+
```
45+
46+
### 3. Install Dependencies (Optional)
47+
48+
If you want to run the TypeScript automation scripts:
49+
50+
```bash
51+
bun install
52+
```
53+
54+
---
55+
56+
## Viewing the Data
57+
58+
All data is stored in human-readable formats:
59+
60+
- **CSV files** - Raw data you can open in Excel, Google Sheets, or any spreadsheet app
61+
- **Markdown files** - Documentation and metadata
62+
63+
No special tools required - just browse the repository!
64+
65+
### Example: View US GDP Data
66+
67+
```bash
68+
# See the data files
69+
ls Data/US-GDP/
70+
71+
# View the CSV in terminal
72+
cat Data/US-GDP/us-gdp-annual.csv | head -20
73+
74+
# Or open in your favorite spreadsheet app
75+
open Data/US-GDP/us-gdp-annual.csv
76+
```
77+
78+
---
79+
80+
## Running Updates
81+
82+
> **Note:** This is optional. The data is already included in the repository.
83+
84+
### Update a Single Dataset
85+
86+
```bash
87+
cd Data/US-GDP
88+
bun run update.ts
89+
```
90+
91+
### Update a Wellbeing Data Source
92+
93+
Most wellbeing sources require API keys:
94+
95+
```bash
96+
# Set your API key
97+
export FRED_API_KEY="your_key_here"
98+
99+
# Run the update
100+
cd Data-Sources/DS-00004—FRED_Economic_Wellbeing
101+
bun run update.ts
102+
```
103+
104+
### Update All Datasets
105+
106+
```bash
107+
bun run scripts/update-all.ts
108+
```
109+
110+
---
111+
112+
## API Keys
113+
114+
Most data sources are free but require registration:
115+
116+
| Data Source | Get Key | Rate Limit |
117+
|-------------|---------|------------|
118+
| **FRED Economic** | [fred.stlouisfed.org/docs/api](https://fred.stlouisfed.org/docs/api/api_key.html) | 120 req/min |
119+
| **Census ACS** | [api.census.gov/data/key_signup](https://api.census.gov/data/key_signup.html) | 500 req/day |
120+
| **EPA Air Quality** | Email: [email protected] | 10 req/min |
121+
| **BLS JOLTS** | [bls.gov/developers/home](https://www.bls.gov/developers/home.htm) | 500 req/day |
122+
| **CDC WONDER** | No key required | Fair use |
123+
124+
### Setting Up Environment Variables
125+
126+
Create a `.env` file in the repository root:
127+
128+
```bash
129+
FRED_API_KEY=your_fred_key
130+
CENSUS_API_KEY=your_census_key
131+
132+
EPA_KEY=your_epa_key
133+
BLS_API_KEY=your_bls_key
134+
```
135+
136+
Or export them in your shell:
137+
138+
```bash
139+
export FRED_API_KEY="your_fred_key"
140+
export CENSUS_API_KEY="your_census_key"
141+
```
142+
143+
---
144+
145+
## Contributing
146+
147+
We welcome contributions! Here's how to add to Substrate:
148+
149+
### What You Can Contribute
150+
151+
| Type | Directory | Description |
152+
|------|-----------|-------------|
153+
| **Problems** | `Problems/` | Documented challenges with evidence |
154+
| **Solutions** | `Solutions/` | Proven approaches with results |
155+
| **Arguments** | `Arguments/` | Reasoning chains with quality scores |
156+
| **Claims** | `Claims/` | Assertions linked to evidence |
157+
| **Data** | `Data/` or `Data-Sources/` | Authoritative datasets |
158+
| **People** | `People/` | Researchers and practitioners |
159+
| **Organizations** | `Organizations/` | Groups working on problems |
160+
| **Projects** | `Projects/` | Active initiatives |
161+
| **Plans** | `Plans/` | Actionable strategies |
162+
| **Ideas** | `Ideas/` | Frameworks and concepts |
163+
| **Values** | `Values/` | Guiding principles |
164+
165+
### How to Submit
166+
167+
1. **Fork the repository** on GitHub
168+
169+
2. **Create a branch** for your contribution:
170+
```bash
171+
git checkout -b add-my-contribution
172+
```
173+
174+
3. **Add your content** following the format in each directory's README
175+
176+
4. **Commit your changes**:
177+
```bash
178+
git add .
179+
git commit -m "Add: description of your contribution"
180+
```
181+
182+
5. **Push and create a Pull Request**:
183+
```bash
184+
git push origin add-my-contribution
185+
```
186+
187+
### Contribution Guidelines
188+
189+
- **Follow existing formats** - Check the README in each directory for templates
190+
- **Include evidence** - Link claims to data sources where possible
191+
- **Use clear IDs** - Follow the naming conventions (e.g., `PR-00001` for problems)
192+
- **Document your sources** - Include provenance and methodology
193+
- **Be specific** - Vague contributions are harder to use
194+
195+
### Quality Standards
196+
197+
We don't gatekeep ideas, but we do maintain quality:
198+
199+
- **Problems** should be specific and evidence-backed
200+
- **Solutions** should include outcomes or expected results
201+
- **Arguments** should have clear reasoning chains
202+
- **Data** should include methodology and limitations
203+
204+
---
205+
206+
## Directory Structure
207+
208+
```
209+
Substrate/
210+
├── Data/ # Core datasets
211+
│ ├── US-GDP/ # US GDP data (1929-present)
212+
│ ├── US-Inflation/ # US inflation data
213+
│ ├── Bay-Area-COVID/ # COVID wastewater monitoring
214+
│ ├── Pulitzer-Prize/ # Pulitzer Prize winners
215+
│ └── Knowledge-Worker-Salaries/
216+
217+
├── Data-Sources/ # Wellbeing data sources
218+
│ ├── DS-00001—WHO_Global_Health_Observatory/
219+
│ ├── DS-00002—UN_SDG_Indicators/
220+
│ ├── DS-00003—World_Bank_Open_Data/
221+
│ ├── DS-00004—FRED_Economic_Wellbeing/
222+
│ ├── DS-00005—CDC_WONDER_Mortality/
223+
│ ├── DS-00006—Census_ACS_Social_Wellbeing/
224+
│ ├── DS-00007—BLS_JOLTS_Labor_Market/
225+
│ └── DS-00008—EPA_Air_Quality_System/
226+
227+
├── Problems/ # Documented challenges
228+
├── Solutions/ # Proven approaches
229+
├── Arguments/ # Reasoning chains
230+
├── Claims/ # Evidence-linked assertions
231+
├── Plans/ # Actionable strategies
232+
├── Ideas/ # Frameworks and concepts
233+
├── People/ # Researchers and practitioners
234+
├── Organizations/ # Groups working on issues
235+
├── Projects/ # Active initiatives
236+
├── Values/ # Guiding principles
237+
238+
├── scripts/ # Automation scripts
239+
├── README.md # Main documentation
240+
├── GETTING_STARTED.md # This file
241+
├── QUICK_REFERENCE.md # Command cheatsheet
242+
└── UPDATES.md # Changelog
243+
```
244+
245+
---
246+
247+
## Next Steps
248+
249+
1. **Explore the data** - Browse `Data/` and `Data-Sources/` to see what's available
250+
2. **Read the documentation** - Each dataset has its own README with methodology
251+
3. **Try the automation** - Run an update script to see how data is refreshed
252+
4. **Contribute** - Add a problem, solution, or data source you care about
253+
254+
---
255+
256+
## Getting Help
257+
258+
- **Issues** - [github.com/danielmiessler/Substrate/issues](https://github.com/danielmiessler/Substrate/issues)
259+
- **Discussions** - Start a discussion in the repository
260+
- **Twitter** - [@danielmiessler](https://twitter.com/danielmiessler)
261+
262+
---
263+
264+
## Related Projects
265+
266+
- **[TELOS](https://github.com/danielmiessler/Telos)** - Goals and strategy framework
267+
- **[Fabric](https://github.com/danielmiessler/fabric)** - AI augmentation framework
268+
269+
---
270+
271+
**[← Back to README](./README.md)** | **[Quick Reference →](./QUICK_REFERENCE.md)**

0 commit comments

Comments
 (0)