Skip to content

Commit dcf06c0

Browse files
guzman109claude
andcommitted
docs: fix stale git/ module references and add missing guide links
- Fix /api/git/ → /api/github/ paths in README.md and architecture.md - Fix POST /api/github/repos/:id → POST /api/github/repos (no :id on create) - Add missing list route GET /api/github/repos to architecture route table - Add http-client-guide.md and async-task-patterns.md to docs/README.md index - Fix all stale git/ namespace/path references in async-task-patterns.md (include/git/ → include/insights/github/, namespace insights::git → insights::github) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 4129b32 commit dcf06c0

File tree

4 files changed

+120
-136
lines changed

4 files changed

+120
-136
lines changed

README.md

Lines changed: 106 additions & 125 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@
55
<img src="assets/logo-light.svg" alt="ICICLE Insights Logo" width="700"/>
66
</picture>
77

8-
<p><strong>A high-performance C++23 HTTP server for collecting and storing metrics about ICICLE project components</strong></p>
8+
<p><strong>A C++23 HTTP server for collecting and storing metrics about ICICLE project components</strong></p>
99

1010
<p>
11-
<a href="#features">Features</a> •
11+
<a href="#overview">Overview</a> •
1212
<a href="#quick-start">Quick Start</a> •
13-
<a href="#api-endpoints">API</a> •
13+
<a href="#api">API</a> •
14+
<a href="#deployment">Deployment</a> •
1415
<a href="#documentation">Documentation</a>
1516
</p>
1617
</div>
@@ -19,192 +20,172 @@
1920

2021
## Overview
2122

22-
ICICLE Insights tracks repositories, accounts, and platform-level statistics across various platforms including Git hosting services (GitHub, GitLab) and container registries (DockerHub, Quay.io).
23+
ICICLE Insights collects and stores metrics for the [ICICLE](https://icicle.osu.edu/) research project — tracking repositories and accounts across Git hosting platforms (GitHub) and container registries.
2324

24-
The system monitors metrics across the ICICLE research project ecosystem:
25+
Metrics tracked include: **stars, forks, clones, views, watchers, followers**.
2526

26-
- **Git Platforms**: stars, forks, clones, views, watchers, followers
27-
- **Container Registries**: pulls, stars
28-
29-
The project tracks ICICLE components across different research thrusts:
30-
- AI4CI (AI for Cyberinfrastructure)
31-
- FoundationAI
32-
- CI4AI (Cyberinfrastructure for AI)
33-
- Software
34-
- Use-inspired domains (Digital Agriculture, Smart Foodsheds, Animal Ecology)
35-
36-
## Features
37-
38-
- **Modern C++23** implementation with `std::expected` for error handling
39-
- **Async I/O** using ASIO for high performance
40-
- **Background task scheduler** for periodic metric collection from Git platforms
41-
- **Type-safe database operations** with PostgreSQL via libpqxx
42-
- **RESTful API** with JSON serialization using glaze
43-
- **Generic CRUD operations** using templates and traits
44-
- **Non-blocking architecture** with thread pool for background tasks
45-
- **Docker support** for easy deployment
27+
The server exposes a REST API for querying stored data and runs a background sync task every two weeks to pull fresh statistics from platform APIs.
4628

4729
## Prerequisites
4830

4931
- C++23 compiler (GCC 13+, Clang 17+, or Apple Clang 15+)
50-
- [Conan 2.x](https://conan.io/) - Package manager
32+
- [Conan 2.x](https://conan.io/)
5133
- [CMake](https://cmake.org/) 3.25+
52-
- [Ninja](https://ninja-build.org/) - Build system
53-
- [just](https://just.systems/) - Command runner
54-
- PostgreSQL - Database
34+
- [Ninja](https://ninja-build.org/)
35+
- [just](https://just.systems/)
36+
- PostgreSQL
5537

5638
## Quick Start
5739

58-
### 1. Clone the repository
59-
6040
```bash
6141
git clone https://github.com/guzman109/icicle-insights.git
6242
cd icicle-insights
6343
```
6444

65-
### 2. Set up environment variables
66-
6745
Create a `.env` file:
6846

6947
```bash
7048
# Required
7149
DATABASE_URL=postgresql://user:password@localhost:5432/icicle_insights
7250
GITHUB_TOKEN=your_github_token
73-
TAPIS_TOKEN=your_tapis_token
7451

7552
# Optional
76-
HOST=127.0.0.1 # Server host (default: 127.0.0.1)
77-
PORT=3000 # Server port (default: 3000)
78-
LOG_LEVEL=info # Log level: trace, debug, info, warn, error (default: info)
53+
HOST=127.0.0.1 # default: 127.0.0.1
54+
PORT=3000 # default: 3000
55+
LOG_LEVEL=info # trace | debug | info | warn | error
56+
SSL_CERT_FILE= # path to CA bundle (macOS: /opt/homebrew/etc/ca-certificates/cert.pem)
7957
```
8058

81-
### 3. Build and run
59+
Build and run:
8260

8361
```bash
84-
# Install dependencies, configure, build, and run
85-
just full-build
86-
just run
87-
88-
# Or individually:
89-
just deps # Install dependencies with Conan
90-
just setup # Configure CMake
91-
just build # Build the project
92-
just run # Run the server
62+
just full-build # install deps, configure, build
63+
just run # start the server on http://localhost:3000
9364
```
9465

95-
The server will start on `http://localhost:3000`.
96-
97-
The background task scheduler will automatically run every 2 weeks to update Git platform metrics. See [docs/async-task-patterns.md](docs/async-task-patterns.md) for details on the async architecture.
98-
99-
## API Endpoints
66+
## API
10067

10168
### Core
10269

103-
- `GET /health` - Health check endpoint (verifies database connectivity)
104-
- `GET /routes` - Lists all available API endpoints
70+
| Method | Path | Description |
71+
|--------|------|-------------|
72+
| `GET` | `/health` | Database connectivity check |
73+
| `GET` | `/routes` | List all registered routes |
10574

106-
### Platforms
75+
### GitHub Accounts
10776

108-
- `GET /api/git/platforms` - Get all platforms
109-
- `POST /api/git/platforms` - Create a platform
110-
- `GET /api/git/platforms/:id` - Get platform by ID
111-
- `PATCH /api/git/platforms/:id` - Update platform
112-
- `DELETE /api/git/platforms/:id` - Delete platform
77+
| Method | Path | Description |
78+
|--------|------|-------------|
79+
| `GET` | `/api/github/accounts` | List all accounts |
80+
| `POST` | `/api/github/accounts` | Create an account |
81+
| `GET` | `/api/github/accounts/:id` | Get account by ID |
82+
| `DELETE` | `/api/github/accounts/:id` | Delete account |
11383

114-
### Accounts
84+
### GitHub Repositories
11585

116-
- `GET /api/git/accounts` - Get all accounts
117-
- `POST /api/git/accounts` - Create an account
118-
- `GET /api/git/accounts/:id` - Get account by ID
119-
- `PATCH /api/git/accounts/:id` - Update account
120-
- `DELETE /api/git/accounts/:id` - Delete account
86+
| Method | Path | Description |
87+
|--------|------|-------------|
88+
| `GET` | `/api/github/repos` | List all repositories |
89+
| `POST` | `/api/github/repos` | Create a repository |
90+
| `GET` | `/api/github/repos/:id` | Get repository by ID |
91+
| `PATCH` | `/api/github/repos/:id` | Update repository |
92+
| `DELETE` | `/api/github/repos/:id` | Delete repository |
12193

122-
### Repositories
94+
## Deployment
12395

124-
- `GET /api/git/repos` - Get all repositories
125-
- `POST /api/git/repos` - Create a repository
126-
- `GET /api/git/repos/:id` - Get repository by ID
127-
- `PATCH /api/git/repos/:id` - Update repository
128-
- `DELETE /api/git/repos/:id` - Delete repository
96+
The CI/CD pipeline (GitHub Actions) builds the binary on `ubuntu-24.04`, packages it into a Docker image, and pushes to GHCR.
12997

130-
## Development
98+
```bash
99+
# Pull and run the latest image
100+
docker pull ghcr.io/icicle-ai/insights:latest
101+
docker run -p 3000:3000 --env-file .env ghcr.io/icicle-ai/insights:latest
102+
```
131103

132-
### Project Structure
104+
To publish a release, push a version tag:
133105

106+
```bash
107+
git tag v1.0.0 && git push origin v1.0.0
134108
```
135-
.
136-
├── include/ # Header files
137-
│ ├── core/ # Foundation utilities
138-
│ ├── db/ # Database layer
139-
│ ├── git/ # Git platform models, routing, and tasks
140-
│ │ ├── models.hpp # Data models
141-
│ │ ├── router.hpp # HTTP route handlers
142-
│ │ ├── tasks.hpp # Background task pipeline
143-
│ │ └── scheduler.hpp # Periodic task scheduler
144-
│ ├── containers/ # Container registry models
145-
│ └── server/ # HTTP server
146-
├── src/ # Implementation files
147-
│ ├── git/ # Git route handlers and task implementations
148-
│ ├── server/ # Server implementation
149-
│ └── insights.cpp # Entry point
150-
├── docs/ # Documentation
151-
├── tests/ # Test files (HTTP client tests)
152-
└── data/ # Component data and fixtures
109+
110+
This triggers the full pipeline: build → Docker image → GitHub Release with binary tarball.
111+
112+
### Local Docker build
113+
114+
```bash
115+
just docker-build # build image locally
116+
just docker-run # run with .env
153117
```
154118

119+
## Development
120+
155121
### Common Commands
156122

157123
```bash
158-
just build # Build the project
159-
just run # Run the server
160-
just clean-build # Clean and rebuild from scratch
161-
just test # Run tests (if configured)
124+
just deps # Install Conan dependencies
125+
just setup # Configure CMake
126+
just build # Compile
127+
just run # Run the server
128+
just full-build # deps + setup + build
129+
just clean-build # wipe build dir and rebuild
130+
131+
just act-linux # Test the linux CI job locally with act
132+
just act-build # Test the docker CI job locally with act
162133
```
163134

164-
### Code Style
135+
### Project Structure
165136

166-
The project follows LLVM naming conventions:
167-
- **Types**: `PascalCase` (e.g., `Platform`, `HttpStatus`)
168-
- **Variables**: `PascalCase` (e.g., `Database`, `Config`)
169-
- **Functions**: `lowerCamelCase` (e.g., `registerRoutes()`, `initServer()`)
170-
- **Enumerators**: `PascalCase` (e.g., `Ok`, `BadRequest`)
137+
```
138+
.
139+
├── include/insights/
140+
│ ├── core/ # Config, error handling, logging, scheduler, HTTP status
141+
│ ├── db/ # Database layer (Database struct, DbTraits, DbEntity concept)
142+
│ ├── github/ # GitHub models, routes, and sync tasks
143+
│ └── server/ # HTTP server setup and middleware
144+
├── src/
145+
│ ├── core/ # Health check and route listing endpoints
146+
│ ├── github/ # Route handlers and sync task implementation
147+
│ └── insights.cpp # Entry point
148+
├── docs/ # Developer documentation
149+
├── .github/
150+
│ └── workflows/
151+
│ └── build.yaml # CI/CD: build → Docker → release
152+
├── Dockerfile # Runtime image (ubuntu:24.04)
153+
├── conanfile.txt # Conan dependencies
154+
├── CMakeLists.txt # Build configuration
155+
└── justfile # Build command runner
156+
```
171157

172-
Code formatting is enforced via `.clang-format` and static analysis via `.clang-tidy`.
158+
### Code Style
173159

174-
## Documentation
160+
LLVM naming conventions throughout:
175161

176-
- [Architecture Guide](docs/architecture.md) - Detailed architecture and design decisions
177-
- [Async Task Patterns](docs/async-task-patterns.md) - Background task scheduling and async patterns
178-
- [CLAUDE.md](CLAUDE.md) - Project instructions for AI assistants and build system details
162+
| Construct | Convention | Example |
163+
|-----------|-----------|---------|
164+
| Types | `PascalCase` | `Platform`, `HttpStatus` |
165+
| Variables | `PascalCase` | `Database`, `Config` |
166+
| Functions | `lowerCamelCase` | `registerRoutes()`, `syncStats()` |
167+
| Enumerators | `PascalCase` | `Ok`, `BadRequest` |
168+
| Members | `PascalCase` | `.Id`, `.Name`, `.AccountId` |
179169

180170
## Dependencies
181171

182172
| Package | Version | Purpose |
183173
|---------|---------|---------|
184-
| asio | 1.36.0 | Async I/O and networking |
185-
| openssl | 3.6.0 | TLS/SSL support |
186-
| glaze | 7.0.0 | JSON serialization and HTTP routing |
187-
| libpq | 17.7 | PostgreSQL C client |
174+
| asio | 1.36.0 | Async I/O, timers, thread pool |
175+
| openssl || TLS for outbound HTTP requests |
176+
| glaze | main | HTTP server, router, JSON |
188177
| libpqxx | 7.10.5 | PostgreSQL C++ client |
189-
| spdlog | 1.17.0 | Logging |
178+
| spdlog | 1.17.0 | Structured logging |
190179

191-
## Docker
192-
193-
Build and run with Docker:
180+
## Documentation
194181

195-
```bash
196-
docker build -t icicle-insights .
197-
docker run -p 3000:3000 --env-file .env icicle-insights
198-
```
182+
- [Architecture](docs/architecture.md) — module structure, core patterns, data model, design decisions
183+
- [Developer Guide](docs/README.md) — how to add routes, tasks, loggers; debugging tips
199184

200185
## License
201186

202-
This project is licensed under the GNU General Public License v3.0 - see the [LICENSE](LICENSE) file for details.
203-
204-
## Contributing
205-
206-
[Add contribution guidelines here]
187+
GNU General Public License v3.0 — see [LICENSE](LICENSE).
207188

208189
## Acknowledgments
209190

210-
This project is part of the [ICICLE (Intelligent Cyberinfrastructure with Computational Learning in the Environment)](https://icicle.osu.edu/) initiative.
191+
Part of the [ICICLE (Intelligent Cyberinfrastructure with Computational Learning in the Environment)](https://icicle.osu.edu/) initiative.

docs/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ Reference documentation for contributing to ICICLE Insights.
88
|-------|-------------|
99
| [architecture.md](architecture.md) | Module structure, core patterns, data model, and design decisions |
1010
| [background-tasks.md](background-tasks.md) | How to schedule recurring background tasks using `scheduleRecurringTask` |
11+
| [http-client-guide.md](http-client-guide.md) | Building reusable HTTP clients with SSL, connection pooling, and caching |
12+
| [async-task-patterns.md](async-task-patterns.md) | Five async patterns from thread pools to C++20 coroutines |
1113
| [logging.md](logging.md) | Per-component named loggers, log file layout, and log configuration |
1214
| [tls-guide.md](tls-guide.md) | Configuring SSL/TLS for outbound HTTP client requests |
1315
| [task-persistence.md](task-persistence.md) | Future: persisting task run history and results to the database |

docs/architecture.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,8 +123,9 @@ Current routes:
123123
| POST | `/api/github/accounts` | Create a GitHub account |
124124
| GET | `/api/github/accounts/:id` | Get a GitHub account by ID |
125125
| DELETE | `/api/github/accounts/:id` | Delete a GitHub account |
126+
| GET | `/api/github/repos` | List all GitHub repositories |
127+
| POST | `/api/github/repos` | Create a GitHub repository |
126128
| GET | `/api/github/repos/:id` | Get a GitHub repository by ID |
127-
| POST | `/api/github/repos/:id` | Create a GitHub repository |
128129
| PATCH | `/api/github/repos/:id` | Update a GitHub repository |
129130
| DELETE | `/api/github/repos/:id` | Delete a GitHub repository |
130131

docs/async-task-patterns.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ sequenceDiagram
9898
### Implementation
9999

100100
```cpp
101-
// include/git/scheduler.hpp
101+
// include/insights/github/scheduler.hpp
102102
#pragma once
103103
#include "core/result.hpp"
104104
#include "db/db.hpp"
@@ -135,12 +135,12 @@ private:
135135
```
136136
137137
```cpp
138-
// src/git/scheduler.cpp
139-
#include "git/scheduler.hpp"
140-
#include "git/tasks.hpp"
138+
// src/github/scheduler.cpp
139+
#include "insights/github/scheduler.hpp"
140+
#include "insights/github/tasks.hpp"
141141
#include <spdlog/spdlog.h>
142142
143-
namespace insights::git {
143+
namespace insights::github {
144144
145145
TaskScheduler::TaskScheduler(asio::io_context &Io,
146146
std::shared_ptr<db::Database> Db,
@@ -244,7 +244,7 @@ void TaskScheduler::runTasksAsync() {
244244
**Concept:** Break `runAll()` into stages that post back to the event loop between steps.
245245

246246
```cpp
247-
// src/git/scheduler.cpp (alternative implementation)
247+
// src/github/scheduler.cpp (alternative implementation)
248248

249249
void TaskScheduler::runTasksAsync() {
250250
spdlog::info("Starting async task pipeline");
@@ -424,12 +424,12 @@ graph TB
424424
### Setup
425425

426426
```cpp
427-
// include/git/scheduler.hpp
427+
// include/insights/github/scheduler.hpp
428428
#include <asio/co_spawn.hpp>
429429
#include <asio/detached.hpp>
430430
#include <asio/awaitable.hpp>
431431

432-
namespace insights::git {
432+
namespace insights::github {
433433

434434
class TaskScheduler {
435435
// ... existing members ...
@@ -444,10 +444,10 @@ private:
444444
```
445445
446446
```cpp
447-
// src/git/scheduler.cpp
447+
// src/github/scheduler.cpp
448448
#include <asio/use_awaitable.hpp>
449449
450-
namespace insights::git {
450+
namespace insights::github {
451451
452452
void TaskScheduler::runTasksAsync() {
453453
// Spawn coroutine on the event loop

0 commit comments

Comments
 (0)