Skip to content

Feat postgres integration #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jul 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
2b8419b
chore: add ht_data_postgres dependency and update logging version
fulleni Jul 6, 2025
9b07fdb
feat(config): add EnvironmentConfig class for managing environment va…
fulleni Jul 6, 2025
e43c5f3
feat(core): create server entrypoint for db connection and logging
fulleni Jul 6, 2025
50f101e
refactor(core): strip middleware of in-memory dependency injection
fulleni Jul 6, 2025
c2aff30
fix(core): correct db connection and provide all repositories
fulleni Jul 6, 2025
6dcbc8a
feat(server): add ht_data_client import for data handling
fulleni Jul 6, 2025
03eca2d
feat(core): initialize and provide application services in server
fulleni Jul 6, 2025
494d83a
refactor(core): rebuild root middleware to consume providers
fulleni Jul 6, 2025
d20b3aa
refactor(dependencies): remove ht_data_inmemory dependency from pubsp…
fulleni Jul 6, 2025
8013166
feat(data): create initial DatabaseSeedingService class
fulleni Jul 6, 2025
8c2c579
feat(data): implement table creation in DatabaseSeedingService
fulleni Jul 6, 2025
04189f3
feat(data): implement global fixture data seeding
fulleni Jul 6, 2025
31bddd2
feat(data): implement seeding for admin user and app config
fulleni Jul 6, 2025
35fa931
fix(data): correct admin user seeding logic
fulleni Jul 6, 2025
dc758e1
fix(data): seed default settings and preferences for admin user
fulleni Jul 6, 2025
745ab9d
feat(core): integrate database seeding into server startup
fulleni Jul 6, 2025
68a2fb5
docs: update README with postgresql setup instructions
fulleni Jul 6, 2025
cccd890
lint: misc
fulleni Jul 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 21 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,23 +66,38 @@ for more details.

1. **Prerequisites:**
* Dart SDK (`>=3.0.0`)
* PostgreSQL (`>=14.0` recommended)
* Dart Frog CLI (`dart pub global activate dart_frog_cli`)
2. **Clone the repository:**

2. **Configuration:**
Before running the server, you must configure the database connection by
setting the `DATABASE_URL` environment variable.

Create a `.env` file in the root of the project or export the variable in
your shell:
```
DATABASE_URL="postgres://user:password@localhost:5432/ht_api_db"
```

3. **Clone the repository:**
```bash
git clone https://github.com/headlines-toolkit/ht-api.git
cd ht-api
```
3. **Get dependencies:**
4. **Get dependencies:**
```bash
dart pub get
```
4. **Run the development server:**
5. **Run the development server:**
```bash
dart_frog dev
```
The API will typically be available at `http://localhost:8080`. Fixture data
from `lib/src/fixtures/` will be loaded into the in-memory repositories on
startup.
The API will typically be available at `http://localhost:8080`. On the
first startup, the server will connect to your PostgreSQL database, create the
necessary tables, and seed them with initial fixture data. This process is
non-destructive; it uses `CREATE TABLE IF NOT EXISTS` and `INSERT ... ON
CONFLICT DO NOTHING` to avoid overwriting existing tables or data.


**Note on Web Client Integration (CORS):**
To allow web applications (like the HT Dashboard) to connect to this API,
Expand Down
33 changes: 33 additions & 0 deletions lib/src/config/environment_config.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import 'dart:io';

/// {@template environment_config}
/// A utility class for accessing environment variables.
///
/// This class provides a centralized way to read configuration values
/// from the environment, ensuring that critical settings like database
/// connection strings are managed outside of the source code.
/// {@endtemplate}
abstract final class EnvironmentConfig {
/// Retrieves the PostgreSQL database connection URI from the environment.
///
/// The value is read from the `DATABASE_URL` environment variable.
///
/// Throws a [StateError] if the `DATABASE_URL` environment variable is not
/// set, as the application cannot function without it.
static String get databaseUrl {
final dbUrl = Platform.environment['DATABASE_URL'];
if (dbUrl == null || dbUrl.isEmpty) {
throw StateError(
'FATAL: DATABASE_URL environment variable is not set. '
'The application cannot start without a database connection.',
);
}
return dbUrl;
}

/// Retrieves the current environment mode (e.g., 'development', 'production').
///
/// The value is read from the `ENV` environment variable.
/// Defaults to 'production' if the variable is not set.
static String get environment => Platform.environment['ENV'] ?? 'production';
}
236 changes: 236 additions & 0 deletions lib/src/config/server.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
import 'dart:io';

import 'package:dart_frog/dart_frog.dart';
import 'package:ht_api/src/config/environment_config.dart';
import 'package:ht_api/src/rbac/permission_service.dart';
import 'package:ht_api/src/services/auth_service.dart';
import 'package:ht_api/src/services/auth_token_service.dart';
import 'package:ht_api/src/services/dashboard_summary_service.dart';
import 'package:ht_api/src/services/database_seeding_service.dart';
import 'package:ht_api/src/services/default_user_preference_limit_service.dart';
import 'package:ht_api/src/services/jwt_auth_token_service.dart';
import 'package:ht_api/src/services/token_blacklist_service.dart';
import 'package:ht_api/src/services/user_preference_limit_service.dart';
import 'package:ht_api/src/services/verification_code_storage_service.dart';
import 'package:ht_data_client/ht_data_client.dart';
import 'package:ht_data_postgres/ht_data_postgres.dart';
import 'package:ht_data_repository/ht_data_repository.dart';
import 'package:ht_email_inmemory/ht_email_inmemory.dart';
import 'package:ht_email_repository/ht_email_repository.dart';
import 'package:ht_shared/ht_shared.dart';
import 'package:logging/logging.dart';
import 'package:postgres/postgres.dart';
import 'package:uuid/uuid.dart';

/// Global logger instance.
final _log = Logger('ht_api');

/// Global PostgreSQL connection instance.
late final Connection _connection;

/// Creates a data repository for a given type [T].
///
/// This helper function centralizes the creation of repositories,
/// ensuring they all use the same database connection and logger.
HtDataRepository<T> _createRepository<T>({
required String tableName,
required FromJson<T> fromJson,
required ToJson<T> toJson,
}) {
return HtDataRepository<T>(
dataClient: HtDataPostgresClient<T>(
connection: _connection,
tableName: tableName,
fromJson: fromJson,
toJson: toJson,
log: _log,
),
);
}

/// The main entry point for the server.
///
/// This function is responsible for:
/// 1. Setting up the global logger.
/// 2. Establishing the PostgreSQL database connection.
/// 3. Providing these dependencies to the Dart Frog handler.
/// 4. Gracefully closing the database connection on server shutdown.
Future<HttpServer> run(Handler handler, InternetAddress ip, int port) async {
// 1. Setup Logger
Logger.root.level = Level.ALL;
Logger.root.onRecord.listen((record) {
// ignore: avoid_print
print(
'${record.level.name}: ${record.time}: '
'${record.loggerName}: ${record.message}',
);
});

// 2. Establish Database Connection
_log.info('Connecting to PostgreSQL database...');
final dbUri = Uri.parse(EnvironmentConfig.databaseUrl);
String? username;
String? password;
if (dbUri.userInfo.isNotEmpty) {
final parts = dbUri.userInfo.split(':');
username = Uri.decodeComponent(parts.first);
if (parts.length > 1) {
password = Uri.decodeComponent(parts.last);
}
}

_connection = await Connection.open(
Endpoint(
host: dbUri.host,
port: dbUri.port,
database: dbUri.path.substring(1), // Remove leading '/'
username: username,
password: password,
),
// Using `require` is a more secure default. For local development against
// a non-SSL database, this may need to be changed to `SslMode.disable`.
settings: const ConnectionSettings(sslMode: SslMode.require),
);
Comment on lines +71 to +93

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The current manual parsing of the database connection URI is fragile. Specifically, dbUri.userInfo.split(':') will not correctly handle passwords that contain a colon (:).

To make the parsing more robust and simplify the code, you should use the Endpoint.fromUri factory constructor provided by the postgres package. It correctly handles all components of a standard PostgreSQL connection URI, including complex usernames and passwords.

  final endpoint = Endpoint.fromUri(Uri.parse(EnvironmentConfig.databaseUrl));

  _connection = await Connection.open(
    endpoint,
    // Using `require` is a more secure default. For local development against
    // a non-SSL database, this may need to be changed to `SslMode.disable`.
    settings: const ConnectionSettings(sslMode: SslMode.require),
  );

_log.info('PostgreSQL database connection established.');

// 3. Initialize and run database seeding
// This runs on every startup. The operations are idempotent (`IF NOT EXISTS`,
// `ON CONFLICT DO NOTHING`), so it's safe to run every time. This ensures
// the database is always in a valid state, especially for first-time setup
// in any environment.
final seedingService = DatabaseSeedingService(
connection: _connection,
log: _log,
);
await seedingService.createTables();
await seedingService.seedGlobalFixtureData();
await seedingService.seedInitialAdminAndConfig();

// 4. Initialize Repositories
final headlineRepository = _createRepository<Headline>(
tableName: 'headlines',
fromJson: Headline.fromJson,
toJson: (h) => h.toJson(),
);
final categoryRepository = _createRepository<Category>(
tableName: 'categories',
fromJson: Category.fromJson,
toJson: (c) => c.toJson(),
);
final sourceRepository = _createRepository<Source>(
tableName: 'sources',
fromJson: Source.fromJson,
toJson: (s) => s.toJson(),
);
final countryRepository = _createRepository<Country>(
tableName: 'countries',
fromJson: Country.fromJson,
toJson: (c) => c.toJson(),
);
final userRepository = _createRepository<User>(
tableName: 'users',
fromJson: User.fromJson,
toJson: (u) => u.toJson(),
);
final userAppSettingsRepository = _createRepository<UserAppSettings>(
tableName: 'user_app_settings',
fromJson: UserAppSettings.fromJson,
toJson: (s) => s.toJson(),
);
final userContentPreferencesRepository =
_createRepository<UserContentPreferences>(
tableName: 'user_content_preferences',
fromJson: UserContentPreferences.fromJson,
toJson: (p) => p.toJson(),
);
final appConfigRepository = _createRepository<AppConfig>(
tableName: 'app_config',
fromJson: AppConfig.fromJson,
toJson: (c) => c.toJson(),
);

// 5. Initialize Services
const uuid = Uuid();
const emailRepository = HtEmailRepository(
emailClient: HtEmailInMemoryClient(),
);
final tokenBlacklistService = InMemoryTokenBlacklistService();
final authTokenService = JwtAuthTokenService(
userRepository: userRepository,
blacklistService: tokenBlacklistService,
uuidGenerator: uuid,
);
final verificationCodeStorageService =
InMemoryVerificationCodeStorageService();
final authService = AuthService(
userRepository: userRepository,
authTokenService: authTokenService,
verificationCodeStorageService: verificationCodeStorageService,
emailRepository: emailRepository,
userAppSettingsRepository: userAppSettingsRepository,
userContentPreferencesRepository: userContentPreferencesRepository,
uuidGenerator: uuid,
);
final dashboardSummaryService = DashboardSummaryService(
headlineRepository: headlineRepository,
categoryRepository: categoryRepository,
sourceRepository: sourceRepository,
);
const permissionService = PermissionService();
final userPreferenceLimitService = DefaultUserPreferenceLimitService(
appConfigRepository: appConfigRepository,
);

// 6. Create the main handler with all dependencies provided
final finalHandler = handler
// Foundational utilities
.use(provider<Uuid>((_) => uuid))
// Repositories
.use(provider<HtDataRepository<Headline>>((_) => headlineRepository))
.use(provider<HtDataRepository<Category>>((_) => categoryRepository))
.use(provider<HtDataRepository<Source>>((_) => sourceRepository))
.use(provider<HtDataRepository<Country>>((_) => countryRepository))
.use(provider<HtDataRepository<User>>((_) => userRepository))
.use(
provider<HtDataRepository<UserAppSettings>>(
(_) => userAppSettingsRepository,
),
)
.use(
provider<HtDataRepository<UserContentPreferences>>(
(_) => userContentPreferencesRepository,
),
)
.use(provider<HtDataRepository<AppConfig>>((_) => appConfigRepository))
.use(provider<HtEmailRepository>((_) => emailRepository))
// Services
.use(provider<TokenBlacklistService>((_) => tokenBlacklistService))
.use(provider<AuthTokenService>((_) => authTokenService))
.use(
provider<VerificationCodeStorageService>(
(_) => verificationCodeStorageService,
),
)
.use(provider<AuthService>((_) => authService))
.use(provider<DashboardSummaryService>((_) => dashboardSummaryService))
.use(provider<PermissionService>((_) => permissionService))
.use(
provider<UserPreferenceLimitService>((_) => userPreferenceLimitService),
);

// 7. Start the server
final server = await serve(finalHandler, ip, port);
_log.info('Server listening on port ${server.port}');

// 8. Handle graceful shutdown
ProcessSignal.sigint.watch().listen((_) async {
_log.info('Received SIGINT. Shutting down...');
await _connection.close();
_log.info('Database connection closed.');
await server.close(force: true);
_log.info('Server shut down.');
exit(0);
});

return server;
}
2 changes: 2 additions & 0 deletions lib/src/registry/model_registry.dart
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
// ignore_for_file: comment_references

import 'package:dart_frog/dart_frog.dart';
import 'package:ht_api/src/rbac/permissions.dart';
import 'package:ht_data_client/ht_data_client.dart';
Expand Down
16 changes: 7 additions & 9 deletions lib/src/services/auth_service.dart
Original file line number Diff line number Diff line change
Expand Up @@ -171,11 +171,7 @@ class AuthService {
// Admin users must be provisioned out-of-band (e.g., via fixtures).
final roles = [UserRoles.standardUser];

user = User(
id: _uuid.v4(),
email: email,
roles: roles,
);
user = User(id: _uuid.v4(), email: email, roles: roles);
user = await _userRepository.create(item: user);
print('Created new user: ${user.id} with roles: ${user.roles}');

Expand All @@ -197,7 +193,9 @@ class AuthService {
}
} on HtHttpException catch (e) {
print('Error finding/creating user for $email: $e');
throw const OperationFailedException('Failed to find or create user account.');
throw const OperationFailedException(
'Failed to find or create user account.',
);
} catch (e) {
print('Unexpected error during user lookup/creation for $email: $e');
throw const OperationFailedException('Failed to process user account.');
Expand All @@ -213,7 +211,7 @@ class AuthService {
throw const OperationFailedException(
'Failed to generate authentication token.',
);
}
}
}

/// Performs anonymous sign-in.
Expand All @@ -227,7 +225,7 @@ class AuthService {
try {
user = User(
id: _uuid.v4(), // Generate new ID
roles: [UserRoles.guestUser], // Anonymous users are guest users
roles: const [UserRoles.guestUser], // Anonymous users are guest users
email: null, // Anonymous users don't have an email initially
);
user = await _userRepository.create(item: user);
Expand Down Expand Up @@ -426,7 +424,7 @@ class AuthService {
final updatedUser = User(
id: anonymousUser.id, // Preserve original ID
email: linkedEmail,
roles: [UserRoles.standardUser], // Now a permanent standard user
roles: const [UserRoles.standardUser], // Now a permanent standard user
);
final permanentUser = await _userRepository.update(
id: updatedUser.id,
Expand Down
Loading
Loading