Skip to content

Conversation

@AronPerez
Copy link
Contributor

@AronPerez AronPerez commented Jul 21, 2025

Ticket

Description

This PR implements a Rust-compatible design pattern system for the DataFed repository management, preparing the codebase for eventual Rust migration while maintaining JavaScript functionality.

Key changes:

  1. Created Repository Type System - Implemented enum-like types (globus, metadata_only) to support different repository behaviors
  2. Replaced Inheritance with Composition - Transformed OOP patterns to use trait-like operations and factory patterns
  3. Added Result Type Error Handling - Implemented Rust-style Result types for explicit error propagation
  4. Created Modular Repository Implementation - Separated concerns into:
    - types.js - Type definitions and data structures
    - factory.js - Type-based repository creation
    - operations.js - Trait-like repository operations
    - validation.js - Pure validation functions
    - globus.js / metadata.js - Type-specific implementations
  5. Added Rust Book References - Comprehensive documentation linking to relevant Rust patterns

How Has This Been Tested?

  • Docker Compose with ArangoDB 3.12.4
  • Node.js Foxx microservices
  • Local development environment
Unit Tests - Created repository_types.test.js covering:

  • Type creation and validation
  • Factory pattern functionality
  • Operation implementations
  • Error handling scenarios

Integration Tests - Created test-foxx-api.sh:

  • API endpoint connectivity
  • Repository CRUD operations
  • Type-specific behaviors
  • Error response validation

Artifacts

API Response Examples:

  // Version endpoint working at /api/1
  {
    "release_year": 2025,
    "api_major": 1,
    "api_minor": 1,
    "component_major": 1
  }

  // Repository with type field
  {
    "_id": "repo/metadata_only_repo",
    "type": "metadata_only",
    "title": "Metadata Only Repository",
    "capacity": 1000000
  }

Test Execution Output:

  === Foxx API Test Suite ===
  Testing: Version endpoint... PASSED
  Testing: Foxx service health... PASSED
  Testing: List repositories endpoint... PASSED
  Tests passed: 3

Summary by Sourcery

Introduce a Rust-compatible repository type system in JavaScript by modularizing repository logic into enum-like types, trait-like operations, and composition-based factories, update the Foxx API to use the new patterns while preserving legacy compatibility, and add supporting tests, examples, and documentation to facilitate a future Rust migration.

New Features:

  • Define a RepositoryType enum and a Rust-style Result type for explicit error handling
  • Implement factory-based repository creation and trait-like operations in types.js, factory.js, operations.js, validation.js, globus.js, and metadata.js
  • Add a new Foxx API router (repo_router_new.js) that uses the repository type system for CRUD and allocation endpoints
  • Provide an ArangoDB migration script to backfill the type field on existing repositories
  • Include usage examples (example.js) and a detailed README documenting the Rust-compatible design patterns

Enhancements:

  • Refactor the legacy Repo class to delegate to the new repository operations internally for backward compatibility

Documentation:

  • Add a comprehensive README in the repository module illustrating the design patterns and migration benefits
  • Include example code demonstrating repository creation, trait-like operations, and error handling

Tests:

  • Add unit tests in repository_types.test.js covering types, factory, validation, and operations
  • Add integration tests for the Foxx API endpoints to verify repository CRUD and allocation behaviors

Chores:

  • Modularize repository code into separate files for better maintainability and future Rust migration

Copy link
Collaborator

@JoshuaSBrown JoshuaSBrown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address these two items.

message: "Missing required repository fields",

@param {string} config.pub_key - Public SSH key for authentication

* @param {string[]} config.admins - Array of admin user IDs
* @param {string} [config.endpoint] - Globus endpoint (required for GLOBUS type)
* @param {string} [config.path] - File path (required for GLOBUS type)
* @param {string} [config.pub_key] - Public SSH key (required for GLOBUS type)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not an ssh key, see below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

megatnt1122 and others added 3 commits August 25, 2025 14:55
* style: Formatted the install scripts

Co-authored-by: Joshua S Brown <[email protected]>
* style: Formatted all remaining scripts within core

* style: Formatted all remaining scripts within common

* style: Formatted all scripts within compose folder

* style: Formatting remaining scripts within DataFed
@AronPerez AronPerez force-pushed the feat-DAPS-1513-rust-js-design branch from ac3a1f3 to de5636b Compare August 26, 2025 17:13
* refactor: run docker foxx with unpriviledged user.

* docs: update documentation on running foxx.

* fix: add security flags to end to end runs, and add group write permissions to folders.

* fix: add .gitlab contents to rules for building foxx image.

* fix: make datafed.sh writable if it is leftover from Dockerfile.dependencies

* fix: improve performance of docker build by scoping permission changes.

* fix: add additional permissions to folders for foxx install

* fix: cleanup pre existing cmake generated files.

* feat: add support for https in install_foxx, as well as detection of unsafe certs.

* feat: support running foxx container on OpenShift with limited permissions, and with ssl enabled.

* feat: run core server container longer on startup before failing on failed database connection.

* fix: prevent configuring proto files if only using foxx.

* tests: add foxx setup scripts, and grant execution mode.

* fix: add BUILD_WEB as part of the conditions for configuring the version.prot file.

* fix: add nvm folder to set of folders that require permissions changes.

* style: fix shell formatting of entrypoint_foxx and install_foxx.sh
@AronPerez AronPerez force-pushed the feat-DAPS-1513-rust-js-design branch from 4d1a056 to 33c787f Compare August 27, 2025 15:49
This reverts commit c5c2874.

Revert "[DAPS-1531] Update comments"

This reverts commit 464b0f1.

Revert "[DAPS-1531] Address CI"

This reverts commit 4d66b75.

Revert "[DAPS-1531] Object to object"

This reverts commit 5c39a8d.

Revert "[DAPS-1531] Address Object -> object, address copilot comments"

This reverts commit ae39a84.
* [DAPS-1514] Add repository validation logic with tests.
Add minimal Result type for Rust-like error handling.
Implement pure validation functions for repository fields.
Add comprehensive unit tests for all validation functions.
Register validation tests in CMake configuration

* [DAPS-1514] Run prettier
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Core Relates to core service

Projects

None yet

7 participants