Skip to content

Commit 1c763fc

Browse files
adrolliadrolli
committed
Connect, Localization, Data
1 parent acf619a commit 1c763fc

File tree

265 files changed

+14468
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

265 files changed

+14468
-0
lines changed

‎packages/connect/.github/FUNDING.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# These are supported funding model platforms
2+
3+
github: [mooxphp]
4+
patreon: # Replace with a single Patreon username
5+
open_collective: # Replace with a single Open Collective username
6+
ko_fi: # Replace with a single Ko-fi username
7+
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
8+
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
9+
liberapay: # Replace with a single Liberapay username
10+
issuehunt: # Replace with a single IssueHunt username
11+
otechie: # Replace with a single Otechie username
12+
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
13+
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

‎packages/connect/.gitignore‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Environment
2+
.env
3+
.env.backup
4+
5+
# Composer
6+
/vendor
7+
composer.lock
8+
auth.json
9+
10+
# NPM / Node
11+
/node_modules
12+
npm-debug.log
13+
package-lock.json
14+
15+
# Laravel
16+
/public/hot
17+
/public/storage
18+
/storage/*.key
19+
20+
# PHPUnit
21+
.phpunit.result.cache
22+
phpunit.xml
23+
24+
# Yarn
25+
yarn-error.log
26+
27+
# PHPStan
28+
/build
29+
phpstan.neon
30+
31+
# Testbench
32+
testbench.yaml
33+
/workbench/*
34+
35+
# PHP CS Fixer
36+
.php-cs-fixer.cache
37+
38+
# Homestead
39+
Homestead.json
40+
Homestead.yaml
41+
42+
# IDEs
43+
/.idea
44+
/.vscode
45+
46+
# MacOS
47+
.DS_Store
48+
49+
# Windows
50+
Thumbs.db

‎packages/connect/CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Changelog
2+
3+
We currently don't track changes in this package. Please refer to the [Moox Monorepo](https://github.com/mooxphp/moox) for the latest changes.
4+
5+
We'll add a changelog in the future.

‎packages/connect/DEVLOG.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Devlog
2+
3+
This is our tasklist in the order we need to do them. Please keep this file automatically updated with the current state of the tasks. If new tasks need to be added, please make a suggestion before adding them.
4+
5+
- [ ] Fix Base API Client, after having models ready
6+
7+
- [ ] Complete missing implementations in auth and client classes:
8+
9+
- [ ] GraphQLAuthStrategy
10+
- Missing applyToRequest() implementation
11+
- Missing token refresh logic
12+
- Missing actual GraphQL authentication flow
13+
- [ ] GraphQLApiClient
14+
- Missing complete query execution
15+
- Missing mutation handling
16+
- Missing GraphQL-specific error handling
17+
- [ ] RestApiClient
18+
- Missing request execution
19+
- Missing response parsing
20+
- Missing header handling
21+
- [ ] BaseSyncJob
22+
- Missing error handling implementation
23+
- Missing notification logic
24+
- Missing sync execution
25+
26+
- [ ] Complete missing support systems:
27+
28+
- [ ] RateLimiter
29+
- Missing implementation of rate limiting strategies
30+
- Missing distributed rate limiting support
31+
- [ ] RetryHandler
32+
- Missing retry strategies implementation
33+
- Missing backoff logic
34+
- [ ] StatusManager
35+
- Missing job status tracking
36+
- Missing API status monitoring
37+
- [ ] WebhookHandler
38+
- Missing complete implementation
39+
40+
- [ ] Complete Request/Response system:
41+
42+
- [ ] ApiRequest
43+
- Missing concrete implementation
44+
- Missing request building logic
45+
- [ ] ApiResponse
46+
- Missing response parsing
47+
- Missing error handling
48+
- [ ] ResponseParser
49+
- Missing implementation
50+
- [ ] RequestBuilder
51+
- Missing implementation
52+
53+
- [ ] Missing validation for transformer priority ranges?
54+
- [ ] Missing type hints in transformer options (could be an interface/enum)?
55+
- [ ] Missing status handling, when do we error an API?
56+
57+
- [ ] Review the complete package, PHPStan and refactor where needed
58+
59+
## Testing
60+
61+
We would be able to create tests that mock the API to test things without actually calling the API or even without having the DB layer ready. But we need to have the DB layer ready to test the whole thing.
62+
63+
- [ ] There is currently a jwt auth test, nothing else
64+
- [ ] Unit Tests (Auth, API Client, Support Systems, Jobs, Transformers)
65+
- [ ] Integration Tests (Auth flows, Request/Response cycle, Error handling, Rate limiting)
66+
- [ ] Feature Tests (Mock API, Real-world scenarios, Error recovery, Rate limiting)
67+
68+
## Database & UI
69+
70+
- [ ] Create database migrations
71+
- [ ] Implement Eloquent models
72+
- [ ] Build Filament resources including repeater for ApiEndpoint
73+
- [ ] Wire the missing models and add missing fields
74+
75+
## Get it running
76+
77+
- [ ] Run on DB values for the first time
78+
- [ ] In this phase, we will add both APIs and hopefully sync the first data.
79+
- [ ] API controller for accessing the API directly including demo components and logging
80+
- [ ] Fine tuning of using the freshness to invoke direct API access
81+
82+
## Enhancements
83+
84+
- [ ] Direct access for Endpoints and Jobs
85+
- [ ] Discuss response time degradation, how to capture it
86+
- [ ] Discuss handling different types of errors (http status codes, etc.)
87+
- [ ] Distinguish between permanent, temporary and transient errors
88+
- [ ] Log level configuration (all including direct, periodic / info,error)
89+
- [ ] Error policy configuration (how many failures before we disable an API?)
90+
- [ ] When and how to reset error count? How to handle?
91+
- [ ] Buffer configuration: 100 requests, 10 seconds
92+
- [ ] Implement buffer for API logging and updating last_used
93+
- [ ] MultiAuth with Token inheritance
94+
- [ ] Add automatic token refresh handling
95+
- [ ] Add lang_param and default_locale support in API requests
96+
- [ ] Auto-refresh detection (401 Unauthorized should trigger refresh).
97+
- [ ] Token inheritance logic (sub-requests inherit main API token).
98+
- [ ] Add WebhookListener to receive API updates.
99+
- [ ] Register webhooks via UI.
100+
- [ ] Adding test buttons for APIs, endpoints, and jobs
101+
- [ ] API Import and Discovery, OpenAPI etc.
102+
- [ ] Add event system for real-time updates
103+
- [ ] Extend logs to track API latency, error counts, and last success time
104+
- [ ] Create dashboard widgets
105+
- [ ] TransformerChain implementation
106+
- [ ] CLI commands for managing APIs, endpoints, and jobs
107+
- [ ] CLI commands for getting status and health
108+
- [ ] An exchange format (JSON) for the complete API configuration from the DB
109+
- [ ] A way to import and export the configuration

‎packages/connect/LICENSE.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
The MIT License (MIT)
2+
3+
Copyright (c) Moox <[email protected]>
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in
13+
all copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21+
THE SOFTWARE.

‎packages/connect/MODELS.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# Database Models
2+
3+
We document each model including all fields, relationships and what they are used for.
4+
5+
## ApiConnection
6+
7+
Stores API configurations including:
8+
9+
- `id` (Primary Key)
10+
- `name` (string) - Human-readable name for the API connection
11+
- `base_url` (string) - Base URL of the API
12+
- `api_type` (enum) - Defines API type (`REST`, `GraphQL`)
13+
- `auth_type` (enum) - Authentication type (e.g., Bearer, Basic, OAuth)
14+
- `auth_credentials` (json) - Stores API keys, tokens, or credentials
15+
- `headers` (json) - Default headers to be sent with requests
16+
- `rate_limit` (integer) - Maximum requests per minute/hour
17+
- `lang_param` (string) - Parameter name used for language selection
18+
- `default_locale` (string) - Default language to be used in API calls
19+
- `status` (enum) - New, Unused, Active, Error and Disabled
20+
- `notify_on_failure` (boolean, default: `true`)
21+
- `notify_on_error` (boolean, default: `true`)
22+
- `notify_email` (string, nullable) - Email to notify on failure
23+
- `last_used` (timestamp) - Last time the API was used
24+
- `last_error` (timestamp) - Last time the API had an error
25+
- `error_window` (integer, default: 3600) - Seconds after which we should reset the error count
26+
- `error_count` (integer, default: 0) - Count of errors per error_window
27+
- `error_limit` (integer, default: 10) - Count of errors before we disable an API
28+
- `created_at` (timestamp)
29+
- `updated_at` (timestamp)
30+
31+
## ApiEndpoint
32+
33+
API endpoints, methods (REST), queries/mutations (GraphQL), expected responses:
34+
35+
- `id` (Primary Key)
36+
- `name` (string) - Human-readable name for the API endpoint
37+
- `direct_access` (boolean, default: false) - Allow direct API calls
38+
- `api_connection_id` (Foreign Key) - Links to `ApiConnection`
39+
- `path` (string, nullable) - API endpoint path (for REST, e.g., `/users`)
40+
- `method` (enum, nullable) - HTTP method (GET, POST, PUT, DELETE, PATCH) for REST APIs
41+
- `query` (text, nullable) - GraphQL query or mutation
42+
- `variables` (json, nullable) - JSON object of GraphQL variables
43+
- `response_map` (json, nullable) - Defines how to extract and store fields from the GraphQL response
44+
- `expected_response` (json) - Schema of the expected response
45+
- `field_mappings` (json, nullable) - Stores Filament Repeater mappings for syncing API data to DB fields
46+
- `lang_override` (string, nullable) - Optional override for language in this specific endpoint
47+
- `rate_limit` (integer, nullable) - Custom rate limit for this endpoint
48+
- `transformers` (json, nullable) - Transformers for this endpoint
49+
- `rate_window` (integer, nullable) - Custom time window for rate limiting
50+
- `status` (enum) - New, Unused, Active, Error and Disabled
51+
- `last_used` (timestamp) - Last time the API was used
52+
- `last_error` (timestamp) - Last time the API had an error
53+
- `error_count` (integer, default: 0) - Count of errors
54+
- `timeout` (integer, default: 30) - Request timeout in seconds
55+
- `created_at` (timestamp)
56+
- `updated_at` (timestamp)
57+
58+
## ApiSyncJob
59+
60+
Define and track API synchronization jobs and their execution statuses:
61+
62+
- `id` (Primary Key)
63+
- `name` (string) - Human-readable name for the API sync job
64+
- `direct_access` (boolean, default: false) - Allow direct job execution
65+
- `api_endpoint_ids` (json) - Array of API Endpoints to process in order
66+
- `status` (enum) - Status of sync (pending, running, completed, failed)
67+
- `last_sync_at` (timestamp) - Last successful sync timestamp
68+
- `chained_job_id`(foreign_key, nulable) - Links to another `ApiSyncJob` for chaining
69+
- `chained_parameters`(json) - stores the source fields to target params
70+
- `batch`(bool) - create a batch
71+
- `batch_size`(int) - create a batch
72+
- `batch_mode`(enum: `sequential`, `parallel`)
73+
- `batch_delay` (integer, nullable) - seconds
74+
- `freshness_ttl` (int) - e. g. 86400
75+
- `last_fresh_at` (timestamp) - updated every time the job successfully completes
76+
- `auto_refresh`(bool) - should the job be called from the frontend when stale
77+
- `auto_refresh_mode`(bool) - queue / sync
78+
- `max_retries` (integer) - after how many errors should we stay in error state
79+
- `retry_strategy` (enum: `fixed`, `exponential`, `linear`)
80+
- `retry_delay` (integer, default 60 sec)
81+
- `notify_on_failure` (boolean, default: `true`)
82+
- `timeout` (integer, default: 300) - Total job timeout
83+
- `created_at` (timestamp)
84+
- `updated_at` (timestamp)
85+
86+
### Chained Parameters
87+
88+
```json
89+
[
90+
{
91+
"source_field": "data[].id",
92+
"target_param": "{id}"
93+
}
94+
]
95+
```
96+
97+
## ApiLog
98+
99+
Logs for API sync jobs and direct API calls:
100+
101+
- `id` (Primary Key)
102+
- `name` (string) - Human-readable name for the API sync job
103+
- `api_sync_job_id` (foreign_key)
104+
- `request_data`
105+
- `response_data`
106+
- `status_code`
107+
- `error_message`
108+
- `created_at` (timestamp)

0 commit comments

Comments
 (0)