Commit f538843
authored
Add an admin client for dataset management (#17)
* *: Add dependencies and model generation for admin client
- Add pydantic>=2.0,<2.12 for data models (constrained for PyIceberg)
- Add datamodel-code-generator for model generation from OpenAPI
- Add respx for HTTP mocking in tests
- Add Makefile target for regenerating models
- Generate 838 lines of Pydantic v2 models from OpenAPI spec
- Add test manifest files for dataset registration testing
* admin: Implement admin client infrastructure
- Create AdminClient base class with HTTP request handling and
error mapping
- Implement DatasetsClient with register/deploy/list/delete operations
- Implement JobsClient with get/list/wait/stop/delete operations
- Implement SchemaClient for SQL validation and schema inference
- Create DeploymentContext for chainable deployment workflows
- Add exception hierarchy with 30+ typed error classes mapped from
API codes
- Support automatic job polling with configurable timeout
* client: Integrate admin client with unified Client class
- Add query_url and admin_url parameters to Client (backward compatible
with url)
- Add datasets, jobs, schema properties for admin operations
- Extend QueryBuilder with with_dependency() for manifest dependencies
- Add to_manifest() for generating dataset manifests from SQL queries
- Add register_as() for one-line registration returning DeploymentContext
- Support fluent API: query → with_dependency → register_as → deploy
- Maintain backward compatibility (existing Client(url=...) still works)
* tests: Add admin client tests
- Add 10 unit tests for error mapping and exception hierarchy
- Add 10 unit tests for Pydantic model validation
- Add 10 integration tests for AdminClient HTTP operations
- Add 10 integration tests for DatasetsClient operations
- Add 18 integration tests for JobsClient operations including polling
- All 48 tests use respx for HTTP mocking (no real server required)
- 0.65s execution time on dev machine
* docs: Update README with admin client features
- Add admin client to feature list
- Add quick start examples for admin operations
- Add links to admin client guide and API reference
- Update overview to highlight dataset management capabilities
* docs: Add admin client documentation and examples
- Add comprehensive admin_client_guide.md with usage patterns and
best practices
- Add complete API reference in docs/api/admin_api.md1 parent 7c66375 commit f538843
File tree
34 files changed
+10819
-10
lines changed- docs
- api
- scripts
- specs
- src/amp
- admin
- tests
- config/manifests
- integration/admin
- unit
- admin
34 files changed
+10819
-10
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
| 2 | + | |
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
62 | 66 | | |
63 | 67 | | |
64 | 68 | | |
65 | 69 | | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
66 | 75 | | |
67 | 76 | | |
68 | 77 | | |
| |||
115 | 124 | | |
116 | 125 | | |
117 | 126 | | |
| 127 | + | |
118 | 128 | | |
119 | 129 | | |
120 | 130 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
11 | 18 | | |
12 | 19 | | |
13 | 20 | | |
| |||
21 | 28 | | |
22 | 29 | | |
23 | 30 | | |
24 | | - | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
25 | 82 | | |
26 | 83 | | |
27 | 84 | | |
| |||
30 | 87 | | |
31 | 88 | | |
32 | 89 | | |
33 | | - | |
| 90 | + | |
34 | 91 | | |
35 | 92 | | |
36 | 93 | | |
37 | 94 | | |
38 | | - | |
| 95 | + | |
39 | 96 | | |
40 | 97 | | |
41 | 98 | | |
42 | 99 | | |
43 | 100 | | |
44 | 101 | | |
45 | 102 | | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
46 | 107 | | |
47 | 108 | | |
48 | 109 | | |
| |||
0 commit comments