Skip to content

Commit 759b8ed

Browse files
dguisingerdguisinger
authored
Feature/json refactor (#87)
* feat(nuget-package-polish): Add comprehensive NuGet package polish across all packages - Add design specification, requirements, and task documentation for NuGet package polish initiative - Create missing README.md files for Oproto.FluentDynamoDb, Logging.Extensions, NewtonsoftJson, and SystemTextJson packages - Update existing README files for consistency and quality across all packages - Add shared package icon (128x128 PNG) in docs/assets directory - Configure all package .csproj files to reference shared icon and include README files in NuGet packages - Update Directory.Build.props with common package metadata (Authors, Copyright, URLs) - Add FluentDynamoDB logo SVG asset for documentation - Update root README.md with comprehensive project overview and package descriptions - Ensure consistent package descriptions and metadata across all 10 packages - Standardize README structure with installation, usage, features, and links sections * feat(json-serializer-refactor): Refactor JSON serialization to runtime configuration - Add IJsonBlobSerializer interface for pluggable JSON serialization in core library - Create SystemTextJsonBlobSerializer and SystemTextJsonOptionsExtensions for System.Text.Json support - Create NewtonsoftJsonBlobSerializer and NewtonsoftJsonOptionsExtensions for Newtonsoft.Json support - Remove compile-time DynamoDbJsonSerializerAttribute and JsonSerializerType enum - Update FluentDynamoDbOptions to accept IJsonBlobSerializer configuration - Refactor source generator to detect and use runtime serializer configuration - Update JsonSerializerContextGenerator to work with runtime configuration - Add comprehensive integration tests for JsonBlob advanced type handling - Update all unit tests to use new runtime serializer configuration - Add design specification and requirements documentation for refactor initiative - Update CHANGELOG and documentation with new serializer configuration patterns - Enable flexible JSON serializer selection at runtime instead of compile-time attributes * feat(options-propagation-fixes): Fix options propagation in batch/transaction responses and documentation - Add options propagation to BatchGetResponse, BatchGetBuilder, TransactionGetResponse, and TransactionGetBuilder to ensure serializer configuration is available during hydration - Update IAsyncEntityHydrator interface to accept FluentDynamoDbOptions parameter for consistent options handling - Fix HydratorGenerator to pass options through generated hydration code - Correct API usage patterns in documentation: replace `.ExecuteAsync()` with `.PutAsync()` and `.ToListAsync()` for Put and Query operations - Update README files in Oproto.FluentDynamoDb.NewtonsoftJson and Oproto.FluentDynamoDb.SystemTextJson packages with corrected API examples - Add comprehensive design specification for options propagation architecture in `.kiro/specs/options-propagation-fixes/` - Update integration test build fixes specification with corrected API patterns - Update CHANGELOG.md and DOCUMENTATION_CHANGELOG.md with changes and corrections - Update GenerateAccessorsAttribute documentation and advanced topics guides with correct API usage - Fixes issue where [JsonBlob] properties would fail during batch/transaction response hydration due to missing serializer configuration
1 parent 166d61b commit 759b8ed

File tree

78 files changed

+5018
-1139
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+5018
-1139
lines changed

‎.kiro/specs/integration-test-build-fixes/design.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -362,8 +362,8 @@ dotnet build Oproto.FluentDynamoDb.IntegrationTests/Oproto.FluentDynamoDb.Integr
362362
await table.Query().PutAsync(entity);
363363

364364
// Correct - separate operations
365-
await table.Put(entity).ExecuteAsync();
366-
var results = await table.Query().ExecuteAsync();
365+
await table.Put(entity).PutAsync();
366+
var results = await table.Query().ToListAsync();
367367
```
368368

369369
### Pattern 3: Scan Type Inference

‎.kiro/specs/json-serializer-refactor/design.md‎

Lines changed: 455 additions & 0 deletions
Large diffs are not rendered by default.

‎.kiro/specs/json-serializer-refactor/requirements.md‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# JSON Serializer Refactor Requirements
2+
3+
## Overview
4+
5+
Refactor the JSON serialization system for `[JsonBlob]` properties to use runtime configuration via `FluentDynamoDbOptions` instead of compile-time assembly attributes. This makes the JSON packages (`Oproto.FluentDynamoDb.SystemTextJson` and `Oproto.FluentDynamoDb.NewtonsoftJson`) provide real value by allowing users to configure serializer options.
6+
7+
## Problem Statement
8+
9+
Currently:
10+
1. The source generator inlines direct calls to `System.Text.Json.JsonSerializer` or `Newtonsoft.Json.JsonConvert`
11+
2. The JSON packages contain wrapper classes (`SystemTextJsonSerializer`, `NewtonsoftJsonSerializer`) that are never used
12+
3. Users cannot customize serializer options (camelCase, null handling, etc.)
13+
4. Configuration is via compile-time `[assembly: DynamoDbJsonSerializer]` attribute
14+
15+
## Requirements
16+
17+
### 1. Interface Changes
18+
19+
1.1. GIVEN the `IDynamoDbEntity` interface, WHEN I review the `ToDynamoDb` and `FromDynamoDb` methods, THEN they SHALL accept `FluentDynamoDbOptions?` instead of `IDynamoDbLogger?`
20+
21+
1.2. GIVEN the `FluentDynamoDbOptions` class, WHEN I configure JSON serialization, THEN there SHALL be an `IJsonBlobSerializer? JsonSerializer` property available
22+
23+
### 2. Core Library Changes
24+
25+
2.1. GIVEN the core library, WHEN I need JSON serialization, THEN there SHALL be an `IJsonBlobSerializer` interface with `Serialize<T>` and `Deserialize<T>` methods
26+
27+
2.2. GIVEN a `[JsonBlob]` property without a configured serializer, WHEN `ToDynamoDb` or `FromDynamoDb` is called, THEN a clear runtime exception SHALL be thrown explaining the requirement
28+
29+
### 3. SystemTextJson Package
30+
31+
3.1. GIVEN the `Oproto.FluentDynamoDb.SystemTextJson` package, WHEN I want to use System.Text.Json, THEN there SHALL be a `SystemTextJsonBlobSerializer` class implementing `IJsonBlobSerializer`
32+
33+
3.2. GIVEN the SystemTextJson package, WHEN I configure options, THEN I SHALL be able to pass `JsonSerializerOptions` for customization
34+
35+
3.3. GIVEN the SystemTextJson package for AOT scenarios, WHEN I configure options, THEN I SHALL be able to pass a `JsonSerializerContext` for AOT-compatible serialization
36+
37+
3.4. GIVEN the SystemTextJson package, WHEN I want simple configuration, THEN there SHALL be a `WithSystemTextJson()` extension method on `FluentDynamoDbOptions`
38+
39+
### 4. NewtonsoftJson Package
40+
41+
4.1. GIVEN the `Oproto.FluentDynamoDb.NewtonsoftJson` package, WHEN I want to use Newtonsoft.Json, THEN there SHALL be a `NewtonsoftJsonBlobSerializer` class implementing `IJsonBlobSerializer`
42+
43+
4.2. GIVEN the NewtonsoftJson package, WHEN I configure options, THEN I SHALL be able to pass `JsonSerializerSettings` for customization
44+
45+
4.3. GIVEN the NewtonsoftJson package, WHEN I want simple configuration, THEN there SHALL be a `WithNewtonsoftJson()` extension method on `FluentDynamoDbOptions`
46+
47+
### 5. Source Generator Changes
48+
49+
5.1. GIVEN the source generator, WHEN generating `ToDynamoDb`/`FromDynamoDb` methods, THEN it SHALL generate code that calls `options?.JsonSerializer?.Serialize()` instead of inlining JSON library calls
50+
51+
5.2. GIVEN the source generator, WHEN a `[JsonBlob]` property is detected but no JSON package is referenced, THEN it SHALL emit a diagnostic warning
52+
53+
5.3. GIVEN the source generator, WHEN generating entity implementations, THEN it SHALL pass `FluentDynamoDbOptions?` to the interface methods
54+
55+
### 6. Cleanup
56+
57+
6.1. GIVEN the `[assembly: DynamoDbJsonSerializer]` attribute, WHEN this refactor is complete, THEN it SHALL be deleted (not deprecated - pre-release)
58+
59+
6.2. GIVEN the `JsonSerializerType` enum, WHEN this refactor is complete, THEN it SHALL be deleted
60+
61+
6.3. GIVEN the `JsonSerializerDetector` in the source generator, WHEN this refactor is complete, THEN it SHALL be updated to only detect package references for diagnostics (not for code generation)
62+
63+
### 7. Documentation
64+
65+
7.1. GIVEN the SystemTextJson package README, WHEN this refactor is complete, THEN it SHALL document the correct usage pattern with `WithSystemTextJson()`
66+
67+
7.2. GIVEN the NewtonsoftJson package README, WHEN this refactor is complete, THEN it SHALL document the correct usage pattern with `WithNewtonsoftJson()`
68+
69+
7.3. GIVEN the main documentation, WHEN this refactor is complete, THEN all `[JsonBlob]` examples SHALL show the options-based configuration
70+
71+
7.4. GIVEN the CHANGELOG.md, WHEN this refactor is complete, THEN it SHALL document this as a breaking change with migration guidance
72+
73+
7.5. GIVEN the docs/DOCUMENTATION_CHANGELOG.md, WHEN documentation is updated, THEN it SHALL include entries for all documentation changes
74+
75+
### 8. Testing
76+
77+
8.1. GIVEN the unit tests, WHEN `ToDynamoDb`/`FromDynamoDb` signatures change, THEN all affected tests SHALL be updated
78+
79+
8.2. GIVEN the JSON serializer packages, WHEN the implementations change, THEN their unit tests SHALL be updated to test the new `IJsonBlobSerializer` implementations
80+
81+
## User Experience
82+
83+
### Before (Current - Incorrect)
84+
```csharp
85+
// Assembly attribute (compile-time)
86+
[assembly: DynamoDbJsonSerializer(JsonSerializerType.SystemTextJson)]
87+
88+
// Entity with JsonBlob
89+
[DynamoDbTable("Documents")]
90+
public partial class Document
91+
{
92+
[JsonBlob]
93+
[DynamoDbAttribute("content")]
94+
public DocumentContent Content { get; set; }
95+
}
96+
97+
// Table usage - no way to customize serializer options
98+
var table = new DocumentTable(client, "Documents");
99+
```
100+
101+
### After (New - Correct)
102+
```csharp
103+
// Entity with JsonBlob (no assembly attribute needed)
104+
[DynamoDbTable("Documents")]
105+
public partial class Document
106+
{
107+
[JsonBlob]
108+
[DynamoDbAttribute("content")]
109+
public DocumentContent Content { get; set; }
110+
}
111+
112+
// Configure at runtime with options
113+
var options = new FluentDynamoDbOptions()
114+
.WithSystemTextJson() // Uses defaults
115+
// OR with custom options
116+
.WithSystemTextJson(new JsonSerializerOptions
117+
{
118+
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
119+
})
120+
// OR for AOT
121+
.WithSystemTextJson(MyJsonContext.Default);
122+
123+
var table = new DocumentTable(client, "Documents", options);
124+
```
125+
126+
## Out of Scope
127+
128+
- Changes to non-JSON blob serialization
129+
- Changes to the `[DynamoDbMap]` attribute behavior
130+
- Changes to encryption or blob storage providers

‎.kiro/specs/json-serializer-refactor/tasks.md‎

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
# JSON Serializer Refactor Tasks
2+
3+
## Task 1: Core Library Interface Changes
4+
- [x] Create `IJsonBlobSerializer` interface in `Oproto.FluentDynamoDb/Storage/`
5+
- [x] Define `Serialize<T>(T value): string` method
6+
- [x] Define `Deserialize<T>(string json): T?` method
7+
- [x] Add XML documentation
8+
- _Requirements: 2.1_
9+
10+
- [x] Update `FluentDynamoDbOptions` class
11+
- [x] Add `JsonSerializer` property of type `IJsonBlobSerializer?`
12+
- [x] Add `WithJsonSerializer(IJsonBlobSerializer?)` builder method
13+
- [x] Update all existing `With*` methods to preserve `JsonSerializer`
14+
- _Requirements: 1.2_
15+
16+
- [x] Update `IDynamoDbEntity` interface
17+
- [x] Change `ToDynamoDb` signature from `IDynamoDbLogger?` to `FluentDynamoDbOptions?`
18+
- [x] Change `FromDynamoDb` (single item) signature from `IDynamoDbLogger?` to `FluentDynamoDbOptions?`
19+
- [x] Change `FromDynamoDb` (multi item) signature from `IDynamoDbLogger?` to `FluentDynamoDbOptions?`
20+
- _Requirements: 1.1_
21+
22+
## Task 2: Delete Obsolete Files
23+
- [x] Delete `Oproto.FluentDynamoDb/Attributes/DynamoDbJsonSerializerAttribute.cs`
24+
- _Requirements: 6.1_
25+
26+
- [x] Delete `Oproto.FluentDynamoDb/Attributes/JsonSerializerType.cs`
27+
- _Requirements: 6.2_
28+
29+
- [x] Delete `Oproto.FluentDynamoDb.SystemTextJson/SystemTextJsonSerializer.cs`
30+
- _Requirements: 3.1 (replaced by new implementation)_
31+
32+
- [x] Delete `Oproto.FluentDynamoDb.NewtonsoftJson/NewtonsoftJsonSerializer.cs`
33+
- _Requirements: 4.1 (replaced by new implementation)_
34+
35+
## Task 3: SystemTextJson Package Implementation
36+
- [x] Create `SystemTextJsonBlobSerializer` class
37+
- [x] Implement `IJsonBlobSerializer` interface
38+
- [x] Add constructor with no parameters (default options)
39+
- [x] Add constructor accepting `JsonSerializerOptions`
40+
- [x] Add constructor accepting `JsonSerializerContext` for AOT
41+
- [x] Implement `Serialize<T>` method
42+
- [x] Implement `Deserialize<T>` method
43+
- [x] Add XML documentation
44+
- _Requirements: 3.1, 3.2, 3.3_
45+
46+
- [x] Create `SystemTextJsonOptionsExtensions` class
47+
- [x] Add `WithSystemTextJson(this FluentDynamoDbOptions)` extension method
48+
- [x] Add `WithSystemTextJson(this FluentDynamoDbOptions, JsonSerializerOptions)` overload
49+
- [x] Add `WithSystemTextJson(this FluentDynamoDbOptions, JsonSerializerContext)` overload
50+
- [x] Add XML documentation
51+
- _Requirements: 3.4_
52+
53+
- [x] Update `Oproto.FluentDynamoDb.SystemTextJson.csproj`
54+
- [x] Add reference to core library for `IJsonBlobSerializer` and `FluentDynamoDbOptions`
55+
- _Requirements: 3.1_
56+
57+
## Task 4: NewtonsoftJson Package Implementation
58+
- [x] Create `NewtonsoftJsonBlobSerializer` class
59+
- [x] Implement `IJsonBlobSerializer` interface
60+
- [x] Add constructor with no parameters (default settings)
61+
- [x] Add constructor accepting `JsonSerializerSettings`
62+
- [x] Define default settings (TypeNameHandling.None, NullValueHandling.Ignore, etc.)
63+
- [x] Implement `Serialize<T>` method
64+
- [x] Implement `Deserialize<T>` method
65+
- [x] Add XML documentation
66+
- _Requirements: 4.1, 4.2_
67+
68+
- [x] Create `NewtonsoftJsonOptionsExtensions` class
69+
- [x] Add `WithNewtonsoftJson(this FluentDynamoDbOptions)` extension method
70+
- [x] Add `WithNewtonsoftJson(this FluentDynamoDbOptions, JsonSerializerSettings)` overload
71+
- [x] Add XML documentation
72+
- _Requirements: 4.3_
73+
74+
- [x] Update `Oproto.FluentDynamoDb.NewtonsoftJson.csproj`
75+
- [x] Add reference to core library for `IJsonBlobSerializer` and `FluentDynamoDbOptions`
76+
- _Requirements: 4.1_
77+
78+
## Task 5: Source Generator Updates
79+
- [x] Update `MapperGenerator.cs`
80+
- [x] Update method signature generation to use `FluentDynamoDbOptions?` instead of `IDynamoDbLogger?`
81+
- [x] Update `GenerateJsonBlobPropertyToAttributeValue` to call `options?.JsonSerializer?.Serialize()`
82+
- [x] Update `GenerateJsonBlobPropertyFromAttributeValue` to call `options?.JsonSerializer?.Deserialize()`
83+
- [x] Add null check with clear exception message when serializer is not configured
84+
- [x] Update logger access to use `options?.Logger`
85+
- _Requirements: 5.1, 5.3, 2.2_
86+
87+
- [x] Update `JsonSerializerDetector.cs`
88+
- [x] Remove code generation logic (no longer needed)
89+
- [x] Keep package detection for diagnostic purposes only
90+
- [x] Remove `AssemblyLevelSerializer` detection
91+
- _Requirements: 6.3_
92+
93+
- [x] Add diagnostic for missing JSON package
94+
- [x] Use existing `DYNDB102` diagnostic descriptor (MissingJsonSerializer)
95+
- [x] Emit warning when `[JsonBlob]` is used but no JSON package is referenced (already implemented in AdvancedTypeValidator)
96+
- _Requirements: 5.2_
97+
98+
- [x] Update all other generators that reference `IDynamoDbLogger` parameter
99+
- [x] `HydratorGenerator.cs` - updated to pass `options: null` instead of `logger: null`
100+
- [x] `TableGenerator.cs` - already uses `FluentDynamoDbOptions`
101+
- [x] `AdvancedTypeAnalyzer.cs` - removed compile-time JSON serializer detection
102+
- [x] `JsonSerializerContextGenerator.cs` - updated to check package reference directly
103+
- _Requirements: 5.3_
104+
105+
## Task 6: Update Request Builders and Extensions
106+
- [x] 6 Update all callers of `ToDynamoDb`/`FromDynamoDb` in request builders
107+
- [x] 6.1 `PutItemRequestBuilder.cs` - pass options instead of logger
108+
- [x] 6.2 `EnhancedExecuteAsyncExtensions.cs` - pass options instead of logger
109+
- [x] 6.3 Any other files calling these methods
110+
- _Requirements: 5.3_
111+
112+
- [x] Update `DynamoDbTableBase.cs` if needed
113+
- [x] Ensure options are passed through to entity mapping calls (verified - already passes Options to all request builders)
114+
- _Requirements: 5.3_
115+
116+
## Task 7: Update Unit Tests
117+
- [x] Update `Oproto.FluentDynamoDb.UnitTests`
118+
- [x] Update all test entity `ToDynamoDb`/`FromDynamoDb` implementations
119+
- [x] Update all tests that call these methods
120+
- [x] Add tests for `IJsonBlobSerializer` null check exception
121+
- _Requirements: 8.1_
122+
123+
- [x] Update `Oproto.FluentDynamoDb.SystemTextJson.UnitTests`
124+
- [x] Delete tests for old `SystemTextJsonSerializer` class
125+
- [x] Add tests for `SystemTextJsonBlobSerializer`
126+
- [x] Add tests for `WithSystemTextJson` extension methods
127+
- [x] Add tests for AOT path with `JsonSerializerContext`
128+
- _Requirements: 8.2_
129+
130+
- [x] Update `Oproto.FluentDynamoDb.NewtonsoftJson.UnitTests`
131+
- [x] Delete tests for old `NewtonsoftJsonSerializer` class
132+
- [x] Add tests for `NewtonsoftJsonBlobSerializer`
133+
- [x] Add tests for `WithNewtonsoftJson` extension methods
134+
- _Requirements: 8.2_
135+
136+
- [x] Update `Oproto.FluentDynamoDb.SourceGenerator.UnitTests`
137+
- [x] Update `AdvancedTypeGenerationTests.cs` - remove assembly attribute tests
138+
- [x] Add tests for new generated code pattern
139+
- [x] Add tests for diagnostic warning
140+
- _Requirements: 8.1_
141+
- ✅ **VERIFIED**: All 37 tests pass
142+
143+
## Task 8: Update Package READMEs
144+
- [x] Update `Oproto.FluentDynamoDb.SystemTextJson/README.md`
145+
- [x] Document `WithSystemTextJson()` extension method
146+
- [x] Document `WithSystemTextJson(JsonSerializerOptions)` overload
147+
- [x] Document `WithSystemTextJson(JsonSerializerContext)` for AOT
148+
- [x] Show complete usage example with `FluentDynamoDbOptions`
149+
L - [x] Remove documentation for old `SystemTextJsonSerializer` class
150+
- _Requirements: 7.1_
151+
152+
- [x] Update `Oproto.FluentDynamoDb.NewtonsoftJson/README.md`
153+
- [x] Document `WithNewtonsoftJson()` extension method
154+
- [x] Document `WithNewtonsoftJson(JsonSerializerSettings)` overload
155+
- [x] Show complete usage example with `FluentDynamoDbOptions`
156+
- [x] Remove documentation for old `NewtonsoftJsonSerializer` class
157+
- _Requirements: 7.2_
158+
159+
## Task 9: Update Main Documentation
160+
- [x] Update `docs/advanced-topics/AdvancedTypes.md`
161+
- [x] Update `[JsonBlob]` section with new configuration pattern
162+
- [x] Remove assembly attribute examples
163+
- [x] Add `FluentDynamoDbOptions` configuration examples
164+
- _Requirements: 7.3_
165+
166+
- [x] Update `docs/reference/AttributeReference.md`
167+
- [x] Update `[JsonBlob]` documentation
168+
- [x] Remove `[DynamoDbJsonSerializer]` section entirely
169+
- _Requirements: 7.3_
170+
171+
- [x] Update `docs/reference/AdvancedTypesQuickReference.md`
172+
- [x] Update JSON blob configuration examples
173+
- _Requirements: 7.3_
174+
175+
- [x] Update `docs/examples/AdvancedTypesExamples.md`
176+
- [x] Update all `[JsonBlob]` examples with new pattern
177+
- _Requirements: 7.3_
178+
179+
- [x] Update `docs/QUICK_REFERENCE.md`
180+
- [x] Update JSON blob configuration section
181+
- _Requirements: 7.3_
182+
183+
## Task 10: Update Changelogs
184+
- [x] Update `CHANGELOG.md`
185+
- [x] Add breaking change entry for `IDynamoDbEntity` interface change
186+
- [x] Add breaking change entry for removed assembly attribute
187+
- [x] Add new feature entry for `IJsonBlobSerializer` and options-based configuration
188+
- [x] Add migration guidance
189+
- _Requirements: 7.4_
190+
191+
- [x] Update `docs/DOCUMENTATION_CHANGELOG.md`
192+
- [x] Add entry for SystemTextJson README changes
193+
- [x] Add entry for NewtonsoftJson README changes
194+
- [x] Add entry for AdvancedTypes.md changes
195+
- [x] Add entry for AttributeReference.md changes
196+
- [x] Add entry for all other documentation updates
197+
- _Requirements: 7.5_
198+
199+
## Task 11: Integration Testing
200+
- [x] 11 Run full test suite
201+
- `dotnet test` on entire solution
202+
- Verify all tests pass
203+
- _Requirements: 8.1, 8.2_
204+
205+
- [x] Manual integration test
206+
- [x] Create test project with `[JsonBlob]` property
207+
- [x] Verify error message when no serializer configured
208+
- [x] Verify SystemTextJson works with `WithSystemTextJson()`
209+
- [x] Verify NewtonsoftJson works with `WithNewtonsoftJson()`
210+
- [x] Verify custom options are respected
211+
- _Requirements: 2.2, 3.1, 4.1_
212+
213+
## Task 12: Build Verification
214+
- [x] Verify solution builds without errors
215+
- [x] `dotnet build` on entire solution
216+
- _Requirements: All_
217+
218+
- [x] Verify NuGet packages build correctly
219+
- [x] `dotnet pack` on solution
220+
- [x] Verify package contents include new files
221+
- _Requirements: All_

0 commit comments

Comments
 (0)