Skip to content

Commit 3ea79e2

Browse files
christoph2806christoph2806
committed
Implement graceful missing fields handling
- Add SilentUndefined class for empty string replacement (default) - Add PropertyMissingUndefined class for '<property missing in json>' replacement - Fix SilentUndefined to properly handle nested attribute access - Add new 'property_missing' undefined_behavior option - Set 'silent' as default behavior instead of 'debug' - Update TemplateRequest model to support new undefined_behavior options - Add comprehensive documentation for missing fields handling - Maintain backward compatibility with existing API Now missing fields in Word templates will show as empty strings by default instead of throwing errors, with configurable behavior options.
1 parent 7c710de commit 3ea79e2

10 files changed

+1233
-220
lines changed

‎.devcontainer/Dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@ COPY requirements.txt requirements-test.txt ./
2222
RUN pip install --no-cache-dir -r requirements.txt \
2323
&& pip install --no-cache-dir -r requirements-test.txt
2424

25-
# Create temp directory for file processing
26-
RUN mkdir -p temp && chmod 755 temp
25+
# Create temp directory for file processing with proper ownership
26+
RUN mkdir -p temp && chmod 755 temp && chown vscode:vscode temp
2727

2828
# Install additional development tools
2929
RUN pip install --no-cache-dir \

‎.devcontainer/devcontainer.json‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
{
22
"name": "Document Template Processing Service",
3-
"image": "mcr.microsoft.com/devcontainers/python:1-3.12-bullseye",
3+
"dockerComposeFile": "docker-compose.yml",
4+
"service": "app",
45
"features": {
56
"ghcr.io/devcontainers/features/docker-in-docker:2": {},
67
"ghcr.io/devcontainers/features/git:1": {}
@@ -59,6 +60,5 @@
5960
"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
6061
],
6162
"remoteUser": "vscode",
62-
"workspaceFolder": "/workspace",
63-
"onCreateCommand": "mkdir -p temp && chmod 755 temp"
63+
"workspaceFolder": "/workspace"
6464
}

‎.devcontainer/docker-compose.yml‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
1-
version: "3.8"
2-
31
services:
42
app:
53
build:
64
context: ..
75
dockerfile: .devcontainer/Dockerfile
86
volumes:
9-
- ../..:/workspaces:cached
7+
- ..:/workspace:cached
108
command: sleep infinity
119
networks:
1210
- dev-network
@@ -19,13 +17,12 @@ services:
1917
gotenberg:
2018
image: gotenberg/gotenberg:8
2119
ports:
22-
- "3001:3000"
20+
- "3002:3000"
2321
networks:
2422
- dev-network
2523
command:
2624
- "gotenberg"
2725
- "--api-timeout=30s"
28-
- "--api-timeout-duration=30s"
2926

3027
networks:
3128
dev-network:

‎docs/missing-fields-handling.md‎

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
# Missing Fields Handling
2+
3+
The Document Template Processing Service now provides robust handling of missing fields in JSON data, preventing errors and allowing graceful degradation when template variables reference non-existent properties.
4+
5+
## Problem Solved
6+
7+
Previously, when a Word document template referenced a field that didn't exist in the provided JSON data, the processor would throw an `UndefinedError` and fail to generate the document. This was particularly problematic when:
8+
9+
- Templates were created with fields that might not always be present
10+
- JSON data structures varied between different data sources
11+
- Optional fields were referenced in templates
12+
13+
## Solution
14+
15+
The service now supports four different behaviors for handling missing fields, configurable via the `undefined_behavior` parameter:
16+
17+
### 1. Silent Mode (`"silent"`) - Default
18+
Missing fields are replaced with empty strings.
19+
20+
**Example:**
21+
- Template: `Email: {{farmer.email}}`
22+
- Missing field result: `Email: `
23+
24+
### 2. Property Missing Mode (`"property_missing"`)
25+
Missing fields are replaced with `<property missing in json>`.
26+
27+
**Example:**
28+
- Template: `Email: {{farmer.email}}`
29+
- Missing field result: `Email: <property missing in json>`
30+
31+
### 3. Debug Mode (`"debug"`)
32+
Missing fields show detailed variable names for debugging.
33+
34+
**Example:**
35+
- Template: `Email: {{farmer.email}}`
36+
- Missing field result: `Email: [Missing variable: farmer.email]`
37+
38+
### 4. Strict Mode (`"strict"`)
39+
Missing fields throw errors (original behavior).
40+
41+
**Example:**
42+
- Template: `Email: {{farmer.email}}`
43+
- Result: HTTP 400 error with detailed error message
44+
45+
## Usage
46+
47+
### Using the Enhanced API
48+
49+
```bash
50+
curl -X POST "http://localhost:8000/api/v1/process-template-document" \
51+
-F "file=@template.docx" \
52+
-F 'request_data={
53+
"template_data": {
54+
"farmer": {
55+
"name": "John Doe",
56+
"phone": "+1234567890"
57+
}
58+
},
59+
"undefined_behavior": "silent"
60+
}'
61+
```
62+
63+
### Using Environment Variable
64+
65+
Set the default behavior for all requests:
66+
67+
```bash
68+
export UNDEFINED_BEHAVIOR=silent
69+
```
70+
71+
### Using Legacy API
72+
73+
The legacy `data` parameter continues to work with the default behavior:
74+
75+
```bash
76+
curl -X POST "http://localhost:8000/api/v1/process-template-document" \
77+
-F "file=@template.docx" \
78+
-F 'data={"farmer": {"name": "John Doe"}}'
79+
```
80+
81+
## Behavior Comparison
82+
83+
| Behavior | Missing Field Output | Use Case |
84+
|----------|---------------------|----------|
85+
| `silent` | (empty string) | Production documents where missing fields should be invisible |
86+
| `property_missing` | `<property missing in json>` | Documents where missing fields should be clearly indicated |
87+
| `debug` | `[Missing variable: field.name]` | Development and debugging templates |
88+
| `strict` | Error thrown | Validation and ensuring all required fields are present |
89+
90+
## Examples
91+
92+
### Template with Missing Fields
93+
94+
```
95+
Farmer Profile Report
96+
97+
Name: {{farmer.name}}
98+
Email: {{farmer.email}} # Missing field
99+
Phone: {{farmer.phone}}
100+
Address: {{farmer.address}} # Missing field
101+
Organization: {{organization.name}}
102+
Type: {{organization.type}} # Missing field
103+
```
104+
105+
### JSON Data
106+
107+
```json
108+
{
109+
"farmer": {
110+
"name": "John Doe",
111+
"phone": "+1234567890"
112+
},
113+
"organization": {
114+
"name": "Farm Co-op"
115+
}
116+
}
117+
```
118+
119+
### Results by Behavior
120+
121+
**Silent (`"silent"`):**
122+
```
123+
Farmer Profile Report
124+
125+
Name: John Doe
126+
Email:
127+
Phone: +1234567890
128+
Address:
129+
Organization: Farm Co-op
130+
Type:
131+
```
132+
133+
**Property Missing (`"property_missing"`):**
134+
```
135+
Farmer Profile Report
136+
137+
Name: John Doe
138+
Email: <property missing in json>
139+
Phone: +1234567890
140+
Address: <property missing in json>
141+
Organization: Farm Co-op
142+
Type: <property missing in json>
143+
```
144+
145+
**Debug (`"debug"`):**
146+
```
147+
Farmer Profile Report
148+
149+
Name: John Doe
150+
Email: [Missing variable: farmer.email]
151+
Phone: +1234567890
152+
Address: [Missing variable: farmer.address]
153+
Organization: Farm Co-op
154+
Type: [Missing variable: organization.type]
155+
```
156+
157+
**Strict (`"strict"`):**
158+
```
159+
HTTP 400 Error:
160+
{
161+
"message": "Undefined variable in template: 'main.DictToObject object' has no attribute 'email'",
162+
"error_type": "undefined_variable",
163+
"details": {
164+
"file": "template.docx",
165+
"undefined_variable": "'main.DictToObject object' has no attribute 'email'",
166+
"suggestion": "Check your template variables match the provided data"
167+
}
168+
}
169+
```
170+
171+
## Implementation Details
172+
173+
The solution uses custom Jinja2 `Undefined` classes:
174+
175+
- `SilentUndefined`: Returns empty strings for missing variables
176+
- `PropertyMissingUndefined`: Returns `<property missing in json>` for missing variables
177+
- `DebugUndefined`: Returns `[Missing variable: name]` for missing variables
178+
- `StrictUndefined`: Throws `UndefinedError` for missing variables (Jinja2 default)
179+
180+
These classes properly handle nested attribute access and dictionary-style access, ensuring that missing fields at any level are handled gracefully.
181+
182+
## Migration Guide
183+
184+
### For Existing Users
185+
186+
- **No breaking changes**: Existing API calls continue to work unchanged
187+
- **Default behavior**: Uses `silent` mode by default (missing fields show as empty)
188+
- **Environment override**: Set `UNDEFINED_BEHAVIOR` environment variable to change default
189+
190+
### For New Implementations
191+
192+
- **Recommended**: Use `request_data` parameter with explicit `undefined_behavior`
193+
- **Production**: Use `"silent"` or `"property_missing"` for end-user documents
194+
- **Development**: Use `"debug"` for template development and testing
195+
- **Validation**: Use `"strict"` when all fields must be present
196+
197+
## Testing
198+
199+
The solution includes comprehensive tests:
200+
201+
```bash
202+
# Test with farmer data
203+
python test_farmer_missing_fields.py
204+
205+
# Test all behaviors
206+
python test_missing_fields_solution.py
207+
208+
# Run existing test suite
209+
python -m pytest tests/test_undefined_variable_handling.py -v
210+
```
211+
212+
## Error Handling
213+
214+
The service maintains backward compatibility with existing error handling while adding the new graceful degradation options. When using `strict` mode, errors are still thrown with detailed information about missing fields, helping developers identify and fix template issues.

0 commit comments

Comments
 (0)