Skip to content

Commit 62bb4de

Browse files
committed
account for the issue coderabbit found with exactly one column errors. update the ai plan documentation for historic documentation. adjust the readme to add a new design philosophies section to further help illustrate, document, and guide.
1 parent 5306111 commit 62bb4de

File tree

6 files changed

+236
-64
lines changed

6 files changed

+236
-64
lines changed

DESIGN_PHILOSOPHIES.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Design Philosophies
2+
3+
This document includes information around design philosophies and decisions made to help document and illustrate scenarios one may encounter when using this package.
4+
5+
## Approach
6+
Carta adopts the "database mapping" approach (described in Martin Fowler's [book](https://books.google.com/books?id=FyWZt5DdvFkC&lpg=PA1&dq=Patterns%20of%20Enterprise%20Application%20Architecture%20by%20Martin%20Fowler&pg=PT187#v=onepage&q=active%20record&f=false)) which is useful among organizations with strict code review processes.
7+
8+
## Comparison to Related Projects
9+
10+
#### GORM
11+
Carta is NOT an an object-relational mapper(ORM).
12+
13+
#### sqlx
14+
Sqlx does not track has-many relationships when mapping SQL data. This works fine when all your relationships are at most has-one (Blog has one Author) ie, each SQL row corresponds to one struct. However, handling has-many relationships (Blog has many Posts), requires running many queries or running manual post-processing of the result. Carta handles these complexities automatically.
15+
16+
## Protection vs. Graceful Handling
17+
18+
A core design principle of the `carta` mapper is to prioritize **user protection and clarity** over attempting a "graceful" but potentially incorrect guess. The library's guiding philosophy is to only proceed if the user's intent is perfectly clear. If there is any ambiguity in the mapping operation, `carta` will **fail fast** by returning an error, forcing the developer to be more explicit.
19+
20+
Making a guess might seem helpful, but it can hide serious, silent bugs. The following scenarios illustrate the balance between failing on ambiguous operations (Protection) and handling well-defined transformations (Graceful Handling).
21+
22+
---
23+
24+
### Scenario 1: Multi-column Query to a Basic Slice (Protection)
25+
26+
- **Query:** `SELECT name, email FROM users`
27+
- **Destination:** `var data []string`
28+
- **Behavior:** `carta.Map` **returns an error immediately**: `carta: when mapping to a slice of a basic type, the query must return exactly one column (got 2)`.
29+
- **Why this is Protection:** The library has no way of knowing if the user intended to map the `name` or the `email` column. A "graceful" solution might be to arbitrarily pick the first column, but this could lead to the wrong data being silently loaded into the slice. By failing fast, `carta` forces the developer to write an unambiguous query (e.g., `SELECT name FROM users`), ensuring the result is guaranteed to be correct.
30+
31+
---
32+
33+
### Scenario 2: SQL `NULL` to a Non-nullable Go Field (Protection)
34+
35+
- **Query:** `SELECT id, NULL AS name FROM users`
36+
- **Destination:** `var users []User` (where `User.Name` is a `string`)
37+
- **Behavior:** `carta.Map` **returns an error during scanning**: `carta: cannot load null value to type string for column name`.
38+
- **Why this is Protection:** A standard Go `string` cannot represent a `NULL` value. A "graceful" but incorrect solution would be to use the zero value (`""`), which is valid data and semantically different from "no data". This can cause subtle bugs in application logic. By failing, `carta` forces the developer to explicitly handle nullability in their Go struct by using a pointer (`*string`) or a nullable type (`sql.NullString`), making the code more robust and correct.
39+
40+
---
41+
42+
### Scenario 3: Merging `JOIN`ed Rows into Structs (Graceful Handling)
43+
44+
- **Query:** `SELECT b.id, p.id FROM blogs b JOIN posts p ON b.id = p.blog_id`
45+
- **Destination:** `var blogs []BlogWithPosts`
46+
- **Behavior:** `carta` **gracefully handles** the fact that the same blog ID appears in multiple rows. It creates one `Blog` object and appends each unique `Post` to its `Posts` slice.
47+
- **Why this is Graceful:** This is the core purpose of the library. There is no ambiguity. The library uses the unique ID of the `Blog` (the `b.id` column) to understand that these rows all describe the same parent entity. This is a well-defined transformation, not a guess.

README.md

Lines changed: 7 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# Carta
22
[![codecov](https://codecov.io/github/hackafterdark/carta/graph/badge.svg?token=TYvbPGGlcL)](https://codecov.io/github/hackafterdark/carta)
33

4-
Dead simple SQL data mapper for complex Go structs.
4+
A simple SQL data mapper for complex Go structs. Load SQL data onto Go structs while keeping track of has-one and has-many relationships.
55

6-
Load SQL data onto Go structs while keeping track of has-one and has-many relationships
6+
Carta is not an object-relational mapper(ORM). With large and complex datasets, using ORMs becomes restrictive and reduces performance when working with complex queries. [Read more about the design philosophy.](#design-philosophy)
77

88
## Examples
99
Using carta is very simple. All you need to do is:
@@ -93,15 +93,6 @@ blogs:
9393
}]
9494
```
9595

96-
97-
## Comparison to Related Projects
98-
99-
#### GORM
100-
Carta is NOT an an object-relational mapper(ORM). Read more in [Approach](#Approach)
101-
102-
#### sqlx
103-
Sqlx does not track has-many relationships when mapping SQL data. This works fine when all your relationships are at most has-one (Blog has one Author) ie, each SQL row corresponds to one struct. However, handling has-many relationships (Blog has many Posts), requires running many queries or running manual post-processing of the result. Carta handles these complexities automatically.
104-
10596
## Guide
10697

10798
### Column and Field Names
@@ -233,19 +224,14 @@ Other types, such as TIME, will will be converted from plain text in future vers
233224
go get -u github.com/hackafterdark/carta
234225
```
235226

227+
## Design Philosophy
236228

237-
## Important Notes
229+
The `carta` package follows a "fail-fast" philosophy to ensure that mapping operations are unambiguous and to protect users from silent bugs. For a detailed explanation of the error handling approach and the balance between user protection and graceful handling, please see the [Design Philosophies](./DESIGN_PHILOSOPHIES.md) document.
230+
231+
## Important Notes
238232

239233
When mapping to **slices of structs**, Carta removes duplicate entities. This is a side effect of the data mapping process, which merges rows that identify the same entity (e.g., a `Blog` with the same ID appearing in multiple rows due to a `JOIN`). To ensure correct mapping, you should always include uniquely identifiable columns (like a primary key) in your query for each struct entity.
240234

241235
When mapping to **slices of basic types** (e.g., `[]string`, `[]int`), every row from the query is treated as a unique element, and **no de-duplication occurs**.
242236

243-
To prevent relatively expensive reflect operations, carta caches the structure of your struct using the column mames of your query response as well as the type of your struct.
244-
245-
## Approach
246-
Carta adopts the "database mapping" approach (described in Martin Fowler's [book](https://books.google.com/books?id=FyWZt5DdvFkC&lpg=PA1&dq=Patterns%20of%20Enterprise%20Application%20Architecture%20by%20Martin%20Fowler&pg=PT187#v=onepage&q=active%20record&f=false)) which is useful among organizations with strict code review processes.
247-
248-
Carta is not an object-relational mapper(ORM). With large and complex datasets, using ORMs becomes restrictive and reduces performance when working with complex queries.
249-
250-
### License
251-
Apache License
237+
To prevent relatively expensive reflect operations, carta caches the structure of your struct using the column mames of your query response as well as the type of your struct.

ai_plans/FIX_DUPLICATE_ROWS.md

Lines changed: 50 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,50 @@
1-
# Problem: Incorrect De-duplication When Mapping to Basic Slices
2-
3-
## Summary
4-
The `carta` library is designed to de-duplicate entities when mapping SQL rows to slices of structs (e.g., `[]User`). This is achieved by generating a unique ID for each entity based on the content of its primary key columns. This behavior is correct for handling `JOIN`s where a single entity might appear across multiple rows.
5-
6-
However, this same logic is incorrectly applied when the mapping destination is a slice of a basic type (e.g., `[]string`, `[]int`). In this scenario, rows with duplicate values are treated as the same entity and are de-duplicated, which is incorrect. The desired behavior is to preserve every row from the result set, including duplicates.
7-
8-
This issue is the root cause for the following problems:
9-
1. The `if m.IsBasic` code path in `load.go` lacks test coverage because no tests exist for mapping to basic slices.
10-
2. Attempts to write such tests lead to infinite loops and incorrect behavior because the column allocation and unique ID generation logic are not designed to handle this case.
11-
12-
## Proposed Solution
13-
The solution is to create a distinct execution path for "basic mappers" (`m.IsBasic == true`) that ensures every row is treated as a unique element.
14-
15-
This will be accomplished in two main steps:
16-
17-
### 1. Fix Column Allocation (`allocateColumns`)
18-
The logic will be modified to enforce a clear rule for basic slices: the source SQL query must return **exactly one column**.
19-
20-
- If `m.IsBasic` is true, the function will bypass the existing name-matching logic.
21-
- It will validate that only one column is present in the query result.
22-
- This single column will be assigned as the `PresentColumn` for the mapper.
23-
- If more than one column is found, the function will return an error to prevent ambiguity.
24-
25-
### 2. Fix Unique ID Generation (`loadRow`)
26-
The logic will be modified to generate a unique ID based on the row's position rather than its content.
27-
28-
- If `m.IsBasic` is true, the call to `getUniqueId(row, m)` will be bypassed.
29-
- A new, position-based unique ID will be generated for each row (e.g., using a simple counter that increments with each row processed).
30-
- This ensures that every row, regardless of its content, is treated as a distinct element to be added to the destination slice.
31-
32-
This approach preserves the existing, correct behavior for struct mapping while introducing a new, robust path for handling basic slices correctly.
33-
34-
## Plan
35-
1. **Modify `column.go`**: Update the `allocateColumns` function to implement the single-column rule for basic mappers.
36-
2. **Modify `load.go`**: Update the `loadRow` function to use a position-based counter for unique ID generation when `m.IsBasic` is true.
37-
3. **Add Tests**: Create a new test case in `mapper_test.go` that maps a query result to a slice of a basic type (e.g., `[]string`) to validate the fix and provide coverage for the `m.IsBasic` code path.
1+
# Plan: Fix Incorrect De-duplication for Basic Slices
2+
3+
## 1. Problem Summary
4+
The `carta` library was incorrectly de-duplicating rows when mapping to a slice of a basic type (e.g., `[]string`). The logic, designed to merge `JOIN`ed rows for slices of structs, was misapplied, causing data loss. This also meant the `m.IsBasic` code path was entirely untested.
5+
6+
The goal was to modify the library to correctly preserve all rows, including duplicates, when mapping to a basic slice, and to add the necessary test coverage.
7+
8+
## 2. Evolution of the Solution
9+
10+
The final solution was reached through an iterative process of implementation and refinement based on code review feedback.
11+
12+
### Initial Implementation
13+
The first version of the fix introduced two key changes:
14+
1. **Position-Based Unique IDs:** In `load.go`, the `loadRow` function was modified. When `m.IsBasic` is true, it now generates a unique ID based on the row's position in the result set (e.g., "row-0", "row-1") instead of its content. This ensures every row is treated as a unique element.
15+
2. **Single-Column Rule:** In `column.go`, the `allocateColumns` function was updated to enforce a strict rule: if the destination is a basic slice, the SQL query must return **exactly one column**. This prevents ambiguity.
16+
17+
### Refinements from Code Review
18+
Feedback from a code review (via Coderabbit) prompted several improvements:
19+
- **Performance:** In `load.go`, `fmt.Sprintf` was replaced with the more performant `strconv.Itoa` for generating the position-based unique ID.
20+
- **Idiomatic Go:** Error creation was changed from `errors.New(fmt.Sprintf(...))` to the more idiomatic `fmt.Errorf`.
21+
- **Clearer Errors:** The error message for the single-column rule was improved to include the actual number of columns found, aiding debugging.
22+
- **Test Coverage:** A negative test case was added to `mapper_test.go` to ensure the single-column rule correctly returns an error.
23+
24+
### Final Fix: Handling Nested Basic Mappers
25+
The most critical refinement came from identifying a flaw in the single-column rule: it did not correctly handle **nested** basic slices (e.g., a struct field like `Tags []string`). The initial logic would have incorrectly failed if other columns for the parent struct were present.
26+
27+
The final patch corrected this by making the logic in `allocateColumns` more nuanced:
28+
- **For top-level basic slices** (`len(m.AncestorNames) == 0`), the query must still contain exactly one column overall.
29+
- **For nested basic slices**, the function now searches the remaining columns for exactly one that matches the ancestor-qualified name (e.g., `tags`). It returns an error if zero or more than one match is found.
30+
31+
This final change ensures the logic is robust for both top-level and nested use cases.
32+
33+
## 3. Summary of Changes Executed
34+
1. **Modified `load.go`**:
35+
- Updated `loadRow` to accept a `rowCount` parameter.
36+
- Implemented logic to generate a unique ID from `rowCount` when `m.IsBasic` is true.
37+
- Refactored error handling and string formatting based on code review feedback.
38+
2. **Modified `column.go`**:
39+
- Updated `allocateColumns` to differentiate between top-level and nested basic mappers, enforcing the correct single-column matching rule for each.
40+
- Improved the error message to be more descriptive.
41+
3. **Modified `mapper.go`**:
42+
- Corrected the logic in `determineFieldsNames` to properly handle casing in `carta` tags, ensuring ancestor names are generated correctly.
43+
4. **Added Tests to `mapper_test.go`**:
44+
- Added a test for a top-level basic slice (`[]string`) to verify that duplicates are preserved.
45+
- Added a negative test to ensure an error is returned for a multi-column query to a top-level basic slice.
46+
- Added a test for a nested basic slice (`PostWithTags.Tags []string`) to verify correct mapping.
47+
- Added negative tests to ensure errors are returned for nested basic slices with zero or multiple matching columns.
48+
5. **Updated Documentation**:
49+
- Updated `README.md` to clarify the difference in de-duplication behavior.
50+
- Created `DESIGN_PHILOSOPHIES.md` to document the "fail-fast" error handling approach.

column.go

Lines changed: 36 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,17 +18,48 @@ type column struct {
1818
func allocateColumns(m *Mapper, columns map[string]column) error {
1919
presentColumns := map[string]column{}
2020
if m.IsBasic {
21-
if len(columns) != 1 {
22-
return fmt.Errorf("carta: when mapping to a slice of a basic type, the query must return exactly one column (got %d)", len(columns))
23-
}
24-
for cName, c := range columns {
21+
if len(m.AncestorNames) == 0 {
22+
// Top-level basic mapper: must map exactly one column overall
23+
if len(columns) != 1 {
24+
return fmt.Errorf(
25+
"carta: when mapping to a slice of a basic type, "+
26+
"the query must return exactly one column (got %d)",
27+
len(columns),
28+
)
29+
}
30+
for cName, c := range columns {
31+
presentColumns[cName] = column{
32+
typ: c.typ,
33+
name: cName,
34+
columnIndex: c.columnIndex,
35+
}
36+
delete(columns, cName)
37+
break
38+
}
39+
} else {
40+
// Nested basic mapper: pick exactly one matching ancestor-qualified column
41+
candidates := getColumnNameCandidates("", m.AncestorNames, m.Delimiter)
42+
var matched []string
43+
for cName := range columns {
44+
if candidates[cName] {
45+
matched = append(matched, cName)
46+
}
47+
}
48+
if len(matched) != 1 {
49+
return fmt.Errorf(
50+
"carta: basic sub-mapper for %v expected exactly one matching column "+
51+
"(ancestors %v), got %d matches",
52+
m.Typ, m.AncestorNames, len(matched),
53+
)
54+
}
55+
cName := matched[0]
56+
c := columns[cName]
2557
presentColumns[cName] = column{
2658
typ: c.typ,
2759
name: cName,
2860
columnIndex: c.columnIndex,
2961
}
3062
delete(columns, cName)
31-
break
3263
}
3364
} else {
3465
for i, field := range m.Fields {

mapper.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -244,7 +244,7 @@ func determineFieldsNames(m *Mapper) error {
244244
if tag := nameFromTag(field.Tag, CartaTagKey); tag != "" {
245245
subMap.Delimiter = "->"
246246
parts := strings.Split(tag, ",")
247-
name = parts[0]
247+
name = strings.TrimSpace(parts[0])
248248
if len(parts) > 1 {
249249
for _, part := range parts[1:] {
250250
option := strings.Split(part, "=")

mapper_test.go

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -571,3 +571,98 @@ func TestMapToBasicSlice_MultipleColumnsError(t *testing.T) {
571571
t.Fatalf("expected error when mapping to []string with multiple columns, got nil")
572572
}
573573
}
574+
575+
type PostWithTags struct {
576+
ID int `db:"id"`
577+
Tags []string `carta:"Tags"`
578+
}
579+
580+
func TestNestedBasicSliceMap(t *testing.T) {
581+
db, mock, err := sqlmock.New()
582+
if err != nil {
583+
t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
584+
}
585+
defer db.Close()
586+
587+
rows := sqlmock.NewRows([]string{"id", "Tags"}).
588+
AddRow(1, "tag1").
589+
AddRow(1, "tag2")
590+
591+
mock.ExpectQuery("SELECT (.+) FROM posts").WillReturnRows(rows)
592+
593+
sqlRows, err := db.Query("SELECT * FROM posts")
594+
if err != nil {
595+
t.Fatalf("error '%s' was not expected when querying rows", err)
596+
}
597+
598+
var posts []PostWithTags
599+
err = Map(sqlRows, &posts)
600+
if err != nil {
601+
t.Errorf("error was not expected while mapping rows: %s", err)
602+
}
603+
604+
if len(posts) != 1 {
605+
t.Fatalf("expected 1 post, got %d", len(posts))
606+
}
607+
608+
if len(posts[0].Tags) != 2 {
609+
t.Fatalf("expected 2 tags, got %d", len(posts[0].Tags))
610+
}
611+
612+
expectedTags := []string{"tag1", "tag2"}
613+
if !reflect.DeepEqual(posts[0].Tags, expectedTags) {
614+
t.Errorf("expected tags to be %+v, but got %+v", expectedTags, posts[0].Tags)
615+
}
616+
617+
if err := mock.ExpectationsWereMet(); err != nil {
618+
t.Errorf("there were unfulfilled expectations: %s", err)
619+
}
620+
}
621+
622+
func TestNestedBasicSliceMap_NoMatchingColumnsError(t *testing.T) {
623+
db, mock, err := sqlmock.New()
624+
if err != nil {
625+
t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
626+
}
627+
defer db.Close()
628+
629+
rows := sqlmock.NewRows([]string{"id", "other_column"}).
630+
AddRow(1, "value")
631+
632+
mock.ExpectQuery("SELECT (.+) FROM posts").WillReturnRows(rows)
633+
634+
sqlRows, err := db.Query("SELECT * FROM posts")
635+
if err != nil {
636+
t.Fatalf("error '%s' was not expected when querying rows", err)
637+
}
638+
639+
var posts []PostWithTags
640+
err = Map(sqlRows, &posts)
641+
if err == nil {
642+
t.Errorf("expected an error when mapping a nested basic slice with no matching columns, but got nil")
643+
}
644+
}
645+
646+
func TestNestedBasicSliceMap_MultipleMatchingColumnsError(t *testing.T) {
647+
db, mock, err := sqlmock.New()
648+
if err != nil {
649+
t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
650+
}
651+
defer db.Close()
652+
653+
rows := sqlmock.NewRows([]string{"id", "tags", "Tags"}).
654+
AddRow(1, "tag1", "tag2")
655+
656+
mock.ExpectQuery("SELECT (.+) FROM posts").WillReturnRows(rows)
657+
658+
sqlRows, err := db.Query("SELECT * FROM posts")
659+
if err != nil {
660+
t.Fatalf("error '%s' was not expected when querying rows", err)
661+
}
662+
663+
var posts []PostWithTags
664+
err = Map(sqlRows, &posts)
665+
if err == nil {
666+
t.Errorf("expected an error when mapping a nested basic slice with multiple matching columns, but got nil")
667+
}
668+
}

0 commit comments

Comments
 (0)