Skip to content

Migrate AttributeValue - #7199

Open
RanVaknin wants to merge 6 commits into
feature/master/DDB-mapperv2from
rvaknin/migrate-converters
Open

Migrate AttributeValue#7199
RanVaknin wants to merge 6 commits into
feature/master/DDB-mapperv2from
rvaknin/migrate-converters

Conversation

@RanVaknin

@RanVaknin RanVaknin commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Scope

This PR ports only the converter/marshalling layer of the v1 DynamoDBMapper to the v2 AttributeValue type (software.amazon.awssdk.services.dynamodb.model.AttributeValue). Every operation on DynamoDBMapper (load, save, query, scan, batch, transaction, tableAdmin) stays on v1 and is ported in later PRs.

The PR does not compile - The files that stay v1 (the two operation files, the paginated list classes that feed v1 results into v2 toParameters, and DynamoDBMapperFieldModel's query condition methods) reference v1 types that no longer line up with the v2 converter surface, so those files are red. This keeps the diff scoped to the converter layer, and the subsequent operation PRs turn it green.

This is intentional in order to make each PR review boundary conceptually separate and limits the diff.

Testing approach

  • Baseline on v1 - Run existing + new test against the v1 source before porting the types to v2

    • Existing converter tests all pass against the v1 source before any change
    • Added a new edge case suite (StandardModelFactoriesEdgeCasesTest) to widen coverage of the boundaries this migration touches (empty vs unset collections, binary roundtrips, boolean encodings, date formats)
  • Migrate the test bodies, not the assertions. For each ported test we changed only the mechanical v1→v2 surface in the test body (new AttributeValue().withS(x)AttributeValue.builder().s(x).build(), getS()s(), and so on) The assertion logic stayed the same (aside from migrating the new types needed for assertion). A green run therefore means the v2 source produces the same result the v1 source did.

  • Running them despite the red module. The maven-compiler-plugin testIncludes allowlist in pom.xml compiles only the converter tests. To execute them we temporarily stubbed the operation files (and their closure), compiled, and ran the suite: 287 passed, 0 failed. The stub was then reverted, restoring the red state.

Changes

Mechanical swap

1. set()/with()builder().build()

  • v1 AttributeValue is mutable (new AttributeValue().withS(x));
  • v2 is an immutable builder (AttributeValue.builder().s(x).build()).

Question should we use the fast createX() factory methods added in #7039 here in this PR?
I decided to not port these here so we can measure performance against a baseline port vs performance improvement branch later.

2. null vs empty — has guards

v2 .l()/.m()/.ss()/.ns()/.bs() return empty collections (not null) when unset. v2 added hasL()/hasM()/hasSs()/hasNs()/hasBs() to distinguish "explicitly empty" from "unset". Guards were added in the parent typeCheck methods (LUnmarshaller, MUnmarshaller, SSUnmarshaller, etc.), which run before unmarshall via ConversionSchemas dispatch. Pattern: value.hasSs() ? value.ss() : null.

3. Boolean legacy "1"/"0" encoding

Old DynamoDB stored booleans as Number "1"/"0" before native BOOL existed. The reader accepts both and produces the same Boolean; the writer produces the number form.

V2CompatibleBool.get: if (o.bool() != null) return o.bool() ? "1" : "0"; return o.n();

Covered by StandardModelFactoriesV2UnconvertTestn("0")/n("1"), bool(false)/bool(true), and boolean set from ns("1")/ns("0")/ns("0","1").

Needs Reviewer Feedback:

3. SdkBytes boundary

v1 binary is java.nio.ByteBuffer; v2 binary is software.amazon.awssdk.core.SdkBytes. On write, SdkBytes.fromByteBuffer(bb) copies the buffer and leaves the caller's position untouched, matching v1.

On read, POJO fields typed ByteBuffer must stay mutable to match v1's getB(). SdkBytes which AttributeValue uses internally, is immutable and exposes its bytes only as a readonly buffer (via asByteBuffer()) so every ByteBuffer read site returns ByteBuffer.wrap(sdkBytes.asByteArray()). The copy costs one O(n) pass per read, so this path is slower than v1

Returning asByteBuffer() is arguably the better, more v2 idiomatic choice, but if a v1 customer tries to mutate a buffer between reads and writes, the immutable implementation would throw ReadOnlyBufferException at runtime.

As part of future work I plan to add an opt-in feature (field annotation or mapper config flag) that returns the readonly zero copy buffer instead of the mutable copy, for callers who never mutate and want to beat v1's read performance. Same ByteBuffer field type, behavior selected by the flag.

Question - Should we pursue backwards compatibility? Or limit potential performance regression? Or add support for it via future opt in flag? (supporting both cases)

5. MapperDateUtils.java (new file)

The date converters format and parse ISO-8601 strings through two methods on v1's com.amazonaws.util.DateUtils. A v2 module can't depend on that class, and v2's software.amazon.awssdk.utils.DateUtils isn't equivalent: it works in java.time.Instant rather than Date, and its formatter omits zero milliseconds (...T00:00:00Z vs v1's ...T00:00:00.000Z), so switching to it would change the stored string form of dates and break round-trip compatibility with v1 data.

MapperDateUtils ports only the two methods the converters use, on v1's Joda-Time formatters, to keep that format identical. A partial port is enough since the converter layer touches nothing else in DateUtils.

Question - The v1 mapper Date types are tightly coupled to Joda-Time (a library that the v2 SDK doesnt use at all). For the sake of backwards compatibility, I ported the APIs the mapper relies on as a thin utils layer. Should we push for a bigger refactor to completely decouple the mapper from Joda time?

6. MapperExceptions.java (new file)

Both marshallers wrapped errors with v1's Throwables.failure, which a v2 module can't depend on. Instead we ported the failure function helper into a new class MapperExceptions.java

Breaking change: checked exceptions now wrap in SdkClientException instead of v1's AmazonClientException. The breaking change is unavoidable, so users who are explicitly error handling the v1 exception would have to update their code beyond the promised "import swap".

Question - Is the name for this new class acceptable? Should we port this method as a standalone and throw it in some other util package we have in the SDK?


DDB mapper v2 roadmap

** BASE PACKAGE SETUP **
+ 1. Source verbatim port ✅
+ 2. Test verbatim port ✅
+ 3. Namespace swap (main + datamodeling tests) ✅ 
+ 4. Namespace swap (remaining test packages) ✅

** PORTING OPERATIONS **
0. converters (AttributeValue seam)  <---- current PR
1. load()
2. save()
3. query() + scan()
4. deleteItem()
5. updateItem()
6. Batch operations
7. Transactions
8. Misc

** PERFORMANCE IMPROVEMENTS **
1. getTableModel caching
2. Wire "fast" createX AV factory methods to convertors
3. ByteBuffer → SdkBytes copy
4. Others

** DEPENDENCY MODERNIZATION **
1. EasyMock -> Mockito
2. Log4j 1.x -> 2.x
3. commons-logging -> SLF4J

@RanVaknin
RanVaknin marked this pull request as ready for review July 28, 2026 18:41
@RanVaknin
RanVaknin requested a review from a team as a code owner July 28, 2026 18:41
import software.amazon.awssdk.mapper.dynamodb.pojos.AutoKeyAndVal;
import software.amazon.awssdk.mapper.dynamodb.pojos.TestClass;

public class StandardModelFactoriesEdgeCasesTest {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a javadoc for this class? What specific edge cases are we testing? why?

* compatibility with dates written by the v1 mapper.
*/
@SdkInternalApi
public final class MapperDateUtils {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there tests for this?

Comment on lines +21 to +25
import org.joda.time.DateTimeZone;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
import org.joda.time.format.ISODateTimeFormat;
import org.joda.time.tz.FixedDateTimeZone;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to keep using Joda time for this internal class?

final List<ByteBuffer> result = new ArrayList<ByteBuffer>(value.bs().size());
for (SdkBytes sb : value.bs()) {
// Writable copy for v1 parity; SdkBytes.asByteBuffer() would be read-only.
result.add(ByteBuffer.wrap(sb.asByteArray()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating a writable ByteBuffer from SdkBytes seems to occur often enough that it might make sense to create a util method

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants