MalformedXML error trying to deleteObjects from S3

[spdaley]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

I'm executing a node.js script to delete some files from an S3 bucket and I'm getting the following error:
MalformedXML: The XML you provided was not well-formed or did not validate against our published schema

This can be observed with the latest @aws-sdk/client-s3 and seems to have started with v3.698.0 (at least my sample script passes before that version).

Regression Issue

• Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/client-s3@3.738

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v20.17.0

Reproduction Steps

const { S3Client, DeleteObjectsCommand } = require("@aws-sdk/client-s3"); 
const client = new S3Client({});
const input = { 
  Bucket: "<replace_with_your_bucket>",
  Delete: { 
    Objects: [ 
      { 
        Key: "test.json", 
//        VersionId: "STRING_VALUE",
//        ETag: "STRING_VALUE",
        Size: 3000
      },
    ],
    Quiet: true
  }
};

const deleteStuff = async () => {
  const command = new DeleteObjectsCommand(input);
  const response = await client.send(command);
  return response;
}

deleteStuff();

If you comment out Size, it will work. If you enable either ETag or Size it will fail.

Observed Behavior

MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
    at throwDefaultError (/Users/seadaley/Downloads/test-s3/node_modules/@smithy/smithy-client/dist-cjs/index.js:844:20)
    at /Users/seadaley/Downloads/test-s3/node_modules/@smithy/smithy-client/dist-cjs/index.js:853:5
    at de_CommandError (/Users/seadaley/Downloads/test-s3/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4829:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /Users/seadaley/Downloads/test-s3/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /Users/seadaley/Downloads/test-s3/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:485:18
    at async /Users/seadaley/Downloads/test-s3/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /Users/seadaley/Downloads/test-s3/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/index.js:263:18
    at async /Users/seadaley/Downloads/test-s3/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22
    at async /Users/seadaley/Downloads/test-s3/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:138:14 {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: '4F4T38E2VDAN70D4',
    extendedRequestId: 'Y/PAriWEA7r0cDYdVg0pW86bZoYUmFzw/DqtxpKJLdH36eyLwHkw+0RAJ4kVM+p+TWMb2YA/m+c=',
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  Code: 'MalformedXML',
  RequestId: '4F4T38E2VDAN70D4',
  HostId: 'Y/PAriWEA7r0cDYdVg0pW86bZoYUmFzw/DqtxpKJLdH36eyLwHkw+0RAJ4kVM+p+TWMb2YA/m+c='
}

Expected Behavior

The script should not blow up with a MalformedXML error.

Possible Solution

No response

Additional Information/Context

In the sample program I provided, if you comment out Size, it will work. If you enable either ETag or Size it will fail.

[zshzbh]

I can reproduce this issue with v3.738

For ETag issue - If you use ETags, it will succeed.

const input = { 
  Bucket: "test-s3-explorer-maggie",
  Delete: { 
    Objects: [ 
      { 
	ETags: "xxxxx",
        Key: "example.txt", 
        // Size: 956.0 
      },
    ],
    Quiet: false
  }
};

I also added middlewareStack to monitor the request and response -

client.middlewareStack.add(
	(next, context) => async (args) => {
	  // Safely access command name
	  const commandName = args.command?.constructor?.name || 'Unknown';
	  
	  console.log('Request's Delete object::', {
		input: args.input.Delete.Objects,
		command: commandName
	  });
	  
	  const response = await next(args);
	  
	  console.log('Response:', {
		metadata: response.response.body,
	  });
	  
	  return response;
	},
	{
	  step: 'build'
	}
  );

And it will display the request and response after running the code -

6857 node index.js
Request: {
  input: [ { ETags: 'xxxxxx', Key: 'example.txt' } ],
  command: 'Unknown'
}
Response: {
  metadata: PassThrough {
    _events: {
      close: undefined,
      error: [Function (anonymous)],
      prefinish: [Function: prefinish],
      finish: undefined,
      drain: undefined,
      data: undefined,
      end: undefined,
      readable: undefined,
      unpipe: undefined
    },
    _readableState: ReadableState {
      highWaterMark: 16384,
      buffer: [],
      bufferIndex: 0,
      length: 0,
      pipes: [],
      awaitDrainWriters: null,
      [Symbol(kState)]: 110626684
    },
    _writableState: WritableState {
      highWaterMark: 16384,
      length: 0,
      corked: 0,
      onwrite: [Function: bound onwrite],
      writelen: 0,
      bufferedIndex: 0,
      pendingcb: 0,
      [Symbol(kState)]: 1091466620,
      [Symbol(kBufferedValue)]: null,
      [Symbol(kWriteCbValue)]: null
    },
    allowHalfOpen: true,
    _maxListeners: undefined,
    _eventsCount: 2,
    [Symbol(shapeMode)]: true,
    [Symbol(kCapture)]: false,
    [Symbol(kCallback)]: null
  }
}

I will talk about it with the team to see if it's the ETag/ETags issue is a documentation issue or the service issue.

[spdaley]

Thanks @zshzbh
Just curious but is it working with ETags because that's not actually a valid attribute? The reason we were passing in ETag is because we are first listing all of the objects in the bucket and then passing those objects on to the delete call.

I just double-checked one of our buckets and this is what is returned from a list API call:

{
  "Key": "test.json",
  "LastModified": "2025-01-27T22:17:19.000Z",
  "ETag": "\"068d50494ac49c6093e9fee8e3ad872d\"",
  "Size": 853,
  "StorageClass": "STANDARD"
}

So the API seems to be using ETag. Also, I think I wrote it earlier but just in case, we get the same error even when passing Size into the request. The only things that do not error are Key and VersionId.

From looking at the type definitions, it looks like the ObjectIdentifier. used to be just a Key and a VersionId. Starting sometime around the time it broke for us, it's now:

export interface ObjectIdentifier {
    Key: string | undefined;
    VersionId?: string | undefined;
    ETag?: string | undefined;
    LastModifiedTime?: Date | undefined;
    Size?: number | undefined;
}

[zshzbh]

Hey @spdaley ,

That makes sense. I will share this with the team.

For Size - This functionality is only supported for directory buckets according to the doc.

Are you using the directory bucket?

[spdaley]

Hey @spdaley ,

That makes sense. I will share this with the team.

For Size - This functionality is only supported for directory buckets according to the doc.

Are you using the directory bucket?

Thanks. That makes sense as well. I missed that facet of the docs. We had just been blindly passing in the response to a list call directly to the delete call. This used to work in the past because it looks like delete only cared about Key and VersionId. Now that it potentially uses ETag, LastModifiedTime, and Size as well, it's definitely correct behavior for us to make sure we only pass in Key and VersionId.

So I think most of this issue is probably more user error now. Though maybe the ETag part of it might still be legit?

[zshzbh]

Hey @spdaley ,

I tried to use cli for this action and saw the same error, it seems the error comes from the service team.

~ $ aws s3api delete-objects --bucket test-s3-explorer-maggie --delete '{
>     "Objects": [
>         {
>             "Key": "example.txt",
>             "ETag": "f3f53a8da359af8e82b72fc47ec7xxxx"
>         }
>     ],
>     "Quiet": false
> }'

An error occurred (MalformedXML) when calling the DeleteObjects operation: The XML you provided was not well-formed or did not validate against our published schema

In Version V3.698, we introduced ETag I will check with the team and cut a ticket to the service team and keep you updated!

[zshzbh]

Hey @spdaley ,

I want to update that the service team is working on fixing the Malformed XML issue where the entire deleteObjects request to S3 general purpose bucket fails when ETag is provided.

I will let you know when there's any updates!

Thanks!