Amplify: I cannot download the correct file content from s3

[josephorng]

Before opening, please confirm:

• I have searched for duplicate or closed issues and discussions.
• I have read the guide for submitting bug reports.
• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

JavaScript Framework

React

Amplify APIs

Storage

Amplify Version

v6

Amplify Categories

storage

Backend

Amplify Gen 2

Environment information

# Put output below this line

  System:
    OS: Windows 11 10.0.26100
    CPU: (24) x64 12th Gen Intel(R) Core(TM) i9-12900K
    Memory: 54.60 GB / 127.75 GB
  Binaries:
    Node: 20.15.0 - C:\Program Files\nodejs\node.EXE
    npm: 10.7.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Chromium (140.0.3485.54)
    Internet Explorer: 11.0.26100.1882
  npmPackages:
    @airwallex/components-sdk: ^1.24.2 => 1.24.2
    @airwallex/node-sdk: ^2.0.0-beta.4 => 2.0.0-beta.4
    @ampproject/toolbox-optimizer:  undefined ()
    @aws-amplify/backend: ^1.16.1 => 1.16.1
    @aws-amplify/backend-cli: ^1.7.2 => 1.7.2
    @aws-amplify/ui-react-ai: ^1.5.0 => 1.5.0
    @aws-sdk/client-cognito-identity-provider: ^3.848.0 => 3.848.0
    @aws-sdk/client-dynamodb: ^3.882.0 => 3.882.0
    @aws-sdk/lib-dynamodb: ^3.882.0 => 3.882.0
    @babel/core:  undefined ()
    @babel/runtime:  7.22.5
    @capacitor/android: ^7.4.2 => 7.4.2
    @capacitor/app: ^7.0.1 => 7.0.1
    @capacitor/cli: ^7.4.2 => 7.4.2
    @capacitor/core: ^7.4.2 => 7.4.2
    @capacitor/ios: ^7.4.2 => 7.4.2
    @edge-runtime/cookies:  6.0.0
    @edge-runtime/ponyfill:  4.0.0
    @edge-runtime/primitives:  6.0.0
    @emotion/react: ^11.14.0 => 11.14.0
    @emotion/styled: ^11.14.0 => 11.14.0
    @ffmpeg/ffmpeg: ^0.12.15 => 0.12.15
    @google-cloud/text-to-speech: ^6.1.0 => 6.1.0
    @hapi/accept:  undefined ()
    @headlessui/react: ^2.2.3 => 2.2.3
    @mswjs/interceptors:  undefined ()
    @mui/icons-material: ^7.1.0 => 7.1.0
    @mui/material: ^7.1.0 => 7.1.0
    @napi-rs/triples:  undefined ()
    @next/font:  undefined ()
    @opentelemetry/api:  undefined ()
    @radix-ui/react-label: ^2.1.6 => 2.1.6
    @radix-ui/react-select: ^2.2.4 => 2.2.4
    @radix-ui/react-separator: ^1.1.7 => 1.1.7
    @radix-ui/react-switch: ^1.2.4 => 1.2.4
    @radix-ui/react-tabs: ^1.1.11 => 1.1.11
    @tailwindcss/postcss: ^4.1.11 => 4.1.11
    @tailwindcss/typography: ^0.5.16 => 0.5.16
    @types/aws-lambda: ^8.10.152 => 8.10.152
    @types/node: ^20 => 20.17.47 (18.19.117)
    @types/react: ^19 => 19.1.4
    @types/react-dom: ^19 => 19.1.5
    @typescript/vfs:  undefined ()
    @vercel/nft:  undefined ()
    @vercel/og:  0.6.8
    acorn:  undefined ()
    amphtml-validator:  undefined ()
    anser:  undefined ()
    assert:  undefined ()
    async-retry:  undefined ()
    async-sema:  undefined ()
    aws-amplify: ^6.14.4 => 6.14.4
    aws-amplify/adapter-core:  undefined ()
    aws-amplify/adapter-core/internals:  undefined ()
    aws-amplify/analytics:  undefined ()
    aws-amplify/analytics/kinesis:  undefined ()
    aws-amplify/analytics/kinesis-firehose:  undefined ()
    aws-amplify/analytics/personalize:  undefined ()
    aws-amplify/analytics/pinpoint:  undefined ()
    aws-amplify/api:  undefined ()
    aws-amplify/api/internals:  undefined ()
    aws-amplify/api/server:  undefined ()
    aws-amplify/auth:  undefined ()
    aws-amplify/auth/cognito:  undefined ()
    aws-amplify/auth/cognito/server:  undefined ()
    aws-amplify/auth/enable-oauth-listener:  undefined ()
    aws-amplify/auth/server:  undefined ()
    aws-amplify/data:  undefined ()
    aws-amplify/data/server:  undefined ()
    aws-amplify/datastore:  undefined ()
    aws-amplify/in-app-messaging:  undefined ()
    aws-amplify/in-app-messaging/pinpoint:  undefined ()
    aws-amplify/push-notifications:  undefined ()
    aws-amplify/push-notifications/pinpoint:  undefined ()
    aws-amplify/storage:  undefined ()
    aws-amplify/storage/s3:  undefined ()
    aws-amplify/storage/s3/server:  undefined ()
    aws-amplify/storage/server:  undefined ()
    aws-amplify/utils:  undefined ()
    aws-cdk-lib: ^2.189.1 => 2.189.1
    axios: ^1.11.0 => 1.11.0
    babel-packages:  undefined ()
    browserify-zlib:  undefined ()
    browserslist:  undefined ()
    buffer:  undefined ()
    bytes:  undefined ()
    ci-info:  undefined ()
    class-variance-authority: ^0.7.1 => 0.7.1
    cli-select:  undefined ()
    client-only:  0.0.1
    clsx: ^2.1.1 => 2.1.1
    commander:  undefined ()
    comment-json:  undefined ()
    compression:  undefined ()
    conf:  undefined ()
    constants-browserify:  undefined ()
    constructs: ^10.4.2 => 10.4.2
    content-disposition:  undefined ()
    content-type:  undefined ()
    cookie:  undefined ()
    cross-env: ^7.0.3 => 7.0.3
    cross-spawn:  undefined ()
    crypto-browserify:  undefined ()
    css.escape:  undefined ()
    data-uri-to-buffer:  undefined ()
    date-fns: ^4.1.0 => 4.1.0
    debug:  undefined ()
    devalue:  undefined ()
    domain-browser:  undefined ()
    edge-runtime:  undefined ()
    esbuild: ^0.25.4 => 0.25.4
    events:  undefined ()
    fast-xml-parser: ^5.2.5 => 5.2.5 (4.4.1, 4.5.3)
    find-up:  undefined ()
    framer-motion: ^12.12.1 => 12.12.1
    fresh:  undefined ()
    glob:  undefined ()
    groq-sdk: ^0.32.0 => 0.26.0
    gzip-size:  undefined ()
    http-proxy:  undefined ()
    http-proxy-agent:  undefined ()
    https-browserify:  undefined ()
    https-proxy-agent:  undefined ()
    icss-utils:  undefined ()
    ignore-loader:  undefined ()
    image-size:  undefined ()
    is-animated:  undefined ()
    is-docker:  undefined ()
    is-wsl:  undefined ()
    jest-worker:  undefined ()
    json5:  undefined ()
    jsonwebtoken:  undefined ()
    lamejs: ^1.2.1 => 1.2.1
    loader-runner:  undefined ()
    loader-utils:  undefined ()
    lodash.curry:  undefined ()
    lru-cache:  undefined ()
    lucide-react: ^0.511.0 => 0.511.0
    mini-css-extract-plugin:  undefined ()
    nanoid:  undefined ()
    native-url:  undefined ()
    neo-async:  undefined ()
    next: 15.3.2 => 15.3.2
    node-html-parser:  undefined ()
    openai: ^5.12.2 => 5.12.2
    opencc-js: ^1.0.5 => 1.0.5
    ora:  undefined ()
    os-browserify:  undefined ()
    p-limit:  undefined ()
    p-queue:  undefined ()
    path-browserify:  undefined ()
    path-to-regexp:  undefined ()
    picomatch:  undefined ()
    postcss: ^8.5.6 => 8.5.6 (8.4.31)
    postcss-flexbugs-fixes:  undefined ()
    postcss-modules-extract-imports:  undefined ()
    postcss-modules-local-by-default:  undefined ()
    postcss-modules-scope:  undefined ()
    postcss-modules-values:  undefined ()
    postcss-preset-env:  undefined ()
    postcss-safe-parser:  undefined ()
    postcss-scss:  undefined ()
    postcss-value-parser:  undefined ()
    process:  undefined ()
    punycode:  undefined ()
    querystring-es3:  undefined ()
    raw-body:  undefined ()
    react: ^19.0.0 => 19.1.0 (18.3.1)
    react-builtin:  undefined ()
    react-dom: ^19.0.0 => 19.1.0 (18.3.1)
    react-dom-builtin:  undefined ()
    react-dom-experimental-builtin:  undefined ()
    react-experimental-builtin:  undefined ()
    react-icons: ^5.5.0 => 5.5.0
    react-is:  19.2.0-canary-3fbfb9ba-20250409
    react-markdown: ^10.1.0 => 10.1.0
    react-refresh:  0.12.0
    react-server-dom-turbopack-builtin:  undefined ()
    react-server-dom-turbopack-experimental-builtin:  undefined ()
    react-server-dom-webpack-builtin:  undefined ()
    react-server-dom-webpack-experimental-builtin:  undefined ()
    regenerator-runtime:  0.13.4
    remark-gfm: ^4.0.1 => 4.0.1
    sass-loader:  undefined ()
    scheduler-builtin:  undefined ()
    scheduler-experimental-builtin:  undefined ()
    schema-utils:  undefined ()
    semver:  undefined ()
    send:  undefined ()
    server-only:  0.0.1
    setimmediate:  undefined ()
    shell-quote:  undefined ()
    source-map:  undefined ()
    source-map08:  undefined ()
    stacktrace-parser:  undefined ()
    stream-browserify:  undefined ()
    stream-http:  undefined ()
    string-hash:  undefined ()
    string_decoder:  undefined ()
    strip-ansi:  undefined ()
    superstruct:  undefined ()
    tailwind-merge: ^3.3.0 => 3.3.0
    tailwindcss: ^4.1.11 => 4.1.11
    tar:  undefined ()
    terser:  undefined ()
    text-table:  undefined ()
    timers-browserify:  undefined ()
    tsx: ^4.19.4 => 4.19.4
    tty-browserify:  undefined ()
    tw-animate-css: ^1.3.0 => 1.3.0
    typescript: ^5.8.3 => 5.8.3 (4.4.4, 4.9.5)
    ua-parser-js:  undefined ()
    unistore:  undefined ()
    util:  undefined ()
    vm-browserify:  undefined ()
    watchpack:  undefined ()
    web-vitals:  undefined ()
    webpack:  undefined ()
    webpack-sources:  undefined ()
    ws:  undefined ()
    zod:  undefined ()
    zod-validation-error:  undefined ()
  npmGlobalPackages:
    @aws-amplify/cli: 12.13.0
    aws-cdk: 2.165.0
    eas-cli: 14.2.0
    expo-cli: 6.3.10
    react-native-cli: 2.0.1
    serve: 14.2.3

Describe the bug

After I successfully upload file to s3 to override the old file version, I then use downloadData function to download the file again. The content body is still the old one. I check the file in the s3 is updated to the new version, but I get the old version everytime when I use amplify downloadData function. This situation happens even after a long time .

# Put output below this line

  private getS3DevicePathFunction(): ({
    identityId,
  }: {
    identityId?: string | undefined;
  }) => string {
    return ({ identityId }) =>
      `private/snapshot/${identityId}/metadata/deviceKey/${this.dataType}.json`;
  }

  /**
   * Get the last used device key from S3
   * @returns Promise<string | null> - The last used device key or null if not found
   */
  private async getLastUsedDeviceFromS3(): Promise<{
    lastUsedDevice: string | null;
    lastUpdated: number | null;
  }> {
    try {
      const s3DataResult = await downloadData({
        path: this.getS3DevicePathFunction(),
      }).result;
      console.log(
        "getLastUsedDeviceFromS3 s3DataResult",
        s3DataResult,
        this.dataType
      );
      const s3Data = await s3DataResult.body.text();
      console.log("getLastUsedDeviceFromS3 s3Data", s3Data, this.dataType);
      const s3DataObj = JSON.parse(s3Data);
      console.log(
        "getLastUsedDeviceFromS3 s3DataObj",
        s3DataObj,
        this.dataType
      );
      if (!s3DataObj.lastUsedDevice || !s3DataObj.lastUpdated) {
        return {
          lastUsedDevice: null,
          lastUpdated: null,
        };
      }
      return {
        lastUsedDevice: s3DataObj.lastUsedDevice,
        lastUpdated: s3DataObj.lastUpdated,
      };
    } catch (error) {
      if (
        (error instanceof Error && error.name === "NotFound") ||
        (error instanceof Error && error.name === "NoSuchKey")
      ) {
        console.log("No device key found in S3");
        return {
          lastUsedDevice: null,
          lastUpdated: null,
        };
      } else if (error instanceof Error && error.name === "AccessDenied") {
        console.log("Access denied to get device key from S3");
        throw error;
      } else {
        console.log("Error getting device key from S3");
        throw error;
      }
    }
  }

  /**
   * Save the last used device key to S3
   * @param deviceKey - The device key to save
   */
  private async saveLastUsedDeviceToS3(deviceKey: string): Promise<void> {
    try {
      const lastUpdated = await this.getPrivateDataLastUpdatedTime();
      const deviceData = {
        lastUsedDevice: deviceKey,
        lastUpdated: lastUpdated == 0 ? Date.now() : lastUpdated,
      };

      const uploadResult = await uploadData({
        path: this.getS3DevicePathFunction(),
        data: JSON.stringify(deviceData),
        options: {
          contentType: "application/json",
        },
      }).result;

      console.log("debug: saveLastUsedDeviceToS3: uploadResult", uploadResult);
      console.log("Device key saved to S3:", deviceKey);
    } catch (error) {
      console.error("Error saving device key to S3:", error);
      throw error;
    }
  }

These are the related code.
I have tried to reopen a new amplify project, but the problem persist.

Expected behavior

I am trying to use a simple s3 object to track the user's latest used device and compare it to the user's local cache value. I am hoping to see that after the upload of the local device, I should get the new updated file from s3 using the function "downloadData".

Reproduction steps

1. create a amplify project
2. use the code
3. check if the downloaded file content is the same as the new updated file

Code Snippet

// Put your code below this line.

  private getS3DevicePathFunction(): ({
    identityId,
  }: {
    identityId?: string | undefined;
  }) => string {
    return ({ identityId }) =>
      `private/snapshot/${identityId}/metadata/deviceKey/${this.dataType}.json`;
  }

  /**
   * Get the last used device key from S3
   * @returns Promise<string | null> - The last used device key or null if not found
   */
  private async getLastUsedDeviceFromS3(): Promise<{
    lastUsedDevice: string | null;
    lastUpdated: number | null;
  }> {
    try {
      const s3DataResult = await downloadData({
        path: this.getS3DevicePathFunction(),
      }).result;
      console.log(
        "getLastUsedDeviceFromS3 s3DataResult",
        s3DataResult,
        this.dataType
      );
      const s3Data = await s3DataResult.body.text();
      console.log("getLastUsedDeviceFromS3 s3Data", s3Data, this.dataType);
      const s3DataObj = JSON.parse(s3Data);
      console.log(
        "getLastUsedDeviceFromS3 s3DataObj",
        s3DataObj,
        this.dataType
      );
      if (!s3DataObj.lastUsedDevice || !s3DataObj.lastUpdated) {
        return {
          lastUsedDevice: null,
          lastUpdated: null,
        };
      }
      return {
        lastUsedDevice: s3DataObj.lastUsedDevice,
        lastUpdated: s3DataObj.lastUpdated,
      };
    } catch (error) {
      if (
        (error instanceof Error && error.name === "NotFound") ||
        (error instanceof Error && error.name === "NoSuchKey")
      ) {
        console.log("No device key found in S3");
        return {
          lastUsedDevice: null,
          lastUpdated: null,
        };
      } else if (error instanceof Error && error.name === "AccessDenied") {
        console.log("Access denied to get device key from S3");
        throw error;
      } else {
        console.log("Error getting device key from S3");
        throw error;
      }
    }
  }

  /**
   * Save the last used device key to S3
   * @param deviceKey - The device key to save
   */
  private async saveLastUsedDeviceToS3(deviceKey: string): Promise<void> {
    try {
      const lastUpdated = await this.getPrivateDataLastUpdatedTime();
      const deviceData = {
        lastUsedDevice: deviceKey,
        lastUpdated: lastUpdated == 0 ? Date.now() : lastUpdated,
      };

      const uploadResult = await uploadData({
        path: this.getS3DevicePathFunction(),
        data: JSON.stringify(deviceData),
        options: {
          contentType: "application/json",
        },
      }).result;

      console.log("debug: saveLastUsedDeviceToS3: uploadResult", uploadResult);
      console.log("Device key saved to S3:", deviceKey);
    } catch (error) {
      console.error("Error saving device key to S3:", error);
      throw error;
    }
  }

Log output

// Put your logs below this line

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

[josephorng]

I hope the info is enough for you to reproduce the behavior, otherwise I can try to create a mini public repo to reproduce the behavior if that is needed.

[Simone319]

Hello @josephorng , thank you for opening this github issue. We will try to reproduce the issue with the information you've given. Will followup if we're able to reproduce, or require additional information from you.

[Simone319]

Hello @josephorng ,

We're not able to reproduce the behavior as you described. The upload and download works expectedly. Have you verified if the app is passing in the correct input for the saveLastUsedDeviceToS3 and getLastUsedDeviceFromS31 function? And is the app waiting for the async function to finish with await keyword?

Also it will help if you can provide a mini repo that reproduces the issue, and we can help to trouble shoot from there.

[josephorng]

https://github.com/josephorng/amplify_storage_bug

I create this mini repo to reproduce the behavior. It does not reproduce the same error stably, but still could happen at a low possibility. After I played with this mini repo, I figured that maybe it's because I didn't add the Amplify.configure(outputs) to the very begining. After I add that, the error is smaller and I can get the right data from s3 after a while.

[Simone319]

Hello @josephorng ,

Glad that moving Amplify.configure(outputs) reduces the chance of reading stale file. It's recommended to call Amplify.configure as early as possible in your application’s life-cycle.

To dive deep into why there's still a chance that you get stale data immediately after upload, the uploadData() api returns a upload task that you can cancel, pause, or resume. (code ref)

So return of the uploadData() does not indicate the object has been uploaded to S3 successfully. The upload could be still in progress. And the time for the upload depends on various factor, such as file size, network condition, etc. That explains why you get the stale data immediately after upload, and can get the right data from s3 after a while.

[josephorng]

I can understand what you describe, but in my bigger project I can still get the stale s3 data after 10 minutes or longer, and I checked the s3 that the file had been updated to the newer version. I guess maybe I should not use s3 to record the device key.