AWS S3: Native crash when calling cancel() on Storage operations

[dkliss]

Description

Description:

Calling cancel() on active storage operations causes the application to crash at the native level with no Dart exception or stack trace. The crash affects both download and upload operations and occurs specifically when:

1. A storage operation is actively transferring data
2. Progress callbacks are being fired
3. cancel() is called on the operation

Environment:

• Platform: macOS, iOS, Android (all native platforms)
• Amplify Flutter version: 2.6.5
  • amplify_flutter: ^2.6.5
  • amplify_auth_cognito: ^2.6.5
  • amplify_storage_s3: ^2.6.5
  • amplify_api: ^2.6.5
• Flutter version: 3.35.3 (stable channel)
• Dart version: 3.9.2

Expected Behavior:

Calling cancel() should gracefully stop the operation and allow the result future to complete with a cancelled state, allowing the application to handle the cancellation properly.

Actual Behavior:

The application crashes immediately with "Lost connection to device" - no Dart exception is caught, indicating a native-level crash in the Amplify SDK that bypasses Dart's exception handling.

Reproduction Steps:

Download Operation (downloadData):

final operation = Amplify.Storage.downloadData(
  path: StoragePath.fromString(key),
  onProgress: (progress) {
    print('Progress: ${progress.transferredBytes}/${progress.totalBytes}');
  },
);

// Start download, let it transfer data for a few seconds
// Then call cancel - app crashes
await operation.cancel();

Download Operation (downloadFile):

final operation = Amplify.Storage.downloadFile(
  localFile: AWSFile.fromPath(localFilePath),
  path: StoragePath.fromString(key),
  onProgress: (progress) {
    print('Progress: ${progress.transferredBytes}/${progress.totalBytes}');
  },
);

// Start download, let it transfer data for a few seconds
// Then call cancel - app crashes
await operation.cancel();

Upload Operation (uploadData):

final payload = StorageDataPayload.bytes(bytes);
final operation = Amplify.Storage.uploadData(
  path: StoragePath.fromString(key),
  data: payload,
  onProgress: (progress) {
    print('Progress: ${progress.transferredBytes}/${progress.totalBytes}');
  },
);

// Start upload, let it transfer data for a few seconds
// Then call cancel - app crashes
await operation.cancel();

Upload Operation (uploadFile):

final operation = Amplify.Storage.uploadFile(
  localFile: AWSFile.fromPath(filePath),
  path: StoragePath.fromString(key),
  onProgress: (progress) {
    print('Progress: ${progress.transferredBytes}/${progress.totalBytes}');
  },
);

// Start upload, let it transfer data for a few seconds
// Then call cancel - app crashes
await operation.cancel();

Frequency: 100% reproducible when cancelling during active data transfer

Categories

• Analytics
• API (REST)
• API (GraphQL)
• Auth
• Authenticator
• DataStore
• Notifications (Push)
• Storage

Steps to Reproduce

See above.

Screenshots

No response

Platforms

• iOS
• Android
• Web
• macOS
• Windows
• Linux

Flutter Version

3.35.3

Amplify Flutter Version

2.6.5

Deployment Method

Amplify Gen 2

Schema

[harsh62]

@dkliss Thanks for brining this issue to our attention. I will bring this up with our team and will post an update as soon as we have one.

[ekjotmultani]

Hi @dkliss, I've attempted to recreate this issue but have been unable to. A few questions:
Are you using physical devices? Is this with your local dev build or a deployed version? If you could share some code snippets around where you see this occurring that would be helpful
Also, can you try to rap the code in a try/catch block, catch any unhandled dart errors with FlutterError.onError, catch any Platform (Android) errors with PlatformDispatcher.instance.onError

[dkliss]

Hi @ekjotmultani, Thanks for looking into it!

For macOS I use a real device, for iOS it's an emulator, and it's in dev (not deployed in prod yet).

I can provide detailed reproduction steps, though they may involve significant setup. In the meantime, I have done some analysis and the crash appears to be missing exception handling in the cancel API. I have proposed brief non-breaking changes below.

If you agree with this analysis and the proposed changes, please provide a patched build and I can test it to confirm it resolves the issue. Otherwise, I'm happy to provide detailed reproduction steps for your team to investigate further.
To save time on the complex setup for reproduction, a patched build would allow me to verify the fix quickly against my scenario.

The testing revealed:

1. Application calls cancel() on the storage operation
2. SDK's cancel() method executes successfully and returns
3. Application's finally blocks complete successfully
4. Application crashes before any SDK callbacks are triggered

The crash occurs after the cancel() method returns but before the SDK's internal whenComplete or error callbacks are invoked. This timing indicates the crash happens within the SDK's internal Future completion mechanism.

Code Analysis

Examining the SDK source code revealed missing exception handling in critical cancellation paths:

Download Task (s3_download_task.dart):

• Line 173: pause() - No try-catch around _bytesSubscription?.cancel()
• Line 230: cancel() - No try-catch around _bytesSubscription?.cancel() or _downloadCompleter.completeError()

Upload Task (s3_upload_task.dart):

• Line 264: pause() - No try-catch around _subtasksStreamSubscription.pause()
• Line 301: cancel() - No try-catch around stream/operation cancellation

When stream cancellations fail during active I/O operations, the resulting exceptions bypass Dart's error handling and cause native-level crashes.

Proposed Fix

Add defensive exception handling around all stream cancellation and Future completion operations:

Fix 1: Download Task - pause() Method

File: packages/storage/amplify_storage_s3_dart/lib/src/storage_s3_service/service/task/s3_download_task.dart

Future<void> pause() async {
  await _getObjectInitiated;

  if (_state != StorageTransferState.inProgress) {
    return;
  }

  _resetPauseCompleter();

  try {
    await _bytesSubscription?.cancel();
  } catch (e) {
    // Swallow exception - pause operation continues regardless
  } finally {
    _bytesSubscription = null;
  }

  _state = StorageTransferState.paused;
  _emitTransferProgress();
  _pauseCompleter?.complete();
}

Fix 2: Download Task - cancel() Method

File: packages/storage/amplify_storage_s3_dart/lib/src/storage_s3_service/service/task/s3_download_task.dart

Future<void> cancel() async {
  if (_state == StorageTransferState.canceled ||
      _state == StorageTransferState.success ||
      _state == StorageTransferState.failure) {
    return;
  }

  _state = StorageTransferState.canceled;

  try {
    await _bytesSubscription?.cancel();
  } catch (e) {
    // Swallow exception - cancellation continues regardless
  } finally {
    _bytesSubscription = null;
  }

  _emitTransferProgress();
  
  try {
    _downloadCompleter.completeError(
      s3_exception.s3ControllableOperationCanceledException,
    );
  } catch (e) {
    // Completer may already be completed or throw during completion
  }
}

Fix 3: Upload Task - pause() Method

File: packages/storage/amplify_storage_s3_dart/lib/src/storage_s3_service/service/task/s3_upload_task.dart

Future<void> pause() async {
  await _uploadModeDetermined;
  if (!_isMultipartUpload || _state != StorageTransferState.inProgress) {
    return;
  }
  _state = StorageTransferState.paused;

  await _uploadPartBatchingCompleted;

  try {
    _subtasksStreamSubscription.pause();
  } catch (e) {
    // Swallow exception - pause operation continues regardless
  }
}

Fix 4: Upload Task - cancel() Method

File: packages/storage/amplify_storage_s3_dart/lib/src/storage_s3_service/service/task/s3_upload_task.dart

Future<void> cancel() async {
  await _uploadModeDetermined;
  if (_state == StorageTransferState.canceled ||
      _state == StorageTransferState.success ||
      _state == StorageTransferState.failure) {
    return;
  }
  _state = StorageTransferState.canceled;

  try {
    if (_isMultipartUpload) {
      await _subtasksStreamSubscription.cancel();
    } else {
      await _putObjectOperation?.cancel();
    }
  } catch (e) {
    // Swallow exception - cancellation continues regardless
  }
}

Justification

These changes are non-breaking - no API changes, and existing behavior is preserved.

Additional Considerations

If this fix does not fully resolve the crash, it would indicate the issue lies deeper in the native platform implementations. However, these try-catch blocks should at minimum prevent the current crash, and further testing can reveal if additional fixes are needed.

[dkliss]

Hi @ekjotmultani, Here are detailed crash logs from my testing.

MACOS Crash Report

MACOS crash logs, could potentially indicate native crash.

CRITICAL: Native Crash During Download Cancellation

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x00007f9fc006a000
Termination: Segmentation fault: 11

Crashed Thread: Thread 0 (main thread)

The crash occurs AFTER all Dart-level cleanup completes successfully:
- S3DownloadTask.cancel() completes
- All error handlers execute
- All finally blocks run
- Then SIGSEGV occurs in Dart VM during microtask execution

Thread 0 Stack (Dart VM internals - symbols unavailable):
Frame 58: FlutterMacOS - dart::DartEntry::InvokeFunction
Frame 59: FlutterMacOS - dart::DartEntry::InvokeCallable
Frame 60: FlutterMacOS - Dart_InvokeClosure
Frame 61: FlutterMacOS - tonic::DartMicrotaskQueue::RunMicrotasks()
Frame 62: FlutterMacOS - flutter::Engine::FlushMicrotaskQueue()

The segfault happens during microtask queue processing, after the 
cancellation exception has been properly caught and handled in Dart code.

Environment:
- macOS 14.7.8
- Flutter/Dart VM (version in FlutterMacOS framework)
- Physical iMac device
- Amplify Storage S3 SDK

Testing Performed

I moved the AWS SDK to a local copy and applied the try-catch exception handling I proposed earlier in this issue. However, the crash still occurs, indicating the root cause may be different than initially suspected.

Modified cancel() Method for Debugging

To isolate the crash location, I added extensive logging to the SDK's cancel() method:

Future<void> cancel() async {
  safePrint('[S3DownloadTask] cancel() called - current state: $_state');
  safePrint('[S3DownloadTask] Stack trace: ${StackTrace.current}');

  if (_state == StorageTransferState.canceled ||
      _state == StorageTransferState.success ||
      _state == StorageTransferState.failure) {
    safePrint(
      '[S3DownloadTask] cancel() skipped - already in terminal state: $_state',
    );
    return;
  }

  safePrint('[S3DownloadTask] Setting state to canceled');
  _state = StorageTransferState.canceled;

  try {
    safePrint('[S3DownloadTask] Attempting to cancel bytes subscription');
    await _bytesSubscription?.cancel();
    _bytesSubscription = null;
    safePrint('[S3DownloadTask] Bytes subscription canceled successfully');

    safePrint('[S3DownloadTask] Emitting transfer progress');
    _emitTransferProgress();
    safePrint('[S3DownloadTask] Transfer progress emitted');

    safePrint(
      '[S3DownloadTask] Checking if completer is completed: ${_downloadCompleter.isCompleted}',
    );
    if (!_downloadCompleter.isCompleted) {
      safePrint('[S3DownloadTask] Completing download completer with error');
      _downloadCompleter.completeError(
        s3_exception.s3ControllableOperationCanceledException,
      );
      safePrint(
        '[S3DownloadTask] Download completer completed with error successfully',
      );
    } else {
      safePrint(
        '[S3DownloadTask] Completer already completed, skipping completeError',
      );
    }

    safePrint('[S3DownloadTask] cancel() completed successfully');
  } on Exception catch (e) {
    safePrint('[S3DownloadTask] Exception during cancel: $e');
  } on Error catch (e) {
    safePrint('[S3DownloadTask] Error during cancel: $e');
  }
}

Key Finding

The crash occurs AFTER the SDK's cancel() method completes successfully and AFTER all cleanup code executes. The cancellation exception never reaches application-level catch blocks, suggesting the crash happens during error propagation after completeError() is called.

Stack Trace and Logs

#5      CloudStorageOperationCancelHelper.cancelSpecificDataDownload (package:cloud_storage_ui/src/cubit/logicHelpers/storage_operations_helper.dart:73:11)
#6      CreateCloudStorageCubit.cancelDataDownload (package:cloud_storage_ui/src/cubit/create_storage_cubit.dart:429:39)
#7      DownloadProgressAndStatusWidget.build.<anonymous closure>.<anonymous closure> (package:cloud_storage_ui/src/templates/support/download_progress.dart:78:57)
#8      MyGenericGestureDetectorButtonCrossPlatform._handleTapDown (package:platform_kit/src/platformWidgets/buttons/my_gesture_detector_icon_button.dart:32:17)
#9      TapGestureRecognizer.handleTapDown.<anonymous closure> (package:flutter/src/gestures/tap.dart:730:61)
#10     GestureRecognizer.invokeCallback (package:flutter/src/gestures/recognizer.dart:345:24)
#11     TapGestureRecognizer.handleTapDown (package:flutter/src/gestures/tap.dart:730:11)
#12     BaseTapGestureRecognizer._checkDown (package:flutter/src/gestures/tap.dart:374:5)
#13     BaseTapGestureRecognizer.didExceedDeadline (package:flutter/src/gestures/tap.dart:344:5)
#14     PrimaryPointerGestureRecognizer.didExceedDeadlineWithEvent (package:flutter/src/gestures/recognizer.dart:749:5)
#15     PrimaryPointerGestureRecognizer.addAllowedPointer.<anonymous closure> (package:flutter/src/gestures/recognizer.dart:691:41)
#16     _rootRun (dart:async/zone.dart:1517:47)
#17     _CustomZone.run (dart:async/zone.dart:1422:19)
#18     _CustomZone.runGuarded (dart:async/zone.dart:1321:7)
#19     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1362:23)
#20     _rootRun (dart:async/zone.dart:1525:13)
#21     _CustomZone.run (dart:async/zone.dart:1422:19)
#22     _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:1345:23)
#23     Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18:15)
#24     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:423:19)
#25     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:454:5)
#26     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:193:12)

flutter: [S3DownloadTask] Setting state to canceled
flutter: [S3DownloadTask] Attempting to cancel bytes subscription
flutter: [S3DownloadTask] Bytes subscription canceled successfully
flutter: [S3DownloadTask] Emitting transfer progress
flutter: [S3DownloadTask] Transfer progress emitted
flutter: [S3DownloadTask] Checking if completer is completed: false
flutter: [S3DownloadTask] Completing download completer with error
flutter: [S3DownloadTask] Download completer completed with error successfully
flutter: [S3DownloadTask] cancel() completed successfully
flutter: 23:14:18 ℹ️ INFO  │ [AmplifyDataDownloadManager] _currentDownloadOperation.cancel() completed
flutter: 23:14:18 ℹ️ INFO  │ [AmplifyDataDownloadManager] cancelDownload() finally block - _isActive set to false
flutter: 23:14:18 ℹ️ INFO  │ [AmplifyDataDownloadManager] Download operation completed
Lost connection to device.

Analysis

1. SDK's cancel() reports: "cancel() completed successfully"
2. Application's finally blocks execute properly
3. SDK's whenComplete callback fires: "Download operation completed"
4. Application's catch blocks never execute - no exception reaches user code
5. Crash occurs immediately after all cleanup completes

This sequence shows the crash is not in the cancellation logic itself, but occurs during exception propagation after completeError() is called.

Reproduction Details

Environment:

• Physical macOS device
• Development build
• Occurs consistently when canceling download during active transfer

Reproduction Steps:

1. Ensure you have a large file in S3 (10+ MB - can upload manually via AWS console, or flutter SDK upload)
2. Update the StoragePath in the code below to point to your test file
3. Run the app and tap "Start Download"
4. Immediately tap "Cancel Download" within 1-2 seconds while transfer is active
5. App crashes immediately

Minimal Reproduction Code

import 'package:amplify_flutter/amplify_flutter.dart';
import 'package:flutter/material.dart';

class DownloadCancelTest extends StatefulWidget {
  @override
  State<DownloadCancelTest> createState() => _DownloadCancelTestState();
}

class _DownloadCancelTestState extends State<DownloadCancelTest> {
  StorageDownloadDataOperation? _downloadOperation;
  bool _isDownloading = false;

  Future<void> startDownload() async {
    setState(() => _isDownloading = true);

    try {
      final operation = Amplify.Storage.downloadData(
        path: StoragePath.fromString('path/to/large-file.mp4'),
      );
      
      _downloadOperation = operation;
      
      final result = await operation.result;
      safePrint('Download complete: ${result.bytes.length} bytes');
      
      setState(() => _isDownloading = false);
    } catch (e) {
      safePrint('Download error: $e');
      setState(() => _isDownloading = false);
    }
  }

  Future<void> cancelDownload() async {
    if (_downloadOperation != null) {
      try {
        safePrint('Canceling download...');
        await _downloadOperation!.cancel();
        safePrint('Cancel completed');
      } catch (e) {
        safePrint('Cancel error: $e');
      } finally {
        setState(() {
          _isDownloading = false;
          _downloadOperation = null;
        });
      }
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Download Cancel Test')),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: [
            ElevatedButton(
              onPressed: _isDownloading ? null : startDownload,
              child: Text('Start Download'),
            ),
            SizedBox(height: 20),
            ElevatedButton(
              onPressed: _isDownloading ? cancelDownload : null,
              child: Text('Cancel Download'),
              style: ElevatedButton.styleFrom(
                backgroundColor: Colors.red,
              ),
            ),
            if (_isDownloading)
              Padding(
                padding: EdgeInsets.only(top: 20),
                child: CircularProgressIndicator(),
              ),
          ],
        ),
      ),
    );
  }
}

Expected vs Actual Behavior

Expected: Cancel should throw a catchable StorageOperationCanceledException

Actual: App crashes with "Lost connection to device" before exception reaches any Dart catch blocks

Additional Context

I've added extensive error handling including:

• Try-catch blocks around all operations
• Finally blocks for cleanup

The exception never reaches any of these handlers. The logs (provided above) show SDK's cancel() completes successfully, but the app terminates at the native level before the exception can be caught in Dart code.

[tyllark]

Hello @dkliss, I'm sorry for the delay, but I've been unable to reproduce the issue on my end. With the sample app you provided I'm receiving the StorageOperationCanceledException as expected.

flutter: Canceling download...
flutter: Cancel completed
flutter: Download error: StorageOperationCanceledException {
"message": "The operation has been canceled.",
"recoverySuggestion": "This is expected when you call cancel() on a storage operation. This exception allows you to take further action when an operation is canceled."
}

To help identify the root cause could you please send your full stack trace as it only starts at number 5. Your stack trace includes references to isolates (which we have an open feature request to support) and several 3rd party packages. Can you temporarily remove all non-Amplify packages from your pubspec.yaml and reproduce the issue with only the below main.dart and amplify_outputs.dart in your project:

import 'package:amplify_auth_cognito/amplify_auth_cognito.dart';
import 'package:amplify_flutter/amplify_flutter.dart';
import 'package:amplify_storage_s3/amplify_storage_s3.dart';
import 'package:flutter/material.dart';

import 'amplify_outputs.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  await _configureAmplify();

  runApp(
    MaterialApp(
      restorationScopeId: 'app',
      title: 'Download Cancel Test',
      theme: ThemeData.dark(),
      home: Scaffold(body: DownloadCancelTest()),
    ),
  );
}

Future<void> _configureAmplify() async {
  try {
    await Amplify.addPlugins([AmplifyAuthCognito(), AmplifyStorageS3()]);

    await Amplify.configure(amplifyConfig);
    safePrint('Success configuring Amplify!');
  } on Exception catch (e) {
    safePrint('Error configuring Amplify: $e');
  }
}

class DownloadCancelTest extends StatefulWidget {
  const DownloadCancelTest({super.key});

  @override
  State<DownloadCancelTest> createState() => _DownloadCancelTestState();
}

class _DownloadCancelTestState extends State<DownloadCancelTest> {
  StorageDownloadDataOperation? _downloadOperation;
  bool _isDownloading = false;

  Future<void> startDownload() async {
    setState(() => _isDownloading = true);

    try {
      final operation = Amplify.Storage.downloadData(
        path: StoragePath.fromString('public/gh6401_1'),
      );

      _downloadOperation = operation;

      final result = await operation.result;
      safePrint('Download complete: ${result.bytes.length} bytes');

      setState(() => _isDownloading = false);
    } catch (e) {
      safePrint('Download error: $e');
      setState(() => _isDownloading = false);
    }
  }

  Future<void> cancelDownload() async {
    if (_downloadOperation != null) {
      try {
        safePrint('Canceling download...');
        await _downloadOperation!.cancel();
        safePrint('Cancel completed');
      } catch (e) {
        safePrint('Cancel error: $e');
      } finally {
        setState(() {
          _isDownloading = false;
          _downloadOperation = null;
        });
      }
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Download Cancel Test')),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: [
            ElevatedButton(
              onPressed: _isDownloading ? null : startDownload,
              child: Text('Start Download'),
            ),
            SizedBox(height: 20),
            ElevatedButton(
              onPressed: _isDownloading ? cancelDownload : null,
              style: ElevatedButton.styleFrom(backgroundColor: Colors.red),
              child: Text('Cancel Download'),
            ),
            if (_isDownloading)
              Padding(
                padding: EdgeInsets.only(top: 20),
                child: CircularProgressIndicator(),
              ),
          ],
        ),
      ),
    );
  }
}

This will help us rule out any 3rd party package incompatibilities. If the cancel is still causing a crash we will need to take a look at your platform setup and how you provisioned your S3 backend.