Merge pull request #10000 from wmontwe/docs/9999/add-manual-testing-checklist

docs: add release testing checklist

Author: rafaeltonholo

docs/SUMMARY.md

@@ -51,6 +51,7 @@ generator, in this case, **mdbook**. It defines the structure and navigation of
   - [Release Process](release/RELEASE.md)
   - [Release Automation](release/AUTOMATION.md)
   - [Developer Release Checklist](release/developer-checklist.md)
+  - [Release Testing Checklist](release/testing-checklist.md)
   - [Manual Release (historical)](release/HISTORICAL_RELEASE.md)
 - [Security]()
   - [Threat Modeling Guide](security/threat-modeling-guide.md)

docs/release/testing-checklist.md

@@ -0,0 +1,199 @@
+### Release Manual Testing Checklist
+
+Purpose
+
+- Ensure consistent, repeatable pre-release validation across core areas: installation/upgrade, UI/UX, functionality regression, performance, connectivity, and security/privacy.
+- Provide a short “Smoke” run and a full “Release” run.
+
+When to use
+
+- Before every Beta/Production release
+- After high‑risk feature merges
+
+Test environments
+
+- Devices: At least one low-end and one mid/high-end device; 1 emulator
+- OS versions: Last 3 major Android versions supported + Lowest supported (API 21)
+- Network: Wi‑Fi, LTE/3G, Offline/Airplane; captive portal if available
+- Accounts: At least 2 IMAP accounts (different providers), 1 POP3 if supported; one account with many messages/folders
+- Locales: English + one non‑English (RTL if possible)
+- Themes: Light and Dark
+
+Data prerequisites
+
+- Test mailboxes with folders: Inbox, Sent, Drafts, Trash, Spam, Archive, custom folders
+- A few threads with 50+ messages for performance/scroll tests
+- At least one account configured with push (if supported)
+
+How to record results
+
+- Mark each item Pass/Fail/Blocked/Not Applicable
+- Record build/commit, device, OS, account provider, network type
+- File defects with clear reproduction steps, logs, and screenshots
+
+## Quick Smoke (15–25 min)
+
+1. Install/Launch
+   - Fresh install: App installs and launches without crash
+   - First‑run: Onboarding visible, no ANRs
+2. Account Setup (1 IMAP)
+   - Manual or auto‑config completes; inbox loads
+3. UI Smoke
+   - Navigate: Account list → Inbox → Message → Back works
+   - Compose: Create draft, send to self, appears in Sent and then Inbox
+4. Sync/Refresh
+   - Pull‑to‑refresh fetches new mail; background sync triggers at expected times
+5. Notifications
+   - Receive mail notification; tap opens message; mark as read behavior correct
+6. Basic Actions
+   - Read, star/flag, delete to Trash; undo/restore if offered
+7. Performance quick check
+   - App cold start acceptable, scrolling in Inbox smooth
+
+## Full Release Checklist
+
+### A. Installation and Upgrade
+
+Top-level verification
+
+- Fresh Install — App installs and runs correctly on a fresh install
+  - Installs and launches without crash
+  - Permissions requested only when needed; rationale shown
+  - First‑run experience shows expected screens/toggles
+- Upgrade to Production — App upgrades correctly to the latest production version (only for releases)
+  - Upgrade to latest production build preserves accounts, settings, and local cache
+  - No duplicate notifications or migration errors on first launch
+- Upgrade to Beta — App upgrades correctly to the latest beta version
+  - Upgrade to latest beta build preserves accounts, settings, and local cache
+  - Feature flags and migrations behave correctly when upgrading to beta
+  - No duplicate notifications or migration errors on first launch
+- Backup/Restore
+  - Backup 1 account setting, delete account from app, restore from backup; account usable
+  - Backup all accounts, uninstall/reinstall, restore settings; accounts usable
+- Settings Import via QR code (only Thunderbird)
+  - From first-run flow, choose "Import settings"; scan a valid QR; settings are imported/applied or account is added/usable
+  - From Settings > Import, choose "Scan QR code"; scan a valid QR; settings are imported/applied or account is added/usable
+  - Try an invalid or expired QR; a clear error is shown and the app does not crash
+
+### B. Account Management
+
+- Add accounts
+  - Auto‑config works for common providers; fallback to manual works
+  - Validation errors are clear (username/password/server settings)
+- Multiple accounts
+  - Switch accounts; titles/subtitles update correctly
+  - Unified Inbox visibility matches setting and number of accounts
+- Delete account
+  - Deleting current account transitions correctly (Unified Inbox if enabled and >1; else default account)
+  - No crash during or after deletion; capability checks safe
+- Special folders
+  - Drafts/Sent/Archive/Trash/Spam configured/created as needed
+
+## C. Core Mail Flows (per account)
+
+- Receiving
+  - New message appears after push/refresh; badge/notification updates
+- Reading
+  - Open message; images loading policy obeyed; external images prompt if applicable
+- Compose/Reply/Forward
+  - From identity correct; quoted content; signatures; attachments; send succeeds
+- Drafts
+  - Auto‑save draft; edit and send later
+- Move/Copy
+  - Move/copy messages between folders; operation succeeds and reflects in UI
+- Search
+  - Local search by sender/subject/body; filters; results correct; remote search if supported
+- Attachments
+  - Download, open with external app, share; large attachment handling
+
+## D. UI and UX
+
+- Navigation
+  - Drawer/tabs/back behavior consistent; deep linking from notifications
+- Message List view
+  - Threaded/flat messages shown; selection mode; swipe actions (customizable to check archive/delete)
+- Message view
+  - Rendering of HTML/plain text; long messages; quoted text expansion; link handling
+  - Image loading per settings; attachment previews
+- Theming and localization
+  - Light/Dark theme correctness; typography and icons;
+  - Non‑English locale strings fit and are translated; RTL layout alignment
+- Accessibility
+  - TalkBack announcements; content descriptions present
+  - Focus order logical; dynamic type/Font scale respected
+
+## E. Performance and Resource Use
+
+- Startup performance
+  - Cold start time acceptable on low‑end device
+- Scrolling and list operations
+  - Smooth scrolling in Inbox with 1000+ items; no jank during load more
+- Sync performance
+  - Time to first sync for a fresh account reasonable; no repeated retries
+- Resource usage
+  - No abnormal CPU usage when idle; memory stable during long scrolling; cache does not balloon unexpectedly
+
+## F. Connectivity and Background Behavior
+
+- Network transitions
+  - Wi‑Fi → Cellular → Offline: app handles gracefully; shows appropriate banners/status
+- Offline usage
+  - Read previously synced messages; queued actions (send/move) execute on reconnect
+
+## G. Notifications
+
+- New mail
+  - Shows correct sender/subject; grouping for multiple messages
+  - Actions: Mark read, Delete, Archive function correctly and reflect in app
+- Quiet hours/Do Not Disturb
+  - Respected according to settings
+
+## H. Settings and Preferences
+
+- General settings
+  - Unified Inbox toggle behavior: only shows when >1 accounts, correct fallback behavior
+  - Sort options persist per account/unified; changes reflected immediately
+- Account settings
+  - Outgoing/incoming server edits persist; re‑auth flows; OAuth if applicable
+
+## I. Security and Privacy
+
+- Permissions
+  - No unexpected permission prompts; revoking a permission shows clear error states
+- Sensitive logging
+  - Disabled by default; enabling shows warnings; logs redact personal data where possible
+- Certificates (if applicable)
+  - Certificate errors surfaced with clear options; pinning remembered
+
+## J. Regression Checks (recently changed areas)
+
+- Check changes in this release
+  - Perform smoke tests around changed screens/use cases
+- Logs, crashes, and ANRs
+  - No new crashes/ANRs; logging remains appropriate and doesn’t expose sensitive data
+- Quick smoke around changed screens
+  - Basic open/close, refresh/sync, action buttons, and navigation work
+
+K. Known Issues Verification
+- Validate that previously documented known issues still match behavior and are noted in release notes
+
+## Sign-off
+
+- All “Smoke” items: Pass
+- No Critical/High defects open; Medium has workarounds; Low acceptable
+- Performance thresholds met on target devices
+
+## Appendix A: Quick result template
+
+- Build/Commit:
+- Device/OS:
+- Accounts:
+- Network:
+- Locale/Theme:
+- Summary:
+- Defects filed:
+
+## Appendix B: Full run log template
+
+- Include timestamps, steps, expected vs actual, screenshots, and logs as needed
+