Skip to content

Conversation

@vascosousamindera
Copy link
Collaborator

@vascosousamindera vascosousamindera commented May 26, 2025

Ticket: ALFMOB-178

This PR introduces major improvements and integrations to support end-to-end performance testing for app startup time across both Android and iOS platforms, backed by full database integration and CI/CD support.


The following features have been implemented:

✅ Test Execution Core

  • Support for running Test Plans and Test Suites.
  • Threshold evaluation for different metric outputs, ensuring metrics meet expected criteria.
  • Fully integrated result tracking (per execution, per metric).

📱 Android & iOS Support

  • Implemented App Startup Time tests for both platforms.
  • Support for passing elementToWaitFor, timeout, thresholdType, and thresholdValue parameters.
  • Integrated real devices and simulators/emulators for dynamic test execution.

💽 Database Integration

  • Refactored and extended the database model for:
  • Test Plans, Test Suites, Devices, Metrics, Thresholds, Results, etc.
  • All test executions now persist detailed metric output.

⚙️ Backend Enhancements

  • Created and updated domain entities, repositories, services, and controllers.
  • Added routes to support:
    • Creating and fetching different data that may be needed.
    • Running tests and retrieving results.

🔧 Driver-Runner (XCTest)

  • Redesigned to:
    • Accept dynamic parameters from the HTTP layer.
    • Handle startup timing with pass/fail logic based on threshold.
    • Return detailed JSON output to the iOS orchestrator.

🚀 CI/CD Integration

  • Created the first version of a GitHub Actions .yml file to facilitate the analysis and integration of automated performance tests into the CI/CD pipeline.

🛠️ Updates to PR on May 31, 2025

  • CI/CD Workflow Enhancement:
    • Delivered stable versions of two GitHub Actions workflow files (.yml) for Android and iOS testing.
    • These workflows enable automated performance testing of both Android and iOS applications directly within CI/CD pipelines, ensuring startup time tests run consistently across platforms.

This PR delivers a stable and production-ready version of the performance testing framework, enabling reliable and automated evaluation of app startup time across both Android and iOS platforms. It also introduces fully functional CI/CD integration, establishing a strong foundation for continuous performance validation within development pipelines.

Next steps will focus on strengthening the CI/CD workflows, expanding support for additional performance metrics, improving result analysis and reporting, and enhancing test orchestration for greater scalability and coverage.

- Updated the DB Model;
- Added DB Configuration in order to track the tests results;
- Fixed some server configurations.
- This additions and updates are crucial for database integration, since it makes it easier to store and extract info.
- Added new routes for creating and fetching data.
- Fixed some errors in some functions;
- Updated some functions;
- Created new methods to help with the app functionalities.
- Integrated the repositories with the database;
- Created new repos and deleted useless ones.
- New Server implementation;
- More dependencies fixed;
- Integration with some services needed in app boot.
- Updated the implementation to evaluate the results;
- Output is now provided in a different way;
- Driver Runner suffered major changes related to test parameters and evaluation;
- Added device and app configuration to the flow of the tests.
- Created the first version of a github actions .yml file to facilitate the analysis and integration of automated performance tests into the CI/CD pipeline.
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@Mindera Mindera deleted a comment from github-actions bot May 31, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jun 2, 2025

📝 Performance Test Results

Performance Test Suite Results (Android)

Suite Execution: 37
Test Suite Version: 29

Test Execution Results

  • App Name: Alfie.apk
  • App Version: 0.8.0
  • App Package: au.com.alfie.ecomm.debug
  • Main Activity: au.com.alfie.ecomm.MainActivity
  • Execution Type: Cold Start

Test Execution: Startup Time Plan

  • Metric: App Startup Time
  • Device: Medium_Phone_API_35
  • App Name: Alfie.apk
  • App Version: 0.8.0
  • App Package: au.com.alfie.ecomm.debug
  • Main Activity: au.com.alfie.ecomm.MainActivity
  • Execution Type: Cold Start
Test Result Start Time End Time
true 2025-06-02T16:11:47.584367086 2025-06-02T16:11:47.584372466
Thresholds
Target Value Threshold Type Metric Output
20000 Max launchTime
Metric Output Results
Metric Output Value
launchTime 17443
elementFound true

@Mindera Mindera deleted a comment from github-actions bot Jun 2, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jun 2, 2025

📝 Performance Test Results

Performance Test Suite Results (iOS)

Suite Execution: 39
Test Suite Version: 30

Test Execution Results

  • App Name: Alfie.app
  • App Version: 0.8.1
  • App Package: com.mindera.alfie.debug
  • Main Activity: N/A
  • Execution Type: Cold Start

Test Execution: Startup Time Plan

  • Metric: App Startup Time
  • Device: iPhone 16 Pro
  • App Name: Alfie.app
  • App Version: 0.8.1
  • App Package: com.mindera.alfie.debug
  • Main Activity: N/A
  • Execution Type: Cold Start
Test Result Start Time End Time
false 2025-06-02T16:35:15.866777 2025-06-02T16:35:15.866784
Thresholds
Target Value Threshold Type Metric Output
5000 Max launchTime
Metric Output Results
Metric Output Value
launchTime 12634
elementFound true

@Mindera Mindera deleted a comment from github-actions bot Jun 3, 2025
@vascosousamindera vascosousamindera merged commit a29d48c into main Jun 3, 2025
2 of 3 checks passed
@vascosousamindera vascosousamindera added the enhancement New feature or request label Jun 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants