Skip to content

MLPerf Mobile App v4.1

Choose a tag to compare

View all tags
@anhappdev anhappdev released this 11 Feb 08:50
v4.1
2dab201
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Description

MLPerf Mobile Inference Benchmark is an open-source benchmark suite for measuring how fast mobile devices (e.g. phones, laptops) can run AI tasks. The benchmark is supported by the MLPerf Mobile App which currently supports Android and iOS. Please see the MLPerf Mobile Inference benchmark paper for a detailed description of the benchmarks along with the motivation and guiding principles behind the benchmark suite.

Release Notes

New features

  • Introduced a new benchmark task: Stable Diffusion
  • Added checksum verification for datasets
  • Provided support for utilizing multiple model files
  • Offered the option to download all resources (models, datasets) on-demand
  • Added support for multiple ML pipelines

Bug fixes and other changes

  • Migrated the Flutter SDK from version 3.7 to version 3.19
  • Implemented Firebase App Check
  • Enhanced the performance of the MobileNetV4 Core ML model
  • Resolved a crash issue caused by permission denied on the Android Play Store version
  • Switched from GitHub to Cloudflare for hosting models and datasets
  • Updated and improved the CI/CD system

Full Changelog: v4.0...v4.1

Supported SOCs

Mediatek

  • Dimensity 9000 series (9000/9000+/9200/9200+/9300/9300+)
  • Dimensity 8000 series (8000/8020/8050/8100/8200)

Qualcomm

  • Snapdragon 8 Gen 3
  • Snapdragon 8s Gen 3
  • Snapdragon 7 Gen 3
  • Snapdragon 8 Gen 2
  • Snapdragon 7+ Gen 2
  • Snapdragon 8+ Gen 1
  • Snapdragon 8 Gen 1
  • Snapdragon 7 Gen 1
  • Snapdragon 888
  • Snapdragon 778

Samsung

  • Exynos 2400
  • Exynos 2300
  • Exynos 2200
  • Exynos 2100
  • Exynos 990

Google Pixel

  • Pixel 8 and Pixel 8 Pro (Tensor G3 SoC)
  • Pixel 7 and Pixel 7 Pro (Tensor G2 ScC)
  • Pixel 6 and Pixel 6 Pro (Tensor G1 ScC)

The MLPerf Mobile App will also run on a host of other devices via our default path, which uses TensorFlow Lite on Android devices.

Installation instructions

  1. Allow installation of unknown apps in Settings > Apps > Special Access.
  2. Download the MLPerf Mobile APK.
  3. Find the APK in 'Downloads' or file browser.
  4. Tap the APK file. Approve installation when prompted.
  5. Confirm 'Install'.
  6. Once installed, tap 'Open' to launch MLPerf Mobile.

SHA256 of mlperfbench-v4.1-qsmgt.apk: 39d3d54c1c8b014aca6a47d49ada51a7c6342b581856e6af61457af98a7e11f9

Assets 3
Loading