Skip to content

Commit 63418b6

Browse files
authored
Merge pull request #139 from arm-university/main
New project, mobile optimisations
2 parents af78740 + 7d7cb12 commit 63418b6

File tree

8 files changed

+784
-6
lines changed

8 files changed

+784
-6
lines changed
Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
title: "Develop a Novel Computational Photography Pipeline"
3+
description: "This project creates and implements a novel computational photography pipeline that is optimized for Arm-based mobile devices using SME2 and neural technology. This project comes with the possibility of a hardware donation to support your efforts"
4+
subjects:
5+
- "ML"
6+
- "Graphics"
7+
requires-team:
8+
- "No"
9+
platform:
10+
- "Laptops and Desktops"
11+
- "AI"
12+
sw-hw:
13+
- "Software"
14+
support-level:
15+
- "Self-Service"
16+
- "Arm Ambassador Support"
17+
- "Direct Support from Arm"
18+
publication-date: 2026-01-07
19+
license:
20+
status:
21+
- "Published"
22+
donation:
23+
24+
---
25+
26+
![super-resolution](./images/Super-resolution_example_closeup.png)
27+
28+
### Description
29+
30+
**Why this is important?**
31+
32+
For many consumers, the camera and its associated features are among the most important factors when choosing a new smartphone. Modern capabilities such as night-mode photography and extreme digital zoom increasingly rely on neural networks to enhance image quality, for example by improving the signal-to-noise ratio (SNR). A notable example is the Google Pixel 10 Pro’ [pro-res zoom](https://blog.google/products/pixel/google-pixel-pro-res-zoom/) which uses on-device diffusion models and generative AI to reconstruct fine detail when zooming beyond the optical range.
33+
34+
With the introduction of Arm’s Scalable Matrix Extension (SME2) in commercial handsets, these advanced neural networks can now run efficiently on the CPU itself. This enables a wider range of devices to incorporate neural networks directly into their photography pipelines, while also promoting more portable software designs that are not tightly coupled to a specific Neural Processing Unit (NPU).
35+
36+
37+
**Project Summary**
38+
39+
This challenge focuses on designing and implementing a novel computational photography pipeline on an Arm-based device. Your solution must implement one of the following example pipelines, or a closely related variant:
40+
41+
• Night-mode imaging: Improve the signal-to-noise ratio (SNR) of photographs captured in low-light conditions.
42+
• Portrait-mode imaging: Apply realistic bokeh blur effects to background regions in portrait photographs.
43+
• Neural ray denoising: Use neural networks to denoise ray-based or rendering-related image data.
44+
• Generative AI–based pipelines: Leverage GenAI models, such as style-transfer or diffusion-based approaches, to enhance or transform images.
45+
46+
For example, your approach could involve fine-tuning an existing model, implementing a novel pipeline. The objective is to optimize your pipeline for both inference latency and perceived image quality and the technical criteria, of your project, on which you will be judged, are as follows:
47+
48+
- Must use a quantized neural network, preferably less that 16-bits (i.e. quantized to less the FP16)
49+
- Preferably use the Executorch runtime, designed for running neural networks with the [Arm Neural Technology](https://developer.arm.com/community/arm-community-blogs/b/ai-blog/posts/arm-neural-technology-in-executorch-1-0) with the [ML Extensions for Vulkan](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/) backend
50+
- Algorithms should run on an Arm-based CPU with single-instruction multiple data (SIMD) such as [SVE](https://developer.arm.com/Architectures/Scalable%20Vector%20Extensions) or [SME](https://www.arm.com/technologies/sme2) or an [Neural-Graphics](https://developer.arm.com/mobile-graphics-and-gaming/neural-graphics) enabled GPU (either simulation or physical handset once available)
51+
52+
## Prerequisites
53+
54+
- Experience with traditional image processing algorithms (e.g., convolution, filters).
55+
- Intermediate experience or willingness to learn about machine learning for graphics and photography (e.g., neural ray denoising).
56+
- Familiarity with build systems and associated tools such as CMake, docker.
57+
- Experience with object-orientated programming and scripting with langauges such as C++ and Python.
58+
- Some experience or willingness to learn about low-level programming and using APIs such as Vulkan.
59+
60+
## Resources from Arm and our partners
61+
62+
- Blog: [Neural Camera Denoising with Arm SME2](https://developer.arm.com/community/arm-community-blogs/b/ai-blog/posts/unlocking-the-power-of-neural-camera-denoising-with-arm-sme2)
63+
- Software: [Reference AI Camera Pipelines](https://gitlab.arm.com/kleidi/kleidi-examples/ai-camera-pipelines)
64+
- Blog: [Neural Graphics](https://developer.arm.com/mobile-graphics-and-gaming/neural-graphics)
65+
- Software: [Arm KleidiCV](https://gitlab.arm.com/kleidi/kleidicv)
66+
- Software: [Vkdt open source photography workflow](https://github.com/hanatos/vkdt)
67+
- Blog: [SME2 enabled devices](https://newsroom.arm.com/blog/arm-c1-cpu-cluster-on-device-ai-performance)
68+
- Blog: [Arm Neural Technolog in Executorch 1.0](https://developer.arm.com/community/arm-community-blogs/b/ai-blog/posts/arm-neural-technology-in-executorch-1-0)
69+
- Tutorial: [Demonstation using Arm VGF backend](https://github.com/pytorch/executorch/blob/main/examples/arm/vgf_minimal_example.ipynb)
70+
71+
## Support Level
72+
73+
This project is designed to be self-serve but comes with opportunity of some community support from Arm Ambassadors, who are part of the Arm Developer program. If you are not already part of our program, [click here to join](https://www.arm.com/resources/developer-program?#register). Additionally, if you would like to engage with this project and need assistance with access to hardware, please email us at Arm-Developer-Labs@arm.com.
74+
75+
## Benefits
76+
77+
Standout project contributions to the community will earn digital badges. These badges can support CV or resumé building and demonstrate earned recognition.
78+
79+
To receive the benefits, you must show us your project through our [online form](https://forms.office.com/e/VZnJQLeRhD). Please do not include any confidential information in your contribution. Additionally if you are affiliated with an academic institution, please ensure you have the right to share your material.
80+
81+
## Attributions
82+
83+
By User:Omegatron - Created by User:Omegatron using the GIMP, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1836856
84+
85+
86+
87+
148 KB
Loading

docs/_data/navigation.yml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -300,6 +300,26 @@ projects:
300300
- Arm Ambassador Support
301301
status:
302302
- Published
303+
- title: Computational-Photography
304+
description: This project creates and implements a novel computational photography
305+
pipeline that is optimized for Arm-based mobile devices using SME2 and neural
306+
technology. This project comes with the possibility of a hardware donation to
307+
support your efforts
308+
url: /2026/01/07/Computational-Photography.html
309+
subjects:
310+
- ML
311+
- Graphics
312+
platform:
313+
- Laptops and Desktops
314+
- AI
315+
sw-hw:
316+
- Software
317+
support-level:
318+
- Self-Service
319+
- Arm Ambassador Support
320+
- Direct Support from Arm
321+
status:
322+
- Published
303323
- title: Academic-Trends-Dashboard
304324
description: "This self-service project creates a web-scraping, database-driven\
305325
\ dashboard that visualizes how computer-science research topics shift over\

docs/_includes/header.html

Lines changed: 40 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,14 @@
2020
{%- endif -%} -->
2121
</div>
2222
{%- if page.url == "/" or page.url == "/index.html" -%}
23-
<button class="button button--secondary mission-button" onclick="document.querySelector('.grid.grid--reverse')?.scrollIntoView({behavior: 'smooth', block: 'start'})">Mission Statement</button>
23+
<button class="button button--secondary mission-button mission-button--desktop" onclick="document.querySelector('.grid.grid--reverse')?.scrollIntoView({behavior: 'smooth', block: 'start'})">Mission Statement</button>
2424
{%- else -%}
25-
<a class="button button--secondary mission-button" href="{{ "/#mission" | relative_url }}">Mission Statement</a>
26-
{%- endif -%}
27-
{%- if site.search.provider -%}
28-
<button class="button button--secondary button--circle search-button js-search-toggle"><i class="fas fa-search"></i></button>
25+
<a class="button button--secondary mission-button mission-button--desktop" href="{{ "/#mission" | relative_url }}">Mission Statement</a>
2926
{%- endif -%}
27+
<!-- Search button removed -->
28+
<button class="mobile-menu-toggle" aria-label="Toggle navigation menu">
29+
<span class="hamburger-icon"></span>
30+
</button>
3031
</div>
3132
{%- if site.data.navigation.header -%}
3233
<nav class="navigation">
@@ -43,6 +44,15 @@
4344
<li class="navigation__item"><a href="{{ _nav_url }}">{%- if _item.title -%}{{ _item.title }}{%- else -%}{{ __return }}{%- endif -%}</a></li>
4445
{%- endif -%}
4546
{%- endfor -%}
47+
{%- if page.url == "/" or page.url == "/index.html" -%}
48+
<li class="navigation__item navigation__item--mobile-only">
49+
<button class="button button--secondary mission-button" onclick="document.querySelector('.grid.grid--reverse')?.scrollIntoView({behavior: 'smooth', block: 'start'})">Mission Statement</button>
50+
</li>
51+
{%- else -%}
52+
<li class="navigation__item navigation__item--mobile-only">
53+
<a class="button button--secondary mission-button" href="{{ "/#mission" | relative_url }}">Mission Statement</a>
54+
</li>
55+
{%- endif -%}
4656
<button class="blog" onclick="window.location.href ='{{"Blog.html" | relative_url }}'">Community</button>
4757
<li>
4858
<a href="https://github.com/arm-university/Arm-Developer-Labs" target="_blank" rel="noopener noreferrer" class="button button--secondary button--circle search-button">
@@ -51,6 +61,31 @@
5161
</li>
5262
</ul>
5363
</nav>
64+
<!--<button class="blog" onclick="window.location.href ='{{"Blog.html" | relative_url }}'">Community</button> -->
5465
{%- endif -%}
5566
</div>
67+
<script>
68+
(function() {
69+
var menuToggle = document.querySelector('.mobile-menu-toggle');
70+
var navigation = document.querySelector('.navigation');
71+
var header = document.querySelector('.header');
72+
73+
if (menuToggle && navigation) {
74+
menuToggle.addEventListener('click', function() {
75+
navigation.classList.toggle('navigation--open');
76+
menuToggle.classList.toggle('is-active');
77+
header.classList.toggle('menu-open');
78+
});
79+
80+
// Close menu when clicking on a nav link
81+
navigation.querySelectorAll('a, button').forEach(function(link) {
82+
link.addEventListener('click', function() {
83+
navigation.classList.remove('navigation--open');
84+
menuToggle.classList.remove('is-active');
85+
header.classList.remove('menu-open');
86+
});
87+
});
88+
}
89+
})();
90+
</script>
5691
</header>

0 commit comments

Comments
 (0)