Skip to content

Commit ccfac04

Browse files
matz-ecristinagrs
matz-e
and
cristinagrs
committed
Add an asset demonstrating the combination of AI Vision and a segmentation model.
Use Vision to detect objects, highlight them with a segmentation model. Co-Authored-By: Cristina Granés <[email protected]>
1 parent 5aa48c0 commit ccfac04

File tree

13 files changed

+484
-0
lines changed

13 files changed

+484
-0
lines changed

‎ai/ai-vision/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ Reviewed: 11.06.2025
2828

2929
## GitHub
3030

31+
- [OCI object detection and image segmentation using SAM2](.ai-vision-and-sam/)
32+
3133
- [OCI image classification using data labeling and vision service](https://github.com/carlgira/oci-image-classification)
3234
- [OCI object detection using data labeling and vision service](https://github.com/carlgira/oci-object-detection)
3335
- [AI vision web client](https://github.com/oracle-devrel/oci-tf-vision-web-client)

‎ai/ai-vision/ai-vision-and-sam/LICENSE‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
Copyright (c) 2025 Oracle and/or its affiliates.
2+
3+
The Universal Permissive License (UPL), Version 1.0
4+
5+
Subject to the condition set forth below, permission is hereby granted to any
6+
person obtaining a copy of this software, associated documentation and/or data
7+
(collectively the "Software"), free of charge and under any and all copyright
8+
rights in the Software, and any and all patent rights owned or freely
9+
licensable by each licensor hereunder covering either (i) the unmodified
10+
Software as contributed to or provided by such licensor, or (ii) the Larger
11+
Works (as defined below), to deal in both
12+
13+
(a) the Software, and
14+
(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
15+
one is included with the Software (each a "Larger Work" to which the Software
16+
is contributed by such licensors),
17+
18+
without restriction, including without limitation the rights to copy, create
19+
derivative works of, display, perform, and distribute the Software and make,
20+
use, sell, offer for sale, import, export, have made, and have sold the
21+
Software and the Larger Work(s), and to sublicense the foregoing rights on
22+
either these or other terms.
23+
24+
This license is subject to the following condition:
25+
The above copyright notice and either this complete permission notice or at
26+
a minimum a reference to the UPL must be included in all copies or
27+
substantial portions of the Software.
28+
29+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
30+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
31+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
32+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
33+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
34+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
35+
SOFTWARE.

‎ai/ai-vision/ai-vision-and-sam/README.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# OCI Vision and SAM2
2+
3+
The following asset is a tool designed to showcase how OCI AI Vision can be connected with SAM2 from Meta for accurate segmentations without needing high amount of data for training.
4+
5+
6+
Reviewed: 18.08.2025
7+
Authors: Matthias Wolf and Cristina Granés
8+
9+
# When to use this asset?
10+
11+
When you need to compute areas from images, detect and segment objects with AI.
12+
13+
# How to use this asset?
14+
15+
See the full setup and usage guide in files/README.md.
16+
17+
18+
# License
19+
20+
Copyright (c) 2025 Oracle and/or its affiliates.
21+
22+
Licensed under the Universal Permissive License (UPL), Version 1.0.
23+
24+
See [LICENSE](LICENSE) for more details.

‎ai/ai-vision/ai-vision-and-sam/files/.config‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
CONFIG_FILE_PATH = "~/.oci/config"
2+
# This needs to be a valid compartment ID within your tenancy
3+
COMPARTMENT_ID = "ocid1.compartment.oc1..aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
4+
# Change the endpoint to match your account's region
5+
ENDPOINT = "https://inference.generativeai.eu-frankfurt-1.oci.oraclecloud.com"

‎ai/ai-vision/ai-vision-and-sam/files/.streamlit/config.toml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
[theme]
2+
primaryColor = "#e83610" # Red
3+
backgroundColor = "#1E1E1E" # Dark gray background
4+
secondaryBackgroundColor = "#2C2C2C" # Slightly lighter sidebar
5+
textColor = "#FFFFFF" # Clean white text
6+
font = "sans serif"
7+
8+
[client]
9+
showSidebarNavigation = false
10+
toolbarMode = "minimal"
11+
12+
[server]
13+
headless = true

‎ai/ai-vision/ai-vision-and-sam/files/README.md‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Image segmentation with OCI AI Vision and SAM2 from Meta
2+
In this demo, you will see how you can segment objects in images by combining **OCI AI Vision** with **Meta’s SAM2 (Segment Anything Model 2)**.
3+
4+
**Segmenting objects in images has never been this easy!**
5+
You can combine both OCI AI Vision to handle image analysis and SAM2 to provide high-precision segmentation, you can quickly identify and isolate objects in any image using just a few clicks or simple prompts. This integration demonstrates how cloud-based vision services and cutting-edge AI models can work together to streamline complex computer vision tasks.
6+
7+
For more accurate results, you can build a custom object detection model in OCI AI Vision (see [the documentation](https://docs.oracle.com/en-us/iaas/Content/vision/using/custom_image_analysis_models_using.htm) for more details).
8+
9+
## Some possible business cases
10+
- Compute areas of roofs to detect the area available for solar panels.
11+
- Segment construction sites to estimate areas from aerial images.
12+
- Environmental monitoring: segment different land types (forest, water, urban areas,...) from satellite imagery and estimate changes in those areas.
13+
- Urban planning: segment and detect roads, bridges, and utilities for urban development projects.
14+
- Crop and weed segmentation to optimize pesticide usage and crop management.
15+
16+
## Requirements
17+
You can install the following demo in a personal PC.
18+
You would need some space and at least a CPU with 300MB of memory, for `sam2.1-hiera-small`.
19+
20+
## Setup
21+
1. Install Python (this project requires Python 3.13.5 or later). You can check your current Python version by running:
22+
</br>
23+
```
24+
python --version
25+
```
26+
or
27+
```
28+
python3 --version
29+
```
30+
2. Install the requirements from `requirements.txt` file.
31+
</br>
32+
```
33+
pip install -r /path/to/requirements.txt
34+
```
35+
3. Create a `.config` file that contains the following variables:
36+
```
37+
CONFIG_FILE_PATH = <path_to_oci_login_config_file>
38+
COMPARTMENT_ID = <compartment_OCID>
39+
# Change the endpoint to match your account's region
40+
ENDPOINT = "https://inference.generativeai.eu-frankfurt-1.oci.oraclecloud.com"
41+
```
42+
4. Run the application using `streamlit run app.py`.
43+
44+
## Technical Details
45+
* The solution leverages Oracle Cloud Infrastructure (OCI) AI Vision Service, an AI service designed to simplify AI adoption.
46+
* Specifically, this demo utilizes:
47+
+ OCI Vision object detection
48+
+ SAM2 segmentation
49+
50+
### About SAM2
51+
SAM2 is an open-source model developed by Meta AI as the next-generation version of the original Segment Anything Model (SAM). It enables fast and flexible image segmentation, allowing users to easily extract precise object masks from images with minimal input. Built to support both interactive and automated segmentation tasks, SAM2 improves on efficiency, accuracy, and generalization across diverse image types and domains.
52+
53+
Key features include:
54+
- State-of-the-art segmentation performance
55+
- Support for promptable segmentation (points, boxes, masks)
56+
- Open-source and ready for integration into custom workflows
57+
58+
You can find more information [here](https://docs.ultralytics.com/models/sam-2/).
59+
And you can find information about the License [here](https://github.com/facebookresearch/sam2/blob/main/LICENSE).
60+
61+
## Project Structure
62+
The repository is organized as follows:
63+
64+
```plaintext
65+
│ .config # File to be added as explained in `Setup`, with your own OCI variables
66+
│ app.py # Main Streamlit application entry point
67+
│ config.py # Variables for the Streamlit application
68+
│ navigation.py # Configuration for the sidebar in the Streamlit application
69+
│ README.md # Project documentation
70+
│ requirements.txt # Python dependencies
71+
│
72+
├───utils
73+
│ │ ai_tools.py # Wrappers for inference on the AI models
74+
│ │ image_utils.py # Wrappers for image functionalities
75+
│
76+
├───app_images
77+
│ │ oracle_logo.png # Logo of Oracle for Streamlit application
78+
│
79+
├───.streamlit # Parameters for UI appearance of the Streamlit application
80+
│
81+
└───uploaded_images # Folder to be used by the Streamlit application
82+
```
83+
84+
## Output
85+
The demo will display an interactive dashboard to upload an image and process it, displaying the resulting detections from OCI Vision and segmentations of those detections.
86+
87+
## Authors
88+
- Matthias Wolf
89+
- Cristina Granés
90+
91+
## Contributing
92+
We welcome contributions to improve and expand the capabilities of this demo. Please fork the repository and submit a pull request with your changes.
93+
94+
## License
95+
Copyright (c) 2025 Oracle and/or its affiliates.
96+
97+
Licensed under the Universal Permissive License (UPL), Version 1.0.
98+
99+
See [LICENSE](../LICENSE) for more details.

‎ai/ai-vision/ai-vision-and-sam/files/app.py‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
import streamlit as st
2+
import config
3+
from PIL import Image
4+
5+
import torch
6+
from sam2.sam2_image_predictor import SAM2ImagePredictor
7+
8+
from utils.image_utils import load_image, save_uploaded_image, draw_detections, draw_masks
9+
from utils.ai_tools import InferencePipeline
10+
from navigation import make_sidebar
11+
12+
@st.cache_resource
13+
def load_sam():
14+
# This will require internet access when deploying the model
15+
# Alternative, larger model: "facebook/sam2.1-hiera-large"
16+
predictor = SAM2ImagePredictor.from_pretrained("facebook/sam2.1-hiera-small", device=torch.device("cpu"))
17+
return predictor
18+
19+
20+
def main():
21+
22+
## Page config
23+
icon = Image.open(config.ORACLE_LOGO)
24+
25+
st.set_page_config(
26+
page_title="Detect and segment",
27+
page_icon=icon,
28+
layout="wide",
29+
)
30+
31+
## Header ##
32+
st.markdown("<h1 style='margin-bottom: 0;'>Segment Images with OCI AI Vision + SAM2 from Meta</h1>", unsafe_allow_html=True)
33+
st.divider()
34+
35+
36+
### Init application ###
37+
38+
sam_model = load_sam()
39+
ai_pipeline = InferencePipeline(config=config, sam_model=sam_model)
40+
41+
# Sidebar for upload and run
42+
uploaded_file, run_button = make_sidebar()
43+
44+
45+
######### finish init
46+
47+
## Upload image & process ##
48+
coli1, coli2 = st.columns([0.5, 0.5])
49+
if uploaded_file:
50+
image = load_image(uploaded_file)
51+
with coli1: st.image(image, caption="Original Image")
52+
53+
if run_button:
54+
with st.spinner("Processing...", show_time=True):
55+
uploaded_file.seek(0)
56+
# Save image:
57+
image_path = save_uploaded_image(uploaded_file, save_dir=config.UPLOAD_PATH)
58+
59+
#detections = ai_pipeline.get_detection(uploaded_file)
60+
detections, masks = ai_pipeline.get_detections_and_masks(image_path)
61+
62+
image_with_boxes = draw_detections(image, detections)
63+
with coli2: st.image(image_with_boxes, caption="OCI Vision Detections")
64+
65+
st.header("Final segmentations")
66+
67+
## Plot masks:
68+
mask_names = [msk["class"] for msk in masks]
69+
tabs_list = st.tabs(mask_names)
70+
for id_msk, msk in enumerate(masks):
71+
ctab = tabs_list[id_msk]
72+
det = detections[id_msk]
73+
image_with_mask = draw_masks(image, msk["mask"])
74+
with ctab:
75+
st.image(image_with_mask,
76+
width=700,
77+
caption=f"{msk["class"]} - Detection confidence score: {det["confidence_score"]:.2f}")
78+
79+
80+
if __name__ == '__main__':
81+
main()

‎ai/ai-vision/ai-vision-and-sam/files/app_images/oracle_logo.png‎

2.97 KB
Loading

‎ai/ai-vision/ai-vision-and-sam/files/config.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
import os
2+
from dotenv import load_dotenv
3+
load_dotenv(dotenv_path=".config")
4+
5+
# config, compartments and endpoints
6+
CONFIG_FILE_PATH = os.getenv("CONFIG_FILE_PATH")
7+
COMPARTMENT_ID = os.getenv("COMPARTMENT_ID")
8+
ENDPOINT= os.getenv("ENDPOINT")
9+
10+
## Other config params
11+
ORACLE_LOGO = "app_images/oracle_logo.png"
12+
UPLOAD_PATH = "uploaded_images"

‎ai/ai-vision/ai-vision-and-sam/files/navigation.py‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
import base64
2+
import streamlit as st
3+
import config
4+
5+
def make_sidebar():
6+
with open(config.ORACLE_LOGO, "rb") as f:
7+
icon_base64 = base64.b64encode(f.read()).decode()
8+
with st.sidebar:
9+
10+
st.markdown(
11+
f"""
12+
<div style="display: flex; align-items: left; gap: 10px;">
13+
<img src="data:image/png;base64,{icon_base64}"
14+
width="90" style="border-radius: 8px;">
15+
<span style="font-size: 30px; font-weight: 600;">Upload & Run</span>
16+
</div>
17+
""",
18+
unsafe_allow_html=True
19+
)
20+
st.markdown("""
21+
<style>
22+
.stButton>button {
23+
background-color: #F80000;
24+
color: white;
25+
padding: 0.5em 1em;
26+
font-size: 16px;
27+
border-radius: 5px;
28+
}
29+
</style>
30+
""", unsafe_allow_html=True)
31+
32+
st.write("")
33+
st.write("")
34+
35+
uploaded_file = st.sidebar.file_uploader("Upload an Image", type=['png', 'jpg'])
36+
run_button = st.sidebar.button("Run")
37+
38+
st.write("")
39+
st.write("")
40+
41+
return uploaded_file, run_button
42+

0 commit comments

Comments
 (0)