Skip to content

Commit 71237fa

Browse files
micaelakaplanmicaelakaplan
committed
Deepgram Backend
1 parent fb0fae1 commit 71237fa

File tree

11 files changed

+568
-0
lines changed

11 files changed

+568
-0
lines changed

‎label_studio_ml/examples/deepgram/Dockerfile‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# syntax=docker/dockerfile:1
2+
ARG PYTHON_VERSION=3.13
3+
4+
FROM python:${PYTHON_VERSION}-slim AS python-base
5+
ARG TEST_ENV
6+
7+
WORKDIR /app
8+
9+
ENV PYTHONUNBUFFERED=1 \
10+
PYTHONDONTWRITEBYTECODE=1 \
11+
PORT=${PORT:-9090} \
12+
PIP_CACHE_DIR=/.cache \
13+
WORKERS=1 \
14+
THREADS=8
15+
16+
# Update the base OS
17+
RUN --mount=type=cache,target="/var/cache/apt",sharing=locked \
18+
--mount=type=cache,target="/var/lib/apt/lists",sharing=locked \
19+
set -eux; \
20+
apt-get update; \
21+
apt-get upgrade -y; \
22+
apt install --no-install-recommends -y \
23+
git; \
24+
apt-get autoremove -y
25+
26+
# install base requirements
27+
COPY requirements-base.txt .
28+
RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \
29+
pip install -r requirements-base.txt
30+
31+
# install custom requirements
32+
COPY requirements.txt .
33+
RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \
34+
pip install -r requirements.txt
35+
36+
# install test requirements if needed
37+
COPY requirements-test.txt .
38+
# build only when TEST_ENV="true"
39+
RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \
40+
if [ "$TEST_ENV" = "true" ]; then \
41+
pip install -r requirements-test.txt; \
42+
fi
43+
44+
COPY . .
45+
46+
EXPOSE 9090
47+
CMD gunicorn --preload --bind :$PORT --workers $WORKERS --threads $THREADS --timeout 0 _wsgi:app
48+

‎label_studio_ml/examples/deepgram/README.md‎

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
<!--
2+
---
3+
title: SAM2 with Images
4+
type: guide
5+
tier: all
6+
order: 15
7+
hide_menu: true
8+
hide_frontmatter_title: true
9+
meta_title: Using SAM2 with Label Studio for Image Annotation
10+
categories:
11+
- Computer Vision
12+
- Image Annotation
13+
- Object Detection
14+
- Segment Anything Model
15+
image: "/tutorials/sam2-images.png"
16+
---
17+
-->
18+
19+
# Using SAM2 with Label Studio for Image Annotation
20+
21+
Segment Anything 2, or SAM 2, is a model released by Meta in July 2024. An update to the original Segment Anything Model,
22+
SAM 2 provides even better object segmentation for both images and video. In this guide, we'll show you how to use
23+
SAM 2 for better image labeling with label studio.
24+
25+
Click on the image below to watch our ML Evangelist Micaela Kaplan explain how to link SAM 2 to your Label Studio Project.
26+
You'll need to follow the instructions below to stand up an instance of SAM2 before you can link your model!
27+
28+
[![Connecting SAM2 Model to Label Studio for Image Annotation ](https://img.youtube.com/vi/FTg8P8z4RgY/0.jpg)](https://www.youtube.com/watch?v=FTg8P8z4RgY)
29+
30+
## Before you begin
31+
32+
Before you begin, you must install the [Label Studio ML backend](https://github.com/HumanSignal/label-studio-ml-backend?tab=readme-ov-file#quickstart).
33+
34+
This tutorial uses the [`segment_anything_2_image` example](https://github.com/HumanSignal/label-studio-ml-backend/tree/master/label_studio_ml/examples/segment_anything_2_image).
35+
36+
Note that as of 8/1/2024, SAM2 only runs on GPU.
37+
38+
## Labeling configuration
39+
40+
The current implementation of the Label Studio SAM2 ML backend works using Interactive mode. The user-guided inputs are:
41+
- `KeypointLabels`
42+
- `RectangleLabels`
43+
44+
And then SAM2 outputs `BrushLabels` as a result.
45+
46+
This means all three control tags should be represented in your labeling configuration:
47+
48+
```xml
49+
<View>
50+
<Style>
51+
.main {
52+
font-family: Arial, sans-serif;
53+
background-color: #f5f5f5;
54+
margin: 0;
55+
padding: 20px;
56+
}
57+
.container {
58+
display: flex;
59+
justify-content: space-between;
60+
margin-bottom: 20px;
61+
}
62+
.column {
63+
flex: 1;
64+
padding: 10px;
65+
background-color: #fff;
66+
border-radius: 5px;
67+
box-shadow: 0 2px 5px rgba(0, 0, 0, 0.1);
68+
text-align: center;
69+
}
70+
.column .title {
71+
margin: 0;
72+
color: #333;
73+
}
74+
.column .label {
75+
margin-top: 10px;
76+
padding: 10px;
77+
background-color: #f9f9f9;
78+
border-radius: 3px;
79+
}
80+
.image-container {
81+
width: 100%;
82+
height: 300px;
83+
background-color: #ddd;
84+
border-radius: 5px;
85+
}
86+
</Style>
87+
<View className="main">
88+
<View className="container">
89+
<View className="column">
90+
<View className="title">Choose Label</View>
91+
<View className="label">
92+
<BrushLabels name="tag" toName="image">
93+
94+
95+
<Label value="defect" background="#FFA39E"/></BrushLabels>
96+
</View>
97+
</View>
98+
<View className="column">
99+
<View className="title">Use Keypoint</View>
100+
<View className="label">
101+
<KeyPointLabels name="tag2" toName="image" smart="true">
102+
103+
104+
<Label value="defect" background="#250dd3"/></KeyPointLabels>
105+
</View>
106+
</View>
107+
<View className="column">
108+
<View className="title">Use Rectangle</View>
109+
<View className="label">
110+
<RectangleLabels name="tag3" toName="image" smart="true">
111+
112+
113+
<Label value="defect" background="#FFC069"/></RectangleLabels>
114+
</View>
115+
</View>
116+
</View>
117+
<View className="image-container">
118+
<Image name="image" value="$image" zoom="true" zoomControl="true"/>
119+
</View>
120+
</View>
121+
</View>
122+
```
123+
124+
## Running from source
125+
126+
1. To run the ML backend without Docker, you have to clone the repository and install all dependencies using pip:
127+
128+
```bash
129+
git clone https://github.com/HumanSignal/label-studio-ml-backend.git
130+
cd label-studio-ml-backend
131+
pip install -e .
132+
cd label_studio_ml/examples/segment_anything_2_image
133+
pip install -r requirements.txt
134+
```
135+
136+
2. Download [`segment-anything-2` repo](https://github.com/facebookresearch/sam2) into the root directory. Install SegmentAnything model and download checkpoints using [the official Meta documentation](https://github.com/facebookresearch/sam2?tab=readme-ov-file#installation)
137+
You should now have the following folder structure:
138+
139+
140+
| root directory
141+
| label-studio-ml-backend
142+
| label-studio-ml
143+
| examples
144+
| segment_anything_2_image
145+
| sam2
146+
| sam2
147+
| checkpoints
148+
149+
150+
3. Then you can start the ML backend on the default port `9090`:
151+
152+
```bash
153+
cd ~/sam2
154+
label-studio-ml start ../label-studio-ml-backend/label_studio_ml/examples/segment_anything_2_image
155+
```
156+
157+
Due to breaking changes from Meta [HERE](https://github.com/facebookresearch/sam2/blob/c2ec8e14a185632b0a5d8b161928ceb50197eddc/sam2/build_sam.py#L20), it is CRUCIAL that you run this command from the sam2 directory at your root directory.
158+
159+
4. Connect running ML backend server to Label Studio: go to your project `Settings -> Machine Learning -> Add Model` and specify `http://localhost:9090` as a URL. Read more in the official [Label Studio documentation](https://labelstud.io/guide/ml#Connect-the-model-to-Label-Studio).
160+
161+
## Running with Docker
162+
163+
1. Start Machine Learning backend on `http://localhost:9090` with prebuilt image:
164+
165+
```bash
166+
docker-compose up
167+
```
168+
169+
2. Validate that backend is running
170+
171+
```bash
172+
$ curl http://localhost:9090/
173+
{"status":"UP"}
174+
```
175+
176+
3. Connect to the backend from Label Studio running on the same host: go to your project `Settings -> Machine Learning -> Add Model` and specify `http://localhost:9090` as a URL.
177+
178+
179+
## Configuration
180+
Parameters can be set in `docker-compose.yml` before running the container.
181+
182+
183+
The following common parameters are available:
184+
- `DEVICE` - specify the device for the model server (currently only `cuda` is supported, `cpu` is coming soon)
185+
- `MODEL_CONFIG` - SAM2 model configuration file (`sam2_hiera_l.yaml` by default)
186+
- `MODEL_CHECKPOINT` - SAM2 model checkpoint file (`sam2_hiera_large.pt` by default)
187+
- `BASIC_AUTH_USER` - specify the basic auth user for the model server
188+
- `BASIC_AUTH_PASS` - specify the basic auth password for the model server
189+
- `LOG_LEVEL` - set the log level for the model server
190+
- `WORKERS` - specify the number of workers for the model server
191+
- `THREADS` - specify the number of threads for the model server
192+
193+
## Customization
194+
195+
The ML backend can be customized by adding your own models and logic inside the `./segment_anything_2` directory.

‎label_studio_ml/examples/deepgram/_wsgi.py‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
import os
2+
import argparse
3+
import json
4+
import logging
5+
import logging.config
6+
7+
logging.config.dictConfig({
8+
"version": 1,
9+
"disable_existing_loggers": False,
10+
"formatters": {
11+
"standard": {
12+
"format": "[%(asctime)s] [%(levelname)s] [%(name)s::%(funcName)s::%(lineno)d] %(message)s"
13+
}
14+
},
15+
"handlers": {
16+
"console": {
17+
"class": "logging.StreamHandler",
18+
"level": os.getenv('LOG_LEVEL'),
19+
"stream": "ext://sys.stdout",
20+
"formatter": "standard"
21+
}
22+
},
23+
"root": {
24+
"level": os.getenv('LOG_LEVEL'),
25+
"handlers": [
26+
"console"
27+
],
28+
"propagate": True
29+
}
30+
})
31+
32+
from label_studio_ml.api import init_app
33+
from model import DeepgramModel
34+
35+
36+
_DEFAULT_CONFIG_PATH = os.path.join(os.path.dirname(__file__), 'config.json')
37+
38+
39+
def get_kwargs_from_config(config_path=_DEFAULT_CONFIG_PATH):
40+
if not os.path.exists(config_path):
41+
return dict()
42+
with open(config_path) as f:
43+
config = json.load(f)
44+
assert isinstance(config, dict)
45+
return config
46+
47+
48+
if __name__ == "__main__":
49+
parser = argparse.ArgumentParser(description='Label studio')
50+
parser.add_argument(
51+
'-p', '--port', dest='port', type=int, default=9090,
52+
help='Server port')
53+
parser.add_argument(
54+
'--host', dest='host', type=str, default='0.0.0.0',
55+
help='Server host')
56+
parser.add_argument(
57+
'--kwargs', '--with', dest='kwargs', metavar='KEY=VAL', nargs='+', type=lambda kv: kv.split('='),
58+
help='Additional LabelStudioMLBase model initialization kwargs')
59+
parser.add_argument(
60+
'-d', '--debug', dest='debug', action='store_true',
61+
help='Switch debug mode')
62+
parser.add_argument(
63+
'--log-level', dest='log_level', choices=['DEBUG', 'INFO', 'WARNING', 'ERROR'], default=None,
64+
help='Logging level')
65+
parser.add_argument(
66+
'--model-dir', dest='model_dir', default=os.path.dirname(__file__),
67+
help='Directory where models are stored (relative to the project directory)')
68+
parser.add_argument(
69+
'--check', dest='check', action='store_true',
70+
help='Validate model instance before launching server')
71+
parser.add_argument('--basic-auth-user',
72+
default=os.environ.get('ML_SERVER_BASIC_AUTH_USER', None),
73+
help='Basic auth user')
74+
75+
parser.add_argument('--basic-auth-pass',
76+
default=os.environ.get('ML_SERVER_BASIC_AUTH_PASS', None),
77+
help='Basic auth pass')
78+
79+
args = parser.parse_args()
80+
81+
# setup logging level
82+
if args.log_level:
83+
logging.root.setLevel(args.log_level)
84+
85+
def isfloat(value):
86+
try:
87+
float(value)
88+
return True
89+
except ValueError:
90+
return False
91+
92+
def parse_kwargs():
93+
param = dict()
94+
for k, v in args.kwargs:
95+
if v.isdigit():
96+
param[k] = int(v)
97+
elif v == 'True' or v == 'true':
98+
param[k] = True
99+
elif v == 'False' or v == 'false':
100+
param[k] = False
101+
elif isfloat(v):
102+
param[k] = float(v)
103+
else:
104+
param[k] = v
105+
return param
106+
107+
kwargs = get_kwargs_from_config()
108+
109+
if args.kwargs:
110+
kwargs.update(parse_kwargs())
111+
112+
if args.check:
113+
print('Check "' + DeepgramModel.__name__ + '" instance creation..')
114+
model = DeepgramModel(**kwargs)
115+
116+
app = init_app(model_class=DeepgramModel, basic_auth_user=args.basic_auth_user, basic_auth_pass=args.basic_auth_pass)
117+
118+
app.run(host=args.host, port=args.port, debug=args.debug)
119+
120+
else:
121+
# for uWSGI use
122+
app = init_app(model_class=DeepgramModel)

0 commit comments

Comments
 (0)