You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Trying to set up a redirect for the API reference section.
* Making the API reference page redirect without having to click a link
* Updating edge docs
* Comment on gitignore.
Starting your model evaluations at the edge reduces latency, cost, network bandwidth, and energy. Once you have downloaded and installed your Groundlight edge models, you can configure the Groundlight SDK to use your edge environment by configuring the 'endpoint' which the SDK connects to. You can do this either directly in code as such:
3
+
If your account has access to edge models, you can download and install them to your edge devices.
4
+
This allows you to run your model evaluations on the edge, reducing latency, cost, network bandwidth, and energy.
5
+
6
+
## How the Edge Endpoint works
7
+
8
+
The Edge Endpoint runs as a set of docker containers on an "edge device". This edge device can be an NVIDIA Jetson device, rack-mounted server, or even a Raspberry Pi. The Edge Endpoint is responsible for downloading and running the models,
9
+
and for communicating with the Groundlight cloud service.
10
+
11
+
To use the edge endpoint, simply configure the Groundlight SDK to use the edge endpoint's URL instead of the cloud endpoint.
12
+
All application logic will work seamlessly and unchanged with the Groundlight Edge Endpoint, except some ML answers will
13
+
return much faster locally. The only visible difference is that image queries answered at the edge endpoint will have the prefix `iqe_` instead of `iq_` for image queries answered in the cloud. `iqe_` stands for "image query edge". Edge-originated
14
+
image queries will not appear in the cloud dashboard.
15
+
16
+
## Configuring the Edge Endpoint
17
+
18
+
To configure the Groundlight SDK to use the edge endpoint, you can either pass the endpoint URL to the Groundlight constructor like:
4
19
5
20
```python
6
21
from groundlight import Groundlight
@@ -12,7 +27,4 @@ or by setting the `GROUNDLIGHT_ENDPOINT` environment variable like:
12
27
```bash
13
28
export GROUNDLIGHT_ENDPOINT=http://localhost:6717
14
29
python your_app.py
15
-
```
16
-
17
-
(Edge model download is not yet generally available. Work with your Solutions Engineer to set up edge inference.)
0 commit comments