Skip to content

Commit ef90ddf

Browse files
pr-Maispr-Mais
authored
Merge pull request #56 from invertase/hf-ext
2 parents 377821d + 2a71fd9 commit ef90ddf

37 files changed

+30897
-53
lines changed

‎_emulator/extensions/firestore-huggingface-inference-api.env

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
COLLECTION_PATH=test
2+
HF_ACCESS_TOKEN=
3+
HF_INFERENCE_ENDPOINT=
4+
LOCATION=us-central1
5+
MODEL_ID=distilbert-base-uncased-finetuned-sst-2-english
6+
TASK=text-classification

‎_emulator/firebase.json

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,14 @@
2323
"extensions": {
2424
"export-user-data": "../extensions/export-user-data",
2525
"firestore-record-acknowledgments": "../extensions/firestore-record-acknowledgments",
26-
"image-processing-api": "../extensions/image-processing-api"
26+
"image-processing-api": "../extensions/image-processing-api",
27+
"firestore-huggingface-inference-api": "../extensions/firestore-huggingface-inference-api"
28+
},
29+
"firestore": {
30+
"rules": "firestore.rules",
31+
"indexes": "firestore.indexes.json"
32+
},
33+
"storage": {
34+
"rules": "storage.rules"
2735
}
2836
}

‎_emulator/storage.rules

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
rules_version = '2';
2+
service firebase.storage {
3+
match /b/{bucket}/o {
4+
match /{allPaths=**} {
5+
allow read, write: if false;
6+
}
7+
}
8+
}

‎docs/firestore-record-acknowledgments/index.mdx

Lines changed: 27 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -50,18 +50,24 @@ To acknowledge a notice, the extension provides two callable functions which acc
5050
For example, to acknowledge a notice:
5151

5252
```js
53-
import { getFunctions, httpsCallable } from "firebase/functions";
53+
import { getFunctions, httpsCallable } from 'firebase/functions';
5454

5555
const functions = getFunctions();
56-
await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-acknowledgeNotice')({
56+
await httpsCallable(
57+
functions,
58+
'ext-firestore-firestore-record-acknowledgments-acknowledgeNotice',
59+
)({
5760
noticeId: 'EA9QhZKKta9KXcckiasc',
5861
});
5962
```
6063

6164
In-case you need to capture custom preferences relating to an acknowledgment, you can provide custom metadata to the function, for example:
6265

6366
```js
64-
await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-acknowledgeNotice')({
67+
await httpsCallable(
68+
functions,
69+
'ext-firestore-firestore-record-acknowledgments-acknowledgeNotice',
70+
)({
6571
noticeId: 'EA9QhZKKta9KXcckiasc',
6672
metadata: { preference1: true, preference2: false },
6773
});
@@ -70,7 +76,10 @@ await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-a
7076
You can also provide a custom “type” of acknowledgment (the default type is “seen”), for example:
7177

7278
```js
73-
await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-acknowledgeNotice')({
79+
await httpsCallable(
80+
functions,
81+
'ext-firestore-firestore-record-acknowledgments-acknowledgeNotice',
82+
)({
7483
noticeId: 'EA9QhZKKta9KXcckiasc',
7584
type: 'seen',
7685
});
@@ -79,7 +88,10 @@ await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-a
7988
If you wish to unacknowledge a notice, call the `unacknowledgeNotice` function:
8089

8190
```js
82-
await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-unacknowledgeNotice')({
91+
await httpsCallable(
92+
functions,
93+
'ext-firestore-firestore-record-acknowledgments-unacknowledgeNotice',
94+
)({
8395
noticeId: 'EA9QhZKKta9KXcckiasc',
8496
metadata: { reason: '...' },
8597
});
@@ -90,19 +102,25 @@ await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-u
90102
To retrieve all previous user notice acknowledgments, call the `getAcknowledgments` callable function. This function will return an ordered array of all acknowledgments along with the notice data:
91103

92104
```js
93-
import { getFunctions, httpsCallable } from "firebase/functions";
105+
import { getFunctions, httpsCallable } from 'firebase/functions';
94106

95107
const functions = getFunctions();
96-
const acknowledgments = await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-getAcknowledgments')();
108+
const acknowledgments = await httpsCallable(
109+
functions,
110+
'ext-firestore-firestore-record-acknowledgments-getAcknowledgments',
111+
)();
97112
```
98113

99114
By default this won’t include unacknowledgment documents, however if those are required you can provide a `includeUnacknowledgments` property to the call:
100115

101116
```js
102-
import { getFunctions, httpsCallable } from "firebase/functions";
117+
import { getFunctions, httpsCallable } from 'firebase/functions';
103118

104119
const functions = getFunctions();
105-
const acknowledgments = await httpsCallable(functions, 'ext-firestore-firestore-record-acknowledgments-getAcknowledgments')({
120+
const acknowledgments = await httpsCallable(
121+
functions,
122+
'ext-firestore-firestore-record-acknowledgments-getAcknowledgments',
123+
)({
106124
includeUnacknowledgments: true,
107125
});
108126
```

‎extensions/firestore-huggingface-inference-api/CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
## Version 0.0.1
2+
3+
Initial release of the **Trigger Hugging Face inference API from Firestore** extension.

‎extensions/firestore-huggingface-inference-api/POSTINSTALL.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
## See it in action
2+
3+
You can test out this extension right away!
4+
5+
Depending on the task you want to run, add a new Firestore document to `${param:COLLECTION_PATH}`.
6+
7+
For example, if you want to run a text classification task using the model [`distilbert-base-uncased-finetuned-sst-2-english`](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english), add a new document to `${param:COLLECTION_PATH}` with the following fields:
8+
9+
```
10+
{
11+
"inputs": "You're amazing!"
12+
}
13+
```
14+
15+
The response will get written back to the same document.
16+
17+
```
18+
{
19+
"inputs": "You're amazing!",
20+
"response": [
21+
{
22+
"label": "POSITIVE",
23+
"score": 0.9998762602806091
24+
},
25+
{
26+
"label": "NEGATIVE",
27+
"score": 0.00012373875781264198
28+
}
29+
]
30+
}
31+
```
32+
33+
## Error handling
34+
35+
If the extension encounters an error, it will write the error message to the document in `${param:COLLECTION_PATH}`.
36+
37+
```
38+
{
39+
"text": "You're amazing!",
40+
"error": "Field `inputs` must be provided and must be a string or a list of strings"
41+
}
42+
```
43+
44+
# Monitoring
45+
46+
As a best practice, you can [monitor the activity](https://firebase.google.com/docs/extensions/manage-installed-extensions#monitor) of your installed extension, including checks on its health, usage, and logs.

‎extensions/firestore-huggingface-inference-api/PREINSTALL.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
Use this extension to run inferences in a Firestore collection using the [Hugging Face inference API](https://huggingface.co/docs/api-inference).
2+
3+
When triggered by a write operation in a Firestore document, this extension performs a Natural Language Processing task on a specified field and updates the document with the results.
4+
5+
The extension supports both using Hugging Face [Hosted Inference API](https://huggingface.co/docs/api-inference) and [Inference Endpoints](https://huggingface.co/docs/inference-endpoints/index).
6+
7+
## Supported tasks
8+
9+
This extension **currently** supports all Natural Language Processing tasks that are supported by HuggingFace, [click here to see the full list of NLP tasks](https://huggingface.co/docs/api-inference).
10+
11+
# Billing
12+
13+
This extension uses other Firebase or Google Cloud Platform services which may have associated charges:
14+
15+
- Cloud Functions
16+
- Cloud Firestore
17+
18+
When you use Firebase Extensions, you're only charged for the underlying resources that you use. A paid-tier billing plan is only required if the extension uses a service that requires a paid-tier plan, for example calling to a Google Cloud Platform API or making outbound network requests to non-Google services. All Firebase services offer a free tier of usage. [Learn more about Firebase billing.](https://firebase.google.com/pricing)

‎extensions/firestore-huggingface-inference-api/README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Trigger Hugging Face inference API from Firestore
2+
3+
**Author**: undefined
4+
5+
**Description**: This extension triggers the Hugging Face inference API when a new document is created in a Firestore collection.
6+
7+
**Details**: Use this extension to run inferences in a Firestore collection using the [Hugging Face inference API](https://huggingface.co/docs/api-inference).
8+
9+
When triggered by a write operation in a Firestore document, this extension performs a Natural Language Processing task on a specified field and updates the document with the results.
10+
11+
The extension supports both using Hugging Face [Hosted Inference API](https://huggingface.co/docs/api-inference) and [Inference Endpoints](https://huggingface.co/docs/inference-endpoints/index).
12+
13+
## Supported tasks
14+
15+
This extension **currently** supports all Natural Language Processing tasks that are supported by HuggingFace, [click here to see the full list of NLP tasks](https://huggingface.co/docs/api-inference).
16+
17+
# Billing
18+
19+
This extension uses other Firebase or Google Cloud Platform services which may have associated charges:
20+
21+
- Cloud Functions
22+
- Cloud Firestore
23+
24+
When you use Firebase Extensions, you're only charged for the underlying resources that you use. A paid-tier billing plan is only required if the extension uses a service that requires a paid-tier plan, for example calling to a Google Cloud Platform API or making outbound network requests to non-Google services. All Firebase services offer a free tier of usage. [Learn more about Firebase billing.](https://firebase.google.com/pricing)
25+
26+
**Configuration Parameters:**
27+
28+
- Hugging Face Access Token: You can find your API token on your [Hugging Face account page](https://huggingface.co/settings/token).
29+
From Hugging Face docs:
30+
You should see a token hf_xxxxx (old tokens are api_XXXXXXXX or api_org_XXXXXXX).
31+
If you do not submit your API token when sending requests to the API, you will not be able to run inference on your private models.
32+
33+
- Model ID: The Model ID from [Hugging Face Model Hub](https://huggingface.co/models).
34+
Check the [recommended models for each ML task available](https://api-inference.huggingface.co/docs/python/html/detailed_parameters.html#detailed-parameters), or the [Tasks](https://huggingface.co/tasks) overview.
35+
36+
- Inference Collection Path: New inferences using the HuggingFace Inference API can be made by easily adding a new document to this collection path.
37+
38+
- The task to run the inference on: The task to run the inference on. [Check more in the Hugging Face docs](https://huggingface.co/docs/api-inference/detailed_parameters), or the [Tasks](https://huggingface.co/tasks) overview.
39+
40+
- Custom Inference Endpoint: If you want to use a custom model hosted on your own server, you can specify the endpoint here.
41+
42+
- Cloud Functions location: Where do you want to deploy the functions created for this extension? For help selecting a location, refer to the [location selection guide](https://firebase.google.com/docs/functions/locations).
43+
44+
**Cloud Functions:**
45+
46+
- **triggerInference:** Firestore onCreate-triggered function that run an inference on ${param:MODEL_ID} when a new document is created in the collection ${param:COLLECTION_PATH}
47+
48+
**Access Required**:
49+
50+
This extension will operate with the following project IAM roles:
51+
52+
- datastore.user (Reason: This role is required to read/write from the Cloud Firestore database.)

0 commit comments

Comments
 (0)