Skip to content

Commit eff0152

Browse files
committed
Update README.md
1 parent 431c9b9 commit eff0152

File tree

1 file changed

+1
-188
lines changed

1 file changed

+1
-188
lines changed

README.md

Lines changed: 1 addition & 188 deletions
Original file line numberDiff line numberDiff line change
@@ -8,196 +8,9 @@
88
Using Oracle JET, create a user-friendly prompt-led user interface (UI) to interact with Oracle's new Generative AI service. This toolkit will configure your Generative AI Service connection so you can begin your journey with AI, or migrate your existing (local or Cloud) LLMs to the Oracle AppDev ecosystem.
99
[Enhance Engagement Using Content Generation with OCI Generative AI](JET.md)
1010

11+
This project deploys an AI pipeline with a multipurpose front end for text generation and summarization. The pipeline integrates with a database to track interactions, enabling fine-tuning and performance monitoring for application optimization. It leverages OCI Generative AI APIs on a Kubernetes cluster.
1112
[Accelerating AI Application Deployment Using Cloud Native Strategies](K8S.md)
1213

13-
Check out [demo here](https://youtu.be/hpRoQ93YeaQ)
14-
15-
![alt text here](images/demo.gif)
16-
17-
## Getting Started
18-
19-
### 0. Prerequisites and setup
20-
21-
- Oracle Cloud Infrastructure (OCI) Account
22-
- Oracle Cloud Infrastructure (OCI) Generative AI Service - [Getting Started with Generative AI](https://docs.oracle.com/en-us/iaas/Content/generative-ai/getting-started.htm)
23-
- Oracle Cloud Infrastructure Documentation - [Generative AI](https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm)
24-
- Oracle Cloud Infrastructure (OCI) Generative AI Service SDK - [Oracle Cloud Infrastructure Python SDK](https://pypi.org/project/oci/)
25-
- Node v16 - [Node homepage](https://nodejs.org/en)
26-
- Oracle JET v15 - [Oracle JET Homepage](https://www.oracle.com/webfolder/technetwork/jet/index.html)
27-
28-
Follow the links below to generate a config file and a key pair in your ~/.oci directory
29-
30-
- [SDK config](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm)
31-
- [API signing key](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm)
32-
- [CLI install](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm#configfile)
33-
34-
After completion, you should have the following 2 things in your `~/.oci directory`
35-
36-
- A config file(where key file point to private key:key_file=`~/.oci/oci_api_key.pem`)
37-
- A key pair named `oci_api_key.pem` and `oci_api_key_public.pem`
38-
- Now make sure you change the reference of the key file in the config file
39-
- Append OCI Generative-AI service compartment and endpoint URL
40-
41-
```console
42-
vim service/python/server.py
43-
```
44-
45-
```Python
46-
#TODO: Update this section with your tenancy details
47-
compartment_id = "ocid1.compartment.oc1.."
48-
CONFIG_PROFILE = "DEFAULT"
49-
config = oci.config.from_file("~/.oci/config", CONFIG_PROFILE)
50-
endpoint = "https://inference.generativeai.<REGION>.oci.oraclecloud.com"
51-
generative_ai_inference_client = (
52-
oci.generative_ai_inference.GenerativeAiInferenceClient(
53-
config=config,
54-
service_endpoint=endpoint,
55-
retry_strategy=oci.retry.NoneRetryStrategy(),
56-
timeout=(10, 240),
57-
)
58-
)
59-
```
60-
61-
### 1. (Optional) Modify websocket ports
62-
63-
- In the root of the project directory run to edit ports
64-
65-
```console
66-
vim app/src/components/content/index.tsx
67-
```
68-
69-
```js
70-
const gateway = ws://${window.location.hostname}:1234;
71-
```
72-
73-
- Update default port in Python websocket server:
74-
75-
```console
76-
vim service/python/server.py
77-
```
78-
79-
```Python
80-
async def start_server():
81-
await websockets.serve(handle_websocket, "localhost", 1234 )
82-
```
83-
84-
### 2. Upload Public Key
85-
86-
- Upload your oci_api_key_public.pem to console:
87-
[API signing key](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm#three)
88-
89-
### 3. Install all dependencies
90-
91-
We suggest you install dependencies in a virtual environment to avoid conflicts on your system.
92-
93-
- Navigate to the server root folder
94-
95-
```console
96-
cd service/python
97-
```
98-
99-
- Create a virtual environment:
100-
101-
```console
102-
python3 -m venv venv
103-
```
104-
105-
- Activate your virtual environment:
106-
107-
```console
108-
. venv/bin/activate
109-
```
110-
111-
- Upgrade pip:
112-
113-
```console
114-
pip3 install --upgrade pip
115-
```
116-
117-
- Install requirements:
118-
119-
```console
120-
pip3 install -r requirements.txt
121-
```
122-
123-
## 4. Start the websocket server app
124-
125-
Once dependencies are installed and your service credentials are updated you can run server.py
126-
127-
```console
128-
python3 server.py
129-
```
130-
131-
## 5. Start JET Client
132-
133-
- Open app directory:
134-
135-
```console
136-
cd ../../app
137-
```
138-
139-
- Install dependencies:
140-
141-
```console
142-
npm install
143-
```
144-
145-
- Run local version:
146-
147-
```console
148-
npx ojet serve
149-
```
150-
151-
- Or package for web deployment
152-
153-
```console
154-
npx ojet build web
155-
```
156-
157-
You can now ask question to generate LLM based response.
158-
![alt text here](images/QandA.png)
159-
160-
Note that sample app can generate markdown.
161-
![alt text here](images/Markdown.png)
162-
163-
## Appendix: Token-based Authentication
164-
165-
Check [Token-based Authentication for the CLI](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/clitoken.htm#Running_Scripts_on_a_Computer_without_a_Browser)
166-
167-
```Python
168-
config = oci.config.from_file('~/.oci/config', profile_name="DEFAULT")
169-
170-
def make_security_token_signer(oci_config):
171-
pk = oci.signer.load_private_key_from_file(oci_config.get("key_file"), None)
172-
with open(oci_config.get("security_token_file")) as f:
173-
st_string = f.read()
174-
return oci.auth.signers.SecurityTokenSigner(st_string, pk)
175-
176-
signer = make_security_token_signer(oci_config=config)
177-
# Service endpoint
178-
endpoint = "https://generativeai.aiservice.<Region>.oci.oraclecloud.com"
179-
180-
generative_ai_client = oci.generative_ai.generative_ai_client.GenerativeAiClient(config=config, service_endpoint=endpoint, retry_strategy=oci.retry.NoneRetryStrategy(), signer=signer)
181-
```
182-
183-
## Notes/Issues
184-
185-
Additional Use Cases like summarization and embedding coming soon.
186-
187-
To change output parameters edit server.py
188-
189-
```Python
190-
cohere_generate_text_request.max_tokens = 500 # choose the number of tokens 1-4000
191-
cohere_generate_text_request.temperature = 0.75 # adjust temperature 0-1
192-
cohere_generate_text_request.top_p = 0.7 # adjust top_p 0-1
193-
cohere_generate_text_request.frequency_penalty = 1.0 # adjust frequency_penalty
194-
```
195-
196-
## URLs
197-
198-
- [Oracle AI](https://www.oracle.com/artificial-intelligence/)
199-
- [AI for Developers](https://developer.oracle.com/technologies/ai.html)
200-
20114
## Contributing
20215

20316
This project is open source. Please submit your contributions by forking this repository and submitting a pull request! Oracle appreciates any contributions that are made by the open-source community.

0 commit comments

Comments
 (0)