Skip to content

Commit 621eab8

Browse files
bobpeulenbobpeulen
authored
Merge branch 'main' into phantompete-patch-2
2 parents f558086 + aa492de commit 621eab8

File tree

11 files changed

+1080
-0
lines changed

11 files changed

+1080
-0
lines changed

‎cloud-infrastructure/ai-infra-gpu/GPU/conda-envs/.LICENSE.swo

Whitespace-only changes.

‎cloud-infrastructure/ai-infra-gpu/GPU/conda-envs/.LICENSE.swp

Whitespace-only changes.

‎cloud-infrastructure/ai-infra-gpu/GPU/conda-envs/LICENSE

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
#Copyright (c) 2024 Oracle and/or its affiliates.
2+
#
3+
#The Universal Permissive License (UPL), Version 1.0
4+
#
5+
#Subject to the condition set forth below, permission is hereby granted to any
6+
#person obtaining a copy of this software, associated documentation and/or data
7+
#(collectively the "Software"), free of charge and under any and all copyright
8+
#rights in the Software, and any and all patent rights owned or freely
9+
#licensable by each licensor hereunder covering either (i) the unmodified
10+
#Software as contributed to or provided by such licensor, or (ii) the Larger
11+
#Works (as defined below), to deal in both
12+
#
13+
#(a) the Software, and
14+
#(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
15+
#one is included with the Software (each a "Larger Work" to which the Software
16+
#is contributed by such licensors),
17+
#
18+
#without restriction, including without limitation the rights to copy, create
19+
#derivative works of, display, perform, and distribute the Software and make,
20+
#use, sell, offer for sale, import, export, have made, and have sold the
21+
#Software and the Larger Work(s), and to sublicense the foregoing rights on
22+
#either these or other terms.
23+
#
24+
#This license is subject to the following condition:
25+
#The above copyright notice and either this complete permission notice or at
26+
#a minimum a reference to the UPL must be included in all copies or
27+
#substantial portions of the Software.
28+
#
29+
#THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
30+
#IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
31+
#FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
32+
#AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
33+
#LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
34+
#OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
35+
#SOFTWARE.

‎cloud-infrastructure/ai-infra-gpu/GPU/conda-envs/OL8/pytorch.yml

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
#Copyright (c) 2024 Oracle and/or its affiliates.
2+
#
3+
#The Universal Permissive License (UPL), Version 1.0
4+
#
5+
#Subject to the condition set forth below, permission is hereby granted to any
6+
#person obtaining a copy of this software, associated documentation and/or data
7+
#(collectively the "Software"), free of charge and under any and all copyright
8+
#rights in the Software, and any and all patent rights owned or freely
9+
#licensable by each licensor hereunder covering either (i) the unmodified
10+
#Software as contributed to or provided by such licensor, or (ii) the Larger
11+
#Works (as defined below), to deal in both
12+
#
13+
#(a) the Software, and
14+
#(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
15+
#one is included with the Software (each a "Larger Work" to which the Software
16+
#is contributed by such licensors),
17+
#
18+
#without restriction, including without limitation the rights to copy, create
19+
#derivative works of, display, perform, and distribute the Software and make,
20+
#use, sell, offer for sale, import, export, have made, and have sold the
21+
#Software and the Larger Work(s), and to sublicense the foregoing rights on
22+
#either these or other terms.
23+
#
24+
#This license is subject to the following condition:
25+
#The above copyright notice and either this complete permission notice or at
26+
#a minimum a reference to the UPL must be included in all copies or
27+
#substantial portions of the Software.
28+
#
29+
#THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
30+
#IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
31+
#FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
32+
#AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
33+
#LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
34+
#OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
35+
#SOFTWARE.
36+
name: pytorch
37+
channels:
38+
- defaults
39+
dependencies:
40+
- _libgcc_mutex=0.1=main
41+
- _openmp_mutex=5.1=1_gnu
42+
- blas=1.0=mkl
43+
- ca-certificates=2024.3.11=h06a4308_0
44+
- cffi=1.16.0=py39h5eee18b_0
45+
- filelock=3.13.1=py39h06a4308_0
46+
- fsspec=2023.10.0=py39h06a4308_0
47+
- gmp=6.2.1=h295c915_3
48+
- gmpy2=2.1.2=py39heeb90bb_0
49+
- intel-openmp=2023.1.0=hdb19cb5_46306
50+
- jinja2=3.1.3=py39h06a4308_0
51+
- ld_impl_linux-64=2.38=h1181459_1
52+
- libffi=3.4.4=h6a678d5_0
53+
- libgcc-ng=11.2.0=h1234567_1
54+
- libgomp=11.2.0=h1234567_1
55+
- libprotobuf=3.20.3=he621ea3_0
56+
- libstdcxx-ng=11.2.0=h1234567_1
57+
- markupsafe=2.1.3=py39h5eee18b_0
58+
- mkl=2023.1.0=h213fc3f_46344
59+
- mkl-service=2.4.0=py39h5eee18b_1
60+
- mkl_fft=1.3.8=py39h5eee18b_0
61+
- mkl_random=1.2.4=py39hdb19cb5_0
62+
- mpc=1.1.0=h10f8cd9_1
63+
- mpfr=4.0.2=hb69a4c5_1
64+
- mpmath=1.3.0=py39h06a4308_0
65+
- ncurses=6.4=h6a678d5_0
66+
- networkx=3.1=py39h06a4308_0
67+
- ninja=1.10.2=h06a4308_5
68+
- ninja-base=1.10.2=hd09550d_5
69+
- numpy=1.26.4=py39h5f9d8c6_0
70+
- numpy-base=1.26.4=py39hb5e798b_0
71+
- openssl=3.0.13=h7f8727e_0
72+
- pip=23.3.1=py39h06a4308_0
73+
- pycparser=2.21=pyhd3eb1b0_0
74+
- python=3.9.19=h955ad1f_0
75+
- readline=8.2=h5eee18b_0
76+
- setuptools=68.2.2=py39h06a4308_0
77+
- sqlite=3.41.2=h5eee18b_0
78+
- sympy=1.12=py39h06a4308_0
79+
- tbb=2021.8.0=hdb19cb5_0
80+
- tk=8.6.12=h1ccaba5_0
81+
- typing_extensions=4.9.0=py39h06a4308_1
82+
- wheel=0.41.2=py39h06a4308_0
83+
- xz=5.4.6=h5eee18b_0
84+
- zlib=1.2.13=h5eee18b_0
85+
- pip:
86+
- accelerate==0.28.0
87+
- aiohttp==3.9.3
88+
- aiosignal==1.3.1
89+
- anyio==4.3.0
90+
- argon2-cffi==23.1.0
91+
- argon2-cffi-bindings==21.2.0
92+
- arrow==1.3.0
93+
- asttokens==2.4.1
94+
- async-lru==2.0.4
95+
- async-timeout==4.0.3
96+
- attrs==23.2.0
97+
- babel==2.14.0
98+
- beautifulsoup4==4.12.3
99+
- bleach==6.1.0
100+
- certifi==2024.2.2
101+
- charset-normalizer==3.3.2
102+
- comm==0.2.2
103+
- datasets==2.18.0
104+
- debugpy==1.8.1
105+
- decorator==5.1.1
106+
- defusedxml==0.7.1
107+
- dill==0.3.8
108+
- docstring-parser==0.16
109+
- exceptiongroup==1.2.0
110+
- executing==2.0.1
111+
- fastjsonschema==2.19.1
112+
- fqdn==1.5.1
113+
- frozenlist==1.4.1
114+
- h11==0.14.0
115+
- httpcore==1.0.4
116+
- httpx==0.27.0
117+
- huggingface-hub==0.22.0
118+
- idna==3.6
119+
- importlib-metadata==7.1.0
120+
- ipykernel==6.29.3
121+
- ipython==8.18.1
122+
- isoduration==20.11.0
123+
- jedi==0.19.1
124+
- json5==0.9.24
125+
- jsonpointer==2.4
126+
- jsonschema==4.21.1
127+
- jsonschema-specifications==2023.12.1
128+
- jupyter-client==8.6.1
129+
- jupyter-core==5.7.2
130+
- jupyter-events==0.10.0
131+
- jupyter-lsp==2.2.4
132+
- jupyter-server==2.13.0
133+
- jupyter-server-terminals==0.5.3
134+
- jupyterlab==4.1.5
135+
- jupyterlab-pygments==0.3.0
136+
- jupyterlab-server==2.25.4
137+
- markdown-it-py==3.0.0
138+
- matplotlib-inline==0.1.6
139+
- mdurl==0.1.2
140+
- mistune==3.0.2
141+
- multidict==6.0.5
142+
- multiprocess==0.70.16
143+
- nbclient==0.10.0
144+
- nbconvert==7.16.3
145+
- nbformat==5.10.3
146+
- nest-asyncio==1.6.0
147+
- notebook-shim==0.2.4
148+
- nvidia-cublas-cu12==12.1.3.1
149+
- nvidia-cuda-cupti-cu12==12.1.105
150+
- nvidia-cuda-nvrtc-cu12==12.1.105
151+
- nvidia-cuda-runtime-cu12==12.1.105
152+
- nvidia-cudnn-cu12==8.9.2.26
153+
- nvidia-cufft-cu12==11.0.2.54
154+
- nvidia-curand-cu12==10.3.2.106
155+
- nvidia-cusolver-cu12==11.4.5.107
156+
- nvidia-cusparse-cu12==12.1.0.106
157+
- nvidia-nccl-cu12==2.19.3
158+
- nvidia-nvjitlink-cu12==12.4.99
159+
- nvidia-nvtx-cu12==12.1.105
160+
- overrides==7.7.0
161+
- packaging==24.0
162+
- pandas==2.2.1
163+
- pandocfilters==1.5.1
164+
- parso==0.8.3
165+
- peft==0.10.0
166+
- pexpect==4.9.0
167+
- pillow==10.2.0
168+
- platformdirs==4.2.0
169+
- prometheus-client==0.20.0
170+
- prompt-toolkit==3.0.43
171+
- psutil==5.9.8
172+
- ptyprocess==0.7.0
173+
- pure-eval==0.2.2
174+
- pyarrow==15.0.2
175+
- pyarrow-hotfix==0.6
176+
- pygments==2.17.2
177+
- python-dateutil==2.9.0.post0
178+
- python-json-logger==2.0.7
179+
- pytz==2024.1
180+
- pyyaml==6.0.1
181+
- pyzmq==25.1.2
182+
- referencing==0.34.0
183+
- regex==2023.12.25
184+
- requests==2.31.0
185+
- rfc3339-validator==0.1.4
186+
- rfc3986-validator==0.1.1
187+
- rich==13.7.1
188+
- rpds-py==0.18.0
189+
- safetensors==0.4.2
190+
- send2trash==1.8.2
191+
- shtab==1.7.1
192+
- six==1.16.0
193+
- sniffio==1.3.1
194+
- soupsieve==2.5
195+
- stack-data==0.6.3
196+
- terminado==0.18.1
197+
- test-pytorch-gpu==0.0.4
198+
- tinycss2==1.2.1
199+
- tokenizers==0.15.2
200+
- tomli==2.0.1
201+
- torch==2.2.1
202+
- torchvision==0.17.1
203+
- tornado==6.4
204+
- tqdm==4.66.2
205+
- traitlets==5.14.2
206+
- transformers==4.39.1
207+
- triton==2.2.0
208+
- trl==0.8.1
209+
- types-python-dateutil==2.9.0.20240316
210+
- tyro==0.7.3
211+
- tzdata==2024.1
212+
- uri-template==1.3.0
213+
- urllib3==2.2.1
214+
- wcwidth==0.2.13
215+
- webcolors==1.13
216+
- webencodings==0.5.1
217+
- websocket-client==1.7.0
218+
- xxhash==3.4.1
219+
- yarl==1.9.4
220+
- zipp==3.18.1
221+
prefix: /home/opc/anaconda3/envs/pytorch

‎cloud-infrastructure/ai-infra-gpu/GPU/conda-envs/OL8/utilities/locustfile.py

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
#Copyright (c) 2024 Oracle and/or its affiliates.
2+
#
3+
#The Universal Permissive License (UPL), Version 1.0
4+
#
5+
#Subject to the condition set forth below, permission is hereby granted to any
6+
#person obtaining a copy of this software, associated documentation and/or data
7+
#(collectively the "Software"), free of charge and under any and all copyright
8+
#rights in the Software, and any and all patent rights owned or freely
9+
#licensable by each licensor hereunder covering either (i) the unmodified
10+
#Software as contributed to or provided by such licensor, or (ii) the Larger
11+
#Works (as defined below), to deal in both
12+
#
13+
#(a) the Software, and
14+
#(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
15+
#one is included with the Software (each a "Larger Work" to which the Software
16+
#is contributed by such licensors),
17+
#
18+
#without restriction, including without limitation the rights to copy, create
19+
#derivative works of, display, perform, and distribute the Software and make,
20+
#use, sell, offer for sale, import, export, have made, and have sold the
21+
#Software and the Larger Work(s), and to sublicense the foregoing rights on
22+
#either these or other terms.
23+
#
24+
#This license is subject to the following condition:
25+
#The above copyright notice and either this complete permission notice or at
26+
#a minimum a reference to the UPL must be included in all copies or
27+
#substantial portions of the Software.
28+
#
29+
#THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
30+
#IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
31+
#FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
32+
#AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
33+
#LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
34+
#OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
35+
#SOFTWARE.
36+
# Import the Locust dependencies
37+
from locust import HttpUser, task
38+
39+
# Import general library from python
40+
import os
41+
import random
42+
43+
# Import lorem ipsum library to generate some random texts
44+
from lorem_text import lorem
45+
46+
# Import dotenv to load the environments variables
47+
from dotenv import load_dotenv
48+
load_dotenv()
49+
50+
# Create a table with some lorem ipsum texts
51+
messages = []
52+
53+
# Add 1000 random texts of 10 paragraphs each (simulate 1000 emails)
54+
for i in range(1000):
55+
messages.append(lorem.paragraphs(10))
56+
57+
# Define the headers of the request. Token is stored as an environment variable here
58+
headers = {
59+
"Authorization": "Bearer {os.getenv('TOKEN')}", "Content-Type": "application/json"}
60+
61+
class HelloWorldUser(HttpUser):
62+
# Definition of the first path where we do our post request
63+
@task
64+
def hello_world(self):
65+
# Define the body with the email choose randomly from the tab of all the emails
66+
body = {"model": "meta-llama/Llama-2-7b-chat-hf","prompt": "the following is a conversation with an AI research assistant. The assistant tone is technical and scientific. Human: Hello, who are you? AI: Greeting! I am an AI research assistant. How can I help you today? Human: Can you tell me about the creation of blackholes? AI:","max_tokens": 20,"temperature": 0}
67+
# Do the post request on the spam detection path
68+
self.client.post("/v1/completions",
69+
headers=headers, json=body)

‎cloud-infrastructure/ai-infra-gpu/GPU/conda-envs/OL8/utilities/test-vllm.py

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#Copyright (c) 2024 Oracle and/or its affiliates.
2+
#
3+
#The Universal Permissive License (UPL), Version 1.0
4+
#
5+
#Subject to the condition set forth below, permission is hereby granted to any
6+
#person obtaining a copy of this software, associated documentation and/or data
7+
#(collectively the "Software"), free of charge and under any and all copyright
8+
#rights in the Software, and any and all patent rights owned or freely
9+
#licensable by each licensor hereunder covering either (i) the unmodified
10+
#Software as contributed to or provided by such licensor, or (ii) the Larger
11+
#Works (as defined below), to deal in both
12+
#
13+
#(a) the Software, and
14+
#(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
15+
#one is included with the Software (each a "Larger Work" to which the Software
16+
#is contributed by such licensors),
17+
#
18+
#without restriction, including without limitation the rights to copy, create
19+
#derivative works of, display, perform, and distribute the Software and make,
20+
#use, sell, offer for sale, import, export, have made, and have sold the
21+
#Software and the Larger Work(s), and to sublicense the foregoing rights on
22+
#either these or other terms.
23+
#
24+
#This license is subject to the following condition:
25+
#The above copyright notice and either this complete permission notice or at
26+
#a minimum a reference to the UPL must be included in all copies or
27+
#substantial portions of the Software.
28+
#
29+
#THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
30+
#IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
31+
#FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
32+
#AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
33+
#LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
34+
#OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
35+
#SOFTWARE.
36+
from vllm import LLM
37+
38+
# Replace the model name with the one you want to use:
39+
# Mixtral simple: mistralai/Mixtral-8x7B-v0.1
40+
# Mixtral instruct: mistralai/Mixtral-8x7B-Instruct-v0.1
41+
# Mistral 7B simple: mistralai/Mistral-7B-v0.1
42+
# Mistral 7B instruct: mistralai/Mistral-7B-Instruct-v0.1
43+
# LLaMA 2 70B: meta-llama/Llama-2-70b-hf
44+
45+
46+
47+
llm = LLM("meta-llama/Llama-2-7b-chat-hf", tensor_parallel_size=1, max_model_len=4096)
48+
49+
print(llm.generate("What is batch inference?"))

0 commit comments

Comments
 (0)