Merge branch 'main' into luisc2024v1

Author: AlexanderHodicke

.gitignore

@@ -3,6 +3,7 @@
 .AppleDouble
 .LSOverride
 .vscode
+.bak
 
 # Icon must end with two \r
 Icon

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2023 Oracle and/or its affiliates.
+Copyright (c) 2024 Oracle and/or its affiliates.
 
 The Universal Permissive License (UPL), Version 1.0
 

ai-and-app-modernisation/ai-services/generative-ai-service/README.md

@@ -11,6 +11,9 @@ Reviewed: 30.01.2024
 
 # Team Publications
 
+- [Enable a Low Code Modular LLM App Engine using Oracle Integration and OCI Generative AI](https://docs.oracle.com/en/solutions/oci-generative-ai-integration/index.html)
+    - This reference architecture lets you understand the necessary considerations and recommendations to enable an AI-based, modular and event-driven  LLM App Engine using a low-code approach with Oracle Integration as the LLM orchestrator, OCI Generative AI and other OCI services
+    - Build enterprise-grade, modular, scalable, secure & maintainable LLM Apps
 - [Oracle Generative AI webinar](https://go.oracle.com/LP=138234?elqCampaignId=489428&src1=:so:ch:or:dg::::&SC=:so:ch:or:dg::::&pcode=WWMK230822P00010)
     - Deep dive into Oracle Generative AI platform
 - [Creating a RAG (Retrieval-Augmented Generation) with Oracle Generative AI Service in just 21 lines of code](https://github.com/oracle-devrel/technology-engineering/tree/main/ai-and-app-modernisation/ai-services/generative-ai-service/rag-genai)

ai-and-app-modernisation/ai-services/generative-ai-service/rag-genai/files/LangChainRAG.py

@@ -0,0 +1,21 @@
+from langchain_community.embeddings import OCIGenAIEmbeddings
+from langchain.chains import RetrievalQA
+from langchain_community.vectorstores import Qdrant
+from langchain_core.prompts import PromptTemplate
+from langchain_community.llms import OCIGenAI
+from langchain_community.document_loaders import UnstructuredURLLoader
+compartment_id = "ocid1.compartment.oc1..aaaaaaaa7ggqkd4ptkeb7ugk6ipsl3gqjofhkr6yacluwj4fitf2ufrdm65q"
+embeddings = OCIGenAIEmbeddings(model_id="cohere.embed-english-light-v3.0",service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",compartment_id= compartment_id,)
+testurls = ['https://docs.oracle.com/iaas/odsaz/odsa-rotate-wallet.html', 'https://docs.oracle.com/iaas/odsaz/odsa-change-password.html', 'https://docs.oracle.com/iaas/odsaz/odsa-database-actions.html']
+loader = UnstructuredURLLoader(urls=testurls)
+docs = loader.load()
+vectorstore = Qdrant.from_documents(docs,embeddings,location=":memory:",prefer_grpc=False,collection_name="test_db")
+retriever = vectorstore.as_retriever()
+rag_prompt_template = """Answer the question based only on the following context:
+{context}
+Question: {question}"""
+rag_prompt = PromptTemplate.from_template(rag_prompt_template)
+llm = OCIGenAI(model_id="cohere.command",service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",compartment_id= compartment_id,model_kwargs={"temperature": 0, "max_tokens": 300})
+rag = RetrievalQA.from_chain_type(llm=llm,retriever=retriever,chain_type_kwargs={"prompt": rag_prompt,},)
+data = rag.invoke("What is rotate a wallet")
+print(data['result'])

ai-and-app-modernisation/ai-services/generative-ai-service/rag-genai/files/README.md

@@ -3,30 +3,27 @@
 ## Introduction
 In this article, we'll explore how to create a Retrieval-Augmented Generation (RAG) model using Oracle Gen AI, llama index, Qdrant Vector Database, and SentenceTransformerEmbeddings. This 21-line code will allow you to scrape through web pages, use llama index for indexing, Oracle Generative AI Service for question generation, and Qdrant for vector indexing.
 
+Find below the code of building a RAG using llamaIndex with Oracle Generative AI Service.
+Also check the file LangChainRAG.py which allows you to create an application (implementing RAG) using Langchain and the file langChainRagWithUI.py which includes a UI build with Streamlit.
+
 <img src="./RagArchitecture.svg">
 </img>
 
-## Limited Availability
-
-Oracle Generative AI Service is in Limited Availability as of today when we are creating this repo.
-
-Customers can easily enter in the LA programs. To test these functionalities you need to enrol in the LA programs and install the proper versions of software libraries.
-
-Code and functionalities can change, as a result of changes and new features
-
 ## Prerequisites
 
 Before getting started, make sure you have the following installed:
 
 - Oracle Generative AI Service
-- llama index
-- qdrant client
+- Llama index
+- Langchain
+- Qdrant client
 - SentenceTransformerEmbeddings
 
 ## Setting up the Environment
 1. Install the required packages:
    ```bash
-   pip install oci==2.112.1+preview.1.1649 llama-index qdrant-client sentence-transformers
+   pip install -U langchain oci
+   pip install langchain llama-index qdrant-client sentence-transformers transformers
    ```
 
 ## Loading data
@@ -41,26 +38,26 @@ sitemap used : https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frpj5kvxry
 ## Entire code
 
    ```bash
-   from genai_langchain_integration.langchain_oci import OCIGenAI
 from llama_index import VectorStoreIndex
 from llama_index import ServiceContext
 from llama_index.vector_stores.qdrant import QdrantVectorStore
 from llama_index.storage.storage_context import StorageContext
 from qdrant_client import qdrant_client
-from langchain.embeddings import SentenceTransformerEmbeddings
+from langchain_community.embeddings import SentenceTransformerEmbeddings
 from llama_hub.web.sitemap import SitemapReader
+from langchain_community.llms import OCIGenAI
 loader = SitemapReader()
-documents = loader.load_data(sitemap_url='https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frpj5kvxryk1/b/thisIsThePlace/o/combined.xml')
+documents = loader.load_data(sitemap_url='https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frpj5kvxryk1/b/thisIsThePlace/o/latest.xml')
 client = qdrant_client.QdrantClient(location=":memory:")
 embeddings = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")
-llm = OCIGenAI(model_id="cohere.command",service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com",compartment_id = "ocid1.tenancy.oc1..aaaaaaaa5hwtrus75rauufcfvtnjnz3mc4xm2bzibbigva2bw4ne7ezkvzha",temperature=0.0)
+llm = OCIGenAI(model_id="cohere.command",service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",model_kwargs={"temperature": 0.0, "max_tokens": 300},compartment_id = "ocid1.compartment.oc1..aaaaaaaa7ggqkd4ptkeb7ugk6ipsl3gqjofhkr6yacluwj4fitf2ufrdm65q")
 system_prompt="As a support engineer, your role is to leverage the information in the context provided. Your task is to respond to queries based strictly on the information available in the provided context. Do not create new information under any circumstances. Refrain from repeating yourself. Extract your response solely from the context mentioned above. If the context does not contain relevant information for the question, respond with 'How can I assist you with questions related to the document?"
 service_context = ServiceContext.from_defaults(llm=llm, chunk_size=1000, chunk_overlap=100, embed_model=embeddings,system_prompt=system_prompt)
 vector_store = QdrantVectorStore(client=client, collection_name="ansh")
 storage_context = StorageContext.from_defaults(vector_store=vector_store)
 index = VectorStoreIndex.from_documents(documents, storage_context=storage_context, service_context=service_context)
 query_engine = index.as_query_engine()
-response = query_engine.query("can i use OCI document understanding for files in french ?")
+response = query_engine.query('What is activity auditing report?')
 print(response)
    ```
 

ai-and-app-modernisation/ai-services/generative-ai-service/rag-genai/files/langChainRagWithUI.py

@@ -0,0 +1,42 @@
+import streamlit as st
+from langchain_community.embeddings import OCIGenAIEmbeddings
+from langchain.chains import RetrievalQA
+from langchain_community.vectorstores import Qdrant
+from langchain_core.prompts import PromptTemplate
+from langchain_community.llms import OCIGenAI
+from langchain_community.document_loaders import UnstructuredURLLoader
+st.title("Oracle QA Chatbot")
+st.text_input("Ask a question:", key="question")  # Input field for questions
+# Data loading (outside any function)
+compartment_id = "ocid1.compartment.oc1..aaaaaaaa7ggqkd4ptkeb7ugk6ipsl3gqjofhkr6yacluwj4fitf2ufrdm65q"
+embeddings = OCIGenAIEmbeddings(model_id="cohere.embed-english-light-v3.0",service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",compartment_id=compartment_id,)
+testurls = ['https://docs.oracle.com/iaas/odsaz/odsa-rotate-wallet.html','https://docs.oracle.com/iaas/odsaz/odsa-change-password.html','https://docs.oracle.com/iaas/odsaz/odsa-database-actions.html',]
+# Cache the loaded documents (outside any function)
+@st.cache_data
+def load_documents():
+    docs = UnstructuredURLLoader(urls=testurls).load()
+    print("Loading data")
+    print(docs)
+    return docs  # Return the loaded documents
+docs = load_documents()
+vectorstore = Qdrant.from_documents(docs, embeddings, location=":memory:", prefer_grpc=False, collection_name="test_db")
+retriever = vectorstore.as_retriever()
+rag_prompt_template = """Answer the question based only on the following context:
+{context}
+Question: {question}"""
+rag_prompt = PromptTemplate.from_template(rag_prompt_template)
+llm = OCIGenAI(
+    model_id="cohere.command",
+    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
+    compartment_id=compartment_id,
+    model_kwargs={"temperature": 0, "max_tokens": 300},
+)
+rag = RetrievalQA.from_chain_type(llm=llm, retriever=retriever, chain_type_kwargs={"prompt": rag_prompt})
+# Answer generation when a question is asked
+if st.button("Get Answer"):
+    question = st.session_state.question
+    # Ensure correct access to cached documents
+    docs = load_documents()  # Call the cached function to retrieve documents
+    data = rag.invoke(question, context=docs)  # Pass documents as context
+    answer = data["result"]
+    st.write("Answer:", answer)

security/security-design/bastion-session-script/LICENSE renamed to ai-and-app-modernisation/ai-services/generative-ai-service/speech-genai/LICENSE

@@ -0,0 +1,85 @@
+# Integrate OCI AI Speech Service and Generative AI Summarization in Visual Builder
+
+# Introduction
+
+OCI Speech is an AI service that applies automatic speech recognition technology to transform audio-based content into text.  Generative AI, The Large Language Model (LLM) analyzes the text input and can generate, summarize, transform, and extract information. Using these AI capabilities, we built a low code application- “Integrate OCI AI Speech Service and Generative AI Summarization in Visual Builder "  to invoke AI Speech REST API to convert audio files into text and then further invoke the Generative AI REST API to Summarize it.       
+
+Reviewed: 20.02.2024
+
+<img src="./files/AISpeechGenAISummary.png"></img>
+
+# Prerequisites
+
+Before getting started, make sure you have access to these services:
+
+- Oracle Speech Service
+- Oracle Generative AI Service
+- Oracle Visual Builder Cloud Service
+- Oracle Visual Builder Service Connection
+
+# AI Speech and OCI Generative AI Service Integration Architecture
+
+1. AI Speech App using VBCS
+
+- Oracle Visual Builder Cloud Service (VBCS) is a hosted environment for your application development infrastructure. It provides an open-source standards-based development service to create, collaborate on, and deploy applications within Oracle Cloud. This application is developed in VBCS.
+
+2.	Transcriptions with OCI AI Speech Service:
+- Speech harnesses the power of spoken language enabling you to easily convert media files containing human speech into highly exact text transcriptions.
+- Produces accurate and easy-to-use JSON and SubRip Subtitle (SRT) files written directly to the Object Storage bucket you choose.
+
+3.	Integration with OCI Generative AI Service:
+- The transcriptions (text) are sent to the OCI Generative AI Service for text summarization.
+
+4.	Integration with OCI AI Vision and OCI Generative AI Service using Visual Builder Service Endpoint:
+- Build a Service Connection Endpoint option is used to integrate the VBCS app and OCI Object Storage, OCI AI Speech Service, and Generative AI Summarization.
+
+5.	Summarization Process:
+- OCI Generative AI Service generates text using the keywords received from OCI Speech service, to create a concise summary of the audio or video.
+
+      
+<img src="./files/AISpeechSummaryAppArch.svg"></img>
+
+# Application Flow in Detail (VBCS, OCI Speech, OCI Generative AI Service)
+
+In this application, the drag-and-drop component in VBCS allows the user to drop the audio or video. 
+- Create a Service Endpoint connection in Visual Builder to handle the communication between Visual Builder and OCI Speech Service.
+- Pass the selected audio or video from Visual Builder to OCI Speech Service to convert it into text.
+- OCI Speech Service analyzes the media (audio or video) file and converts it into text.
+- The OCI Speech Service returns the transcription to the AI Speech Service Endpoint and returns the results to the Visual Builder app.
+- The transcription further passes to the Generative AI Service Endpoint and returns the Summarization results to the Visual Builder app.
+
+         User (Visual Builder) --> (Drag and Drop File) --> |Media File (adudio or video) --> (Service Endpoint) --> |OCI Speech Service| --> |Speech to Text| --> (Service Endpoint) --> |Result| --> (Visual Builder) --> (Gen AI Service Endpoint) --> |Result| --> (Visual Builder) 
+
+   <img src="./files/AISpeechEngine.png"></img>
+
+# Service Endpoint call - Invoke OCI Object Storage
+
+      uploadfile - /n/{namespaceName}/b/{bucketName}/o/{objectName}
+      getObject - /n/{namespaceName}/b/{bucketName}/o/{outputFolderName}/{outputObjectName}
+
+
+# Service Endpoint call - Invoke AI Speech Service 
+
+      create transcription - /transcriptionJobs
+      get transcription - transcriptionJobs/{transcriptionJobId}
+
+# Service Endpoint call - Invoke Generative AI Service
+
+      create summary - /20231130/actions/summarizeText
+
+
+# Conclusion
+
+In this article, we've covered how to utilize Oracle AI Speech Service features to provide a transription and summarize using Generative AI service.  
+
+Feel free to modify and expand upon this template according to your specific use case and preferences.
+
+
+# License
+ 
+Copyright (c) 2024 Oracle and/or its affiliates.
+ 
+Licensed under the Universal Permissive License (UPL), Version 1.0.
+ 
+See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details.
+