Merge branch 'main' into hyperion-update-february

Author: velrajkuppu

ai-and-app-modernisation/ai-services/generative-ai-service/rag-genai/files/README.md

@@ -26,7 +26,7 @@ Before getting started, make sure you have the following installed:
 ## Setting up the Environment
 1. Install the required packages:
    ```bash
-   pip install oci==2.112.1+preview.1.1649 llama-index qdrant-client sentence-transformers
+   pip install oci==2.118.1+preview.1.1697 llama-index qdrant-client sentence-transformers
    ```
 
 ## Loading data

ai-and-app-modernisation/ai-services/generative-ai-service/speech-genai/LICENSE

@@ -0,0 +1,35 @@
+Copyright (c) 2024 Oracle and/or its affiliates.
+
+The Universal Permissive License (UPL), Version 1.0
+
+Subject to the condition set forth below, permission is hereby granted to any
+person obtaining a copy of this software, associated documentation and/or data
+(collectively the "Software"), free of charge and under any and all copyright
+rights in the Software, and any and all patent rights owned or freely
+licensable by each licensor hereunder covering either (i) the unmodified
+Software as contributed to or provided by such licensor, or (ii) the Larger
+Works (as defined below), to deal in both
+
+(a) the Software, and
+(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
+one is included with the Software (each a "Larger Work" to which the Software
+is contributed by such licensors),
+
+without restriction, including without limitation the rights to copy, create
+derivative works of, display, perform, and distribute the Software and make,
+use, sell, offer for sale, import, export, have made, and have sold the
+Software and the Larger Work(s), and to sublicense the foregoing rights on
+either these or other terms.
+
+This license is subject to the following condition:
+The above copyright notice and either this complete permission notice or at
+a minimum a reference to the UPL must be included in all copies or
+substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ai-and-app-modernisation/ai-services/generative-ai-service/speech-genai/README.md

@@ -0,0 +1,85 @@
+# Integrate OCI AI Speech Service and Generative AI Summarization in Visual Builder
+
+# Introduction
+
+OCI Speech is an AI service that applies automatic speech recognition technology to transform audio-based content into text.  Generative AI, The Large Language Model (LLM) analyzes the text input and can generate, summarize, transform, and extract information. Using these AI capabilities, we built a low code application- “Integrate OCI AI Speech Service and Generative AI Summarization in Visual Builder "  to invoke AI Speech REST API to convert audio files into text and then further invoke the Generative AI REST API to Summarize it.       
+
+Reviewed: 20.02.2024
+
+<img src="./files/AISpeechGenAISummary.png"></img>
+
+# Prerequisites
+
+Before getting started, make sure you have access to these services:
+
+- Oracle Speech Service
+- Oracle Generative AI Service
+- Oracle Visual Builder Cloud Service
+- Oracle Visual Builder Service Connection
+
+# AI Speech and OCI Generative AI Service Integration Architecture
+
+1. AI Speech App using VBCS
+
+- Oracle Visual Builder Cloud Service (VBCS) is a hosted environment for your application development infrastructure. It provides an open-source standards-based development service to create, collaborate on, and deploy applications within Oracle Cloud. This application is developed in VBCS.
+
+2.	Transcriptions with OCI AI Speech Service:
+- Speech harnesses the power of spoken language enabling you to easily convert media files containing human speech into highly exact text transcriptions.
+- Produces accurate and easy-to-use JSON and SubRip Subtitle (SRT) files written directly to the Object Storage bucket you choose.
+
+3.	Integration with OCI Generative AI Service:
+- The transcriptions (text) are sent to the OCI Generative AI Service for text summarization.
+
+4.	Integration with OCI AI Vision and OCI Generative AI Service using Visual Builder Service Endpoint:
+- Build a Service Connection Endpoint option is used to integrate the VBCS app and OCI Object Storage, OCI AI Speech Service, and Generative AI Summarization.
+
+5.	Summarization Process:
+- OCI Generative AI Service generates text using the keywords received from OCI Speech service, to create a concise summary of the audio or video.
+
+      
+<img src="./files/AISpeechSummaryAppArch.svg"></img>
+
+# Application Flow in Detail (VBCS, OCI Speech, OCI Generative AI Service)
+
+In this application, the drag-and-drop component in VBCS allows the user to drop the audio or video. 
+- Create a Service Endpoint connection in Visual Builder to handle the communication between Visual Builder and OCI Speech Service.
+- Pass the selected audio or video from Visual Builder to OCI Speech Service to convert it into text.
+- OCI Speech Service analyzes the media (audio or video) file and converts it into text.
+- The OCI Speech Service returns the transcription to the AI Speech Service Endpoint and returns the results to the Visual Builder app.
+- The transcription further passes to the Generative AI Service Endpoint and returns the Summarization results to the Visual Builder app.
+
+         User (Visual Builder) --> (Drag and Drop File) --> |Media File (adudio or video) --> (Service Endpoint) --> |OCI Speech Service| --> |Speech to Text| --> (Service Endpoint) --> |Result| --> (Visual Builder) --> (Gen AI Service Endpoint) --> |Result| --> (Visual Builder) 
+
+   <img src="./files/AISpeechEngine.png"></img>
+
+# Service Endpoint call - Invoke OCI Object Storage
+
+      uploadfile - /n/{namespaceName}/b/{bucketName}/o/{objectName}
+      getObject - /n/{namespaceName}/b/{bucketName}/o/{outputFolderName}/{outputObjectName}
+
+
+# Service Endpoint call - Invoke AI Speech Service 
+
+      create transcription - /transcriptionJobs
+      get transcription - transcriptionJobs/{transcriptionJobId}
+
+# Service Endpoint call - Invoke Generative AI Service
+
+      create summary - /20231130/actions/summarizeText
+
+
+# Conclusion
+
+In this article, we've covered how to utilize Oracle AI Speech Service features to provide a transription and summarize using Generative AI service.  
+
+Feel free to modify and expand upon this template according to your specific use case and preferences.
+
+
+# License
+ 
+Copyright (c) 2024 Oracle and/or its affiliates.
+ 
+Licensed under the Universal Permissive License (UPL), Version 1.0.
+ 
+See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details.
+	

ai-and-app-modernisation/ai-services/generative-ai-service/speech-genai/files/AISpeechEngine.png

@@ -0,0 +1,35 @@
+Copyright (c) 2024 Oracle and/or its affiliates.
+
+The Universal Permissive License (UPL), Version 1.0
+
+Subject to the condition set forth below, permission is hereby granted to any
+person obtaining a copy of this software, associated documentation and/or data
+(collectively the "Software"), free of charge and under any and all copyright
+rights in the Software, and any and all patent rights owned or freely
+licensable by each licensor hereunder covering either (i) the unmodified
+Software as contributed to or provided by such licensor, or (ii) the Larger
+Works (as defined below), to deal in both
+
+(a) the Software, and
+(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
+one is included with the Software (each a "Larger Work" to which the Software
+is contributed by such licensors),
+
+without restriction, including without limitation the rights to copy, create
+derivative works of, display, perform, and distribute the Software and make,
+use, sell, offer for sale, import, export, have made, and have sold the
+Software and the Larger Work(s), and to sublicense the foregoing rights on
+either these or other terms.
+
+This license is subject to the following condition:
+The above copyright notice and either this complete permission notice or at
+a minimum a reference to the UPL must be included in all copies or
+substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ai-and-app-modernisation/ai-services/generative-ai-service/speech-genai/files/AISpeechGenAISummary.png

@@ -0,0 +1,95 @@
+# Describe an image using OCI AI Vision Service and OCI Generative AI Service
+
+# Introduction
+In this article, we'll explore how to describe an image using OCI AI Vision Service and OCI Generative AI Service.
+The application is developed using Oracle VBCS, OIC, OCI AI Vision service, and OCI Generative AI Service.
+This integrated approach combines the strength of OCI AI Vision and OCI Generative AI Service, allowing for efficient and insightful summarization of image content.
+
+
+Reviewed: 21.02.2024
+
+<img src="./files/AIVisionApp.jpg"></img>
+
+
+# Prerequisites
+
+Before getting started, make sure you have access to these services:
+
+- Oracle Generative AI Service
+- Oracle Vision Service
+- Oracle Visual Builder Cloud Service
+- Oracle Integration Cloud
+
+# AI Vision and OCI Generative AI Service Integration Architecture
+
+1. AI Vision App using VBCS
+- Oracle Visual Builder Cloud Service (VBCS) is a hosted environment for your application development infrastructure. It provides an open-source standards-based development service to create, collaborate on, and deploy applications within Oracle Cloud. This application is developed in VBCS.
+
+2. Image Analysis with OCI AI Vision Service:
+- The AI Vision service is employed to analyze images.
+- It identifies objects within the image by using advanced computer vision algorithms.
+
+3. Integration with OCI Generative AI Service:
+- The extracted object keywords are sent to the OCI Generative AI Service
+
+4. Integration with OCI AI Vision and OCI Generative AI Service using OIC:
+- Oracle Integration Cloud integrates the VBCS app and OCI AI Services.
+
+5. Summarization Process:
+- OCI Generative AI Service generates text using the keywords received from OCI Vision service, to create a concise summary of the image.
+
+<img src="./files/AIVisionAppArch.svg"></img>
+
+# Application Flow in Detail (VBCS, OIC, OCI Vision, OCI Generative AI Service)
+
+In this application,
+-	The File Picker action in VBCS allows the user to select the image. 
+-	Create an integration process in Oracle Integration Cloud (OIC) to handle the communication between   VBCS and OCI Vision Service.
+-	Pass the selected image from VBCS to OCI Vision Service to analyze the image.
+-	OCI Vision Service analyzes the image and identifies objects within it.
+-	The OCI Vision Service returns the detected objects (keywords) to the OIC integration process and returns the results to VBCS.
+
+         User (VBCS) --> (File Picker) --> |Image| --> (OIC) --> |OCI Vision Service| --> |Detected Objects| --> (OIC) --> |Result| --> (VBCS)
+
+   <img src="./files/VBCS_Vision.jpg">
+      </img>
+
+      OIC call - Invoke OCI Vision Service
+      Endpoint - /actions/analyzeImage
+
+   <img src="./files/OIC_VisionService.jpg">
+      </img>
+
+-	User clicks the "Generate" button in the app to initiate the summary generation.
+-	Configure the OIC integration process to invoke the GenAI service.
+-	Pass the keywords returned by the OCI Vision Service along with any additional relevant information.
+-	Generative AI Service processes the received keywords and generates a summary of the image content.
+
+         User (VBCS) --> (File Picker) --> |Image| --> (OIC) --> |OCI Vision Service| --> |Detected Keywords| --> (OIC) --> | OCI Generative AI Service  --> |Summary| --> (OIC) --> |Result| --> (VBCS)
+
+   <img src="./files/VBCS_GenerateSummary.jpg">
+      </img>
+
+      OIC call - Invoke OCI Generative AI Service
+      Endpoint - /20231130/actions/generatText
+   <img src="./files/OIC_GenerateSummary.jpg">
+      </img>
+
+# Code
+      VBCS app -  ImageClassification-1.0.zip
+      OIC Vision Integration - RESTVISION_01.00.0000.iar
+      OIC Generate Summary - IMAGEDESCRIPTIONGENERATION_01.00.0000.iar
+
+# Conclusion
+
+In this article, we've covered how to utilize Oracle AI Vision Service features to provide a summary of an image using Generative AI service.  
+Feel free to modify and expand upon this template according to your specific use case and preferences.
+
+# License
+ 
+Copyright (c) 2024 Oracle and/or its affiliates.
+ 
+Licensed under the Universal Permissive License (UPL), Version 1.0.
+ 
+See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details.
+