diff --git a/Convai Innovations/Videos/CustomLLMBasedChatbot.mp4 b/Convai Innovations/Videos/CustomLLMBasedChatbot.mp4 new file mode 100644 index 00000000..6b231270 Binary files /dev/null and b/Convai Innovations/Videos/CustomLLMBasedChatbot.mp4 differ diff --git a/Convai Innovations/Videos/VRCoach.mp4 b/Convai Innovations/Videos/VRCoach.mp4 new file mode 100644 index 00000000..8efc8873 Binary files /dev/null and b/Convai Innovations/Videos/VRCoach.mp4 differ diff --git a/Convai Innovations/screenshots/1.png b/Convai Innovations/screenshots/1.png new file mode 100644 index 00000000..d86535b1 Binary files /dev/null and b/Convai Innovations/screenshots/1.png differ diff --git a/Convai Innovations/screenshots/2.png b/Convai Innovations/screenshots/2.png new file mode 100644 index 00000000..df04f180 Binary files /dev/null and b/Convai Innovations/screenshots/2.png differ diff --git a/Convai Innovations/screenshots/3.png b/Convai Innovations/screenshots/3.png new file mode 100644 index 00000000..20afb1e1 Binary files /dev/null and b/Convai Innovations/screenshots/3.png differ diff --git a/Convai Innovations/screenshots/4.png b/Convai Innovations/screenshots/4.png new file mode 100644 index 00000000..d53089ac Binary files /dev/null and b/Convai Innovations/screenshots/4.png differ diff --git a/Convai Innovations/screenshots/5.png b/Convai Innovations/screenshots/5.png new file mode 100644 index 00000000..a3da6f88 Binary files /dev/null and b/Convai Innovations/screenshots/5.png differ diff --git a/Convai Innovations/screenshots/6.png b/Convai Innovations/screenshots/6.png new file mode 100644 index 00000000..fccf3bb2 Binary files /dev/null and b/Convai Innovations/screenshots/6.png differ diff --git a/Convai Innovations/screenshots/metrabs.png b/Convai Innovations/screenshots/metrabs.png new file mode 100644 index 00000000..997207b1 Binary files /dev/null and b/Convai Innovations/screenshots/metrabs.png differ diff --git a/Convai Innovations/screenshots/team.png b/Convai Innovations/screenshots/team.png new file mode 100644 index 00000000..0cf4afff Binary files /dev/null and b/Convai Innovations/screenshots/team.png differ diff --git a/README.md b/README.md index 04eafd41..9c64ff0d 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,88 @@ -# ICC Eliminator Submisison -Teams will be required to submit their code base on GitHub by forking this repository. -Follow the following steps to submit your code base. -1. **Fork** this repository. -1. Create a **folder with your team name** -1. **Upload** your code base and videos to the repository. -1. Create a **pull request** to this repository. Make sure you raise your **pull request** as per your **team name**. -1. You can also add you team details in the README.md of your forked repository -For Example: -### Team Information ------------- -###### Team Name - -###### Track - -###### Brief Description and Snapshots - + +# Team Name - +## Convai Innovations + +![alt text](https://github.com/NandhaKishorM/icc-eliminator-submission/blob/main/Convai%20Innovations/screenshots/team.png) + +## Hackathon Participants + +1. **Nandakishor M** +2. **Anjali M** + +# Track - +## AR/VR +## Brief Description and Snapshots - + +# Virtual Reality Cricket Coaching with Cricketers Avatar +## Overview +This project provides virtual reality coaching for cricket with a 3D avatar of Virat Kohli, a famous Indian cricketer. The coaching covers multiple master batting classes, including the flick, cover drive, pull shot, and more. Users can access the training on the Meta Quest headset. The platform uses advanced technologies like Unreal Engine, MetaHuman 3D avatar creator platform, and AI based pose estimation using MeTRAbs(https://arxiv.org/abs/2007.07227) to provide an immersive and engaging experience to the users. +We have employed MeTRAbs, a reliable 3D pose estimation technique, to streamline control rig animation. To facilitate our upcoming **chatbot** AI integration, **we have developed a PyTorch-based API, which is deployed on an AWS EC2 GPU instance for improved performance.** + +## Demo Video + +Check out the demo video + +[![Alt text](https://img.youtube.com/vi/i4E9pcAUN5c/0.jpg)](https://www.youtube.com/watch?v=i4E9pcAUN5c) + +## Requirements +To run this project, you will need: + +1. A Meta Quest headset +2. Unreal Engine 4.27 or higher +3. MetaHuman 3D avatar creator platform + +## Usage +To use the platform, follow these steps: + +### Download and install the app on Meta Quest 2 + +**APK Link** : https://drive.google.com/file/d/1uqhaYZOzGVh08LTyucLwd6O-Bfb6z8m0/view?usp=share_link + +Put on the Meta Quest headset +Follow the instructions provided by the 3D avatar to learn the techniques + +## Installation + +1. Clone the repository to your local machine +2. Install Unreal Engine 4.27 or higher +3. Open the project in Unreal Engine : NOTE THAT DUE TO HIGHER SIZE OF THE GAME(> 5GB) DIDN'T UPLOADED IN THE GITHUB REPO. +4. Connect the Meta Quest headset to your computer +5. Build and run the project + +## Features +1. Virtual reality coaching for cricket with a 3D avatar of Virat Kohli +2. Multiple master batting classes included, such as the flick, cover drive, and pull shot +3. Advanced technologies like Unreal Engine, MetaHuman 3D avatar creator platform, and Pose estimation +4. Immersive and engaging experience + +## Advanced video to 3d pose conversion for control rig animation Unreal Engine +![alt text](https://github.com/NandhaKishorM/icc-eliminator-submission/blob/main/Convai%20Innovations/screenshots/metrabs.png) + +## Todo List +1. We have recently introduced **CUTY**, a **3D avatar powered by our state-of-the-art 400M large language model**. The model leverages a Transformer-based architecture similar to **GPT** and includes **internet search capability**. It took us **two years** to develop and incorporates a **Cross platform API** for intelligent interaction with a **persona of cricket coach**. The model has been **fine-tuned on a cricket dataset** to enhance its accuracy. To experience its full potential, **kindly turn on the volume in the video demonstration**. CUTY is available both on the **web and VR platforms**. PS: Server has been turned off due to development and server cost(Nvidia A10 GPU costs) + +https://user-images.githubusercontent.com/48623612/222485817-e996263d-aced-4b49-9180-1f27246d3fb7.mp4 + + +You can find more details on the CUTY on : https://convaiinnovations.com/ + +2. Implement deep reinforcement learning to create an intelligent 3D avatar that can improve the user experience +3. Add more master classes for batting +4. Create an analytics dashboard to track user progress and performance +5. Integrate social media sharing features to allow users to share their experience with others +## Screenshots + +
+ + +
+ + +
+ +## Reference + +1. Thanks to the original video for the reference : https://www.youtube.com/shorts/OmR4WMIQcek +2. https://arxiv.org/abs/2007.07227 + +