Shara Proactive Robot

THIS IS A SECOND VERSION OF THE EVA ROBOT.
Shara is a social 🗣 and affective ❤️ robot. It is not only a passive assistant, but an active one: proactive behaviour is incorporated in the robot. It can start conversations and show concern about the user, making the interaction more natural and affective.
This repo contains the brain 🧠 structure of the robot, the proactive and interaction behavior themselves.

Be different 😎

With SHARA, you can have conversations in the more natural way. It is activated by a novel method called wakeface, in which the robot activates/listens the user by looking at it. Also, it is able to start conversations by using proactive questions.

Highlighted proactive questions ✨ how are you, who are you

Main components 🤖🛠️

Shara hardware structure is based on the existing EVA robotic platform, but with several modifications. This EVA affective and proactive version is constructed using the following elements:

• 🖥️ Screen Waveshare 7inch HDMI LCD (H) for 👀 displaying
• 📷 IMX219-160IR Camera
• 🎙️ ReSpeaker Mic Array v2.0
• 🔊 Speakers
• 🤖 Raspberry Pi 5 16GB (remember to use cooler heatsinks 🔥)
• 🔃 2x Stepper motor 28BYJ-48 for the neck
• 💡 Neopixel LED RGB ring 24 bits WS2812, controlled by ESP32 ESP-WROOM-32 and WLED project!
• 🪭 Noctua NF-A8 5V PWM fan
• Plastic structure printed using 3D printer [repository of the 3D models here!] [WIP]

Installation ⚙️

Requirements

Use the package manager pip to install all the pre-requisites.

pip3 install -r requirements.txt

Google and OpenAI services

Ensure you have Google Cloud and OpenAI accounts.

It is necessary to have apikeys 🔑 for the following services:

• Google Speech to Text
• Google Text to Speech
• OpenAI Text Generation for conversation generation with Shara

Google project credentials' file files must be stored in a credentials/ directory, located outside the main project directory:

$ any_directory
.
├── credentials
│   └── google_credentials.json
└── Proactive-Shara-Robot/

In this case, by-default environment variables provided by Google and OpenAI (GOOGLE_APPLICATION_CREDENTIALS and OPENAI_API_KEY) are used for automatic services apikey authentication (easier!).

Usage 🚀

For executing SHARA, you only have to run from root repo directory:

python3 main.py

➡️Note: shara_prompt.txt contain instructions totally in spanish, so if you want SHARA to speak in a different language, teach it your language by changing the necessary files in your language (prompt and google lang). She will be happy to learn it 😊

And that's how you construct your own affective social robot! 🤖❤️👩🏻

Authors 📝

• Laura Villa 🦁