ROS Noetic, Python 3 Updates by zacharykratochvil · Pull Request #31 · furushchev/respeaker_ros

zacharykratochvil · 2022-04-28T13:42:33Z

After installing on RaspberryPi 4B, Ubuntu Impish, ROS 1 Noetic, Python 3.9.7, the Respeaker array would not initialize. It required two additional dependencies not listed in package.xml and had old python 2 syntax and function calls in respeaker_node.py. After these corrections it initializes and runs well. I believe these changes are necessary for anyone running newer versions of python and ROS so others might benefit from this branch being publicly available. Not sure about their backwards compatibility though, so it may be best to leave them in a separate branch.

…yntax for python 3.9.7

MirkoFerrati

Hello! I was looking for someone who made respeaker_ros work in python3 and found this PR. I did a quick review and asked some questions and I am sorry if they are unclear or sound rude, I find this PR very helpful and just want to contribute!
Mirko

MirkoFerrati · 2022-08-23T13:20:48Z

scripts/speech_to_text.py

        # format of input audio data
-        self.sample_rate = rospy.get_param("~sample_rate", 16000)
-        self.sample_width = rospy.get_param("~sample_width", 2)
+        self.sample_rate = 16000 #rospy.get_param("~sample_rate", 16000)


why remove parameter? is this a leftover?

not sure why I hardcoded this, maybe I was changing the rate during debugging

MirkoFerrati · 2022-08-23T13:21:10Z

scripts/speech_to_text.py

        self.pub_speech = rospy.Publisher(
            "speech_to_text", SpeechRecognitionCandidates, queue_size=1)
-        self.sub_audio = rospy.Subscriber("audio", AudioData, self.audio_cb)
+        self.sub_audio = rospy.Subscriber("speech_audio", AudioData, self.audio_cb)


public topic name change, meaning API changed. intended?

yes, this is important, the package appears to be set up to publish raw audio to "audio" and processed, clean audio to "speech_audio", so the speech to text feature ought to use the cleaned up audio for better recognition

MirkoFerrati · 2022-08-23T13:21:15Z

scripts/speech_to_text.py

            if stamp - self.last_tts > self.tts_tolerance:
                rospy.logdebug("END CANCELLATION")
-                self.is_canceling = False
+                self.is_canceling = Falser


MirkoFerrati · 2022-08-23T13:21:40Z

scripts/speech_to_text.py

            return
-        data = SR.AudioData(msg.data, self.sample_rate, self.sample_width)
+        data = SR.AudioData(bytes(msg.data), self.sample_rate, self.sample_width)
+        with open(str(len(msg.data)) + ".wav","wb") as f:


change of behavior by writing to file, I think this is not related to python3

casting msg.data to bytes before sending to AudioData is indeed related to python3. opening the wave file is just from debugging. again, sorry I should have cleaned this up.

MirkoFerrati · 2022-08-23T13:22:28Z

scripts/speech_to_text.py

-            result = self.recognizer.recognize_google(
-                data, language=self.language)
-            msg = SpeechRecognitionCandidates(transcript=[result])
+            result = self.recognizer.recognize_sphinx(


not an expert on this, but google->sphinx may change user experience somehow

this should be removed now. specific to my use case

MirkoFerrati · 2022-08-23T13:22:46Z

scripts/respeaker_node.py


    def on_audio(self, data, channel):
+
+        if channel == 0:


also just debugging

MirkoFerrati · 2022-08-23T13:23:09Z

scripts/respeaker_node.py

        self.timer_led = None
        self.sub_led = rospy.Subscriber("status_led", ColorRGBA, self.on_status_led)
+        self.big_data0 = []
+        self.out = wave.open("/home/pi/Desktop/test.wav", 'wb')


is this self.out some debug code? hardcoded path will not work anywhere else

yes debug code, sorry!

zacharykratochvil · 2022-08-27T11:37:19Z

Hello! I was looking for someone who made respeaker_ros work in python3 and found this PR. I did a quick review and asked some questions and I am sorry if they are unclear or sound rude, I find this PR very helpful and just want to contribute! Mirko

Hi, glad I could help! Apologies it seems I forgot to clean up my code before making the PR. There are also some changes specific to my use case in here that I forgot to make a new branch for. I just reset the branch to before that commit and also removed some of the debugging lines. Still have to test. I'll respond to your comments directly as well. Let me know if you have any more questions!

JohannaPrinz · 2022-09-02T10:06:27Z

Hey,
thanks for debugging the code, so it works with ROS Noetic and Python3
I have applied all changes, but still get the following error messages:

File "/opt/ros/noetic/lib/python3/dist-packages/tf2_py/init.py", line 38, in
from ._tf2 import *
ImportError: dynamic module does not define init function (init_tf2)

File "/opt/ros/noetic/lib/python3/dist-packages/rospy/impl/tcpros_base.py", line 167
(e_errno, msg, *_) = e.args
^
SyntaxError: invalid syntax

Can you may help me fixing this?! Thanks so much
Best regards

zacharykratochvil · 2022-09-02T12:11:14Z

Hi, Happy to help but the lines you listed don't exist in the respeaker_ros code. Either the errors are with your ros installation or you haven't given the relevant lines from the error messages. Best, Zach

…

On Fri, Sep 2, 2022, 6:06 AM JohannaPrinz ***@***.***> wrote: Hey, thanks for debugging the code, so it works with ROS Noetic and Python3 I have applied all changes, but still get the following error messages: 1. from ._tf2 import * ImportError: dynamic module does not define init function (init_tf2) 1. (e_errno, msg, *_) = e.args ^ SyntaxError: invalid syntax Can you may help me fixing this?! Thanks so much Best regards — Reply to this email directly, view it on GitHub <#31 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APSFFUDGB4QIQY7SIWEGE7TV4HGS3ANCNFSM5USORCZQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

S-Fichtl · 2022-09-08T07:25:33Z

requirements.txt

 PyAudio==0.2.8
 SpeechRecognition==3.8.1
 click==6.7
-numpy==1.16.2


Are you sure? I believe numpy is still required. See scripts/respeaker_node.py line 11

S-Fichtl · 2022-09-08T07:25:55Z

scripts/respeaker_node.py

+        self.rate = rospy.get_param("~sample_rate", 16000)
+        self.bitwidth = rospy.get_param("~sample_width", 2)
        self.bitdepth = 16
+        self.i = 0


this is also a debugging left-over?

S-Fichtl · 2022-09-08T08:15:31Z

scripts/respeaker_node.py

 import usb.core
 import usb.util
 import pyaudio
+import wave


this should be removed again as well, right? that was for the wav file you used for debugging?

zacharykratochvil · 2022-09-08T10:50:48Z

All correct, thank you!

…

On Thu, Sep 8, 2022, 4:15 AM Severin Fichtl ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In scripts/respeaker_node.py <#31 (comment)> : > @@ -7,6 +7,7 @@ import usb.core import usb.util import pyaudio +import wave this should be removed again as well, right? that was for the wav file you used for debugging? — Reply to this email directly, view it on GitHub <#31 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APSFFUFUCUHKRVNBZD2NWBDV5GOC5ANCNFSM5USORCZQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

hello-binit · 2023-01-04T08:55:46Z

Hi @zacharykratochvil, thank you for creating this PR! I gave it a try and it needed a few fixes to get past catkin_virtualenv related build errors. I've open a PR towards your fork to fix these build errors, address the feedback left by @S-Fichtl in this PR, and add a few additional tweaks. Let me know what you think!

zacharykratochvil added 3 commits April 28, 2022 09:26

added necessary dependencies and updated to preferred functions and s…

e771068

…yntax for python 3.9.7

additional python 3 fix

5237497

fixed speech recognition

415fb74

MirkoFerrati reviewed Aug 23, 2022

View reviewed changes

zacharykratochvil force-pushed the python3.9.7 branch from e73482b to 415fb74 Compare August 27, 2022 12:03

zacharykratochvil added 3 commits August 27, 2022 08:28

cleaning up code after debugging

5427c94

edited lanugage

dd8af5c

fixed bug

ef9aabe

S-Fichtl reviewed Sep 8, 2022

View reviewed changes

hello-binit mentioned this pull request Jan 4, 2023

Fixed build errors and addressed PR feedback zacharykratochvil/respeaker_ros#1

Open

nickswalker mentioned this pull request Aug 29, 2023

ROS2 support #34

Open

Conversation

zacharykratochvil commented Apr 28, 2022

Uh oh!

MirkoFerrati left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zacharykratochvil commented Aug 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JohannaPrinz commented Sep 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zacharykratochvil commented Sep 2, 2022 via email

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zacharykratochvil commented Sep 8, 2022 via email

Uh oh!

hello-binit commented Jan 4, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

zacharykratochvil commented Aug 27, 2022 •

edited

Loading

JohannaPrinz commented Sep 2, 2022 •

edited

Loading