Skip to content

Commit aaf72ea

Browse files
shiralalshiralal
committed
Node.js backend server and riva-frontend app for testing
1 parent 232c737 commit aaf72ea

28 files changed

+20257
-0
lines changed

‎.gitignore

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,3 +186,22 @@ tests/integration/tts/outputs
186186
riva/client/proto/*_pb2.py
187187
riva/client/proto/*_pb2_grpc.py
188188

189+
190+
# Downloaded/generated proto files and repositories
191+
app-backend/common/
192+
app-backend/riva/proto/
193+
194+
# Node.js specific ignores for app-backend
195+
app-backend/node_modules/
196+
app-backend/coverage/
197+
app-backend/.nyc_output/
198+
app-backend/logs/
199+
app-backend/*.log
200+
app-backend/.env
201+
app-backend/.env.local
202+
app-backend/.env.development.local
203+
app-backend/.env.test.local
204+
app-backend/.env.production.local
205+
app-backend/npm-debug.log*
206+
app-backend/yarn-debug.log*
207+
app-backend/yarn-error.log*

‎app-backend/README.md

Lines changed: 243 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,243 @@
1+
# Riva App Backend
2+
3+
This is a Node.js proxy server that connects to the Riva API server. It provides API endpoints for automatic speech recognition (ASR) and text-to-speech (TTS) services.
4+
5+
## Features
6+
7+
- Direct connection to Riva server using official proto files
8+
- ASR (Automatic Speech Recognition) endpoint
9+
- TTS (Text-to-Speech) endpoint
10+
- WAV file support with header analysis and proper processing
11+
- Configurable via environment variables
12+
- WebSocket support for real-time streaming recognition
13+
14+
## Setup
15+
16+
1. Ensure you have Node.js installed (v14 or higher recommended)
17+
18+
2. Install dependencies:
19+
```
20+
npm install
21+
```
22+
23+
3. Download the proto files:
24+
```
25+
npm run download-protos
26+
```
27+
This script will clone the nvidia-riva/common repository and copy the necessary proto files to the `riva/proto` directory.
28+
29+
## Configuration
30+
31+
Create a `.env` file in the root directory with the following variables:
32+
33+
```
34+
PORT=3002
35+
RIVA_API_URL=localhost:50051
36+
```
37+
38+
- `PORT`: The port on which the proxy server will run
39+
- `RIVA_API_URL`: The URL of the Riva API server
40+
41+
## Running the Server
42+
43+
Start the server:
44+
45+
```
46+
npm start
47+
```
48+
49+
This will automatically run the `download-protos` script before starting the server if the proto files are not already present.
50+
51+
## Testing the Application
52+
53+
### Prerequisites
54+
55+
Before testing:
56+
1. Ensure the Riva API server is running at the configured URL
57+
2. Verify that the proto files have been downloaded successfully
58+
3. Make sure the Node.js server is running (check for "Server listening on port 3002" message)
59+
4. Have sample audio files available for testing
60+
61+
### Testing the API Endpoints Directly
62+
63+
#### Testing the Health Endpoint
64+
65+
```bash
66+
curl http://localhost:3002/health
67+
```
68+
69+
Expected response:
70+
```json
71+
{
72+
"status": "ok",
73+
"services": {
74+
"asr": {
75+
"available": true
76+
},
77+
"tts": {
78+
"available": true
79+
}
80+
}
81+
}
82+
```
83+
84+
#### Testing ASR with a WAV File
85+
86+
You can use the included test script:
87+
88+
```bash
89+
# If you have a sample WAV file
90+
node test-asr.js /path/to/your/audio.wav
91+
```
92+
93+
Or test manually with curl:
94+
95+
```bash
96+
# Convert WAV to base64 first
97+
base64 -w 0 /path/to/your/audio.wav > audio.b64
98+
99+
# Send the request
100+
curl -X POST http://localhost:3002/api/recognize \
101+
-H "Content-Type: application/json" \
102+
-d @- << EOF
103+
{
104+
"audio": "$(cat audio.b64)",
105+
"config": {
106+
"encoding": "LINEAR_PCM",
107+
"sampleRateHertz": 16000,
108+
"languageCode": "en-US",
109+
"enableAutomaticPunctuation": true
110+
}
111+
}
112+
EOF
113+
```
114+
115+
### Testing with the Frontend
116+
117+
The best way to test the complete functionality is using the provided frontend application:
118+
119+
1. Start this backend server
120+
2. Start the Riva frontend application
121+
3. Use the frontend to upload audio files or test streaming recognition
122+
123+
### Debugging and Log Information
124+
125+
The server provides detailed logging for audio processing. When processing WAV files, it will:
126+
127+
1. Log detection of WAV headers
128+
2. Display information about:
129+
- Sample rate
130+
- Number of channels
131+
- Bits per sample
132+
- Audio format
133+
134+
When issues occur, check the console output for detailed error messages.
135+
136+
## Troubleshooting Proto Files Download
137+
138+
If you encounter issues downloading proto files:
139+
140+
1. Check your internet connection
141+
2. Verify that git is installed and accessible
142+
3. Look for specific errors in the console output
143+
4. Make sure the `riva_common.proto` file is included in the filter (the download script now includes this file)
144+
5. Try running the download script manually:
145+
```
146+
node download-protos.js
147+
```
148+
6. If problems persist, you can manually clone the repository and copy the proto files:
149+
```
150+
git clone https://github.com/nvidia-riva/common.git
151+
mkdir -p riva/proto
152+
cp common/riva/proto/*.proto riva/proto/
153+
```
154+
155+
## API Endpoints
156+
157+
### Status
158+
159+
- **GET** `/health`
160+
- Returns the status of the ASR and TTS services
161+
162+
### Speech Recognition (ASR)
163+
164+
- **POST** `/api/recognize`
165+
- Request body:
166+
```json
167+
{
168+
"audio": "<base64-encoded audio data>",
169+
"config": {
170+
"encoding": "LINEAR_PCM",
171+
"sampleRateHertz": 16000,
172+
"languageCode": "en-US",
173+
"maxAlternatives": 1,
174+
"enableAutomaticPunctuation": true,
175+
"audioChannelCount": 1
176+
}
177+
}
178+
```
179+
- Response:
180+
```json
181+
{
182+
"results": [
183+
{
184+
"alternatives": [
185+
{
186+
"transcript": "recognized text",
187+
"confidence": 0.98
188+
}
189+
]
190+
}
191+
],
192+
"text": "recognized text",
193+
"confidence": 0.98
194+
}
195+
```
196+
197+
### WebSocket Streaming (ASR)
198+
199+
- **WebSocket** `/streaming/asr`
200+
- First message (config):
201+
```json
202+
{
203+
"sampleRate": 16000,
204+
"encoding": "LINEAR_PCM",
205+
"languageCode": "en-US",
206+
"maxAlternatives": 1,
207+
"enableAutomaticPunctuation": true
208+
}
209+
```
210+
- Subsequent messages: Binary audio data (16-bit PCM)
211+
- Server responses:
212+
```json
213+
{
214+
"results": [
215+
{
216+
"alternatives": [
217+
{
218+
"transcript": "recognized text"
219+
}
220+
]
221+
}
222+
],
223+
"isPartial": true|false
224+
}
225+
```
226+
227+
### Text to Speech (TTS)
228+
229+
- **POST** `/api/synthesize`
230+
- Request body:
231+
```json
232+
{
233+
"text": "Text to be synthesized",
234+
"voice": "en-US-Scarlett",
235+
"language": "en-US"
236+
}
237+
```
238+
- Response:
239+
```json
240+
{
241+
"audio": "<base64-encoded audio data>"
242+
}
243+
```

‎app-backend/download-protos.js

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
#!/usr/bin/env node
2+
3+
const fs = require('fs');
4+
const path = require('path');
5+
const { execSync } = require('child_process');
6+
7+
// Paths for proto files and repositories
8+
const SCRIPT_DIR = __dirname;
9+
const COMMON_DIR = path.join(SCRIPT_DIR, 'common');
10+
const PROTO_DIR = path.join(SCRIPT_DIR, 'riva/proto');
11+
12+
console.log('Proto file downloader script');
13+
console.log(`Script directory: ${SCRIPT_DIR}`);
14+
console.log(`Common repository target: ${COMMON_DIR}`);
15+
console.log(`Proto files target: ${PROTO_DIR}`);
16+
17+
// Create directories if they don't exist
18+
[COMMON_DIR, PROTO_DIR].forEach(dir => {
19+
if (!fs.existsSync(dir)) {
20+
console.log(`Creating directory: ${dir}`);
21+
fs.mkdirSync(dir, { recursive: true });
22+
}
23+
});
24+
25+
// Function to download proto files from nvidia-riva/common repository
26+
function downloadProtoFiles() {
27+
try {
28+
console.log('Checking for existing proto files...');
29+
30+
// Check if proto directory already contains proto files
31+
if (fs.existsSync(path.join(PROTO_DIR, 'riva_asr.proto')) &&
32+
fs.existsSync(path.join(PROTO_DIR, 'riva_tts.proto'))) {
33+
console.log('Proto files already exist in proto directory, skipping download');
34+
return true;
35+
}
36+
37+
// Check if common repository is already cloned
38+
const commonRepoExists = fs.existsSync(path.join(COMMON_DIR, '.git'));
39+
40+
if (!commonRepoExists) {
41+
console.log('Cloning nvidia-riva/common repository...');
42+
execSync(`git clone https://github.com/nvidia-riva/common.git ${COMMON_DIR}`, {
43+
stdio: 'inherit'
44+
});
45+
} else {
46+
console.log('Common repository already exists, pulling latest changes...');
47+
execSync(`cd ${COMMON_DIR} && git pull`, {
48+
stdio: 'inherit'
49+
});
50+
}
51+
52+
// Check if the riva/proto directory exists in the cloned repo
53+
const rivaProtoPath = path.join(COMMON_DIR, 'riva', 'proto');
54+
if (!fs.existsSync(rivaProtoPath)) {
55+
console.error(`Error: Expected directory not found: ${rivaProtoPath}`);
56+
return false;
57+
}
58+
59+
// Copy proto files to our proto directory
60+
console.log('Copying proto files to proto directory...');
61+
const protoFiles = fs.readdirSync(rivaProtoPath);
62+
63+
// Filter for relevant proto files (ASR and TTS)
64+
const relevantProtos = protoFiles.filter(file =>
65+
file.includes('riva_asr') || file.includes('riva_tts') || file.includes('riva_audio') || file.includes('riva_common')
66+
);
67+
68+
if (relevantProtos.length === 0) {
69+
console.error('No relevant proto files found in the repository');
70+
return false;
71+
}
72+
73+
// Copy each proto file
74+
relevantProtos.forEach(file => {
75+
const sourcePath = path.join(rivaProtoPath, file);
76+
const targetPath = path.join(PROTO_DIR, file);
77+
fs.copyFileSync(sourcePath, targetPath);
78+
console.log(`Copied: ${file}`);
79+
});
80+
81+
console.log('Successfully downloaded and copied proto files');
82+
console.log('Available proto files:');
83+
fs.readdirSync(PROTO_DIR).forEach(file => {
84+
console.log(`- ${file}`);
85+
});
86+
87+
return true;
88+
} catch (error) {
89+
console.error('Failed to download proto files:', error);
90+
return false;
91+
}
92+
}
93+
94+
// Execute the download function
95+
const success = downloadProtoFiles();
96+
97+
// Exit with appropriate code
98+
process.exit(success ? 0 : 1);

0 commit comments

Comments
 (0)