Dialog/lines

[dmytrove]

Voice Templates

The engine provides character-specific voice templates that generate synthesized audio during dialog text rendering. Each voice has distinct audio characteristics to match different character archetypes.

Available Voice Templates

Template	Frequency	Waveform	Character	Audio Description
HUMAN	800Hz	Sine	Default human	Clean, natural tone
ROBOT	100Hz	Sawtooth	Mechanical being	Deep, synthetic, gentle
GHOST	1200Hz	Sine	Ethereal spirit	High-pitched, haunting
MONSTER	300Hz	Sawtooth	Creature/beast	Low, growling tone
ALIEN	1000Hz	Triangle	Extraterrestrial	Sharp, otherworldly
NARRATOR	600Hz	Sine	Storyteller	Smooth, authoritative
RANDOM	Variable	Variable	Dynamic character	Random variation per character

Voice Configuration

type VoiceConfig = {
    template?: 'HUMAN' | 'ROBOT' | 'GHOST' | 'MONSTER' | 'ALIEN' | 'NARRATOR' | 'RANDOM'
    seed?: number | null
}

Seed Behavior:

• seed: 42 - Fixed seed for consistent sound
• seed: null - Random seed per dialog (consistent within dialog)
• seed: undefined - Uses template default
• template: 'RANDOM' - New random seed for each character

Dialog Formats

Simple String Dialog

templates: {
    'npc': {
        dialog: "Hello, player!"
    }
}

Dialog Array Format

For multi-speaker conversations with voice assignment:

// Speaker configuration
speakers: {
    'hero': { voice: { template: 'HUMAN' } },
    'villain': { voice: { template: 'MONSTER' } }
}

// Dialog array
templates: {
    'cutscene': {
        dialog: [
            ['hero', 'I must stop you!'],
            ['villain', 'You cannot defeat me!'],
            ['hero', 'We shall see about that.']
        ]
    }
}

Voice Assignment Priority

1. Voice Override (passed to openDialog)
2. Speaker Configuration (from speakers object)
3. Actor Default Voice (from actor template)
4. No Voice (silent)

API Usage

Game Configuration

createGame({
    speakers: {
        'narrator': { voice: { template: 'NARRATOR' } },
        'robot': { voice: { template: 'ROBOT' } }
    },
    templates: {
        'character': {
            dialog: "I have a voice!",
            voice: { template: 'HUMAN', seed: 42 }
        }
    }
})

Runtime Dialog

// Use actor's default voice
game.openDialog("Hello world!")

// Override with specific voice
game.openDialog("Beep boop!", { template: 'ROBOT' })

// Dialog array with speaker voices
game.openDialog([
    ['narrator', 'Once upon a time...'],
    ['robot', 'SYSTEM INITIALIZED']
])

Actor Templates with Voices

templates: {
    'robot': {
        sprite: 'R',
        dialog: "I AM A ROBOT",
        voice: { template: 'ROBOT' }
    },
    'ghost': {
        sprite: 'G', 
        dialog: "Boooooo...",
        voice: { template: 'GHOST', seed: null } // Random per dialog
    }
}

Advanced Voice Features

Variation Modes

// Fixed sound - same every time
voice: { template: 'HUMAN', seed: 42 }

// Random per dialog - consistent within conversation
voice: { template: 'HUMAN', seed: null }

// Random per character - different for each letter
voice: { template: 'RANDOM' }

Voice Overrides

// Override actor's default voice
game.openDialog("Special announcement!", { template: 'NARRATOR' })

// Override speaker voice in dialog array
game.openDialog([
    ['character', 'Normal voice'],
    ['character', 'Robot voice!']
], null, { template: 'ROBOT' })

Technical Details

Audio Generation

• Pre-generated audio buffers for performance
• 5 frequency variations per template (±100Hz range)
• 15ms throttling between voice sounds
• Independent volume control for voices
• Browser autoplay policy handling

Performance

• Voice sounds are synthesized at initialization
• No real-time audio generation during gameplay
• Optimized for rapid character-by-character playback
• Separate audio context gain node for voice isolation

Browser Compatibility

• Requires Web Audio API support
• Handles audio context suspension/resumption
• Graceful fallback for audio failures
• No external audio file dependencies

[achtaitaipai]

Thank's, it's amazing! I'm going to merge it tomorrow!

[achtaitaipai]

Thanks again! I would’ve never come up with this feature myself, and now it already feels essential!

I've made two modifications to the voice system:

1. The voice configuration now mirrors the sound syntax for consistency.
   You can now use:

   • voice: 'ROBOT'
   • voice: ['ROBOT', 33]

   Same goes for openDialog:

   • game.openDialog('Hello', 'ROBOT')
   • game.openDialog('Hello', ['ROBOT', 44])

2. Temporary removal of the speakers system
   I've removed the speakers mapping for now, as I plan to reintroduce it with a more flexible syntax, inspired by dialog effects.
   For example:
   dialog: "I'm <MONSTER>angry<MONSTER>"

This would allow inline voice changes, giving more control over character expression within a single line of dialog.

Let me know if that all sounds good to you, if so, I'll go ahead and merge it!

[achtaitaipai]

This also made me realize I should update pfxr to support AudioBuffer directly.
That way, I could move the voice logic into pfxr and maybe even pave the way for adding a music system later on!

[achtaitaipai]

@dmytrove does this work for you as a temporary solution?