question about callback api

[shakfu]

Hi, first of all, thanks for sharing your excellent project. There's really is nothing as comprehensive out there!

I have a question about the callback api since I'm in the midst of a preliminary effort to try to cythonize libpd and enable sound input/ouput via portaudio.

Your example, 'test_playback.py', which I got to work, shows how numpy generated data can be use to generate sound.

In my case, I have an libpd api sound generation function libpd_process_double which I think I have to embed in the portaudio callback as follows:

cimport libpd
cimport libportaudio
from cpython cimport array

from libc.stdio cimport printf, fprintf, stderr, FILE

DEF N_TICKS = 1
DEF SAMPLE_RATE = 44100
DEF CHANNELS_IN = 1
DEF CHANNELS_OUT = 2
DEF BLOCKSIZE = 64
DEF IN_BUF = CHANNELS_IN * BLOCKSIZE
DEF OUT_BUF = CHANNELS_OUT * BLOCKSIZE


cdef struct UserAudioData:
    # one input channel, two output channels
    double inbuf[N_TICKS * BLOCKSIZE * CHANNELS_IN]
    # block size 64, one tick per buffer
    double outbuf[N_TICKS * BLOCKSIZE * CHANNELS_OUT]
    # stereo outputs are interlaced, s[0] = RIGHT, s[1] = LEFT, etc..

# globals
cdef UserAudioData data

cdef int audio_callback(const void *inputBuffer, void *outputBuffer,
    unsigned long framesPerBuffer,
    const libportaudio.PaStreamCallbackTimeInfo* timeInfo,
    libportaudio.PaStreamCallbackFlags statusFlags,
    void *userData ):
    """Called by the PortAudio engine when audio is needed.
    
    It may called at interrupt level on some machines so don't do anything
    that could mess up the system like calling malloc() or free().
    """
    # Cast data passed through stream to our structure.
    cdef UserAudioData *data = <UserAudioData*>userData
    cdef float *out = <float*>outputBuffer
    cdef unsigned int i;
    
    libpd.libpd_process_double(N_TICKS, data.inbuf, data.outbuf)
    
    # dsp perform routine
    for i in range(framesPerBuffer * CHANNELS_OUT):
        if (i % 2):
            out[i] = data.outbuf[i]
        else:
            out[i] = data.outbuf[i]
    return 0

Any attempts to have libpd_process_double write to UserAudioData outside the callback just doesn't work properly in this case.

I would love to use your library which has a much more robust cythonized wrapping of the portaudio api. However I can't decide if I need to write my own version of your <PaStreamCallback*>_stream_callback which is handled by StreamCallback to address my use case. I would appreciate any advice to this end.

Thanks in advance for any help!

S

[nocarryr]

Thanks for your interest in this little project. I actually just created it for my own use and decided to put it out there, so I'm glad you found it useful.

The PortAudio callback is quite picky, which is why I chose to use the buffering mechanisms. Even with Cython, you pretty much have to treat it as though it's completely isolated from other code, so if your libpd.libpd_process_double() does any memory allocations or requires the GIL, it won't work within the callback.

Is it possible to process at least one frame behind PortAudio, or do you need it to be completely real-time?

Also, sorry for the delay in response. I was out on vacation.

[nocarryr]

Oh, one suggestion... Try using the decorators:

@cython.wraparound(False)
@cython.boundscheck(False)

And mark the function with nogil if possible.

Plus, it looks like you're mixing double and float types in the out and data.outbuf variables.

(Ok, that was three suggestions)

[shakfu]

Thanks very much for your response. Perhaps I can clarify things a bit more:

The original c function libpd_process_double from libpd is defined via a macro as follows in https://github.com/libpd/libpd/blob/master/libpd_wrapper/z_libpd.c#L172 which has some locking calls and a memset call. Not sure if that answers your question:

#define PROCESS(_x, _y) \
  int i, j, k; \
  t_sample *p0, *p1; \
  sys_lock(); \
  sys_pollgui(); \
  for (i = 0; i < ticks; i++) { \
    for (j = 0, p0 = STUFF->st_soundin; j < DEFDACBLKSIZE; j++, p0++) { \
      for (k = 0, p1 = p0; k < STUFF->st_inchannels; k++, p1 += DEFDACBLKSIZE) \
        { \
        *p1 = *inBuffer++ _x; \
      } \
    } \
    memset(STUFF->st_soundout, 0, \
        STUFF->st_outchannels*DEFDACBLKSIZE*sizeof(t_sample)); \
    SCHED_TICK(pd_this->pd_systime + STUFF->st_time_per_dsp_tick); \
    for (j = 0, p0 = STUFF->st_soundout; j < DEFDACBLKSIZE; j++, p0++) { \
      for (k = 0, p1 = p0; k < STUFF->st_outchannels; k++, p1 += DEFDACBLKSIZE) \
        { \
        *outBuffer++ = *p1 _y; \
      } \
    } \
  } \
  sys_unlock(); \
  return 0;

//...

int libpd_process_double(const int ticks, const double *inBuffer, double *outBuffer) {
  PROCESS(,)
}

This function was simply used in the code from my initial post after cimporting from libpd.pxd

cdef extern from "../libpd_wrapper/z_libpd.h":

# ...

    # process interleaved double samples from inBuffer -> libpd -> outBuffer
    # buffer sizes are based on # of ticks and channels where:
    #     size = ticks * libpd_blocksize() * (in/out)channels
    # returns 0 on success
    int libpd_process_double(const int ticks, const double *inBuffer, double *outBuffer)

Just to be clear, I was able to successfully generate sound as expected using the code in the initial post, and this without any decorators or using the nogil suffix in the callback function definition. I just intuitively assumed that it was suboptimal and hence my search for a more robust portaudio wrapping code and my question on how to properly define the portaudio callback in your code base.

If you did not intend (as per your use cases) to to have a replaceable callback then that's fine too and I can tweak my original code as per your recommendation (decorator and nogil) in case I have any issues going forward.

S

[nocarryr]

I'm surprised that it worked successfully, even with what looks like a tick to the UI loop, but if it works, it works!

I did originally intend to have a functioning callback override mechanism and still do. It just hasn't been implemented yet.

With that in mind, I don't think it would make things any easier to use this project as opposed you took initially, but if I think of anything, I'll let you know.

[shakfu]

Yes. if it works, then that's something (-:

I'll keep at it and experiment some more with some more extreme cases.

Thanks again for your help and for sharing your project!