support C buffer protocol

[teepark]

unfortunately it changed dramatically between python 2 and 3, but there is a protocol by which objects implemented in C can support memory sharing. this could be used with file objects for example to dump a json stream straight to the file object's buffer rather than creating a PyString and calling .read() through python.

this would need to fall back to current behavior as pure-python file-like objects should be supported.

see http://docs.python.org/release/2.6.5/c-api/typeobj.html#buffer-object-structures and http://docs.python.org/release/2.6.5/c-api/objbuffer.html

[rtyler]

I don't entirely understand what supporting buffer protocols would look like within py-yajl? Could you include some psuedo-code of an API you'd like?

(I'm woefully unfamiliar with the new fancy things in Python 3)

[teepark]

it appears the 2.x buffer protocol is actually still in python 3. http://docs.python.org/py3k/c-api/objbuffer.html

the modifications to load() are much easier. in _internal_stream_load:

if (PyObject_AsCharBuffer(stream, &char_ptr, &length) {
  /* clear the exception and proceed with .read() as before */
} else {
  /* jump down to the _internal_decode() call, you've got your input and its length */
}

I jumped around a bit through encoder.c and realized it would take some different py_yajl_printer plumbing as it really expects to be dumping to a string/bytes.

EDIT: ah i misread about the pseudo-code you were asking for. this is just an idea for an optimization, it wouldn't necessarily change the API.