Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions doc/aerospike.rst
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,14 @@ Other
digest = aerospike.calc_digest("test", "demo", 1 )
pp.pprint(digest)

.. py:function:: get_partition_id(digest) -> int

Calculate the partition ID using a digest.

:param bytes-like object digest: a record digest. It can be calculated using :py:meth:`aerospike.calc_digest`.
:return: the partition ID for the digest
:rtype: :class:`int`

.. _client_config:

Client Configuration
Expand Down
2 changes: 1 addition & 1 deletion src/main/aerospike.c
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ static PyMethodDef aerospike_methods[] = {
METH_VARARGS | METH_KEYWORDS, "Calculate the digest of a key"},

//Get partition ID for given digest
{"get_partition_id", (PyCFunction)Aerospike_Get_Partition_Id, METH_VARARGS,
{"get_partition_id", (PyCFunction)Aerospike_Get_Partition_Id, METH_O,
"Get partition ID for given digest"},

{NULL}};
Expand Down
40 changes: 30 additions & 10 deletions src/main/calc_digest.c
Original file line number Diff line number Diff line change
Expand Up @@ -131,20 +131,40 @@ PyObject *Aerospike_Calc_Digest(PyObject *self, PyObject *args, PyObject *kwds)
return Aerospike_Calc_Digest_Invoke(py_ns, py_set, py_key);
}

PyObject *Aerospike_Get_Partition_Id(PyObject *self, PyObject *args)
PyObject *Aerospike_Get_Partition_Id(PyObject *self, PyObject *arg)
{
// Python Function Arguments
as_digest_value digest;
Py_buffer py_buffer;
PyObject *py_retval = NULL;

// Python Function Argument Parsing
if (PyArg_Parse(args, "(s)", &digest) == false) {
return NULL;
if (PyArg_Parse(arg, "y*", &py_buffer) == false) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To optimize, you should change to "y#" in order to avoid creating an entire py_buffer object:

Since you don't need to write to the buffer, using read only mode should be faster.
Snippet:

`
const uint8_t *buf;
Py_ssize_t len;

if (!PyArg_ParseTuple(args, "y#", &buf, &len)) {
    return NULL;  // PyArg_ParseTuple already sets an exception
}

`

Also requires changing METH_O back to METH_VARARGS.

Copy link
Collaborator Author

@juliannguyen4 juliannguyen4 Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think switching to "y#" would make get_partition_id() and calc_digest() harder to use. The user can no longer pass a digest from aerospike.calc_digest() directly into aerospike.get_partition_id(). calc_digest() returns a bytearray which is mutable, and "y#" only accepts a "read-only bytes-like object"

y# documentation: https://docs.python.org/3/c-api/arg.html#:~:text=accept%20binary%20data.-,y#,-(read%2Donly%20bytes

aerospike.calc_digest() has returned a bytearray for a while now: https://aerospike-python-client.readthedocs.io/en/8.0.0/aerospike.html#aerospike.calc_digest

To give more context, this PR is for aerospike.get_partition_id() debugging QE's bank test. (the Jira ticket explains more about what aerospike.get_partition_id() is used for) I don't believe customers are using this in production.

We could make a breaking change to aerospike.calc_digest() to return a bytes object instead of bytearray; bytes is immutable whereas the latter is mutable. But I feel like that's outside the scope of this PR

goto exit;
}

uint32_t part_id = 0;
as_error err;
as_error_init(&err);

if (py_buffer.len != 20) {
as_error_update(&err, AEROSPIKE_ERR_PARAM,
"Digest must be 20 bytes long");
goto CLEANUP_AND_EXIT;
}

part_id = as_partition_getid(digest, 4096);
uint32_t part_id = as_partition_getid(py_buffer.buf, 4096);

// Invoke Operation
return PyLong_FromLong(part_id);
py_retval = PyLong_FromLong(part_id);
if (!py_retval) {
as_error_update(&err, AEROSPIKE_ERR_CLIENT,
"Failed to retrieve partition id");
goto CLEANUP_AND_EXIT;
}

CLEANUP_AND_EXIT:
PyBuffer_Release(&py_buffer);

if (err.code != AEROSPIKE_OK) {
raise_exception(&err);
}

exit:
return py_retval;
}
25 changes: 25 additions & 0 deletions test/new_tests/test_get_partition_id.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
from aerospike import exception as e
import aerospike
import pytest

# This isn't a correctness test. It's only for code coverage purposes
# and to make sure the API is aligned with the documentation
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will QE be testing the correctness of this? Should be right, but we should confirm with testing. Should be easy to verify with AS_POLICY_KEY_DIGEST.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll just manually check using gdb. I don't think it's necessary to write a correctness test since customers shouldn't be using this (it's only meant for internal testing)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm starting to see that maybe this API call isn't necessary.. we can just use gdb to print the partition id...

class TestGetPartitionID:
def test_basic_usage(self):
digest = aerospike.calc_digest("test", "demo", 1)
part_id = aerospike.get_partition_id(digest)
assert type(part_id) == int

@pytest.mark.parametrize(
"digest, expected_exception",
[
# Digests must be exactly 20 bytes long
(bytearray([0] * 21), e.ParamError),
(bytearray([0] * 19), e.ParamError),
# Does not accept strings
("1" * 20, TypeError)
]
)
def test_invalid_digest(self, digest, expected_exception):
with pytest.raises(expected_exception):
aerospike.get_partition_id(digest)
Loading