MLE-22707 : Vector base64 encode and decode functions in the MarkLogic Node.js client #938

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

anu3990 merged 1 commit into marklogic:develop from anu3990:june-27

Jul 17, 2025

Contributor

anu3990 commented Jul 16, 2025

No description provided.

anu3990 requested review from BillFarber, Copilot and rjrudin

July 16, 2025 19:10

Copilot AI reviewed

View reviewed changes

Copilot AI left a comment

Pull Request Overview

This PR implements vector base64 encoding and decoding functionality for the MarkLogic Node.js client. The changes add client-side utilities to convert between floating-point vector arrays and base64-encoded binary representations, enabling interoperability with MarkLogic's server-side vector functions.

Added vector-util.js library with base64Encode and base64Decode functions
Implemented comprehensive test suite covering client-side encoding/decoding and cross-compatibility with server-side functions
Added validation for unsupported vector versions in the decode function

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 10 comments.

File	Description
`lib/vector-util.js`	Core implementation of vector base64 encoding/decoding utilities with buffer manipulation
`test-basic/vector-util-test.js`	Complete test suite validating encoding/decoding accuracy and server-client interoperability

test-basic/vector-util-test.js Outdated

+                      const input = 'AAAAAAMAAADD9UhAH4XLP5qZKUA=';
+                      const decoded = vectorUtil.base64Decode(input);
+                      try {
+                          assert(Object.getPrototypeOf(decoded) == Array.prototype);

Copilot AI Jul 16, 2025

Use Array.isArray(decoded) instead of Object.getPrototypeOf(decoded) == Array.prototype for more reliable array type checking.

Suggested change

      
                        assert(Object.getPrototypeOf(decoded) == Array.prototype);
          
                        assert(Array.isArray(decoded));

Copilot uses AI. Check for mistakes.

test-basic/vector-util-test.js Outdated

+                              const input = response.rows[0].t.value;
+                              const decoded = vectorUtil.base64Decode(input);
+                              assert(input =='AAAAAAEAAABvEgM7');
+                              assert(Object.getPrototypeOf(decoded) == Array.prototype);

Copilot AI Jul 16, 2025

Use Array.isArray(decoded) instead of Object.getPrototypeOf(decoded) == Array.prototype for more reliable array type checking.

Suggested change

      
                            assert(Object.getPrototypeOf(decoded) == Array.prototype);
          
                            assert(Array.isArray(decoded));

Copilot uses AI. Check for mistakes.

test-basic/vector-util-test.js Outdated

+                          .then(function(response) {
+                              const input = response.rows[0].t.value;
+                              const decoded = vectorUtil.base64Decode(input);
+                              assert(input =='AAAAAAEAAABvEgM7');

Copilot AI Jul 16, 2025

Use assert.strictEqual() instead of loose equality comparison for more precise testing.

Suggested change

      
                            assert(input =='AAAAAAEAAABvEgM7');
          
                            assert.strictEqual(input, 'AAAAAAEAAABvEgM7');

Copilot uses AI. Check for mistakes.

test-basic/vector-util-test.js Outdated

Comment on lines 50 to 53

+                          assert(Object.getPrototypeOf(decoded) == Array.prototype);
+                          assert(decoded[0] == 3.140000104904175);
+                          assert(decoded[1] == 1.590000033378601);
+                          assert(decoded[2] == 2.6500000953674316);

Copilot AI Jul 16, 2025

Use assert.strictEqual() instead of loose equality comparison for more precise testing.

Suggested change

      
                        assert(Object.getPrototypeOf(decoded) == Array.prototype);
          
                        assert(decoded[0] == 3.140000104904175);
          
                        assert(decoded[1] == 1.590000033378601);
          
                        assert(decoded[2] == 2.6500000953674316);
          
                        assert(Object.getPrototypeOf(decoded) === Array.prototype);
          
                        assert(Math.abs(decoded[0] - 3.140000104904175) < delta, 'Value mismatch for decoded[0]');
          
                        assert(Math.abs(decoded[1] - 1.590000033378601) < delta, 'Value mismatch for decoded[1]');
          
                        assert(Math.abs(decoded[2] - 2.6500000953674316) < delta, 'Value mismatch for decoded[2]');

Copilot uses AI. Check for mistakes.

test-basic/vector-util-test.js Outdated

Comment on lines 51 to 53

+                          assert(decoded[0] == 3.140000104904175);
+                          assert(decoded[1] == 1.590000033378601);
+                          assert(decoded[2] == 2.6500000953674316);

Copilot AI Jul 16, 2025

Use assert.strictEqual() instead of loose equality comparison for more precise testing.

Suggested change

      
                        assert(decoded[0] == 3.140000104904175);
          
                        assert(decoded[1] == 1.590000033378601);
          
                        assert(decoded[2] == 2.6500000953674316);
          
                        assert.strictEqual(decoded[0], 3.140000104904175);
          
                        assert.strictEqual(decoded[1], 1.590000033378601);
          
                        assert.strictEqual(decoded[2], 2.6500000953674316);

Copilot uses AI. Check for mistakes.

test-basic/vector-util-test.js Outdated

+                          assert(Object.getPrototypeOf(decoded) == Array.prototype);
+                          assert(decoded[0] == 3.140000104904175);
+                          assert(decoded[1] == 1.590000033378601);
+                          assert(decoded[2] == 2.6500000953674316);

Copilot AI Jul 16, 2025

Use assert.strictEqual() instead of loose equality comparison for more precise testing.

Suggested change

      
                        assert(decoded[2] == 2.6500000953674316);
          
                        assert(Math.abs(decoded[2] - 2.6500000953674316) < delta, 'Value mismatch for decoded[2]');

Copilot uses AI. Check for mistakes.

test-basic/vector-util-test.js Outdated

Comment on lines 70 to 71

		assert(Object.getPrototypeOf(decoded) == Array.prototype);
		assert(decoded[0] == 0.0020000000949949026);

Copilot AI Jul 16, 2025

Use assert.strictEqual() instead of loose equality comparison for more precise testing.

Suggested change

      
                            assert(Object.getPrototypeOf(decoded) == Array.prototype);
          
                            assert(decoded[0] == 0.0020000000949949026);
          
                            assert(Object.getPrototypeOf(decoded) === Array.prototype);
          
                            assert(Math.abs(decoded[0] - 0.0020000000949949026) < delta, 'Value mismatch');

Copilot uses AI. Check for mistakes.

test-basic/vector-util-test.js Outdated

+                      const input = vectorUtil.base64Encode(vector);
+                      const vectorString = '[ '+vector[0].toString()+', '+vector[1].toString()+', '+vector[2].toString()+' ]';
+                      dbWriter.eval(`vec.base64Decode('${input}')`).result(res=>{
+                          assert(res[0].value == vectorString);

Copilot AI Jul 16, 2025

Use assert.strictEqual() instead of loose equality comparison for more precise testing.

Suggested change

      
                        assert(res[0].value == vectorString);
          
                        assert.strictEqual(res[0].value, vectorString);

Copilot uses AI. Check for mistakes.

lib/vector-util.js

+              'use strict';
+              const { Buffer } = require('buffer');
+              const base64Encode = (vector) => {

This comment was marked as off-topic.

Sign in to view

lib/vector-util.js

		};

		const base64Decode = (encodedVector) => {

This comment was marked as off-topic.

Sign in to view

rjrudin reviewed

View reviewed changes

Contributor

rjrudin left a comment

Looks good, though I think all the Copilot suggestions are good ones, especially the ones to assert using "strict equals".


          MLE-20906 : Update Node Client based on changes to functions in Optic

8fa1138

anu3990 force-pushed the june-27 branch from d80ad2f to 8fa1138 Compare

July 16, 2025 21:16

anu3990 requested a review from rjrudin

July 16, 2025 21:17

rjrudin approved these changes

View reviewed changes

anu3990 merged commit c75be94 into marklogic:develop

1 of 2 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet