Port paginate json docs tests to Elixir by AlbaHerrerias · Pull Request #19 · neighbourhoodie/couchdb

AlbaHerrerias · 2025-11-14T16:37:36Z

Overview

Testing recommendations

Related Issues or Pull Requests

Checklist

Code is written and works correctly
Changes are covered by tests
Any new configurable parameters are documented in rel/overlay/etc/default.ini
Documentation changes were made in the src/docs folder
Documentation changes were backported (separated PR) to affected branches

refactor(basic-find-test): add sort to _find calls for limit and skip options

test/elixir/test/mango/14_json_pagination_test.exs

… depends on

…m Python to Elixir

… value

Support safe secret rotation

@janl

Credits to @janl who identified the need for this change

Updated test for SSL support to reflect API changes

These started to become a bit more flaky after the apache#5754 pr merged.

This helps with startup race conditions in the test suite where the secret is set in config but the gen_server in couch_secrets hasn't received it via config_change callback yet.

use secret from config if ETS not populated

Previously, when the purge checkpoints were first created concurrently with compaction running, it was possible for compaction to finish first and remove too many purge infos before the internal replicator checkpointed. In that case we could end up with a "hole" between a minimum (checkpointed) purge sequence, and the oldest purge sequence. Subsequently, internal replicator would start crashing since when fetching the minimum purge sequence it will correctly detect that one of the purge clients is asking for a sequence that's too low (that is it "skipped" and hasn't processed intermediate purge sequences). The tell-tale sign of this in production is repeated `invalid_start_purge_seq` errors emitted in the logs. One way to get out of would be to delete the checkpoints docs and let them be re-created. To fix the race condition, when compaction starts check if all the expected checkpoints from the other shard copies are created first, and only then use the minimum version, otherwise use the oldest purge sequence version.

This allows debugging of test runs in a unified log. It also retains individual test run state dirs if `make eunit` fails. The next successful run will clean it up

feat: concat eunit test logs and remove test tmp dirs in Makefile

Noticed a few timeouts running eunit -j8 tests

The main improvement is to separate the per-app makefile targets early, during templating phase, so they never share a common app structure during runtime. That lets us avoid reconfiguring the data directory at runtime, which effectively is an almost full restart internally since we're tearing down all the couch servers. So we're saving an extra app shutdown + restart cycle per app with the new scheme and improving the isolation. The new scheme uses the `tmp/$app` namespace for each app. After the tests run any unsuccessful `tmp/$app` directories will be left behind so they can be inspected. This is especially useful for looking at the failed couch.log entries. There is a separate catlogs target to concatenate failed app couch.log files though maybe keeping them separately may be even easier to inspect. Since if app a failed, we'd just rather look at app a's logs and not app c's logs. Since setup_eunit rebar template now takes an app argument, opted to remove some older eunit targets keeping only the new ones supporting parallel execution. Otherwise, would have to add a special case statement in the plugin for no-app vs one app cases. Previously, `make eunit apps=a,b,c` did not run apps `a`, `b` and `c` as separate goals in parallel execution. They were passed in as the `a,b,c` goal. To fix that parse the apps into individual space separated goals, so they can be sped up by parallel execution as well. To keep the apps as isolated from each other as possible make build_eunit_config also create separate etc folders, use separate BUILDDIR macros and each apps only uses its own fixtures. Also use 0-valued ports the eunit template to avoid any chance of port collision and resulting flakiness. Rename SUBDIRS to EUNIT_SUBDIRS since they used for eunit tests currently only. Also, exclude docs and fauxton apps from the target list, since they are not proper erlang apps and don't have eunit tests. Newer Erlangs have `file:del_dir_r/1` so use that in cleanup_dirs to save a few lines of code.

Previous PR [1] failed to account for shards db itself. Shards db (`_dbs`) is managed differently than regular shard copies. Its `mem3:shards(Dbs)` result is a single element shard list with a `#shard{}` having `node = node()` and `range = [0, ff..]`. They are replicated in a ring across all nodes, we expect to find a purge checkpoint pushing changes to the "next" node in a ring only. [1] apache#5827

In the previous PR [1] we added special handling for shards dbs since it has a custom ring replication topology. In PR [1] we considered only the checkpoint for pushes from the current node to the "next" none in the ring. However, we should also consider the checkpoints created by the "previous" node when it pull purges from current node, so that what we fix in this PR. As a reminder, a replication job from node A to node B will: 1) pull purges from B to A (checkpoint on B with a B->A purge checkpoint doc) 2) push purges from A to B (checkpoint on A with a A->B purge checkpoint doc) 3) push do updates from A to B (checkpoint on A with A->B sync checkpoint doc, and on B also with an A->B sync checkpoint doc) [1] apache#5832

* Add regexp duplicate named groups bellard/quickjs@24379bf * Fix fast array extension optimization when there are multiple realms bellard/quickjs@e5fd391 * Remove memory leak in case of error in `cpool_add()` bellard/quickjs@fcd33c1 * Remove use after free in `js_create_module_bytecode_function()` bellard/quickjs@1dbba8a * Don't call well-known Symbol methods for RegExp on primitive values bellard/quickjs@c73a435 * Slightly faster lexical variable assignment bellard/quickjs@31ef02b * `\x{N}` is a syntax error bellard/quickjs@7bd1ae2 * Removed `alloca()` is `lre_exec()` bellard/quickjs@f113949

The issues is that homebrew now sometimes prints download metadata to stderr that we are not expecting when we run `brew prefix`. Filtering this out should help for now. Note that the issue is that this happens despite the `--quiet` option present, hence the redirect workaround. Homebrew/brew#20976

re-add macOS CI

These are to help the users understand how it works so they can use it more effectively. It includes some examples, and two rendered histogram images.

Normally we don't run our eunit or ci tests as root but rarely it happens, such as when when Ronny was investing ARM64 perf issues on one of the CI nodes. There we noticed two smoosh state persisting tests failed. In order to test lack of fs access, those tests set up a file so it's unreadable or unwritable by current user. That doesn't work, however, if that user is root or admin, as they can't remove their own access with plain chmods. So add a quick exception for those cases to skip part of the tests if the setup cannot be accomplished as planned.

If we get a node down plus a node in maintenance mode, and we just figured out we can't make progress, try to return any existing results if we got them, instead of always bailing with the last error (an mm or nodedown).

jcoglan force-pushed the port-python-tests branch from 9d983c3 to edf35c0 Compare November 17, 2025 13:34

hulkoba force-pushed the port-python-tests branch 8 times, most recently from c413a13 to 25f1f55 Compare November 19, 2025 14:21

ninetteadhikari added the ready for review label Nov 20, 2025

ninetteadhikari added 2 commits November 21, 2025 13:11

refactor(basic-find-test): Port test for basic-find-test to elixir

7f2d9c7

refactor(basic-find-test): add sort to _find calls for limit and skip options

refactor: remove python test basic-find-test

672a8c6