Release 3.0.9 (#1084)

* Logging of 2i query performance

* Update deps

* Add release notes

Initial preparation for Riak 3.0.9 release

Author: martinsumner

RELEASE-NOTES.md

@@ -1,3 +1,17 @@
+# Riak KV 3.0.9 Release Notes
+
+This release contains stability, monitoring and performance improvements.
+
+- Fix to the [riak_core coverage planner](https://github.com/basho/riak_kv/issues/1801) to significantly reduce the CPU time required to produce a coverage plan, especially with larger ring sizes.  This improves both the mean and tail-latency of secondary index queries.  As a result of this change, it is recommended that larger ring sizes should be used by default, even when running relatively small clusters - for example in standard volume tests a ring size of 512 is outperforming lower ring sizes even on small (8-node) clusters.
+
+- Further monitoring stats have been added to track the performance of coverage queries, in particular secondary index queries.  For each worker queue (e.g. vnode_worker_pool, af1_pool etc) the queue_time and work_time is now monitored with results available via riak stats.  The result counts, and overall query time for secondary index queries are now also monitored via riak stats. See the [PR](https://github.com/basho/riak_kv/pull/1802#issuecomment-966160925) for a full list of stats added in this release.
+
+- Change to some default settings to be better adapted to running with higher concentrations of vnodes per nodes.  The per-vnode cache sizes in leveled are reduced, and the [default size of the vnode_worker_pool has been reduced from 10 to 4](https://github.com/basho/riak_kv/commit/39e3d84082d2144ab300c0a6250b8ec2e9e8db65) and is now configurable via riak.conf.  Exceptionally heavy users of secondary index queries (i.e. > 1% of transactions), should consider monitoring the [new queue_time and work_time statistics](https://github.com/basho/riak_kv/pull/1802#issuecomment-966160925) before accepting this new default.
+
+- Fix to an issue in the leveled backend when a key and (internal) sequence number would [hash to 0](https://github.com/martinsumner/leveled/issues/361).  It is recommended that users of leveled uplift to this version as soon as possible to reoslve this issue.  The risk is mitigated in Riak as it can generally be expected that most keys will have different sequence numbers in different vnodes, and will always have different sequence numbers when re-written - so normally anti-entropy process will recover from any occurrence.
+
+- More time is now given to the legacy AAE `kv_index_hashtree` process to [shut down](https://github.com/basho/riak_kv/pull/1803), to handle delays as multiple vnodes are shutdown concurrently and contend for disk and CPU resources.
+
 # Riak KV 3.0.8 Release Notes
 
 This release contains a number of stability  improvements.

eunit.log

@@ -11,13 +11,13 @@
 {deps, [
         {lager_syslog, {git, "https://github.com/basho/lager_syslog.git", {tag, "3.1.0"}}},
         {cluster_info, {git, "https://github.com/basho/cluster_info.git", {tag, "2.1.0"}}},
-        {riak_kv, {git, "https://github.com/basho/riak_kv.git", {tag, "riak_kv-3.0.8"}}},
+        {riak_kv, {git, "https://github.com/basho/riak_kv.git", {branch, "develop-3.0"}}},
         {riak_auth_mods, {git, "https://github.com/basho/riak_auth_mods.git", {tag, "riak_kv-3.0.0"}}},
         {riaknostic, {git, "https://github.com/basho/riaknostic.git", {tag, "riak_kv-3.0.2"}}},
 
         % {yokozuna, {git, "https://github.com/basho/yokozuna.git", {branch, "develop-3.0"}}},
 
-        {riak_repl, {git, "https://github.com/basho/riak_repl.git", {tag, "riak_kv-3.0.8"}}}
+        {riak_repl, {git, "https://github.com/basho/riak_repl.git", {branch, "develop-3.0"}}}
       ]}.
 
 {project_plugins, [
@@ -46,7 +46,7 @@
     ]}
 ]}.
 
-{relx, [{release, {riak, "3.0.8"},
+{relx, [{release, {riak, "3.0.9"},
     [kernel,
      stdlib,
      lager,

rebar.config

@@ -53,7 +53,7 @@
   2},
  {<<"kv_index_tictactree">>,
   {git,"https://github.com/martinsumner/kv_index_tictactree.git",
-       {ref,"1c297426e86542fcb9d2dd61400853a42a6e104d"}},
+       {ref,"e77b381e5c1e608b15756b9f183ac2b364201749"}},
   1},
  {<<"lager">>,
   {git,"git://github.com/erlang-lager/lager.git",
@@ -65,7 +65,7 @@
   0},
  {<<"leveled">>,
   {git,"https://github.com/martinsumner/leveled",
-       {ref,"8fa373d367e33179da12fd0b7fb1ca8ddce23ffd"}},
+       {ref,"086e06979e6f6d0d37b15df883c0ce2eb3c15892"}},
   2},
  {<<"lz4">>,
   {git,"https://github.com/martinsumner/erlang-lz4",
@@ -106,15 +106,15 @@
   1},
  {<<"riak_api">>,
   {git,"https://github.com/basho/riak_api.git",
-       {ref,"4c1dbe06c31c93b6947ec9ab0150ea668811f726"}},
+       {ref,"11e23d47f25cf01648e2219392a30e95f37162e1"}},
   1},
  {<<"riak_auth_mods">>,
   {git,"https://github.com/basho/riak_auth_mods.git",
        {ref,"1eccaef8e9e6200f0b75ea9312f0a731c8d572bf"}},
   0},
  {<<"riak_core">>,
   {git,"https://github.com/basho/riak_core.git",
-       {ref,"b36a7aedcc152b3070748972730fb2461ab49c14"}},
+       {ref,"7e0aa310ecf5ec7b9fd03df61b996c2823591950"}},
   1},
  {<<"riak_dt">>,
   {git,"https://github.com/basho/riak_dt.git",
@@ -126,19 +126,19 @@
   2},
  {<<"riak_kv">>,
   {git,"https://github.com/basho/riak_kv.git",
-       {ref,"ef092fb28d1cf32cf10cd61a4ceb3d4467701c62"}},
+       {ref,"44849c1bee170804d68df7ad9d244dc89236db7b"}},
   0},
  {<<"riak_pb">>,
   {git,"https://github.com/basho/riak_pb.git",
        {ref,"b7abca90e4c708073021d2c4e18c896f1a10b838"}},
   2},
  {<<"riak_pipe">>,
   {git,"https://github.com/basho/riak_pipe.git",
-       {ref,"8dd2a6352e58a7d8215ba5afad4cf7b9e4807714"}},
+       {ref,"229965f8c90ff2168800a68b8fb47fa44722f67d"}},
   1},
  {<<"riak_repl">>,
   {git,"https://github.com/basho/riak_repl.git",
-       {ref,"89ffda17e1fcd4f596a5c72e4ff6fc9405616327"}},
+       {ref,"0e0ed28210b4720a99475802e79bb8eae999d946"}},
   0},
  {<<"riak_sysmon">>,
   {git,"https://github.com/basho/riak_sysmon.git",