v21.1.4

release 21.1.4

Major release:
This work includes 420 commits.

Major features:

* Initial feature of coprocessor with coproc wasm (WASM) feature
  exposed under `rpk wasm {deploy,remove}` commands.
  We note that this feature while in the tree is marked experimental.
  Documentation to follow.

* Initial feature of archival (s3) backups.
  We landed the metadata description and s3 archival functions
  alongside all of the secure hashing schemes needed by s3.

* Cross segmen compaction finally landed in tree which allows for
  compaction of adjacent segments. with an initially exposed max file handle size of 5G
  for adjacent segment compaction

* RPK landed a huge number of bugfixes and simplifications, most notably namespacing
  the new namespacing is
  rpk {redpanda, debug, topic} - which should hopefully allow users to
  understand RPK better.

* First patch of idempotent producer also landed with full compatibility
  targetted for the next release :) - We note that this is the first step in turn
  to support transactions.

* A new `kafka::client` written in seastar was ported out of the http proxy
  into the top level kafka namespace

* First pass for live partition migration landed, which included node id
  virtualization. Need to hook this up to the `controller backend` to
  expose it to users.

* This build includes a HUGE amount of performance improvements where
  we can now gurantee hardware saturation in all modes of acknowledgement.

* This change also comes with priority-based raft leader election, among
  RPC reordering optimization, amortized flushes by a 'look-ahead' flush
  on the followers. The TCP clients were increased to 8 from 3 to allow
  for saturation on large machines.

* Added GCP specific tuner to `rpk tuner` to maximize throughput on
  GCP after chatting with the storage team at google

Notable bug fixes:

* rpk is effectively revamped and should be much less buggy from tuners
  to logging, to namespacing, to containers and configuration re-write

* graceful recovery of an administration removing a directory manually
  on /var/lib/redpanda/data/*