elasticsearch: install Java first on CentOS too

elasticsearch-oss package fails to install if we install it with Java:

INFO:kolla.common.utils.elasticsearch:could not find java; set JAVA_HOME
INFO:kolla.common.utils.elasticsearch:error: %prein(elasticsearch-oss-0:6.8.23-1.noarch) scriptlet failed, exit status 1
INFO:kolla.common.utils.elasticsearch:�[91mError in PREIN scriptlet in rpm package elasticsearch-oss

Backport down to ussuri needed.

This change also includes a backport of
I7d4788f5b63fba24e3b2f9b15c16866ff811d83e:

Always use the distro-provided libvirt-python

This patch switches masakari-monitors image to follow nova-compute
and ceilometer-compute images.
This will be used and required after [1] merges.
Usage of libvirt-python from PyPI has already proven to be
problematic on CentOS Stream in our stable branches. [2]
With this patch we avoid those issues as well.

[1] https://review.opendev.org/c/openstack/masakari-monitors/+/804913
[2] https://review.opendev.org/c/openstack/kolla/+/797102

Depends-On: https://review.opendev.org/c/openstack/masakari-monitors/+/804913

NOTE(hrw): added removal of libvirt-python from upper-constraints as we have
never version of libvirt that Victoria used.

This change also includes a backport of
I5efab66e487e06abd1a56af97d7e7caa1ebc880d:

Use jinja2.pass_context instead of contextfilter

The contextfilter decorator was deprecated in jinja2 3.0.0, and has been
dropped in 3.1.0. This results in the following warning, and failed
attempts to use filters:

    [WARNING]: Skipping plugin (filters.py) as it seems to be invalid:
    module 'jinja2' has no attribute 'contextfilter'

This change switches to use the pass_context decorator. The minimum
version of Jinja2 is raised to 3 to ensure pass_context is present.

This change has been updated to also support Jinja2 2.x releases,
since the Wallaby upper constraints specify 2.11.3. In practice, most
users will not use UC to install kolla.

CoAuthored-by: Mark Goddard <[email protected]>

Finally, amended for Debian Buster to also skip the lower
constraint from masakari-monitors.

Finally, finally, includes also hotfix for contextfunction.
Ie4bccb9ed3f4d8782730c5929abbfa73da215a8c

Change-Id: I72d7920acd8d15941c8c57a4186186212b273a38
(cherry picked from commit 976465c)

Author: Marcin Juszkiewicz

docker/elasticsearch/elasticsearch/Dockerfile.j2

@@ -12,8 +12,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" build-date="{{ build
 {{ macros.enable_extra_repos(['elasticsearch']) }}
 
 {% if base_package_type == 'rpm' %}
+
+# NOTE(hrw): post-install script of elasticsearch fails when trying to
+# install elasticsearch and java together.
+{{ macros.install_packages(['java-11-openjdk-headless']) }}
+
     {% set elasticsearch_packages = [
-        'java-11-openjdk-headless',
         'elasticsearch-oss',
     ] %}
 

docker/masakari/masakari-monitors/Dockerfile.j2

@@ -19,15 +19,15 @@ RUN echo '{{ install_type }} not yet available for {{ base_distro }}' \
     {% if base_package_type == 'rpm' %}
 
         {% set masakari_monitors_packages = [
-            'libvirt-devel',
+            'python3-libvirt',
             'pacemaker-cli',
             'tcpdump',
         ] %}
 
     {% elif base_package_type == 'deb' %}
 
         {% set masakari_monitors_packages = [
-            'libvirt-dev',
+            'python3-libvirt',
             'pacemaker-cli-utils',
             'tcpdump',
         ] %}
@@ -42,8 +42,10 @@ ADD masakari-monitors-archive /masakari-monitors-source
     '/masakari-monitors'
 ] %}
 
+# NOTE(hrw): distros may provide other version of libvirt
 RUN ln -s masakari-monitors-source/* masakari-monitors \
-    {% if distro_package_manager == 'dnf' %}&& sed -i -e 's/libvirt-python===.*/libvirt-python===6.10.0/' /requirements/upper-constraints.txt {% endif %}\
+    && sed -i -e "/^libvirt-python/d" /requirements/upper-constraints.txt \
+    && sed -i -e "/^libvirt-python/d" /masakari-monitors/requirements.txt \
     && {{ macros.install_pip(masakari_monitors_pip_packages | customizable("pip_packages")) }} \
     && mkdir -p /etc/masakari-monitors \
     && chown -R masakari: /etc/masakari-monitors

kolla/template/filters.py

@@ -12,11 +12,16 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from jinja2 import contextfilter
+# NOTE: jinja2 3.1.0 dropped contextfilter in favour of pass_context.
+try:
+    from jinja2 import pass_context
+except ImportError:
+    from jinja2 import contextfilter as pass_context
+
 from jinja2 import Undefined
 
 
-@contextfilter
+@pass_context
 def customizable(context, val_list, call_type):
     # NOTE(mgoddard): Don't try to customise undefined values. There are cases
     # where this might happen, for example using a generic template overrides

kolla/template/methods.py

@@ -15,7 +15,11 @@
 import os
 import yaml
 
-from jinja2 import contextfunction
+# NOTE: jinja2 3.1.0 dropped contextfunction in favour of pass_context.
+try:
+    from jinja2 import pass_context
+except ImportError:
+    from jinja2 import contextfunction as pass_context
 
 
 def debian_package_install(packages, clean_package_cache=True):
@@ -71,7 +75,7 @@ def debian_package_install(packages, clean_package_cache=True):
     return ' && '.join(cmds)
 
 
-@contextfunction
+@pass_context
 def handle_repos(context, reponames, mode):
     """NOTE(hrw): we need to handle CentOS, Debian and Ubuntu with one macro.
 

releasenotes/notes/jinja2-pass-context-3f3febcd944e3a51.yaml

@@ -0,0 +1,4 @@
+---
+fixes:
+  - |
+    Fixes an issue seen when using Jinja2 3.1.0.