Skip to content

feat: Enable CI for Redis CE 8.0 #3274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Feb 28, 2025
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
bdb19f5
chore: extract benchmark tests
ndyakov Feb 10, 2025
c031034
wip
ndyakov Feb 10, 2025
deb84a6
enable pubsub tests
ndyakov Feb 10, 2025
e422646
enable ring tests
ndyakov Feb 10, 2025
9fec123
stop tests with build redis from source
ndyakov Feb 10, 2025
f8f0dd5
start all tests
ndyakov Feb 10, 2025
11bdcc8
mix of makefile and action
ndyakov Feb 10, 2025
a94513b
add sentinel configs
ndyakov Feb 11, 2025
b87a20b
fix example test
ndyakov Feb 11, 2025
4e890ba
stop debug on re
ndyakov Feb 11, 2025
3842e2e
Merge remote-tracking branch 'origin/master' into ndyakov/extract-ben…
ndyakov Feb 11, 2025
05e052c
wip
ndyakov Feb 11, 2025
4f410ad
enable gears for redis 7.2
ndyakov Feb 11, 2025
140b88d
wip
ndyakov Feb 11, 2025
804a1f7
enable sentinel, they are expected to fail
ndyakov Feb 11, 2025
3646787
fix: linter configuration
ndyakov Feb 18, 2025
48e3c14
chore: update re versions
ndyakov Feb 18, 2025
43f557b
return older redis enterprise version
ndyakov Feb 18, 2025
bce2ff7
add basic codeql
ndyakov Feb 18, 2025
75c20f7
Merge branch 'master' into ndyakov/extract-benchmark-tests
ndyakov Feb 18, 2025
6d277a4
Merge branch 'master' into ndyakov/extract-benchmark-tests
ndyakov Feb 18, 2025
6c78cea
wip: increase timeout, focus only sentinel tests
ndyakov Feb 18, 2025
17f31fc
sentinels with docker network host
ndyakov Feb 18, 2025
82b9467
enable all tests
ndyakov Feb 18, 2025
81c267e
Merge branch 'master' into ndyakov/extract-benchmark-tests
ndyakov Feb 18, 2025
ee0fb71
fix flanky test
ndyakov Feb 18, 2025
5961017
enable example tests
ndyakov Feb 18, 2025
7c498dc
tidy docker compose
ndyakov Feb 18, 2025
a4b2906
add debug output
ndyakov Feb 18, 2025
164f44a
stop shutingdown masters
ndyakov Feb 18, 2025
6bd9923
don't test sentinel for re
ndyakov Feb 18, 2025
2ccd81a
Merge branch 'master' into ndyakov/extract-benchmark-tests
ndyakov Feb 20, 2025
8a544fc
Merge remote-tracking branch 'origin/master' into ndyakov/extract-ben…
ndyakov Feb 26, 2025
ce09492
Merge branch 'master' into ndyakov/extract-benchmark-tests
ndyakov Feb 26, 2025
6818620
skip unsuported addscores
ndyakov Feb 26, 2025
3c6b79e
Update README
ndyakov Feb 27, 2025
65a8f4a
Merge branch 'master' into ndyakov/extract-benchmark-tests
ndyakov Feb 27, 2025
055ca37
Update README.md
ndyakov Feb 27, 2025
00a31cc
Update CONTRIBUTING.md
ndyakov Feb 27, 2025
e50eacd
Merge branch 'master' into ndyakov/extract-benchmark-tests
ndyakov Feb 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 5 additions & 15 deletions .github/actions/run-tests/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,7 @@ runs:
CLIENT_LIBS_TEST_IMAGE: "redislabs/client-libs-test:${{ inputs.redis-version }}"
run: |
set -e
redis_major_version=$(echo "$REDIS_VERSION" | grep -oP '^\d+')
if (( redis_major_version < 8 )); then
echo "Using redis-stack for module tests"
else
echo "Using redis CE for module tests"
fi
redis_version_np=$(echo "$REDIS_VERSION" | grep -oP '^\d+.\d+')

# Mapping of redis version to redis testing containers
declare -A redis_version_mapping=(
Expand All @@ -36,27 +31,22 @@ runs:
)

if [[ -v redis_version_mapping[$REDIS_VERSION] ]]; then
echo "REDIS_MAJOR_VERSION=${redis_major_version}" >> $GITHUB_ENV
echo "REDIS_VERSION=${redis_version_np}" >> $GITHUB_ENV
echo "REDIS_IMAGE=redis:${{ inputs.redis-version }}" >> $GITHUB_ENV
echo "CLIENT_LIBS_TEST_IMAGE=redislabs/client-libs-test:${redis_version_mapping[$REDIS_VERSION]}" >> $GITHUB_ENV
else
echo "Version not found in the mapping."
exit 1
fi
sleep 10 # time to settle
shell: bash
- name: Set up Docker Compose environment with redis ${{ inputs.redis-version }}
run: docker compose --profile all up -d
run: |
make docker.start
shell: bash
- name: Run tests
env:
RCE_DOCKER: "true"
RE_CLUSTER: "false"
run: |
go test \
--ginkgo.skip-file="ring_test.go" \
--ginkgo.skip-file="sentinel_test.go" \
--ginkgo.skip-file="pubsub_test.go" \
--ginkgo.skip-file="gears_commands_test.go" \
--ginkgo.label-filter="!NonRedisEnterprise"
make test.ci
shell: bash
62 changes: 48 additions & 14 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,19 @@ permissions:
contents: read

jobs:
build:
name: build

benchmark:
name: benchmark
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
go-version: [1.21.x, 1.22.x, 1.23.x]
redis-version:
- "8.0-M03" # 8.0 milestone 4
- "7.4.2" # should use redis stack 7.4
go-version:
- "1.23.x"
- "1.24.x"

steps:
- name: Set up ${{ matrix.go-version }}
Expand All @@ -27,15 +33,38 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4

- name: Test
run: make test
- name: Setup Test environment
env:
REDIS_VERSION: ${{ matrix.redis-version }}
CLIENT_LIBS_TEST_IMAGE: "redislabs/client-libs-test:${{ matrix.redis-version }}"
run: |
set -e
redis_version_np=$(echo "$REDIS_VERSION" | grep -oP '^\d+.\d+')

# Mapping of redis version to redis testing containers
declare -A redis_version_mapping=(
["8.0-M03"]="8.0-M04-pre"
["7.4.2"]="rs-7.4.0-v2"
)
if [[ -v redis_version_mapping[$REDIS_VERSION] ]]; then
echo "REDIS_VERSION=${redis_version_np}" >> $GITHUB_ENV
echo "REDIS_IMAGE=redis:${{ matrix.redis-version }}" >> $GITHUB_ENV
echo "CLIENT_LIBS_TEST_IMAGE=redislabs/client-libs-test:${redis_version_mapping[$REDIS_VERSION]}" >> $GITHUB_ENV
else
echo "Version not found in the mapping."
exit 1
fi
shell: bash
- name: Set up Docker Compose environment with redis ${{ matrix.redis-version }}
run: make docker.start
shell: bash
- name: Benchmark Tests
env:
RCE_DOCKER: "true"
RE_CLUSTER: "false"
run: make bench
shell: bash

- name: Upload to Codecov
uses: codecov/codecov-action@v5
with:
files: coverage.txt
token: ${{ secrets.CODECOV_TOKEN }}

test-redis-ce:
name: test-redis-ce
runs-on: ubuntu-latest
Expand All @@ -47,11 +76,10 @@ jobs:
- "7.4.2" # should use redis stack 7.4
- "7.2.7" # should redis stack 7.2
go-version:
- "1.22.x"
- "1.23.x"
- "1.24.x"

steps:

- name: Checkout code
uses: actions/checkout@v4

Expand All @@ -60,4 +88,10 @@ jobs:
with:
go-version: ${{matrix.go-version}}
redis-version: ${{ matrix.redis-version }}


- name: Upload to Codecov
uses: codecov/codecov-action@v5
with:
files: coverage.txt
token: ${{ secrets.CODECOV_TOKEN }}

68 changes: 68 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: [ master ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://git.io/codeql-language-support

steps:
- name: Checkout repository
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
4 changes: 2 additions & 2 deletions .github/workflows/doctests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
strategy:
fail-fast: false
matrix:
go-version: [ "1.21", "1.22", "1.23" ]
go-version: ["1.24"]

steps:
- name: Set up ${{ matrix.go-version }}
Expand All @@ -38,4 +38,4 @@ jobs:

- name: Test doc examples
working-directory: ./doctests
run: go test
run: go test -v
4 changes: 2 additions & 2 deletions .github/workflows/test-redis-enterprise.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
strategy:
fail-fast: false
matrix:
go-version: [1.23.x]
go-version: [1.24.x]
re-build: ["7.4.2-54"]

steps:
Expand Down Expand Up @@ -47,7 +47,7 @@ jobs:
- name: Test
env:
RE_CLUSTER: true
REDIS_MAJOR_VERSION: 7
REDIS_VERSION: "7.4"
run: |
go test \
--ginkgo.skip-file="ring_test.go" \
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,6 @@ testdata/*
*.tar.gz
*.dic
redis8tests.sh
.vscode
coverage.txt
**/coverage.txt
.vscode
42 changes: 16 additions & 26 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,45 +1,35 @@
GO_MOD_DIRS := $(shell find . -type f -name 'go.mod' -exec dirname {} \; | sort)
export REDIS_MAJOR_VERSION := 7

test: testdeps
docker start go-redis-redis-stack || docker run -d --name go-redis-redis-stack -p 6379:6379 -e REDIS_ARGS="--enable-debug-command yes --enable-module-command yes" redis/redis-stack-server:latest
$(eval GO_VERSION := $(shell go version | cut -d " " -f 3 | cut -d. -f2))
docker.start:
docker compose --profile all up -d --quiet-pull

docker.stop:
docker compose --profile all down

test:
$(MAKE) docker.start
$(MAKE) test.ci
$(MAKE) docker.stop

test.ci:
set -e; for dir in $(GO_MOD_DIRS); do \
if echo "$${dir}" | grep -q "./example" && [ "$(GO_VERSION)" = "19" ]; then \
echo "Skipping go test in $${dir} due to Go version 1.19 and dir contains ./example"; \
continue; \
fi; \
echo "go test in $${dir}"; \
(cd "$${dir}" && \
go mod tidy -compat=1.18 && \
go test && \
go test ./... -short -race && \
go test ./... -run=NONE -bench=. -benchmem && \
env GOOS=linux GOARCH=386 go test && \
go test -coverprofile=coverage.txt -covermode=atomic ./... && \
go vet); \
go vet && \
go test -coverprofile=coverage.txt -covermode=atomic ./... -race); \
done
cd internal/customvet && go build .
go vet -vettool ./internal/customvet/customvet
docker stop go-redis-redis-stack

testdeps: testdata/redis/src/redis-server

bench: testdeps
bench:
go test ./... -test.run=NONE -test.bench=. -test.benchmem

.PHONY: all test testdeps bench fmt
.PHONY: all test bench fmt

build:
go build .

testdata/redis:
mkdir -p $@
wget -qO- https://download.redis.io/releases/redis-7.4.2.tar.gz | tar xvz --strip-components=1 -C $@

testdata/redis/src/redis-server: testdata/redis
cd $< && make all

fmt:
gofumpt -w ./
goimports -w -local github.com/redis/go-redis ./
Expand Down
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,20 @@
> See [OpenTelemetry](https://github.com/redis/go-redis/tree/master/example/otel) example which
> demonstrates how you can use Uptrace to monitor go-redis.
## Supported versions

In `go-redis` we are aiming to support the last three releases of Redis. Currently, this means we do support:
- [Redis 7.2](https://raw.githubusercontent.com/redis/redis/7.2/00-RELEASENOTES) - using Redis Stack 7.2 for modules support
- [Redis 7.4](https://raw.githubusercontent.com/redis/redis/7.4/00-RELEASENOTES) - using Redis Stack 7.4 for modules support
- [Redis 8.0](https://raw.githubusercontent.com/redis/redis/8.0/00-RELEASENOTES) - using Redis CE 8.0 where modules are included

Although the `go.mod` states it requires at minimum `go 1.18`, our CI is configured to run the tests against all three
versions of Redis and latest two versions of Go ([1.23](https://go.dev/doc/devel/release#go1.23.0),
[1.24](https://go.dev/doc/devel/release#go1.24.0)). We observe that some modules related test may not pass with
Redis Stack 7.2 and some commands are changed with Redis CE 8.0.
Please do refer to the documentation and the tests if you experience any issues. We do plane to update the go version
in the `go.mod` to `go 1.24` in one of the next releases.

## How do I Redis?

[Learn for free at Redis University](https://university.redis.com/)
Expand Down
6 changes: 3 additions & 3 deletions acl_commands_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@ var _ = Describe("ACL permissions", Label("NonRedisEnterprise"), func() {
})

It("set permissions for module commands", func() {
SkipBeforeRedisMajor(8, "permissions for modules are supported for Redis Version >=8")
SkipBeforeRedisVersion(8, "permissions for modules are supported for Redis Version >=8")
Expect(client.FlushDB(ctx).Err()).NotTo(HaveOccurred())
val, err := client.FTCreate(ctx, "txt", &redis.FTCreateOptions{}, &redis.FieldSchema{FieldName: "txt", FieldType: redis.SearchFieldTypeText}).Result()
Expect(err).NotTo(HaveOccurred())
Expand Down Expand Up @@ -322,7 +322,7 @@ var _ = Describe("ACL permissions", Label("NonRedisEnterprise"), func() {
})

It("set permissions for module categories", func() {
SkipBeforeRedisMajor(8, "permissions for modules are supported for Redis Version >=8")
SkipBeforeRedisVersion(8, "permissions for modules are supported for Redis Version >=8")
Expect(client.FlushDB(ctx).Err()).NotTo(HaveOccurred())
val, err := client.FTCreate(ctx, "txt", &redis.FTCreateOptions{}, &redis.FieldSchema{FieldName: "txt", FieldType: redis.SearchFieldTypeText}).Result()
Expect(err).NotTo(HaveOccurred())
Expand Down Expand Up @@ -419,7 +419,7 @@ var _ = Describe("ACL Categories", func() {
})

It("lists acl categories and subcategories with Modules", func() {
SkipBeforeRedisMajor(8, "modules are included in acl for redis version >= 8")
SkipBeforeRedisVersion(8, "modules are included in acl for redis version >= 8")
aclTestCase := map[string]string{
"search": "FT.CREATE",
"bloom": "bf.add",
Expand Down
Loading
Loading