20240914 weekly

[dingxiaoshuai123]

todo: OpenAtomFoundation/pikiwidb#379

Summer Code

题目	负责人	进度
PikiwiDB 缓存层实现冷热数据分离	沈梦菊	redis cache https://github.com/pikiwidb/rediscache/pull/7/files https://github.com/arana-db/kiwi/pull/6/files 1、0817 KV 类型改造完成；List 已经改造完毕
PikiwiDB 实现集群模式	@longfar-ncy	完成了 multi-raft 的改造，并将 PD 模块搭建起来了，目前已经完成了 store 上线的注册功能。
实现 Pika 高性能流量转发代理	@黄恺一	完成了命令框架；路由转发还在开发；client sdk 开发中
向量数据库	@郭同学	已完成存储层开发，指令层待添加

core

Item	Priority	Issue/Pr	Owner	Description
[TCL] 插入大 key 会卡主不动		arana-db/kiwi#29	Mingxuan	待提交 PR
PikiwiDB 支持单 DB	P2	OpenAtomFoundation/pikiwidb#426	Lyc	预计下周完成
TCL 测试：插入大 key 会卡主不动	P0	OpenAtomFoundation/pikiwidb#424	lyc
执行 raft on_apply 方法附近会崩溃	P0	OpenAtomFoundation/pikiwidb#421	lyc
Raft pre vote 的时候，突然崩溃	P0	OpenAtomFoundation/pikiwidb#420	lyc
pikiwidb 运行过程中突然崩溃	P0	OpenAtomFoundation/pikiwidb#416	lyc
jepsen 测试遇到的一些问题	PO	OpenAtomFoundation/pikiwidb#414
support rocky CI	P0	OpenAtomFoundation/pikiwidb#407	文一	PR 可以 review
PClient::TransferToSlaveThreads empty function body	P0	OpenAtomFoundation/pikiwidb#405	QX
feat: Add blpop cmd	P1	OpenAtomFoundation/pikiwidb#373	shiminggxuan	待提交代码
[WIP]feat: multi raft support	P1	OpenAtomFoundation/pikiwidb#364	小帅、潘磊、月财	转提交到 pikiwidb/pikiwidb
support echo, auth commnd	P1	OpenAtomFoundation/pikiwidb#356	tangruilin
test raft cluster by python	P1	. OpenAtomFoundation/pikiwidb#393	liuhuan	redis raft 的 python 集成测试搬过来
update task manager	P1	OpenAtomFoundation/pikiwidb#397	trl
fix: Resolve the inconsistency between member configuration and log index in snapshot meta	P1	OpenAtomFoundation/pikiwidb#386	pl	修复打快照把成员信息弄丢的问题
feat: support dbsize, bgsave, lastsave command	P1	OpenAtomFoundation/pikiwidb#379	renzhenfeng	draft
feat: add monitor cmd	P1	OpenAtomFoundation/pikiwidb#374	gukj	待 review
feat: supports the master-slave mode	P2	OpenAtomFoundation/pikiwidb#342	Pl	先 close 掉，后面再考虑 master-slave mode
loading pikiwidb.conf overuses atomic variables	P0	OpenAtomFoundation/pikiwidb#329	gukj	待提交 PR
feat: add info section: server stats cpu commandstats	P1	OpenAtomFoundation/pikiwidb#338	shenmengju	@dxs @LanaO Review 待 A
feat: add client command	P0	OpenAtomFoundation/pikiwidb#361	gukj	待 A
add info section: server stats cpu commandstats	P1	OpenAtomFoundation/pikiwidb#338	smj
loading pikiwidb.conf overuses atomic variables	P1	OpenAtomFoundation/pikiwidb#329	lihuan
将存储引擎中直接写 Rocksdb 的部分换成调用封装的 Batch 接口	P1	OpenAtomFoundation/pikiwidb#280		持续跟进
feat: Add online profiling functionality	P1	OpenAtomFoundation/pikiwidb#256	待认领	性能分析工具，优先级不高
modify readme	P1	OpenAtomFoundation/pikiwidb#230
Pika 主从复制一些经验教训	P1	OpenAtomFoundation/pikiwidb#176		经验分享 Issue
command develop tasks	P1	OpenAtomFoundation/pikiwidb#30
降低 Pika 碎片化监控，提升 compact 性能	P1	OpenAtomFoundation/pikiwidb#2255	gukaijie	将 Pika 侧的 PR 拿过来

todo:

• 1 binlog for master-slave; raft; slot
• 2 master -> slave replication： (1) using redis command SYNC&PSYNC; (2) using RocksDB WAL instead of pika binlog by RocksDB API GetUpdateSince()
• 3 redis cache
• 4 info command
• 5 hello command

存储模型 floyd

• issue bugfix config set maxclients don't work OpenAtomFoundation/pikiwidb#269

Item	Priority	Issue/Pr	Owner	Description
添加关闭 RocksDB wal 的接口	P0		dxs	done
先把 floyd 移植过来	P0		dxs	done
一个 db 内一个 key 只能有一个 类型	P0

raft 集群进度

• issue bugfix SRandmemberCmd::Do() OpenAtomFoundation/pikiwidb#277

State	Item	Priority	Issue/Pr	Owner	Description
已合并	引入 braft	p0	OpenAtomFoundation/pikiwidb#130	@panlei-coder, @longfar-ncy , @dingxiaoshuai123
已合并	redis raft 指令	p0	OpenAtomFoundation/pikiwidb#136	@panlei-coder, @KKorpse, @hotjump
已合并	braft log index 映射	p0	OpenAtomFoundation/pikiwidb#246	@longfar-ncy, @hotjump
已合并	pikiwidb checkpoint	p0	OpenAtomFoundation/pikiwidb#226	@dingxiaoshuai123, @hotjump
已合并	apply 支持	p0	OpenAtomFoundation/pikiwidb#213	@longfar-ncy, @hotjump
merged	braft 快照自定义日志截断点	p0	arana-db/braft#2	@panlei-coder, @hotjump
merged	raft 快照支持	p0	OpenAtomFoundation/pikiwidb#238, OpenAtomFoundation/pikiwidb#245	@dingxiaoshuai123, @panlei-coder, @hotjump	第二个 pr 包含第一个 pr
merged	将 import-braft 合并到 unstable	p0	OpenAtomFoundation/pikiwidb#300
	braft 主从模式	p1	arana-db/braft#1	@KKorpse
	pikiwidb 主从命令	p1		@panlei-coder	依赖于 braft 主从代码
	优雅关闭 pikiwidb	p1		@dingxiaoshuai123	和 snapshot 相关
	logindex 维护的多队列优化	p1	OpenAtomFoundation/pikiwidb#428	@刘月财	思路见OpenAtomFoundation/pikiwidb#265
	单独 cf 保存其它 cf flush 位点	p1	OpenAtomFoundation/pikiwidb#430
	braft 新增接口，判断两个 logindex 之间数据量	p1	OpenAtomFoundation/pikiwidb#429
	raft 集群操作文档	p1	OpenAtomFoundation/pikiwidb#427
	raft 测试	p1			参考 redis-raft https://github.com/RedisLabs/redisraft/tree/master/tests

Pika binlog + braft -> 1 经典模式； 2 raft 模式。

可观测

Item	Priority	Issue/Pr	Owner	Description

工具集

Item	Priority	Issue/Pr	Owner	Description

云原生

Item	Priority	Issue/Pr	Owner	Description

质量建设

Item	Priority	Issue/Pr	Owner	Description
feature: add go test for pikiwidb from pika	P0	OpenAtomFoundation/pikiwidb#75
TCL 测试错误汇总	P0	OpenAtomFoundation/pikiwidb#266

summer code

Item	Priority	Issue/Pr	Owner	Description
pika binlog 重构		https://summer-ospp.ac.cn/org/prodetail/247e50197?list=org&navpage=org	chenjunhua	PR: OpenAtomFoundation/pikiwidb#2906
Pika-Operator		https://summer-ospp.ac.cn/org/prodetail/247e50199?list=org&navpage=org	xuhaoliang	PR:OpenAtomFoundation/pikiwidb#2860 OpenAtomFoundation/pikiwidb#2903 1 近期把 kubeblcoks 升级到 0.9.0； 2 添加缩容； 3 长期工作是把工作流从 Kubeblocks 迁移到 codis【向 Kubeblocks 下达启动组件的命令】，Kubeblocks 只做资源管理【Kubeblocks 只启动相关组件即可】； 0809. PR 已经提交，需要内部帮忙测试 0830. 要主从切换逻辑
Pika Vector		https://summer-ospp.ac.cn/org/prodetail/247e50200?list=org&navpage=org	guoweijian（郭伟建）&zhuhaiyong	1 cjh 帮忙建分支； 2 在官网提交结项书；
Pika Transfer Proxy		https://summer-ospp.ac.cn/org/prodetail/247e50201?list=org&navpage=org	lihao	PR: OpenAtomFoundation/pikiwidb#2855 1 这个工作比 dump&restore 更有价值；2 librdkafka 0809. 已经提交一个框架 PR，待补充说明文档
PikiwiDB Redis Cache		https://summer-ospp.ac.cn/org/prodetail/247e50203?list=org&navpage=org	沈梦菊	PR: OpenAtomFoundation/pikiwidb#440
PikiwiDB Cluster		https://summer-ospp.ac.cn/org/prodetail/247e50205?list=org&navpage=org	宁长远	PR 和项目结项书都还没提交
Codis Proxy(C++)		https://summer-ospp.ac.cn/org/prodetail/247e50206?list=org&navpage=org	黄恺一	PR 和项目结项书都还没提交
Improve Pika		https://summer-ospp.ac.cn/org/prodetail/247e50207?list=org&navpage=org	吴启龙	https://github.com/MalikHou/pika/pull/2； https://github.com/OpenAtomFoundation/pika/issues/2566；RTC；使用 RocksDB mget 优化 Pika 的 mget 和 hmget

Members

ID	Name
Tangruilin	TangRuilin
happy-v587	LiuHuan
gukj-spel	GuKaijie
hotjump	HongJunwang
hahahashen	ShenMengju
panlei-coder	PanLei
chienguo	GuoWeijian
SamuelSze1	ShiMingxuan
mirthfullee	LiHuan
KyeeHuang	HuangKaiyi
longfar-ncy	NingChangyuan
dingxiaoshuai123	DingXiaoshuai
Iam-WenYi	WenYi
JW-Gao	ZhangRongkuan
qinggu(1468449777)	MengXiangrui