New px-fuse forked from v2.6.0 with limited set of changes from v2.7.0 by prabirpaul · Pull Request #199 · portworx/px-fuse

prabirpaul · 2021-03-12T07:41:49Z

Cherry-picked only limited set of required changes from v2.7.0
Disables fastpath - stubs only
Includes fuse headers required for px-storage build (iouring)
Includes all compilation changes from v2.7.0

Sanity and perf regression test done.
Smoke pass: https://jenkins.portworx.dev/job/DEV/job/Porx-03/533
BVT running: https://jenkins.portworx.dev/job/DEV/job/Porx-03/534/
Upgrade test running: 2.6 to 2.7, 2.7 to master
Px-fuse build test done covering the following:
3.10.0-1127.el7.x86_64
3.10.0-1160.11.1.el7.x86_64
3.10.0-327.36.3.el7.x86_64
3.10.0-957.21.3.el7.x86_64
4.12.2-1.el7.elrepo.x86_64
4.16.3-301.fc28.x86_64
4.18.0-1.el7.elrepo.x86_64
4.18.0-193.29.1.el8_2.x86_64
4.18.0-193.el8.x86_64
4.18.0-240.1.1.el8_3.x86_64
4.18.0-257.el8.x86_64
4.18.0-259.el8.x86_64
4.18.0-269.el8.x86_64
4.20.13-1.el7.elrepo.x86_64
5.4.101-1.el7.elrepo.x86_64
5.6.15-1.el7.elrepo.x86_64
5.8.11-1.el8.elrepo.x86_64
5.8.9-1.el7.elrepo.x86_64
5.9.6-1.el7.elrepo.x86_64
3.10.0-1160.15.2.el7.x86_64
4.13.9-300.fc27.x86_64
4.14.186-146.268.amzn2.x86_64
4.14.214-160.339.amzn2.x86_64
4.18.0-147.20.1.el8_1.x86_64
4.18.0-147.el8.x86_64
4.18.0-193.14.2.el8_2.x86_64
4.18.0-193.23.1.el8_2.x86_64
4.18.0-193.24.1.el8_2.dt1.x86_64
4.18.0-240.15.1.el8_3.x86_64
4.18.0-80.1.2.el8_0.x86_64
4.18.0-80.11.1.el8_0.x86_64
5.0.16-100.fc28.x86_64
5.10.0-1.el7.elrepo.x86_64
5.10.0-1.el8.elrepo.x86_64
5.10.15-1.el7.elrepo.x86_64
5.4.12-1.el7.elrepo.x86_64
5.7.0-1.el7.elrepo.x86_64
5.7.0-1.el8.elrepo.x86_64
5.7.12-1.el7.elrepo.x86_64
5.7.7-1.el7.elrepo.x86_64
5.8.0-1.el7.elrepo.x86_64
5.8.0-1.el8.elrepo.x86_64
5.8.15-301.fc33.x86_64
5.9.0-1.el7.elrepo.x86_64
5.9.0-1.el8.elrepo.x86_64
4.9.0-4-amd64
4.9.0-11-amd64
4.9.0-7-amd64
4.9.0-13-amd64
4.9.0-14-amd64
4.19.0-14-cloud-amd64
4.19.0-8-amd64
4.12.14-120-default
4.12.14-122.12-default
4.12.14-122.17-default
4.12.14-122.20-default
4.12.14-122.23-default
4.12.14-122.26-default
4.12.14-122.29-default
4.12.14-122.32-default
4.12.14-122.37-default
4.12.14-122.41-default
4.12.14-122.46-default
4.12.14-122.51-default
4.12.14-122.54-default
4.12.14-122.57-default
4.12.14-122.60-default
4.12.14-122.7-default
4.12.14-95.13-default
4.12.14-95.16-default
4.12.14-95.19-default
4.12.14-95.24-default
4.12.14-95.29-default
4.12.14-95.32-default
4.12.14-95.37-default
4.12.14-95.40-default
4.12.14-95.45-default
4.12.14-95.48-default
4.12.14-95.51-default
4.12.14-95.54-default
4.12.14-95.6-default
4.4.140-94.42-default
4.4.175-94.79-default
4.4.176-94.88-default
4.4.178-94.91-default
4.4.180-94.100-default
4.4.180-94.97-default
4.12.14-150.14-default
4.12.14-150.17-default
4.12.14-150.22-default
4.12.14-150.27-default
4.12.14-150.32-default
4.12.14-150.35-default
4.12.14-150.38-default
4.12.14-150.41-default
4.12.14-150.47-default
4.12.14-195-default
4.12.14-197.10-default
4.12.14-197.15-default
4.12.14-197.18-default
4.12.14-197.21-default
4.12.14-197.26-default
4.12.14-197.29-default
4.12.14-197.34-default
4.12.14-197.37-default
4.12.14-197.4-default
4.12.14-197.40-default
4.12.14-197.45-default
4.12.14-197.48-default
4.12.14-197.51-default
4.12.14-197.56-default
4.12.14-197.61-default
4.12.14-197.64-default
4.12.14-197.67-default
4.12.14-197.7-default
4.12.14-197.72-default
4.12.14-197.75-default
4.12.14-197.78-default
4.12.14-25.28-default
5.3.18-22-default
5.3.18-24.12-default
5.3.18-24.15-default
5.3.18-24.24-default
5.3.18-24.29-default
5.3.18-24.34-default
5.3.18-24.37-default
5.3.18-24.43-default
5.3.18-24.46-default
5.3.18-24.49-default
5.3.18-24.9-default
4.4.0-185-generic
5.4.0-1029-gke
4.4.0-112-generic
5.4.0-1026-azure
5.4.0-1036-gke
4.4.0-170-generic
5.4.0-1028-gcp
4.4.0-116-generic
4.4.0-190-generic
4.4.0-177-generic
4.15.0-1082-azure
4.4.0-178-generic
4.4.0-184-generic
5.4.0-29-generic
5.4.0-1024-gcp
4.4.0-186-generic
4.4.0-31-generic
4.4.0-187-generic
4.4.0-176-generic
4.4.0-174-generic
5.4.0-1036-azure
4.4.0-193-generic
5.4.0-1036-gcp
5.4.0-65-generic

Co-authored-by: jrivera-px <jose@portworx.com>

…w interface blk_alloc_queue_rh. Add explicit check for this api for now. (#184)

* compilation fixes in 5.9/5.10 kernel Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

… submit ios (#196) (#197) to pxd instead of the request queue

* SLES/Suse 4.12.x compile fixes. * Use a 'defined' entry instead of 'not defined' entry * Consolidate duplicate code to pxd_compat.h and wrap cond to 80 cols * Fix compilation issue. Co-authored-by: latrel.stack@gmail.com <latrel.stack@gmail.com>

Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

* Use the existing pxd_io_tracker obj. * Change version check to Fix undefined declarations for 5.8 kernels. Fix iot not declared error. * Use the existing pxd_io_tracker obj.

* address race between restart and new req processing Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

…193)" This reverts commit c83c546.

* address race between restart and new req processing Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

Makefile.in

pxd.h

Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

sulakshm · 2021-03-12T12:22:11Z

pxd_fastpath_stub.c

@@ -1,11 +1,9 @@
 /* Enable stub action if fastpath is not enabled */
-#ifndef __PX_FASTPATH__


this line should be included back. makefile is fixed to not defined PX_FASTPATH

Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

sulakshm

changes look good to me.

Makefile.in

pxd.h

adikulkarni0 · 2021-03-12T18:08:11Z

pxd_io_uring.h

@@ -0,0 +1,160 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */


Can we add a note on such header files indicating that they are included for build purpose only?

This reverts commit 2485f5f.

…193)" This reverts commit 525a5ee.

adikulkarni0 · 2021-03-12T18:10:58Z

pxd_fastpath_stub.c

+void pxd_fastpath_adjust_limits(struct pxd_device *pxd_dev, struct request_queue *topque) {}
+int pxd_suspend_state(struct pxd_device *pxd_dev) {return 0;}
+
+void pxd_suspend_io(struct pxd_device* pxd_dev) { }
+void pxd_resume_io(struct pxd_device* pxd_dev) { }
+int pxd_switch_fastpath(struct pxd_device* pxd_dev) {return -1;}
+int pxd_switch_nativepath(struct pxd_device* pxd_dev) {return -1;}


traces for these functions where we dont expect them to be called? I would have said ASSERTs but I am not sure if a kernel panic is what we want. Is there any other way to ensure these functions are not getting called, perhaps a test only module which will have asserts here.

it's not that they can't be called, we are not going to advertise fastpath.

pxd.h

sulakshm · 2021-03-13T03:59:05Z

pxd_fastpath_stub.c

+#if LINUX_VERSION_CODE >= KERNEL_VERSION(5,9,0)
+#define BLK_QC_RETVAL BLK_QC_T_NONE
+blk_qc_t pxd_make_request_fastpath(struct bio *bio)
+{


This function should definitely assert hard. When fastpath function is stubbed, IO cannot be received/processed.

sulakshm · 2021-03-15T15:30:23Z

pxd.c


-	spin_lock(&fc->lock);
 	fuse_end_queued_requests(fc);
-	spin_unlock(&fc->lock);


This is the rootcause of https://jenkins.portworx.dev/job/PX2-RELEASE/view/PX2-RELEASE-Cloudsnap/job/FVT-MN-ObjectStore-NEXT-PX2-RELEASE/

prabirpaul and others added 22 commits March 11, 2021 10:56

Everything needed to build storage

c7332cd

Allow greater than 300sec timeout values (#132)

82202cf

Compilation changes for kernel 5.7 (#137)

9316aee

support making a process into a IO flusher

98151e8

addressed review comments from prabir

5dc75c2

addressed review comments

205d53b

Compilation issues with 5.9.8-flatcar. (#168) (#169)

c9a4cff

Co-authored-by: jrivera-px <jose@portworx.com>

PWX-16992: Fix compile issues for 5.9.6-1.el7.elrepo.x86_64 kernel. (#…

c75e50d

…174)

Centos 8 Stream 4.18.0-257 contains kernel 5.x backports and added ne…

ed60132

…w interface blk_alloc_queue_rh. Add explicit check for this api for now. (#184)

PWX-18829: Fix device queue depth setting (#191) (#192)

80ce975

PWX-18535: compilation fixes upto 5.10 kernel (#189) (#194)

9db4369

* compilation fixes in 5.9/5.10 kernel Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

PWX-19006: Do not set pxd_bd_ops.submit_bio, 5.10 kernel uses this to…

2f085e6

… submit ios (#196) (#197) to pxd instead of the request queue

Suse compile fixes (#147)

1ee26a1

* SLES/Suse 4.12.x compile fixes. * Use a 'defined' entry instead of 'not defined' entry * Consolidate duplicate code to pxd_compat.h and wrap cond to 80 cols * Fix compilation issue. Co-authored-by: latrel.stack@gmail.com <latrel.stack@gmail.com>

Compile fixes for 5.8.x, 4.12.x & 4.13.x EL7 kernels. (#145)

40fd6c3

address recursive fc->lock path (#183)

89b149d

Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

5.8.x fixes master (#186)

050c82b

* Use the existing pxd_io_tracker obj. * Change version check to Fix undefined declarations for 5.8 kernels. Fix iot not declared error. * Use the existing pxd_io_tracker obj.

Compile fixes... (#198)

245b638

Always use stub for fastpath

78d7ed2

remove pxd_fastpath.c

83c49b8

compile fix

262a36a

Update mod version so that px 2.6->2.7 upgrade attemtpts module upgrade

e52fe10

Update debug log

c301000

prabirpaul requested review from adikulkarni0, jrivera-px and sulakshm March 12, 2021 07:42

sulakshm and others added 3 commits March 11, 2021 23:44

PWX-18756 fix race in restart and new req processing (#190) (#193)

c83c546

* address race between restart and new req processing Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

Revert "PWX-18756 fix race in restart and new req processing (#190) (#…

581d754

…193)" This reverts commit c83c546.

PWX-18756 fix race in restart and new req processing (#190) (#193)

525a5ee

* address race between restart and new req processing Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

sulakshm reviewed Mar 12, 2021

View reviewed changes

Makefile.in Outdated Show resolved Hide resolved

sulakshm reviewed Mar 12, 2021

View reviewed changes

pxd.h Show resolved Hide resolved

px-fastpath makefile cleanup - clean disable

2d040d0

Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

sulakshm force-pushed the v2.7.0-limited branch from fdb53d6 to 2d040d0 Compare March 12, 2021 08:27

prabirpaul added the do-not-merge label Mar 12, 2021

Lakshmi Narasimhan Sundararajan added 2 commits March 12, 2021 14:19

zero initialize all fieds of pxd_dev_init

6f40b8a

Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

cleanup on fc->connected|allow_disconnected

2485f5f

Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

sulakshm reviewed Mar 12, 2021

View reviewed changes

minor fastpath stub edit

1cef21b

Signed-off-by: Lakshmi Narasimhan Sundararajan <lns@portworx.com>

sulakshm approved these changes Mar 12, 2021

View reviewed changes

adikulkarni0 reviewed Mar 12, 2021

View reviewed changes

Makefile.in Outdated Show resolved Hide resolved

adikulkarni0 reviewed Mar 12, 2021

View reviewed changes

pxd.h Outdated Show resolved Hide resolved

adikulkarni0 reviewed Mar 12, 2021

View reviewed changes

prabirpaul added 2 commits March 12, 2021 10:08

Revert "cleanup on fc->connected|allow_disconnected"

af5b6d7

This reverts commit 2485f5f.

Revert "PWX-18756 fix race in restart and new req processing (#190) (#…

879a7c9

…193)" This reverts commit 525a5ee.

adikulkarni0 reviewed Mar 12, 2021

View reviewed changes

pxd.h Show resolved Hide resolved

Review comments

d4ba933

sulakshm reviewed Mar 13, 2021

View reviewed changes

sulakshm reviewed Mar 15, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New px-fuse forked from v2.6.0 with limited set of changes from v2.7.0#199

New px-fuse forked from v2.6.0 with limited set of changes from v2.7.0#199
prabirpaul wants to merge 32 commits intov2.6.0from
v2.7.0-limited

prabirpaul commented Mar 12, 2021 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

sulakshm Mar 12, 2021

Uh oh!

sulakshm Mar 12, 2021

Uh oh!

sulakshm left a comment

Uh oh!

Uh oh!

Uh oh!

adikulkarni0 Mar 12, 2021

Uh oh!

adikulkarni0 Mar 12, 2021

Uh oh!

prabirpaul Mar 12, 2021

Uh oh!

Uh oh!

sulakshm Mar 13, 2021

Uh oh!

sulakshm Mar 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		@@ -1,11 +1,9 @@
		/* Enable stub action if fastpath is not enabled */
		#ifndef __PX_FASTPATH__

		@@ -0,0 +1,160 @@
		/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */

Conversation

prabirpaul commented Mar 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sulakshm Mar 12, 2021

Choose a reason for hiding this comment

Uh oh!

sulakshm Mar 12, 2021

Choose a reason for hiding this comment

Uh oh!

sulakshm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

adikulkarni0 Mar 12, 2021

Choose a reason for hiding this comment

Uh oh!

adikulkarni0 Mar 12, 2021

Choose a reason for hiding this comment

Uh oh!

prabirpaul Mar 12, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sulakshm Mar 13, 2021

Choose a reason for hiding this comment

Uh oh!

sulakshm Mar 15, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

prabirpaul commented Mar 12, 2021 •

edited

Loading