[Issue]
I use og-1.13.2 on Linux box with AlmaLinux release 8.9. I do try to test mixed load of multipart_write, read, list, metadata, delete S3 operations using og-1.13.2. But, I find if concurrency > 6, I hit the following errors constantly like the followings.
2024-04-01 12:52:24.769 INFO [client-3] - I/O exception (java.net.SocketException) caught when processing request to {}->http://cxo-s3-cluster39:8443: Broken pipe (Write failed)
2024-04-01 12:52:24.769 INFO [client-3] - Retrying request to {}->http://cxo-s3-cluster39:8443
2024-04-01 12:52:26.819 INFO [client-2] - I/O exception (java.net.SocketException) caught when processing request to {}->http://cxo-s3-cluster39:8443: Broken pipe (Write failed)
2024-04-01 12:52:26.819 INFO [client-2] - Retrying request to {}->http://cxo-s3-cluster39:8443
But, I don't see "Broken pipe (Write failed)" issue at all when I test the mixed load of write, read, list, metadata, delete S3 operations using og-1.13.2 with concurrency 24 even 48.
[Questions]
1. Is it the right behavior or not for mixed load of multipart_write, read, list, metadata, delete S3 operations using og?
2. Is there any way to avoid "Broken pipe (Write failed)" issue?
[Reference mixed load profile of og used]
[root# cat mpu_04_01_2024_100req_6t_wl_og.json
{
"host": {
"selection": "roundrobin",
"choices": ["<S3 storage endpoint IP>:Port"]
},
"api": "s3",
"uri_root": "/",
"container": {
"selection": "roundrobin",
"prefix": "bkt-pl10a-",
"min_suffix": 0,
"max_suffix": 31
},
"multipart_write": {
"upload": {
"part_size": {
"choices": [
{
"choice": 5242880,
"weight": 1
}
],
"selection": "roundrobin"
#"selection": "random"
},
"parts_per_session": {
"choices": [],
"selection": "roundrobin"
#"selection": "random"
},
"target_sessions": 1
},
"weight": 100
},
"read": {
"weight": 50,
"object": {
"selection": "roundrobin"
}
},
"list": {
"weight": 20,
"parameters": {
"list-type": 2
},
"object": {
"selection": "random"
}
},
"metadata":{
"weight": 10,
"object": {
"selection": "roundrobin"
}
},
"delete": {
"weight": 50,
"object": {
"selection": "roundrobin"
}
},
"filesize": {
"selection": "random",
"choices": [
{
"choice": {
"distribution": "uniform",
"average": 64,
"average_unit": "mb",
"spread": 5,
"spread_unit": "mb"
},
"weight": 7.0
},
{
"choice": {
"distribution": "normal",
"average": 32,
"average_unit": "mb"
},
"weight": 1.0
},
{
"choice": {
"distribution": "normal",
"average": 100,
"average_unit": "mb"
},
"weight": 1.0
}
]
},
"concurrency": {
"type": "threads",
"count": 6
},
"authentication": {
"type": "awsv4",
"username": "testuser",
"password": "testuser"
},
"stopping_conditions": {
"operations": 100
},
"client": {
"connect_timeout": 30000,
"so_timeout": 30000,
"retry_count": 40
}
}
Thank you very much.
[Issue]
I use og-1.13.2 on Linux box with AlmaLinux release 8.9. I do try to test mixed load of multipart_write, read, list, metadata, delete S3 operations using og-1.13.2. But, I find if concurrency > 6, I hit the following errors constantly like the followings.
But, I don't see "Broken pipe (Write failed)" issue at all when I test the mixed load of write, read, list, metadata, delete S3 operations using og-1.13.2 with concurrency 24 even 48.
[Questions]
[Reference mixed load profile of og used]
Thank you very much.