Write performance problem, can not reach 1000qps #27
Description
Run Xapiand as a test, and I found the write throughput can hardly reach 1000 qps, even as cluster. Write with golang and msgpack, and can hold about 800k qps when those services write to Elasticsearch. The number is about 900 qps at beginning, and slow down to 500 qps as document amount increase to 500k.
But the cpu usage of Xapiand service stay below 1000%(cpu core amount is 64) at the same time. Server built by Xeon Gold 5218 x2, 256G, Intel SSD DC P4510 2.0TB x2, the system is Debian 9. Which seems not the performance bottleneck.
I try to perf Xapiand service (svg send with this issue), and found doc_preparer thread wait for spin lock at enqueue and dequeue. I read the code and try to do some thing, but my C++ skill is very poor. Is there any place to do some optimization?
Thank you for your awesome work, and look forward for your help.
