Dokan.Net performance

[mlacamera]

Hi,
I am using the dokan.net version 2.0.4.1 wrapper with its dokan library. To test the performance I use a software (already used in production) that converts pdf with MagickNet (ImageMagick .NET wrapper) using Ghostscript. The software takes a set of pdf, reads them from the filesystem and transforms them into png (multiimage if the pdf has multiple images). To improve performance the software runs in production with multiple threads working on different pdf files. For example if there are 5 threads the software reads 5 files and processes them for each thread with different temporary directories (used by MagickNet). Now if I run the software on normal file system everything works correctly with 5 threads, if instead I use it on Dokan mirror file system the software works fine with only one thread but on multiple threads (on desired file / directory) some times it fails some reading operations (MagickNet gives me an error on read but Dokan.NET doesn't). Could it be a Dokan.NET performance problem?

[Liryna]

Hi @mlacamera , You could try with the native C samples to see whether it is a .NET issue or not.
You could also use Procmon to see which MagickNet call is failing.
It is probably not a performance issue but some race condition in the .NET code or a call incorrectly handled.

Are you running against your own filesystem that use Dokan.Net ?

[mlacamera]

Thanks @Liryna, I try with C samples and the result it is same, read error from MagicNet with 5 thread but no error with 1 thread. With ProcMon i can to see this when the software fails with 5 thread:

I running Dokan.Net e mirror.exe (c samples) in my filesystem

[Liryna]

Could you export the procmon logs and share them here ?
How can we reproduce the issue ?

[mlacamera]

There he is Logfile.zip

[Liryna]

Looks like this is the read failing

There is so much request failing that I cannot even say what is working and expected. Without more context or how to cleanly reproduce it with the native sample, it is going to be difficulty to make a correct analysis.

[mlacamera]

Can you send a sample code?

[Liryna]

See #313 (comment). The samples are package with the installer. You can find how to use them in the dokany repository wiki.

[mlacamera]

The log that I sended you is run with the native C sample of the #313 (comment). I have develop a dedicate console application that use only MagickNet with 5 thread on differnte files and I run the console application without mirror of the #313 (comment) and with mirror of the #313 (comment) (C Native Sample). In first case (without mirror) the console end correctly, in second case the console crash. I am attaching you teh procmon log of the second case I hoping it will be useful to you.
Logfile_withDOKANFileSystemMirrorSample.zip

[Liryna]

Are you able to find out what function is failing in you app and maybe debug Magicknet? The second log stop on a read that succeeded.

[mlacamera]

This is proc log without mirror (first case) that end correctly dirctly on windows file system
Logfile_withoutDOKANFileSystem.zip. Later I will send you stack trace and info on which MagickNet function fails

[mlacamera]

This is stack trace when I run with five thread

Unhandled Exception: System.Runtime.InteropServices.SEHException: External component has thrown an exception.
at ImageMagick.MagickImageCollection.NativeMethods.X64.MagickImageCollection_ReadFile(IntPtr settings, IntPtr& exception)
at ImageMagick.MagickImageCollection.NativeMagickImageCollection.ReadFile(IMagickSettings1 settings) at ImageMagick.MagickImageCollection.AddImages(String fileName, IMagickReadSettings1 readSettings, Boolean ping)
at TestDokan.Program.InfoFile.ConvertPDF(String pdfFile, Double density, String tmpPath) in D:\Tecnoter\TFS_Cloud\OLSA\OLSAOVDfs\TestDokan\TestDokan\TestDokan\Program.cs:line 72
at TestDokan.Program.InfoFile.ConvertPDFAndSave(String pdfFile, Double density, String tmpPath) in D:\Tecnoter\TFS_Cloud\OLSA\OLSAOVDfs\TestDokan\TestDokan\TestDokan\Program.cs:line 40
at TestDokan.Program.<>c__DisplayClass4_0.

b__5(Object o) in D:\Tecnoter\TFS_Cloud\OLSA\OLSAOVDfs\TestDokan\TestDokan\TestDokan\Program.cs:line 136
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart(Object obj)

If I run with one thread all works fine but if I run with two thread or more crash with the above error.

I remember you that software that use MagickNet component run in production environment with more five thread on windows file system without error

[LTRData]

It is a bit strange that a call into native ImageMagick library throws a native exception like that. I would debug this by either calling the same functions from a native application and catch the exception with a filter or __try/__except, or run it within a debugger with native debugging turned on. It seems to me that either that library is getting unexpected data back from Dokan, or it does something else internally that causes for example a null pointer dereference. It is difficult to tell without a native debugger and native stack trace.

[mlacamera]

@LTRData I can send you the solution Net Framework that use MagickNet and a example of files but, the strange thing, is that with one thread work fine, with 2 or more thread (they use different file e temporany directory) do not work, instead on Windows file system work fine with multi thread

[LTRData]

That happens sometimes. Threads introduce a complexity that can cause issues to only show up in multi thread scenarios.

Yes I could maybe take a look at what happens. Since the exception happens in native code, I need to run the managed solution with native libraries in debugger with native debugging turned on and preferably native symbol files available.

[mlacamera]

In this zip TestDokan.zip you found the code test on Visual Studio Solution, for MagickNet yuo remember to install GostScript (I use this https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs9561/gs9561w64.exe). In this zip
RootExample.zip you found three directory that you must copy on Dokan File System Root.