The execution has failed due to the bug in the SevenZipSharp. (code: -2147024882)

[deltaone]

when compress large amount of data (about 1GB) with using 7-zip.dll v16.00 / v16.04 exception is up after 2-3 program run ...
when use 7-zip.dll v9.20 exception has gone ...
when set compressor.CompressionLevel = CompressionLevel.Fast; exeption has gone ...

SevenZip.SevenZipException was unhandled
HResult=-2146233088
Message=The execution has failed due to the bug in the SevenZipSharp. (code: -2147024882)
Source=main
StackTrace:
в SevenZip.SevenZipBase.ThrowException(CallbackBase handler, Exception[] e) в d:\work\fdsb\src\lib\SevenZip\Common.cs:строка 324
в SevenZip.SevenZipBase.CheckedExecute(Int32 hresult, String message, CallbackBase handler) в d:\work\fdsb\src\lib\SevenZip\Common.cs:строка 351
в SevenZip.SevenZipCompressor.CompressFilesEncrypted(Stream archiveStream, Int32 commonRootLength, String password, String[] fileFullNames) в d:\work\fdsb\src\lib\SevenZip\SevenZipCompressor.cs:строка 1270
в SevenZip.SevenZipCompressor.CompressDirectory(String directory, Stream archiveStream, String password, String searchPattern, Boolean recursion) в d:\work\fdsb\src\lib\SevenZip\SevenZipCompressor.cs:строка 1545
в SevenZip.SevenZipCompressor.CompressDirectory(String directory, String archiveName, String password, String searchPattern, Boolean recursion) в d:\work\fdsb\src\lib\SevenZip\SevenZipCompressor.cs:строка 1472
в SevenZip.SevenZipCompressor.CompressDirectory(String directory, String archiveName) в d:\work\fdsb\src\lib\SevenZip\SevenZipCompressor.cs:строка 1314
в Core.GM.Main(String[] args) в d:\work\fdsb\src\Main.cs:строка 183
в System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
в System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
в Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
в System.Threading.ThreadHelper.ThreadStart_Context(Object state)
в System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
в System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
в System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
в System.Threading.ThreadHelper.ThreadStart()
InnerException:

[pablopioli]

I'm the maintainer of a project that uses this library to create big files (https://github.com/ColiseoSoftware/hypervbackup). In the past I tinkered with the abandoned source code because it gave us a lot for errors.
Until I found this fork, that works without flaws for me (until now).

I will be glad to help in what I can, do you have some way to reproduce this error?

[StevenBonePgh]

Apologies - I somehow missed this issue. The error code of '-2147024882' is a Win32 error code for ERROR_OUTOFMEMORY 'Not enough storage is available to complete this operation.'

A few thoughts:

1. There may be some difference in how memory is being managed between these versions of 7-Zip.
2. The code is leaking resources somewhere. Is it the C# code or the 7-zip code - I have no idea.

I haven't really looked into the code all that much other than to fix the more terrible bugs and apply various patches found in the discussion forums. I'd concentrate efforts on the second point as I'd guess there is a leak of a native handle somewhere. Good repro test cases would be repeated compressing of the same large file in a loop until it breaks which would confirm the latter case, then it is a matter of figuring out who and where the leak is. I'm fairly busy at the moment, but will look into it when I have the chance.

[pablopioli]

Thanks for your response.

Based on it I made a quick tests. Compressed a virtual machine of 350 GB.

Using the 7zip dll included in the original project (doesn't say wich version is) used 25 MB of RAM. With version 16.04 of 7zip the same program used 1.6 GB of RAM!

Guess the problem is in the 7zip dll, but I'll take a look at the code anyway, if a found something will make a PR.

For now, if anyone has the memory problem, the solution is to replace the built-in version.

[pablopioli]

Made another set of tests, with version 16.04 and without
CustomParameters.Add("mt", "on")

So the RAM usage is:

• Original version: 25 MB
• Version 16.04 without MT: 135 MB.
• Version 16.04 with MT: 1.6 GB.

So, I believe it gives me a place to start, but I'm still trying to build a repro set.

For now there are options to avoid this error.

[ebolefeysot]

Same error here: compressing a 64MB file from a memorystream into an empty 7z archive (using CompressStreamDictionary method, 32b).
CompressionLevel Ultra and high -> error, Normal, low, fast and none -> ok.
Note that I get the same result with all 7z.dll (9.2, 16.4, 17 beta, original).
Exception raised on each run.
The error code is sent back by the 7z.dll library.
No solution yet :(
Any idea ?

[pablopioli]

Have been testing this for over a month, some things I found

Do not use
CustomParameters.Add("mt", "on")

Use Normal compression level

With this it failed once in 100 executions

[ebolefeysot]

I did some tests, and wrote a unit test for sevenzipsharp solution. The exception is raised on every test.
Can someone have a look?
It seems the error comes from 7z.dll library, can someone confirm this point ?
UnitTest.zip

[ebolefeysot]

7zip thread here: https://sourceforge.net/p/sevenzip/bugs/2069/

[ebolefeysot]

Did you see the answer on the other topic ?

It seems I have to find the better parameter depending of the available ram. It is said that a value of ram/10 should be ok as dictionary size. For a 32b process, it would be 2GB/10 -> 200MB.

I tried to set the dictionary size, but with no success.
CustomParameters.Add("d", "200000");
CustomParameters.Add("d", "200000b");
CustomParameters.Add("d", "128m");
Nothing is accepted :(
What is the range for this value ?

[pablopioli]

The igor solution didn't worked out?

"x9" with "mt2" - is OK in most cases.

[ebolefeysot]

I can't use mt2. mt seems to accept only on/off

I use this code:

            if (!Environment.Is64BitProcess)
            {
                if (stream.Length>=16*1024*1024)
                {
                    int sizeG = (int)(stream.Length / 1024 / 1024 / 1024);
                    int param;

                    // from 16MB to 2GB: use 23
                        if (sizeG <= 2)
                        {
                            param = 23;
                        }
                        else
                        {
                            // from 2GB : use 24 - size(GB)
                            param = Math.Max(24 - sizeG, 1);
                        }
                    szc.CustomParameters.Add("d", param.ToString());
                }
            }

Tested with files from 16MB to 8GB.

Could it be included directly in the sevenzipsharp source code ?

[pablopioli]

mt is on/off only. I set it on off, and helps a lot. Is slightly slower but taxes the processor a lot less, with mt on it can take almost 100% of the processor giving a slugish feeling.

I can include your code in my fork until is included in the official source.

But, I have found the same problem in 64 bits processes, so it will not cover all uses. I would apply to 64 bits too. It can be a little slower but a lot more stable.

[pablopioli]

The d parameter didn't work when I compressed 150 GB of data.
What's the meaning of this parameter? As the machine has 32GB of RAM I used 24 as the value to avoid negative numbers.

[ebolefeysot]

Does it work with 'd 24' and 150GB files ?

[pablopioli]

Y used d24, you mean d 24 with an space between?