diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index edb30274..d5b6c119 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -31,17 +31,17 @@ jobs: BenchmarkMode: Automatic BenchmarkSection: query - name: Renaming result file - run: mv MiniExcelLibs.Benchmarks.XlsxBenchmark-report-github.md query-benchmark.md - working-directory: ./benchmarks/MiniExcel.Benchmarks/BenchMarkDotNet.Artifacts/results + run: mv MiniExcelLibs.Benchmarks.BenchmarkSections.QueryXlsxBenchmark-report-github.md query-benchmark.md + working-directory: ./benchmarks/MiniExcel.Benchmarks/BenchmarkDotNet.Artifacts/results - name: Save benchmark results uses: actions/upload-artifact@v4 with: name: query-benchmark-result - path: ./benchmark/MiniExcel.Benchmarks/BenchMarkDotNet.Artifacts/results/*.md - + path: ./benchmarks/MiniExcel.Benchmarks/BenchmarkDotNet.Artifacts/results/*.md + CreateBenchmark: runs-on: ubuntu-latest - + steps: - uses: actions/checkout@v4 - name: Setup .NET @@ -61,17 +61,17 @@ jobs: BenchmarkMode: Automatic BenchmarkSection: create - name: Renaming result file - run: mv MiniExcelLibs.Benchmarks.XlsxBenchmark-report-github.md create-benchmark.md - working-directory: ./benchmarks/MiniExcel.Benchmarks/BenchMarkDotNet.Artifacts/results + run: mv MiniExcelLibs.Benchmarks.BenchmarkSections.CreateXlsxBenchmark-report-github.md create-benchmark.md + working-directory: ./benchmarks/MiniExcel.Benchmarks/BenchmarkDotNet.Artifacts/results - name: Save benchmark results uses: actions/upload-artifact@v4 with: name: create-benchmark-result - path: ./benchmark/MiniExcel.Benchmarks/BenchMarkDotNet.Artifacts/results/*.md - + path: ./benchmarks/MiniExcel.Benchmarks/BenchmarkDotNet.Artifacts/results/*.md + TemplateBenchmark: runs-on: ubuntu-latest - + steps: - uses: actions/checkout@v4 - name: Setup .NET @@ -91,18 +91,18 @@ jobs: BenchmarkMode: Automatic BenchmarkSection: template - name: Renaming result file - run: mv MiniExcelLibs.Benchmarks.XlsxBenchmark-report-github.md template-benchmark.md - working-directory: ./benchmarks/MiniExcel.Benchmarks/BenchMarkDotNet.Artifacts/results + run: mv MiniExcelLibs.Benchmarks.BenchmarkSections.TemplateXlsxBenchmark-report-github.md template-benchmark.md + working-directory: ./benchmarks/MiniExcel.Benchmarks/BenchmarkDotNet.Artifacts/results - name: Save benchmark results uses: actions/upload-artifact@v4 with: name: template-benchmark-result - path: ./benchmark/MiniExcel.Benchmarks/BenchMarkDotNet.Artifacts/results/*.md + path: ./benchmarks/MiniExcel.Benchmarks/BenchmarkDotNet.Artifacts/results/*.md PushBenchmarksResults: runs-on: ubuntu-latest needs: [ QueryBenchmark, CreateBenchmark, TemplateBenchmark ] - + steps: - uses: actions/checkout@v4 with: @@ -110,8 +110,8 @@ jobs: - name: Fetch benchmark results uses: actions/download-artifact@v4 with: - path: ./benchmarks/results - merge-multiple: true + path: ./benchmarks/results + merge-multiple: true - name: Commit reports working-directory: ./benchmarks/results run: | diff --git a/README-NuGet.md b/README-NuGet.md index ffc25289..6d9fb9f5 100644 --- a/README-NuGet.md +++ b/README-NuGet.md @@ -15,9 +15,9 @@ At present, most popular frameworks need to load all the data into the memory to ### Features - Low memory consumption, avoid OOM (out of memory) and full GC -- Support `real-time` operation of each row of data -- Support LINQ deferred execution, it can do low-consumption, fast paging and other complex queries -- Lightweight, without Microsoft Office installed, no COM+, DLL size is less than 150KB +- Supports real time operation of each row of data +- Supports LINQ deferred execution, it can do low-consumption, fast paging and other complex queries +- Lightweight, without Microsoft Office installed, no COM+, DLL size is less than 400KB - Easy API style to read/write/fill excel ### Get Started @@ -48,55 +48,52 @@ Please Check [TODO](https://github.com/mini-software/MiniExcel/projects/1?fulls ### Performance -Benchmarks logic can be found in [MiniExcel.Benchmarks](benchmarks/MiniExcel.Benchmarks/Program.cs) , and test cli +The code for the benchmarks can be found in [MiniExcel.Benchmarks](https://github.com/mini-software/MiniExcel/tree/master/benchmarks/MiniExcel.Benchmarks). +To run all the benchmarks use: ```bash -dotnet run -p .\benchmarks\MiniExcel.Benchmarks\ -c Release -f netcoreapp3.1 -- -f * --join +dotnet run -project .\benchmarks\MiniExcel.Benchmarks -c Release -f net9.0 -filter * --join ``` -Output from the latest run is : - -```bash -BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19042 -Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores - [Host] : .NET Framework 4.8 (4.8.4341.0), X64 RyuJIT - Job-ZYYABG : .NET Framework 4.8 (4.8.4341.0), X64 RyuJIT -IterationCount=3 LaunchCount=3 WarmupCount=3 +Hardware and settings used are the following: +``` +BenchmarkDotNet v0.15.0, Linux Ubuntu 24.04.2 LTS (Noble Numbat) +AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores +.NET SDK 9.0.300 + [Host] : .NET 9.0.5 (9.0.525.21509), X64 RyuJIT AVX2 + ShortRun : .NET 9.0.5 (9.0.525.21509), X64 RyuJIT AVX2 ``` - -Benchmark History : [Link](https://github.com/mini-software/MiniExcel/issues/276) - - #### Import/Query Excel -Logic : [**Test1,000,000x10.xlsx**](benchmarks/MiniExcel.Benchmarks/Test1%2C000%2C000x10.xlsx) as performance test basic file, 1,000,000 rows * 10 columns "HelloWorld" cells, 23 MB file size +The file used to test performance is [**Test1,000,000x10.xlsx**](https://github.com/mini-software/MiniExcel/tree/master/benchmarks/MiniExcel.Benchmarks/Test1%2C000%2C000x10.xlsx), a 32MB document containing 1,000,000 rows * 10 columns whose cells are filled with the string "HelloWorld". +| Method | Mean | StdDev | Error | Gen0 | Gen1 | Gen2 | Allocated | +|--------------------------------------|-----------------:|---------------:|-----------------:|------------:|------------:|----------:|--------------:| +| 'MiniExcel QueryFirst' | 63.70 μs | 0.337 μs | 6.144 μs | 2.9297 | 2.7669 | - | 49.67 KB | +| 'ExcelDataReader QueryFirst' | 5,010,679.51 μs | 53,245.186 μs | 971,390.400 μs | 105000.0000 | 333.3333 | - | 1717272.56 KB | +| 'MiniExcel Query' | 9,172,286.91 μs | 12,805.326 μs | 233,616.824 μs | 448500.0000 | 4666.6667 | - | 7327883.36 KB | +| 'ExcelDataReader Query' | 10,609,617.09 μs | 29,055.953 μs | 530,088.745 μs | 275666.6667 | 68666.6667 | - | 4504691.87 KB | +| 'Epplus QueryFirst' | 13,770,656.24 μs | 45,909.809 μs | 837,565.827 μs | 174333.3333 | 88833.3333 | 4333.3333 | 3700587.76 KB | +| 'Epplus Query' | 19,257,306.83 μs | 63,117.956 μs | 1,151,506.486 μs | 452333.3333 | 90500.0000 | 5333.3333 | 8223933.16 KB | +| 'ClosedXml Query' | 31,070,263.83 μs | 342,973.671 μs | 6,257,116.502 μs | 401666.6667 | 104166.6667 | 3333.3333 | 6822559.68 KB | +| 'ClosedXml QueryFirst' | 31,141,877.48 μs | 21,006.538 μs | 383,237.459 μs | 402166.6667 | 104833.3333 | 3833.3333 | 6738357.8 KB | +| 'OpenXmlSDK QueryFirst' | 31,750,686.63 μs | 263,328.569 μs | 4,804,093.357 μs | 374666.6667 | 374500.0000 | 3166.6667 | 6069266.96 KB | +| 'OpenXmlSDK Query' | 32,919,119.46 μs | 411,395.682 μs | 7,505,388.691 μs | 374666.6667 | 374500.0000 | 3166.6667 | 6078467.83 KB | -| Library | Method | Max Memory Usage | Mean | -| ---------------------------- | -------------: | ---------------: | ---------------: | -| MiniExcel | 'MiniExcel QueryFirst' | 0.109 MB | 0.0007264 sec | -| ExcelDataReader | 'ExcelDataReader QueryFirst' | 15.24 MB | 10.66421 sec | -| MiniExcel | 'MiniExcel Query' | 17.3 MB | 14.17933 sec | -| ExcelDataReader | 'ExcelDataReader Query' | 17.3 MB | 22.56508 sec | -| Epplus | 'Epplus QueryFirst' | 1,452 MB | 18.19801 sec | -| Epplus | 'Epplus Query' | 1,451 MB | 23.64747 sec | -| OpenXmlSDK | 'OpenXmlSDK Query' | 1,412 MB | 52.00327 sec | -| OpenXmlSDK | 'OpenXmlSDK QueryFirst' | 1,413 MB | 52.34865 sec | -| ClosedXml | 'ClosedXml QueryFirst' | 2,158 MB | 66.18897 sec | -| ClosedXml | 'ClosedXml Query' | 2,184 MB | 191.43412 sec | #### Export/Create Excel -Logic : create a total of 10,000,000 "HelloWorld" excel +Logic: create a total of 10,000,000 "HelloWorld" cells Excel document -| Library | Method | Max Memory Usage | Mean | -| ------------------------ | -------------: | ---------------: | -----------: | -| MiniExcel | 'MiniExcel Create Xlsx' | 15 MB | 11.53181 sec | -| Epplus | 'Epplus Create Xlsx' | 1,204 MB | 22.50971 sec | -| OpenXmlSdk | 'OpenXmlSdk Create Xlsx' | 2,621 MB | 42.47399 sec | -| ClosedXml | 'ClosedXml Create Xlsx' | 7,141 MB | 140.93992 sec | +| Method | Mean | StdDev | Error | Gen0 | Gen1 | Gen2 | Allocated | +|----------------------------------------------|---------:|---------:|---------:|------------:|------------:|----------:|----------:| +| 'MiniExcel Create Xlsx' | 4.427 s | 0.0056 s | 0.1023 s | 251666.6667 | 1833.3333 | 1666.6667 | 3.92 GB | +| 'OpenXmlSdk Create Xlsx by DOM mode' | 22.729 s | 0.1226 s | 2.2374 s | 307000.0000 | 306833.3333 | 3833.3333 | 6.22 GB | +| 'ClosedXml Create Xlsx' | 22.851 s | 0.0190 s | 0.3473 s | 195500.0000 | 54500.0000 | 4166.6667 | 4.48 GB | +| 'Epplus Create Xlsx' | 23.027 s | 0.0088 s | 0.1596 s | 89000.0000 | 17500.0000 | 6000.0000 | 2.51 GB | +Warning: these results may be outdated. You can find the benchmarks for the latest release [here](https://github.com/mini-software/MiniExcel/tree/master/benchmarks/results). ### Documents diff --git a/README.md b/README.md index 288c5308..d9b4d01c 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -
+

NuGet Build status star GitHub stars @@ -11,19 +11,19 @@ [](https://www.dotnetfoundation.org/) -

+

This project is part of the .NET Foundation and operates under their code of conduct.

--- -
+

English | 简体中文 | 繁體中文

--- -
+
Your Star and Donate can make MiniExcel better
@@ -43,9 +43,9 @@ At present, most popular frameworks need to load all the data into the memory to ### Features - Low memory consumption, avoid OOM (out of memory) and full GC -- Support `real-time` operation of each row of data -- Support LINQ deferred execution, it can do low-consumption, fast paging and other complex queries -- Lightweight, without Microsoft Office installed, no COM+, DLL size is less than 150KB +- Supports real time operation of each row of data +- Supports LINQ deferred execution, it can do low-consumption, fast paging and other complex queries +- Lightweight, without Microsoft Office installed, no COM+, DLL size is less than 400KB - Easy API style to read/write/fill excel ### Get Started @@ -76,54 +76,18 @@ Please Check [TODO](https://github.com/mini-software/MiniExcel/projects/1?fulls ### Performance -Benchmarks logic can be found in [MiniExcel.Benchmarks](benchmarks/MiniExcel.Benchmarks/Program.cs) , and test cli +The code for the benchmarks can be found in [MiniExcel.Benchmarks](benchmarks/MiniExcel.Benchmarks/Program.cs). -```bash -dotnet run -p .\benchmarks\MiniExcel.Benchmarks\ -c Release -f netcoreapp3.1 -- -f * --join -``` +The file used to test performance is [**Test1,000,000x10.xlsx**](benchmarks/MiniExcel.Benchmarks/Test1%2C000%2C000x10.xlsx), a 32MB document containing 1,000,000 rows * 10 columns whose cells are filled with the string "HelloWorld". -Output from the latest run is : +To run all the benchmarks use: ```bash -BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19042 -Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores - [Host] : .NET Framework 4.8 (4.8.4341.0), X64 RyuJIT - Job-ZYYABG : .NET Framework 4.8 (4.8.4341.0), X64 RyuJIT -IterationCount=3 LaunchCount=3 WarmupCount=3 +dotnet run -project .\benchmarks\MiniExcel.Benchmarks -c Release -f net9.0 -filter * --join ``` -Benchmark History : [Link](https://github.com/mini-software/MiniExcel/issues/276) - - - -#### Import/Query Excel - -Logic : [**Test1,000,000x10.xlsx**](benchmarks/MiniExcel.Benchmarks/Test1%2C000%2C000x10.xlsx) as performance test basic file, 1,000,000 rows * 10 columns "HelloWorld" cells, 23 MB file size - - -| Library | Method | Max Memory Usage | Mean | -| ---------------------------- | -------------: | ---------------: | ---------------: | -| MiniExcel | 'MiniExcel QueryFirst' | 0.109 MB | 0.0007264 sec | -| ExcelDataReader | 'ExcelDataReader QueryFirst' | 15.24 MB | 10.66421 sec | -| MiniExcel | 'MiniExcel Query' | 17.3 MB | 14.17933 sec | -| ExcelDataReader | 'ExcelDataReader Query' | 17.3 MB | 22.56508 sec | -| Epplus | 'Epplus QueryFirst' | 1,452 MB | 18.19801 sec | -| Epplus | 'Epplus Query' | 1,451 MB | 23.64747 sec | -| OpenXmlSDK | 'OpenXmlSDK Query' | 1,412 MB | 52.00327 sec | -| OpenXmlSDK | 'OpenXmlSDK QueryFirst' | 1,413 MB | 52.34865 sec | -| ClosedXml | 'ClosedXml QueryFirst' | 2,158 MB | 66.18897 sec | -| ClosedXml | 'ClosedXml Query' | 2,184 MB | 191.43412 sec | - -#### Export/Create Excel - -Logic : create a total of 10,000,000 "HelloWorld" excel +You can find the benchmarks' results for the latest release [here](benchmarks/results). -| Library | Method | Max Memory Usage | Mean | -| ------------------------ | -------------: | ---------------: | -----------: | -| MiniExcel | 'MiniExcel Create Xlsx' | 15 MB | 11.53181 sec | -| Epplus | 'Epplus Create Xlsx' | 1,204 MB | 22.50971 sec | -| OpenXmlSdk | 'OpenXmlSdk Create Xlsx' | 2,621 MB | 42.47399 sec | -| ClosedXml | 'ClosedXml Create Xlsx' | 7,141 MB | 140.93992 sec | ### Excel Query/Import @@ -156,9 +120,9 @@ using (var stream = File.OpenRead(path)) * dynamic key is `A.B.C.D..` -| MiniExcel | 1 | -| -------- | -------- | -| Github | 2 | +| MiniExcel | 1 | +|-----------|---| +| Github | 2 | ```csharp @@ -182,10 +146,10 @@ note : same column name use last right one Input Excel : -| Column1 | Column2 | -| -------- | -------- | -| MiniExcel | 1 | -| Github | 2 | +| Column1 | Column2 | +|-----------|---------| +| MiniExcel | 1 | +| Github | 2 | ```csharp @@ -372,10 +336,10 @@ MiniExcel.SaveAs(path, values); Create File Result : -| Column1 | Column2 | -| -------- | -------- | -| MiniExcel | 1 | -| Github | 2 | +| Column1 | Column2 | +|-----------|---------| +| MiniExcel | 1 | +| Github | 2 | #### 3. IDataReader @@ -993,11 +957,11 @@ When the template is rendered, the `$` prefix will be removed and `$enumrowstart ##### 2. Other Example Formulas: -| | | -|------------|-----------------------------------------------------------------------------------------| -|Sum |`$=SUM(C{{$enumrowstart}}:C{{$enumrowend}})` | -|Alt. Average|`$=SUM(C{{$enumrowstart}}:C{{$enumrowend}}) / COUNT(C{{$enumrowstart}}:C{{$enumrowend}})`| -|Range |`$=MAX(C{{$enumrowstart}}:C{{$enumrowend}}) - MIN(C{{$enumrowstart}}:C{{$enumrowend}})` | +| | | +|--------------|-------------------------------------------------------------------------------------------| +| Sum | `$=SUM(C{{$enumrowstart}}:C{{$enumrowend}})` | +| Alt. Average | `$=SUM(C{{$enumrowstart}}:C{{$enumrowend}}) / COUNT(C{{$enumrowstart}}:C{{$enumrowend}})` | +| Range | `$=MAX(C{{$enumrowstart}}:C{{$enumrowend}}) - MIN(C{{$enumrowstart}}:C{{$enumrowend}})` | #### 11. Other