Skip to content

Commit 0c9ffe0

Browse files
authored
Tidy readme
1 parent c1afaa1 commit 0c9ffe0

File tree

1 file changed

+35
-25
lines changed

1 file changed

+35
-25
lines changed

README.md

Lines changed: 35 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,19 @@
11
# EnumerableAsyncProcessor
2+
23
Process Multiple Asynchronous Tasks in Various Ways - One at a time / Batched / Rate limited / Concurrently
34

45
[![nuget](https://img.shields.io/nuget/v/EnumerableAsyncProcessor.svg)](https://www.nuget.org/packages/EnumerableAsyncProcessor/)
56
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/9c57d16dc4a841629560707c5ab3019d)](https://www.codacy.com/gh/thomhurst/EnumerableAsyncProcessor/dashboard?utm_source=github.com&utm_medium=referral&utm_content=thomhurst/EnumerableAsyncProcessor&utm_campaign=Badge_Grade)
67
[![CodeFactor](https://www.codefactor.io/repository/github/thomhurst/enumerableAsyncProcessor/badge)](https://www.codefactor.io/repository/github/thomhurst/enumerableAsyncProcessor)
78
<!-- ![Nuget](https://img.shields.io/nuget/dt/EnumerableAsyncProcessor) -->
89

9-
## Support
10-
11-
If this library helped you, consider buying me a coffee
12-
13-
<a href="https://www.buymeacoffee.com/tomhurst" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: auto !important;width: auto !important;" ></a>
14-
1510
## Installation
16-
.NET 6 Required
1711

1812
Install via Nuget
1913
`Install-Package EnumerableAsyncProcessor`
2014

2115
## Why I built this
16+
2217
Because I've come across situations where you need to fine tune the rate at which you do things.
2318
Maybe you want it fast.
2419
Maybe you want it slow.
@@ -28,6 +23,7 @@ Maybe you just don't want to write all the boilerplate code that comes with mana
2823
### Rate Limited Parallel Processor
2924

3025
**Types**
26+
3127
| Type | Source Object | Return Object | Method 1 | Method 2 |
3228
|--------------------------------------------------|---------------|---------------|--------------------| ------------------ |
3329
| `RateLimitedParallelAsyncProcessor` ||| `.WithExecutionCount(int)` | `.ForEachAsync(delegate)` |
@@ -36,30 +32,32 @@ Maybe you just don't want to write all the boilerplate code that comes with mana
3632
| `ResultRateLimitedParallelAsyncProcessor<TInput, TOutput>` ||| `.WithItems(IEnumerable<TInput>)` | `.SelectAsync(delegate)` |
3733

3834
**How it works**
39-
Processes your Asynchronous Tasks in Parallel, but honouring the limit that you set. As one finishes, another will start.
35+
Processes your Asynchronous Tasks in Parallel, but honouring the limit that you set. As one finishes, another will start.
4036

4137
E.g. If you set a limit of 100, only 100 should ever run at any one time
4238

4339
This is a hybrid between Parallel Processor and Batch Processor (see below) - Trying to address the caveats of both. Increasing the speed of batching, but not overwhelming the system by using full parallelisation.
4440

4541
**Usage**
42+
4643
```csharp
4744
var ids = Enumerable.Range(0, 5000).ToList();
4845

4946
// SelectAsync for if you want to return something
50-
var results = await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
47+
var results = await ids
5148
.SelectAsync(id => DoSomethingAndReturnSomethingAsync(id), CancellationToken.None)
5249
.ProcessInParallel(levelOfParallelism: 100);
5350

5451
// ForEachAsync for when you have nothing to return
55-
await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
52+
await ids
5653
.ForEachAsync(id => DoSomethingAsync(id), CancellationToken.None)
5754
.ProcessInParallel(levelOfParallelism: 100);
5855
```
5956

6057
### Timed Rate Limited Parallel Processor (e.g. Limit RPS)
6158

6259
**Types**
60+
6361
| Type | Source Object | Return Object | Method 1 | Method 2 |
6462
|--------------------------------------------------|---------------|---------------|--------------------| ------------------ |
6563
| `TimedRateLimitedParallelAsyncProcessor` ||| `.WithExecutionCount(int)` | `.ForEachAsync(delegate)` |
@@ -68,29 +66,31 @@ await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToA
6866
| `ResultTimedRateLimitedParallelAsyncProcessor<TInput, TOutput>` ||| `.WithItems(IEnumerable<TInput>)` | `.SelectAsync(delegate)` |
6967

7068
**How it works**
71-
Processes your Asynchronous Tasks in Parallel, but honouring the limit that you set over the timespan that you set. As one finishes, another will start, unless you've hit the maximum allowed for the current timespan duration.
69+
Processes your Asynchronous Tasks in Parallel, but honouring the limit that you set over the timespan that you set. As one finishes, another will start, unless you've hit the maximum allowed for the current timespan duration.
7270

73-
E.g. If you set a limit of 100, and a timespan of 1 second, only 100 operation should ever run at any one time over the course of a second. If the operation finishes sooner than a second (or your provided timespan), it'll wait and then start the next operation once that timespan has elapsed.
71+
E.g. If you set a limit of 100, and a timespan of 1 second, only 100 operation should ever run at any one time over the course of a second. If the operation finishes sooner than a second (or your provided timespan), it'll wait and then start the next operation once that timespan has elapsed.
7472

7573
This is useful in scenarios where, for example, you have an API but it has a request per second limit
7674

7775
**Usage**
76+
7877
```csharp
7978
var ids = Enumerable.Range(0, 5000).ToList();
8079

8180
// SelectAsync for if you want to return something
82-
var results = await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
81+
var results = await ids
8382
.SelectAsync(id => DoSomethingAndReturnSomethingAsync(id), CancellationToken.None)
8483
.ProcessInParallel(levelOfParallelism: 100, TimeSpan.FromSeconds(1));
8584

8685
// ForEachAsync for when you have nothing to return
87-
await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
86+
await ids
8887
.ForEachAsync(id => DoSomethingAsync(id), CancellationToken.None)
8988
.ProcessInParallel(levelOfParallelism: 100, TimeSpan.FromSeconds(1));
9089
```
9190

9291
**Caveats**
93-
- If your operations take longer than your provided TimeSpan, you probably won't get your desired throughput. This processor ensures you don't go over your rate limit, but will not increase parallel execution if you're below it.
92+
93+
- If your operations take longer than your provided TimeSpan, you probably won't get your desired throughput. This processor ensures you don't go over your rate limit, but will not increase parallel execution if you're below it.
9494

9595
### One At A Time
9696

@@ -107,26 +107,29 @@ await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToA
107107
Processes your Asynchronous Tasks One at a Time. Only one will ever progress at a time. As one finishes, another will start
108108

109109
**Usage**
110+
110111
```csharp
111112
var ids = Enumerable.Range(0, 5000).ToList();
112113

113114
// SelectAsync for if you want to return something
114-
var results = await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
115+
var results = await ids
115116
.SelectAsync(id => DoSomethingAndReturnSomethingAsync(id), CancellationToken.None)
116117
.ProcessOneAtATime();
117118

118119
// ForEachAsync for when you have nothing to return
119-
await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
120+
await ids
120121
.ForEachAsync(id => DoSomethingAsync(id), CancellationToken.None)
121122
.ProcessOneAtATime();
122123
```
123124

124125
**Caveats**
125-
- Slowest method
126+
127+
- Slowest method
126128

127129
### Batch
128130

129131
**Types**
132+
130133
| Type | Source Object | Return Object | Method 1 | Method 2 |
131134
|--------------------------------------------------|---------------|---------------|--------------------| ------------------ |
132135
| `BatchAsyncProcessor` ||| `.WithExecutionCount(int)` | `.ForEachAsync(delegate)` |
@@ -138,27 +141,30 @@ await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToA
138141
Processes your Asynchronous Tasks in Batches. The next batch will not start until every Task in previous batch has finished
139142

140143
**Usage**
144+
141145
```csharp
142146
var ids = Enumerable.Range(0, 5000).ToList();
143147

144148
// SelectAsync for if you want to return something
145-
var results = await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
149+
var results = await ids
146150
.SelectAsync(id => DoSomethingAndReturnSomethingAsync(id), CancellationToken.None)
147151
.ProcessInBatches(batchSize: 100);
148152

149153
// ForEachAsync for when you have nothing to return
150-
await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
154+
await ids
151155
.ForEachAsync(id => DoSomethingAsync(id), CancellationToken.None)
152156
.ProcessInBatches(batchSize: 100);
153157
```
154158

155159
**Caveats**
156-
- If even just 1 Task in a batch is slow or hangs, this will prevent the next batch from starting
157-
- If you set a batch of 100, and 70 have finished, you'll only have 30 left executing. This could slow things down
160+
161+
- If even just 1 Task in a batch is slow or hangs, this will prevent the next batch from starting
162+
- If you set a batch of 100, and 70 have finished, you'll only have 30 left executing. This could slow things down
158163

159164
### Parallel
160165

161166
**Types**
167+
162168
| Type | Source Object | Return Object | Method 1 | Method 2 |
163169
|--------------------------------------------------|---------------|---------------|--------------------| ------------------ |
164170
| `ParallelAsyncProcessor` ||| `.WithExecutionCount(int)` | `.ForEachAsync(delegate)` |
@@ -170,29 +176,32 @@ await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToA
170176
Processes your Asynchronous Tasks as fast as it can. All at the same time if it can
171177

172178
**Usage**
179+
173180
```csharp
174181
var ids = Enumerable.Range(0, 5000).ToList();
175182

176183
// SelectAsync for if you want to return something
177-
var results = await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
184+
var results = await ids
178185
.SelectAsync(id => DoSomethingAndReturnSomethingAsync(id), CancellationToken.None)
179186
.ProcessInParallel();
180187

181188
// ForEachAsync for when you have nothing to return
182-
await AsyncProcessorBuilder.WithItems(ids) // Or Extension Method: await ids.ToAsyncProcessorBuilder()
189+
await ids
183190
.ForEachAsync(id => DoSomethingAsync(id), CancellationToken.None)
184191
.ProcessInParallel();
185192
```
186193

187194
**Caveats**
188-
- Depending on how many operations you have, you could overwhelm your system. Memory and CPU and Network usage could spike, and cause bottlenecks / crashes / exceptions
195+
196+
- Depending on how many operations you have, you could overwhelm your system. Memory and CPU and Network usage could spike, and cause bottlenecks / crashes / exceptions
189197

190198
### Processor Methods
191199

192200
As above, you can see that you can just `await` on the processor to get the results.
193201
Below shows examples of using the processor object and the various methods available.
194202

195203
This is for when you need to Enumerate through some objects and use them in your operations. E.g. Sending notifications to certain ids
204+
196205
```csharp
197206
var httpClient = new HttpClient();
198207

@@ -229,6 +238,7 @@ This is for when you need to Enumerate through some objects and use them in your
229238
```
230239

231240
This is for when you need to don't need any objects - But just want to do something a certain amount of times. E.g. Pinging a site to warm up multiple instances
241+
232242
```csharp
233243
var httpClient = new HttpClient();
234244

0 commit comments

Comments
 (0)