You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+61-55Lines changed: 61 additions & 55 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
ObscurCore
2
2
==========
3
3
4
-
A general purpose, easy to use, highly customisable and extensible encryption and data packaging library.
4
+
A general purpose, customisable, and extensible encryption and data packaging library.
5
5
6
6
Thanks
7
7
------
@@ -14,6 +14,8 @@ This work is built heavily on top of theirs, and likely would not have been poss
14
14
15
15
Also a big thanks to **Marc Gravell** for his excellent [protobuf-net](https://code.google.com/p/protobuf-net/) and its associated serialisation assembly precompiler. It is used extensively throughout (especially for the packaging features) and provides great flexibility and performance.
16
16
17
+
And finally, to [LZ4 for .NET](https://lz4net.codeplex.com/). It is used, optionally, for compression of package manifests, where it has excellent speed.
18
+
17
19
18
20
Why use this?
19
21
-------------
@@ -39,18 +41,45 @@ Now that that's out of the way, here's how you can use it for doing some stuff!
39
41
ObscurCore Packaging System
40
42
---------------------------
41
43
42
-
**Please note that this part of the API is new and is subject to change on its way to a stable release. I'll try to minimise this, but the fact remains.**
44
+
**Please note that this part of the API is subject to change on its way to a stable release. I'll try to minimise this, but the fact remains. Now it has reached v0.9, it should change very little.**
45
+
*****
46
+
47
+
This feature is an automated system that acts somewhat like an very paranoid archive/packaging format.
48
+
It allows you to bundle together collections of data, text, and cryptographic keys. You need not care about the details if you don't want to (conservative defaults are used), but if want to, you can heavily customise.
49
+
50
+
*Note: If you don't want to use the packager, but are instead interested in the general-purpose crypto, skip this entire section.*
51
+
43
52
44
-
This feature is an automated system that acts somewhat like an very paranoid archive format.
45
-
It allows you to bundle together collections of messages, data, and key agreements/actions.
53
+
### What does it look like to use? ###
46
54
47
-
The index of the archive is encrypted too, as with the actual contents. Each item has its own unique encryption configuration, so you can have plausible deniability by using different keys on different items, and/or including decoy/fake items if needed.
55
+
using (var output = new MemoryStream()) {
56
+
var package = new Package(key);
57
+
package.AddFile(filePath);
58
+
package.Write(output);
59
+
}
60
+
61
+
you can also add an entire directory (optionally including subdirectories) at once:
62
+
package.AddDirectory(path)
63
+
package.AddDirectory(string path, search : SearchOption.AllDirectories) // including subdirectories
64
+
65
+
66
+
Pretty easy?
67
+
68
+
You *can* stop reading here, if you have no interest in the technical parts. You really don't need to know, but it might be easier with some knowledge of the technicalities.
69
+
70
+
71
+
### How does it work? ###
72
+
73
+
At basics, a package consists of a Manifest and Payload. The manifest (a sort of description of contents, an index) is encrypted, as with the actual contents.
74
+
The payload consists of items, and each item has its own unique encryption configuration, so you can have plausible deniability by using different keys on different items, and/or including decoy/fake items if needed.
48
75
If deniability is not needed, one could use this feature by distributing a package to multiple recipients, with the different items encrypted with recipient-specific keys, so each sees only that which they are able to decrypt.
49
76
50
-
The index of the archive (where all the information about the items in the archive is kept), termed a *manifest* in ObscurCore nonclemature, is encrypted with a choice of schemes:
77
+
The manifest is encrypted with a choice of schemes:
51
78
52
79
+ Symmetric cipher (participants must share secret key securely somehow)
+ UM1-hybrid (ephemeral-static-static elliptic curve Diffie-Hellman, so-called *Unified Model 1-pass*)
81
+
82
+
The latter doesn't use a secret key which has to be somehow communicated to the other party, instead you just exchange 'public keys', so named because you don't need to keep them secret.
54
83
55
84
The hybrid scheme derives a key from the public key cryptosystem, derives it further with a KDF, and uses this as the key for a symmetric cipher.
56
85
The symmetric-only scheme derives the supplied shared key further with a KDF, and uses this as the key for a symmetric cipher.
@@ -65,32 +94,27 @@ Simple just concatenates them together in varied (or sequential) order.
65
94
Frameshift does the same but inserts variable (or fixed) lengths of bytes before and after each item.
66
95
Fabric multiplexes variable (or fixed) length stripes of each item, mixing them all up, much like the Rubberhose file system.
67
96
68
-
Where variable ordering/lengths are used, these are assigned by the use of a CSPRNG, so the recipient can decrypt - if it was actually random, this would be impossible.
69
-
70
-
All items and their configurations, along with the manifest and its configuration, are authenticated with a MAC in the **Encrypt-then-MAC (EtM)** scheme, which provides strong verification and minimum information leakage. Upon detection of any alteration, all operations are aborted.
97
+
Where variable ordering/lengths are used, these are assigned by the use of a cryptographically-secure pseudorandom number generator.
71
98
99
+
All data is authenticated with a MAC in the **Encrypt-then-MAC (EtM)** scheme, which provides strong verification and minimum information leakage. Upon detection of any alteration, all operations are aborted.
72
100
73
-
Here's an example using the packager:
74
101
75
-
using (var output = new MemoryStream()) {
76
-
var package = new Package(key);
77
-
package.AddFile(filePath);
78
-
package.Write(output);
79
-
}
80
-
81
-
The above...
82
-
* Creates the symmetric-encryption package, the manifest being configured to use XSalsa20 encryption, Keccak256 key confirmation, Keccak256 authentication, and scrypt key derivation
102
+
In the code example above/before, the packager performs the following actions automatically...
103
+
* Creates the symmetric-encryption package, the manifest being configured to use XSalsa20 encryption, Keccak-256 (SHA-3-256) key confirmation, Keccak-512 (SHA-3-512) authentication, and Scrypt key derivation
83
104
*The package is set up with frameshifting payload layout (default) and a Salsa20-based CSPRNG
84
-
* Adds a file to a package with HC-128 encryption (random key & IV, default) and Poly1305-AES EtM authentication (random key and nonce, default). Key confirmation and derivation is not used due to the keys being random and stored in the manifest. If keys are to be supplied by recipient, then the defaults are Keccak256 key confirmation and scrypt key derivation.
85
-
* Derives a package/manifest key (from the supplied one) with scrypt KDF
105
+
* Adds a file to a package with HC-128 encryption (random key & IV, default) and Poly1305-AES EtM authentication (random key and nonce, default). Key confirmation and derivation is not used due to the keys being random and stored in the manifest, in this instance. If keys are to be supplied by recipient, then the defaults are Keccak-256 key confirmation and scrypt key derivation.
106
+
* Derives a package/manifest key (from the supplied one) with Scrypt KDF
86
107
* Writes it out to the output stream
87
108
88
109
89
-
All schemes offer key confirmation capability with a choice of algorithms (e.g. MAC, KDF, etc.)
110
+
All schemes offer key confirmation capability with a choice between MAC or KDF method.
90
111
91
112
If a package is recieived from a sender for which you hold multiple keys on file for (whatever kind they may be), all them will be verified with the key confirmation data (if present, which it is by default - generated automatically) to determine the correct one to proceed with.
92
113
93
-
Packages include the capability to communicate new keys (of any kind), or request invalidation of keys for subsequent communications. For example, you could send a symmetric key for manifests, so as to reduce overhead incurred by public key schemes.
114
+
115
+
In future: packages will include the capability to communicate new keys (of any kind), or request invalidation of keys for subsequent communications. For example, you could send a symmetric key for manifests, so as to reduce overhead incurred by public key schemes.
116
+
117
+
94
118
95
119
96
120
Functionality exposed through streams
@@ -100,9 +124,9 @@ Functionality exposed through streams
100
124
101
125
### Encryption/decryption ###
102
126
103
-
var config = SymmetricCipherConfigurationFactory.CreateBlockCipherConfiguration(SymmetricBlockCipher.Aes,
127
+
var config = CipherConfigurationFactory.CreateBlockCipherConfiguration(BlockCipher.Aes,
104
128
BlockCipherMode.Ctr, BlockCipherPadding.None);
105
-
using (var cs = new SymmetricCryptoStream(destStream, encrypting:true, config, keyBytes, closeOnDispose:false) ) {
129
+
using (var cs = new CipherStream(destStream, encrypting:true, config, keyBytes, closeOnDispose:false) ) {
106
130
sourceStream.CopyTo(cs);
107
131
}
108
132
@@ -189,13 +213,12 @@ These are in serious need of a convenience method. It's on the list.
189
213
### Key agreements ###
190
214
191
215
Please note that currently, perfect-forward-secrecy ECDH algorithms (such as 3-pass Full Unified Model; UM3) are not implemented. Sorry!
192
-
Work has started on implementing the *Axolotl Ratchet* for another manifest cryptography scheme.
193
216
194
217
There are, however, implemented UM1-type agreements, which provide unilateral forward secrecy - which is much better than nothing.
195
218
196
219
J-PAKE password-authenticated key agreement is also available, using ECC instead of DSA, making for very secure and fast agreements.
197
220
198
-
Elliptic curves provided are from the Brainpool Consortium and SEC2 (secp and sect curves; also called NIST curves). These are the most popular choices.
221
+
Elliptic curves provided are from the Brainpool Consortium, SEC2 (secp and sect curves; also called NIST curves), and Daniel J. Bernstein. These are the most popular choices.
199
222
200
223
201
224
Creating keys:
@@ -205,27 +228,27 @@ Creating keys:
205
228
And calculating shared secret:
206
229
207
230
EcKeyConfiguration ephemeral;
208
-
byte[] initiatorSS = UM1Exchange.Initiate(senderKeypair.ExportPublicKey(), senderKeypair.GetPrivateKey(), out ephemeral);
No concrete implementation is yet in place. ECDSA is being added.
215
-
The preferred example of this is Ed25519.
237
+
No concrete implementation is yet in place. ECDSA is being added - the preferred example of this is Ed25519.
216
238
DSA proper (using RSA) will most likely not be added due to concerns with security and efficiency.
217
239
Watch this space.
218
240
219
241
220
242
Some words on Streams
221
243
---------------------
222
244
223
-
SymmetricCryptoStream**encrypts only when being written to**, and **decrypts only when being read from**. The other core stream types (for example, the HashStream) do not enforce directionality like this.
245
+
CipherStream**encrypts only when being written to**, and **decrypts only when being read from**. The other core stream types (for example, the HashStream) do not enforce directionality like this.
224
246
225
247
ObscurCore uses stream decorators for I/O, so anything that is derived from the abstract Stream class of the .NET BCL can be plugged into the constructor of an ObscurCore *SymmetricCryptoStream/HashStream/MacStream*. This means in practice pretty much anything, since pretty much anything can be serialised to a byte array.
226
248
249
+
Please note that ObscurCore's main stream classes **close on dispose by default**. The reason for this is to ensure the stream that it's bound to gets closed. Most (all?) of the .NET BCL streams are the same, the difference here is that you get a choice, because you may only want to finish the cipher operation but retain the underlying bound stream(s).
227
250
228
-
Please note that ObscurCore's main stream classes **close on dispose by default**. This means the stream they were bound to on construction will be closed with it, when the wrapping ObscurCore stream is closed/disposed.
251
+
This means the stream they were bound to on construction will be closed with it, when the wrapping ObscurCore stream is closed/disposed.
229
252
Don't make the mistake of thinking stuff isn't working when binding on a MemoryStream, and wondering why the data is missing afterward.
230
253
231
254
The main stream classes have a parameter, *closeOnDispose*, that controls this. Set it to **false** if you *don't* want them to close. They're set to *true* by default to try and ensure that if they're bound to a FileStream, the OS hooks get disposed of properly.
@@ -239,43 +262,26 @@ Recommendations
239
262
240
263
The author's recommendations, among block ciphers:
241
264
242
-
+ AES/CTR
243
-
+ Twofish/CTR
244
-
+ Serpent/CTR
265
+
+ AES in CTR mode
266
+
+ Twofish in CTR mode
267
+
+ Serpent in CTR mode
245
268
246
269
and in stream ciphers:
247
270
248
-
+ HC-128
249
-
+ SOSEMANUK
250
271
+ XSalsa20
272
+
+ HC-128
251
273
252
274
In hash and MAC functions:
253
275
276
+
+ Keccak/SHA-3
254
277
+ BLAKE2B
255
278
+ Poly1305-AES
256
-
+ Keccak/SHA-3
257
279
258
280
In KDFs:
259
281
260
282
+ Scrypt
261
283
262
-
Advanced
263
-
--------
264
-
265
-
Want to play around with the cryptographic primitives instead? This is not recommended, but you can.
266
-
Have a look around the **Source** object. It provides easy instantiation and configuration of the most useful classes.
267
-
268
-
Some features of ObscurCore are only accessible this way currently, but that's soon to change. If you don't want to deal with low-level features, you shouldn't have to. That's the point of automation.
269
-
270
-
What's next?
271
-
------------
272
-
273
-
Mostly, the work to come is tidying up the API. Style of use will ideally become more consistent, and less contact with intermediate objects will be required.
274
-
275
-
The next major planned feature is automatic nonce management.
276
-
The idea is that you simply pass in a collection that conforms to a particular interface (whether this is just a reference to a simple in-memory database that's repopulated from disk on startup, or a *real* database), ObscurCore checks this DB whenever you reference a key, and makes sure it doesn't use the same nonce again when it's using a cryptographic scheme for which this is essential to maintain its security assurances.
0 commit comments