Skip to content

Commit 4a671a8

Browse files
U/sgriffin/block (#205)
* simplify more * fix recursion bug * Fix MetaPropValueGetPartial tags * Convert more rops * Convert more rops * Allow copy of full untruncated block strings * move files * break up classes in files/folders * convert some more structs * Convert more * Convert more * fix up guids * Convert PropertyName * cleanup * Convert RecipientFlags * gut HelpMember, rebuild as extension * Convert RecipientRow * simplify * breakout files * Convert more * Encapsulate guid block chaining * better string typing * Better debug output * Fix empty node, Add red to debug on empty nodes * rename dir * break out classes * break out classes * remove this pointers * Convert more * Convert more * Convert more * break out classes * Convert more * break out classes * pull out block adders and document * simplify adders * fix docs * update comments * Simplify chaining BlockBytes * convert more * fix usings * add implicit cast to T for BlockT * tree doesn't like fonts * break out classes * fix extra spaces in comments * fix some parsing * consume all of fx buffers * simplify junk parsing * Fix time * Fix up string parsing * Add Partial clock comment data * Fix ServerObjectHandleTable * add debug note * add cch for strings * Hex T * remove extra cast * factor partial information out * cleanup * reorg more * more reorg * cleanup * fix partial unicode fx buffers * take List out of BlockBytes, remove redundant conversions * Only compute FullString on demand. It's only for testing now * fix unicode/wide partial parsing in more places * arrange better * more cleanup * remove tabs * break out classes * Convert nspi * fix offset in parseas remove dead field label AddressBookPropertyValue * fix mv ab props * add text decoding of hex blocks * fix junk test * fix more bin test * fix AddressBookPropertyValue values * Break up classes * Convert more * break up classes * break out classes * small cleanup * convert more * reorg * cleanup * convert table * minor fixes * Convert more * convert more * rename Size * convert more * rename size * convert more * Convert more * convert more * Add line by line W/A parsing * Convert more * remove more dead code * Removed dead code * fix linefeed * fix linefeeds * Simplify response parsing * Convert ROPOutputBuffer * Simplify * Fix loop * Fix parsing offset calculations * handle compression * Pass through expected exceptions * better debug nodes * Handle payload offsets better * add some comments * simplify before conversion * Convert more * add comments * convert more * convert more * convert more, simplify * convert more * fix node offsets * convert more * convert more * simplify * fix offset calc * cut dead code * remove dead code * typo * remove dead code * simplify * convert more * simplify * convert more * fix typo * simplify * simplify * Simplify * more cleanup * ROPBufferServerObjectTable didn't parse enough * remove dead code * fix array error * fix cache failures * Fix typo * Fix exports * Turn off debug output * fix bytes highlighting * move blockbytes subheader around * move blockt formatting into blockt * BlockT ToString * Simplify AddChildBlockT * simplify AddLabeledChildren * update rules * Fix PropertyName * fix GroupPropertyName * add missing text * fix public/private on blocks * clean up string prop parsing, unify with ab * format error returns * remove dead code * move tags abovevalues in row data * fix comments * fix typo * Add docs * handle extra data better * better prop tag output * Toggle debug via context menu * fix line endings * clean up designer code * get control working in designer * Remove rich text box, make splitter wider * stub in search * add f3 * better key handling * fix names * remove test exception * keyboard shortcuts * fix hex box copy * ditch this. * rename helper * ctrl+A select all * hex is read only * change fonts to segoe * better layout * fix layout * fix tab order * fix test case * better exception output * Simplify * simplify * Add missing spaces * fix test cases * remove test exceptions * More structured exception output, flag exceptions as root node * clean up error parsing * simplify * simplify * simplify * fix test * add docs * add comments * clean up exception * simplify * simplify * simplify * int32 output * Handle no rops correctly. Better error output on missing context information * clean up comments * Update comments * update comments * more doc updates * more docs * Add docs * remove using * more docs * Add more docs * more docs * Fix notification parsing * fix rpcheader lookup * fix mv bin parsing in AB * fix ropLogonTime parsing * fix comments * Add exception trace parsing * better handle missing information * Add session nevaigator to simplify walking forward and backward in sessions * Simplify * Finish removing AllSessions * color junk blocks * Fix notification parsing * fix enum names * Augment NotificationFlags parsing * cut dead code * Remove EnumToString * simplify * parse more named prop names * For speed, start tree only partially expanded. Add Ctrl+Right to expand all * Add more missing exception info * remove hash code * update .net * remove analyzer * convert parser backing to byte array * simplify * cut dead function * simplify * simplify * simplify * update test framework to 4.8 * not expanded is annoying - better to take the hit * fix property name parsing * test sourceling * add nuget config * just test deterministic for now * try sourcelink again
1 parent 1a69ed6 commit 4a671a8

File tree

698 files changed

+57055
-51996
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

698 files changed

+57055
-51996
lines changed

.gitignore

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,10 @@
1313
/FSSHTTPWOPIInspector/Test/WOPIautomation/WOPIautomation/obj/
1414
/MAPIInspector/Test/MAPIAutomationTest/.vs/
1515
/MAPIInspector/Test/MAPIAutomationTest/MAPIAutomationTest/bin/
16-
/MAPIInspector/Test/MAPIAutomationTest/MAPIAutomationTest/obj/
16+
/MAPIInspector/Test/MAPIAutomationTest/MAPIAutomationTest/obj/
17+
/MAPIInspector/Source/BlockParser/bin/
18+
/MAPIInspector/Source/BlockParser/obj/
19+
/MAPIInspector/Source/BlockParserTests/bin/
20+
/MAPIInspector/Source/BlockParserTests/obj/
21+
/MAPIInspector/Source/packages/
22+
/MAPIInspector/Source/TestResults/

FSSHTTPWOPIInspector/Source/FSSHTTPandWOPIInspector.csproj

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,9 @@
1010
<AppDesignerFolder>Properties</AppDesignerFolder>
1111
<RootNamespace>FSSHTTPandWOPIInspector</RootNamespace>
1212
<AssemblyName>FSSHTTPandWOPIFiddlerInspector</AssemblyName>
13-
<TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
13+
<TargetFrameworkVersion>v4.8</TargetFrameworkVersion>
1414
<FileAlignment>512</FileAlignment>
15+
<Deterministic>true</Deterministic>
1516
<FileUpgradeFlags>
1617
</FileUpgradeFlags>
1718
<OldToolsVersion>3.5</OldToolsVersion>
@@ -36,6 +37,10 @@
3637
<SccLocalPath>SAK</SccLocalPath>
3738
<SccAuxPath>SAK</SccAuxPath>
3839
<SccProvider>SAK</SccProvider>
40+
<PublishRepositoryUrl>true</PublishRepositoryUrl>
41+
<EmbedUntrackedSources>true</EmbedUntrackedSources>
42+
<IncludeSymbols>true</IncludeSymbols>
43+
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
3944
</PropertyGroup>
4045
<PropertyGroup>
4146
<FiddlerPath Condition="$(FiddlerPath) == '' AND Exists('$(LocalAppData)\Programs\Fiddler\Fiddler.exe')">$(LocalAppData)\Programs\Fiddler</FiddlerPath>
@@ -155,6 +160,9 @@
155160
<DependentUpon>FSSHTTPAndWOPIControl.cs</DependentUpon>
156161
</EmbeddedResource>
157162
</ItemGroup>
163+
<ItemGroup>
164+
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0" PrivateAssets="All" />
165+
</ItemGroup>
158166
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
159167
<PropertyGroup>
160168
<PostBuildEvent>start "CopyDLLs" /B /wait start xcopy "$(SolutionDir)$(OutDir)*.*" "$(FiddlerPath)\Inspectors" /Q /Y</PostBuildEvent>

FSSHTTPWOPIInspector/Test/WOPIautomation/WOPIautomation/WOPIautomation.csproj

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
<AssemblyName>WOPIautomation</AssemblyName>
1111
<TargetFrameworkVersion>v4.8</TargetFrameworkVersion>
1212
<FileAlignment>512</FileAlignment>
13+
<Deterministic>true</Deterministic>
1314
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
1415
<VisualStudioVersion Condition="'$(VisualStudioVersion)' == ''">10.0</VisualStudioVersion>
1516
<VSToolsPath Condition="'$(VSToolsPath)' == ''">$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)</VSToolsPath>
Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
using System;
2+
using System.Collections.Generic;
3+
using System.Diagnostics;
4+
5+
namespace BlockParser
6+
{
7+
/// <summary>
8+
/// Helper class for parsing binary data from a stream or buffer,
9+
/// providing safe access and offset management, including support for artificial caps.
10+
/// </summary>
11+
public class BinaryParser
12+
{
13+
/// <summary>
14+
/// Gets whether the parser has reached the end of the buffer.
15+
/// </summary>
16+
public bool Empty => Offset == size;
17+
18+
/// <summary>
19+
/// Gets or sets the current offset within the buffer.
20+
/// </summary>
21+
public int Offset { get; set; }
22+
23+
/// <summary>
24+
/// Gets the number of bytes remaining from the current offset to the end of the buffer.
25+
/// Returns 0 if the offset is out of bounds.
26+
/// </summary>
27+
public int RemainingBytes => Offset > size ? 0 : size - Offset;
28+
29+
private readonly byte[] bin;
30+
private int size; // When uncapped, this is bin.Length. When capped, this is our artificial capped size.
31+
private readonly Stack<int> sizes = new Stack<int>();
32+
33+
/// <summary>
34+
/// Initializes a new, empty BinaryParser.
35+
/// </summary>
36+
public BinaryParser()
37+
{
38+
bin = Array.Empty<byte>();
39+
size = 0;
40+
Offset = 0;
41+
}
42+
43+
/// <summary>
44+
/// Initializes a BinaryParser with a byte array and a specified count.
45+
/// If the array is longer than the count, only the first count bytes are used.
46+
/// </summary>
47+
/// <param name="cb">The number of bytes to use from the array.</param>
48+
/// <param name="_bin">The byte array to parse.</param>
49+
public BinaryParser(int cb, byte[] _bin)
50+
{
51+
if (_bin != null && cb > 0)
52+
{
53+
if (_bin.Length > cb)
54+
{
55+
bin = new byte[cb];
56+
Buffer.BlockCopy(_bin, 0, bin, 0, cb);
57+
}
58+
else
59+
{
60+
bin = new byte[_bin.Length];
61+
Buffer.BlockCopy(_bin, 0, bin, 0, _bin.Length);
62+
}
63+
}
64+
else
65+
{
66+
bin = Array.Empty<byte>();
67+
}
68+
69+
size = bin.Length;
70+
Offset = 0;
71+
}
72+
73+
/// <summary>
74+
/// Initializes a BinaryParser with a byte array.
75+
/// </summary>
76+
/// <param name="_bin">The byte array to parse.</param>
77+
public BinaryParser(byte[] _bin)
78+
{
79+
if (_bin != null)
80+
{
81+
bin = new byte[_bin.Length];
82+
Buffer.BlockCopy(_bin, 0, bin, 0, _bin.Length);
83+
}
84+
else
85+
{
86+
bin = Array.Empty<byte>();
87+
}
88+
89+
size = bin.Length;
90+
Offset = 0;
91+
}
92+
93+
/// <summary>
94+
/// Advances the current offset by the specified number of bytes.
95+
/// </summary>
96+
/// <param name="cb">The number of bytes to advance.</param>
97+
public void Advance(int cb) => Offset += cb;
98+
99+
/// <summary>
100+
/// Resets the current offset to the beginning of the buffer.
101+
/// </summary>
102+
public void Rewind() => Offset = 0;
103+
104+
/// <summary>
105+
/// Pushes a cap onto the size stack, limiting the accessible buffer size to the current offset plus the specified cap.
106+
/// Used to temporarily restrict parsing to a subsection of the buffer.
107+
/// </summary>
108+
/// <param name="cap">The number of bytes to cap from the current offset.</param>
109+
public void PushCap(int cap)
110+
{
111+
sizes.Push(size);
112+
if (cap != 0 && Offset + cap < bin.Length)
113+
{
114+
size = Offset + cap;
115+
}
116+
}
117+
118+
/// <summary>
119+
/// Pops the most recent cap from the size stack, restoring the previous accessible buffer size.
120+
/// </summary>
121+
public void PopCap()
122+
{
123+
if (sizes.Count == 0)
124+
{
125+
size = bin.Length;
126+
}
127+
else
128+
{
129+
size = sizes.Pop();
130+
}
131+
}
132+
133+
/// <summary>
134+
/// Checks if the specified number of bytes can be read from the current offset without exceeding the buffer.
135+
/// </summary>
136+
/// <param name="cb">The number of bytes to check.</param>
137+
/// <returns>True if the bytes can be read; otherwise, false.</returns>
138+
public bool CheckSize(int cb) => cb <= RemainingBytes;
139+
140+
/// <summary>
141+
/// Reads the specified number of bytes from the current offset in the binary stream.
142+
/// Advances the offset by the number of bytes read.
143+
/// If there are not enough bytes remaining, returns an empty array.
144+
/// </summary>
145+
/// <param name="cb">The number of bytes to read.</param>
146+
/// <returns>
147+
/// A byte array containing the bytes read, or an empty array if there are not enough bytes remaining.
148+
/// </returns>
149+
public byte[] ReadBytes(int cb)
150+
{
151+
if (CheckSize(cb))
152+
{
153+
byte[] bytes = new byte[cb];
154+
Buffer.BlockCopy(bin, Offset, bytes, 0, cb);
155+
Advance(cb);
156+
return bytes;
157+
}
158+
159+
return Array.Empty<byte>();
160+
}
161+
162+
/// <summary>
163+
/// Returns the entire binary stream as a hexadecimal string.
164+
/// Only used for debugging purposes.
165+
/// </summary>
166+
/// <returns>A string representation of the binary data in hexadecimal format.</returns>
167+
public string PeekBytes()
168+
{
169+
return Strings.BinToHexString(bin, bin.Length);
170+
}
171+
172+
/// <summary>
173+
/// Outputs a sample of bytes from the current offset in the binary stream to the debug output.
174+
/// Only used for debugging purposes.
175+
/// </summary>
176+
/// <param name="cb">The number of bytes to output (default is 20).</param>
177+
public void SampleBytes(int cb = 20)
178+
{
179+
var offset = Offset;
180+
var length = Math.Min(cb, RemainingBytes);
181+
var _data = ReadBytes(length);
182+
var hex = Strings.BinToHexString(_data, 0);
183+
var text = Strings.BinToTextStringA(_data, true);
184+
Debug.WriteLine($"cb: 0x{length:X}={length} lpb: {hex}={text}");
185+
Offset = offset;
186+
}
187+
}
188+
}
Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
using System.Collections.Generic;
2+
using System.IO;
3+
4+
namespace BlockParser
5+
{
6+
// TODO: Make use of these
7+
public static class Constants
8+
{
9+
public const uint _MaxBytes = 0xFFFF;
10+
public const uint _MaxDepth = 25;
11+
public const uint _MaxEID = 500;
12+
public const uint _MaxEntriesSmall = 500;
13+
public const uint _MaxEntriesLarge = 1000;
14+
public const uint _MaxEntriesExtraLarge = 1500;
15+
public const uint _MaxEntriesEnormous = 10000;
16+
}
17+
18+
public abstract partial class Block
19+
{
20+
public long Size { get; set; }
21+
public long Offset { get; set; }
22+
private string _text;
23+
public string Text
24+
{
25+
get => _text;
26+
set => _text = value ?? "";
27+
}
28+
public uint Source
29+
{
30+
get => _source;
31+
set
32+
{
33+
_source = value;
34+
foreach (var child in Children)
35+
{
36+
child.Source = value;
37+
}
38+
}
39+
}
40+
41+
public IReadOnlyList<Block> Children => children.AsReadOnly();
42+
public bool IsHeader => Size == 0 && Offset == 0;
43+
public bool HasData => !string.IsNullOrEmpty(Text) || Children.Count > 0;
44+
45+
protected BinaryParser parser;
46+
public bool Parsed { get; protected set; } = false;
47+
protected bool EnableJunk { get; set; } = true;
48+
protected virtual bool UsePipes() => false;
49+
50+
private List<Block> children { get; } = new List<Block>();
51+
private uint _source;
52+
53+
// Overrides
54+
/// <summary>
55+
/// When implemented in a derived class, parses the current block from the associated <see cref="BinaryParser"/>.
56+
/// This method should set up the block's data and state based on the binary input.
57+
/// </summary>
58+
protected abstract void Parse();
59+
/// <summary>
60+
/// When overridden in a derived class, parses and adds any child blocks to this block, building a tree of blocks
61+
/// No default implementation is provided, as this method is expected to be specific to the derived class's structure.
62+
/// Do NOT attempt to parse data from the stream or parser here; that should be done in the <see cref="Parse"/> method.
63+
/// </summary>
64+
protected abstract void ParseBlocks();
65+
66+
public void ShiftOffset(long shift)
67+
{
68+
Offset += shift;
69+
foreach (var child in Children)
70+
{
71+
child.ShiftOffset(shift);
72+
}
73+
}
74+
75+
public void Parse(BinaryParser parser, bool enableJunk = false) => Parse(parser, 0, enableJunk);
76+
77+
private void Parse(BinaryParser parser, int cbBin, bool enableJunk = false)
78+
{
79+
this.parser = parser;
80+
parser.PushCap(cbBin);
81+
EnableJunk = enableJunk;
82+
EnsureParsed();
83+
parser.PopCap();
84+
}
85+
86+
protected void EnsureParsed()
87+
{
88+
if (!Parsed && parser != null && !parser.Empty)
89+
{
90+
Parsed = true; // parse can unset this if needed
91+
Offset = parser.Offset;
92+
93+
// Parse and ParseBlocks are seperate so that if one throws an exception, the other can still
94+
// run and we can still get the tree layout, even if the data parsing failed.
95+
try
96+
{
97+
Parse();
98+
}
99+
catch (System.Exception e)
100+
{
101+
var typeName = e.GetType().FullName;
102+
if (typeName == "MAPIInspector.Parsers.MissingInformationException") throw e;
103+
if (typeName == "MAPIInspector.Parsers.MissingPartialInformationException") throw e;
104+
children.Add(BlockException.Create("Buffer Parsing Exception", e, Offset));
105+
}
106+
107+
// Compute Size so ParseBlocks can use it
108+
Size = parser.Offset - Offset;
109+
110+
try
111+
{
112+
ParseBlocks();
113+
}
114+
catch (System.Exception e)
115+
{
116+
children.Add(BlockException.Create("Tree Layout Exception", e, Offset));
117+
}
118+
119+
if (HasData && EnableJunk && parser.RemainingBytes > 0)
120+
{
121+
AddChild(ParseJunk("Unparsed data"));
122+
}
123+
124+
// Recompute Size just in case ParseBlocks changed it
125+
Size = parser.Offset - Offset;
126+
}
127+
}
128+
129+
// This is purely for testing now
130+
public string FullString()
131+
{
132+
EnsureParsed();
133+
var stringArray = ToStringsInternal();
134+
var _stringBlock = Strings.TrimWhitespace(string.Join(string.Empty, stringArray));
135+
_stringBlock = _stringBlock.Replace('\0', '.');
136+
return _stringBlock;
137+
}
138+
139+
public override string ToString() => Text;
140+
141+
private List<string> ToStringsInternal()
142+
{
143+
var strings = new List<string>(Children.Count + 1);
144+
if (!string.IsNullOrEmpty(Text)) strings.Add(Text + "\r\n");
145+
146+
foreach (var child in Children)
147+
{
148+
var childStrings = child.ToStringsInternal();
149+
if (!string.IsNullOrEmpty(Text)) childStrings = Strings.TabStrings(childStrings, UsePipes());
150+
strings.AddRange(childStrings);
151+
}
152+
153+
return strings;
154+
}
155+
}
156+
}

0 commit comments

Comments
 (0)