Add TOON format encoding and decoding functionality by 239573049 · Pull Request #2 · toon-format/toon-dotnet

239573049 · 2025-11-06T08:14:18Z

Implement StringUtils for string manipulation, including escaping and unescaping.
Create ValidationShared for key validation and safety checks.
Introduce ToonDecodeOptions and ToonEncodeOptions for customizable encoding/decoding settings.
Develop ToonDecoder for parsing TOON format strings into JsonNode objects.
Implement ToonEncoder for converting data structures into TOON format strings.
Add ToonFormatException for error handling during parsing and encoding.
Create unit tests for ToonDecoder and ToonEncoder to ensure functionality and data integrity.
Remove outdated UnitTest1.cs file.

Description

Type of Change

Bug fix (non-breaking change that fixes an issue)
New feature (non-breaking change that adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update
Refactoring (no functional changes)
Performance improvement
Test coverage improvement

Related Issues

Closes #

Changes Made

SPEC Compliance

This PR implements/fixes spec compliance
Spec section(s) affected:
Spec version:

Testing

All existing tests pass
Added new tests for changes
Tested on .NET 8.0
Tested on .NET 9.0

Checklist

My code follows the project's coding standards
I have run dotnet format
I have added tests that prove my fix/feature works
New and existing tests pass locally
I have updated documentation (if needed)
My changes do not introduce new dependencies

Additional Context

- Implement StringUtils for string manipulation, including escaping and unescaping. - Create ValidationShared for key validation and safety checks. - Introduce ToonDecodeOptions and ToonEncodeOptions for customizable encoding/decoding settings. - Develop ToonDecoder for parsing TOON format strings into JsonNode objects. - Implement ToonEncoder for converting data structures into TOON format strings. - Add ToonFormatException for error handling during parsing and encoding. - Create unit tests for ToonDecoder and ToonEncoder to ensure functionality and data integrity. - Remove outdated UnitTest1.cs file.

Copilot

Pull Request Overview

This PR implements the complete TOON format encoder and decoder functionality, replacing the placeholder implementations. TOON is a data serialization format that provides human-readable encoding/decoding of structured data with support for arrays, objects, and primitive types.

Key changes:

Implements encoding logic to convert .NET objects to TOON format strings
Implements decoding logic to parse TOON format strings back into JSON structures
Adds comprehensive test coverage for encoding, decoding, and round-trip scenarios
Introduces encoding/decoding options with customizable indentation, delimiters, and validation modes

Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
tests/ToonFormat.Tests/UnitTest1.cs	Removed placeholder tests that checked for NotImplementedException
tests/ToonFormat.Tests/ToonRoundTripTests.cs	Added round-trip tests verifying encoding and decoding preserve data integrity
tests/ToonFormat.Tests/ToonEncoderTests.cs	Added comprehensive encoder tests covering primitives, arrays, objects, and formatting options
tests/ToonFormat.Tests/ToonDecoderTests.cs	Added decoder tests for parsing TOON strings into JSON structures
src/ToonFormat/ToonFormatException.cs	Added custom exception type with error categorization and contextual error messages
src/ToonFormat/ToonFormat.csproj	Added empty folder placeholders for internal encoder/decoder organization
src/ToonFormat/ToonEncoder.cs	Implemented TOON encoder with multiple overloads for encoding to strings, bytes, and streams
src/ToonFormat/ToonDecoder.cs	Implemented TOON decoder with multiple overloads for decoding from strings, bytes, and streams
src/ToonFormat/Options/ToonEncodeOptions.cs	Added encoding configuration options (indent, delimiter, length marker)
src/ToonFormat/Options/ToonDecodeOptions.cs	Added decoding configuration options (indent, strict mode)
src/ToonFormat/Internal/Shared/ValidationShared.cs	Added validation utilities for key and value quoting rules
src/ToonFormat/Internal/Shared/StringUtils.cs	Added string escaping/unescaping and quote handling utilities
src/ToonFormat/Internal/Shared/LiteralUtils.cs	Added literal type detection for booleans, null, and numbers
src/ToonFormat/Internal/Encode/Primitives.cs	Implemented primitive value encoding and header formatting
src/ToonFormat/Internal/Encode/Normalize.cs	Implemented .NET object to JsonNode normalization
src/ToonFormat/Internal/Encode/LineWriter.cs	Added utility for building indented TOON output
src/ToonFormat/Internal/Encode/Encoders.cs	Implemented main encoding logic for objects, arrays, and nested structures
src/ToonFormat/Internal/Decode/Validation.cs	Added decoding validation logic for strict mode checks
src/ToonFormat/Internal/Decode/Scanner.cs	Implemented source text scanning into structured lines with depth tracking
src/ToonFormat/Internal/Decode/Parser.cs	Implemented TOON format parsing for headers, values, and tokens
src/ToonFormat/Internal/Decode/Decoders.cs	Implemented main decoding logic for converting TOON to JSON structures
src/ToonFormat/Internal/Converters/SingleNamedFloatToNullConverter.cs	Added JSON converter to normalize float NaN/Infinity to null
src/ToonFormat/Internal/Converters/DoubleNamedFloatToNullConverter.cs	Added JSON converter to normalize double NaN/Infinity to null
src/ToonFormat/Constants.cs	Added shared constants for TOON format structural characters and delimiters
src/ToonFormat/Class1.cs	Removed placeholder implementation file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/ToonFormat/Constants.cs

src/ToonFormat/Internal/Encode/Encoders.cs

src/ToonFormat/Internal/Encode/Normalize.cs

src/ToonFormat/Internal/Encode/Encoders.cs

src/ToonFormat/Internal/Decode/Parser.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

ghost1face · 2025-11-07T13:47:12Z

Hi @239573049 thanks for your contribution, we've recently just formed this repo and team, so we're forming some initial plans before we start taking contributions. We will definitely keep this for consideration!

ghost1face · 2025-11-17T18:07:11Z

@239573049 I'm creating a new pull request which builds on your changes to add support for PathExpansion and KeyFolding with passing test coverage according to the V2 spec

johannschopplich · 2025-11-17T18:09:42Z

@ghost1face Thanks for being on it! @239573049 Would you mind commenting on the Copilot issues? If the teams wants, they can merge this PR and then build upon it.

I'd love to see the repo take shape, so thanks for all the contributions.

src/ToonFormat/Internal/Decode/Scanner.cs

239573049 · 2025-11-17T18:46:09Z

I will first attempt to merge the content provided by Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Optimize the performance of the `ParseDelimitedValues` and `ToParsedLines` methods: - Reduce memory allocation by using `StringBuilder` and `ReadOnlySpan<char>`. - Allocate capacity in advance to avoid dynamic resizing. - Replace string operations with more efficient logic. Fix the code formatting and logic of `IsArrayHeaderAfterHyphen` and `ParseStringLiteral`: - Adjust the format of the `return` statement. - Eliminate redundant code. New `MapRowValuesToPrimitives` method: - Supports mapping string arrays to JSON primitive values. Other minor optimizations: - Fixed the logic for handling blank lines. - Replaced some string operations with more efficient implementations. - Fixed the issue of character encoding in comments, improving code readability.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…rs, and a new FloatUtils utility class has been added. Introduce the FloatUtils utility class and provide the NormalizeSignedZero method, to uniformly handle the logic of negated zero normalization for floating-point numbers, replacing the original repetitive implementations. Update the relevant code in Parser.cs and Normalize.cs to enhance readability. Add the NearlyEqual method for approximate comparison of floating-point numbers. Delete the references to the Internal\Shared\ folder.

ghost1face

Will approve and merge, I will be aligning several pieces for full spec validation and implementing missing pieces.

We'll also start to take contribution for minor enhancements such as @Daniel-iel 's recommendations as well.

Thanks for the help everyone!

239573049 requested a review from a team as a code owner November 6, 2025 08:14

Copilot AI review requested due to automatic review settings November 6, 2025 08:14

Copilot AI reviewed Nov 6, 2025

View reviewed changes

239573049 and others added 2 commits November 6, 2025 16:55

Update src/ToonFormat/Constants.cs

0fce37f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/ToonFormat/Internal/Encode/Normalize.cs

9a1517c

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

saapo-ka-baadshah mentioned this pull request Nov 15, 2025

What is the roadmap & strategy for the official .NET implementation (toon-dotnet)? #5

Closed

ghost1face mentioned this pull request Nov 17, 2025

feat: Toon SpecGenerator for building unit tests to verify against spec #6

Merged

20 tasks

Daniel-iel reviewed Nov 17, 2025

View reviewed changes

src/ToonFormat/Internal/Decode/Scanner.cs Outdated Show resolved Hide resolved

239573049 and others added 6 commits November 18, 2025 02:46

Update src/ToonFormat/Internal/Encode/Encoders.cs

12a80fd

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/ToonFormat/Internal/Encode/Encoders.cs

532ee00

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/ToonFormat/Internal/Encode/Normalize.cs

670ae1f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Merge branch 'main' of https://github.com/239573049/toon-dotnet

ac006fe

ghost1face added the enhancement New feature or request label Nov 18, 2025

ghost1face approved these changes Nov 18, 2025

View reviewed changes

ghost1face self-assigned this Nov 18, 2025

ghost1face merged commit 4caaa2c into toon-format:main Nov 18, 2025
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add TOON format encoding and decoding functionality#2

Add TOON format encoding and decoding functionality#2
ghost1face merged 9 commits intotoon-format:mainfrom
239573049:main

239573049 commented Nov 6, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ghost1face commented Nov 7, 2025

Uh oh!

ghost1face commented Nov 17, 2025

Uh oh!

johannschopplich commented Nov 17, 2025

Uh oh!

Uh oh!

239573049 commented Nov 17, 2025

Uh oh!

ghost1face left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

239573049 commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of Change

Related Issues

Changes Made

SPEC Compliance

Testing

Checklist

Additional Context

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ghost1face commented Nov 7, 2025

Uh oh!

ghost1face commented Nov 17, 2025

Uh oh!

johannschopplich commented Nov 17, 2025

Uh oh!

Uh oh!

239573049 commented Nov 17, 2025

Uh oh!

ghost1face left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

239573049 commented Nov 6, 2025 •

edited

Loading