Skip to content

Commit 854bce8

Browse files
committed
Initial commit: ASON - Token-optimized JSON compression for LLMs
ASON (Aliased Serialization Object Notation) is a compression format designed to reduce token consumption in Large Language Model contexts.
0 parents  commit 854bce8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+16745
-0
lines changed

.DS_Store

12 KB
Binary file not shown.

.github/workflows/npm-publish.yml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Publish to NPM
2+
3+
on:
4+
release:
5+
types: [created]
6+
7+
jobs:
8+
build:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- uses: actions/checkout@v4
12+
13+
- name: Setup Node.js
14+
uses: actions/setup-node@v4
15+
with:
16+
node-version: '18.x'
17+
registry-url: 'https://registry.npmjs.org'
18+
19+
- name: Install dependencies
20+
run: |
21+
cd nodejs-compressor
22+
npm ci
23+
24+
- name: Run tests
25+
run: |
26+
cd nodejs-compressor
27+
npm test
28+
29+
- name: Publish to NPM
30+
run: |
31+
cd nodejs-compressor
32+
npm publish --access public
33+
env:
34+
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/test.yml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
13+
strategy:
14+
matrix:
15+
node-version: [16.x, 18.x, 20.x]
16+
17+
steps:
18+
- uses: actions/checkout@v4
19+
20+
- name: Use Node.js ${{ matrix.node-version }}
21+
uses: actions/setup-node@v4
22+
with:
23+
node-version: ${{ matrix.node-version }}
24+
25+
- name: Install dependencies
26+
run: |
27+
cd nodejs-compressor
28+
npm ci
29+
30+
- name: Run tests
31+
run: |
32+
cd nodejs-compressor
33+
npm test
34+
35+
- name: Run benchmarks
36+
run: |
37+
cd nodejs-compressor
38+
npm run benchmark

CHANGELOG.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [1.1.0] - 2025-01-10
9+
10+
### Added
11+
- **CLI Tool** - Command-line interface for JSON ↔ ASON conversion
12+
- Auto-detection of format from file extension or content
13+
- `--stats` flag for visual token count comparison with ASCII table
14+
- `--delimiter` option for comma/tab/pipe separators
15+
- Stdin/stdout support for piping workflows (e.g., `cat data.json | npx ason`)
16+
- Comprehensive help documentation with `-h/--help`
17+
- Support for `--no-references` and `--no-dictionary` flags
18+
- **TypeScript Support** - Full type definitions included
19+
- `.d.ts` files for ESM (48.2 KB)
20+
- `.d.cts` files for CommonJS (48.2 KB)
21+
- Full IntelliSense and autocomplete support in VSCode and other IDEs
22+
- Proper type exports for both `import` and `require`
23+
- **Build System** - Integrated tsup for optimized bundling
24+
- Minified ESM build (14.5 KB) - 70% smaller than source
25+
- Minified CJS build (14.5 KB) for Node.js legacy compatibility
26+
- Tree-shaking enabled for smaller bundle sizes
27+
- Dual package exports for seamless ESM/CJS interop
28+
29+
### Changed
30+
- Package now ships with built/minified code instead of source files
31+
- `package.json` configured with proper `exports` field for ESM/CJS resolution
32+
- Updated `files` field to include only dist/, src/cli.js, and README.md
33+
- Repository URL format corrected to use `git+` prefix (eliminates npm publish warning)
34+
35+
### Improved
36+
- **README** - Premium documentation matching Toon's quality
37+
- Head-to-head comparison table: ASON vs Toon showing ASON superiority
38+
- ASCII bar charts for visual benchmark comparison
39+
- Complete API reference with TypeScript examples
40+
- CLI usage guide with real-world examples
41+
- Professional badges (npm, TypeScript, License, Node.js)
42+
- Detailed table of contents with deep anchor links
43+
- "When to use" guide comparing ASON, Toon, JSON, and CSV
44+
- **Package Distribution** - Optimized for production
45+
- Tarball size: 37.8 KB (includes types, CLI, and docs)
46+
- Unpacked size: 148 KB
47+
- Only 7 files published (down from previous bloat)
48+
49+
### Performance
50+
- Benchmarks updated showing ASON achieves **+4.94% average token reduction**
51+
- Toon shows **-6.75% average** (worse than JSON in some cases)
52+
- ASON wins on **3 out of 4** tested real-world datasets
53+
- CLI `--stats` flag provides instant compression metrics
54+
55+
## [1.0.0] - 2025-01-10
56+
57+
### Added
58+
- Initial release of ASON (Aliased Serialization Object Notation)
59+
- Complete Node.js implementation with SmartCompressor
60+
- Automatic pattern detection (zero configuration required)
61+
- Object references for repeated structures (`&obj0`, `&obj1`, etc.)
62+
- Inline-first value dictionary optimized for LLM readability (`#0`, `#1`, etc.)
63+
- Uniform array compression with `@keys` notation
64+
- Path flattening for nested single-property objects
65+
- Configurable indentation (0, 1, or 2 spaces)
66+
- Lossless compression with perfect round-trip fidelity
67+
- Interactive web visualizer with compress/decompress modes
68+
- Comprehensive documentation site
69+
- Benchmark suite comparing against Toon and JSON
70+
- Complete test suite with Jest
71+
- Community files: CODE_OF_CONDUCT, CONTRIBUTING, SECURITY
72+
- MIT License
73+
74+
### Performance
75+
- Achieves 33.26% token reduction on real-world data (28KB JSON)
76+
- Beats Toon format by 8 tokens (1,808 vs 1,816)
77+
- Processes ~35ms for compression + decompression cycle
78+
- Memory efficient: ~10MB for large datasets
79+
80+
### Documentation
81+
- Web visualizer at `/docs/index.html`
82+
- Complete API documentation at `/docs/docs.html`
83+
- Interactive benchmarks at `/docs/benchmarks.html`
84+
- README with quick start guide
85+
- Examples in `/nodejs-compressor/examples/`
86+
87+
[1.1.0]: https://github.com/SeanLuis/ason/releases/tag/v1.1.0
88+
[1.0.0]: https://github.com/SeanLuis/ason/releases/tag/v1.0.0

CODEOWNERS

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# CODEOWNERS file for ASON project
2+
# This file defines code ownership for the repository
3+
# Learn more: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
4+
5+
# Global owners - will be requested for review on all PRs
6+
* @SeanLuis
7+
8+
# Node.js implementation
9+
/nodejs-compressor/ @SeanLuis
10+
11+
# Web visualizer and documentation
12+
/docs/ @SeanLuis
13+
14+
# Documentation files
15+
*.md @SeanLuis
16+
LICENSE @SeanLuis
17+
18+
# Test files
19+
/nodejs-compressor/tests/ @SeanLuis
20+
/nodejs-compressor/benchmarks/ @SeanLuis
21+
22+
# Configuration files
23+
package.json @SeanLuis
24+
*.config.js @SeanLuis
25+
26+
# Security and community files
27+
/CODE_OF_CONDUCT.md @SeanLuis
28+
/CONTRIBUTING.md @SeanLuis
29+
/SECURITY.md @SeanLuis

CODE_OF_CONDUCT.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Contributor Covenant Code of Conduct
2+
3+
## Our Pledge
4+
5+
We as members, contributors, and leaders pledge to make participation in our
6+
community a harassment-free experience for everyone, regardless of age, body
7+
size, visible or invisible disability, ethnicity, sex characteristics, gender
8+
identity and expression, level of experience, education, socio-economic status,
9+
nationality, personal appearance, race, caste, color, religion, or sexual
10+
identity and orientation.
11+
12+
We pledge to act and interact in ways that contribute to an open, welcoming,
13+
diverse, inclusive, and healthy community.
14+
15+
## Our Standards
16+
17+
Examples of behavior that contributes to a positive environment for our
18+
community include:
19+
20+
* Demonstrating empathy and kindness toward other people
21+
* Being respectful of differing opinions, viewpoints, and experiences
22+
* Giving and gracefully accepting constructive feedback
23+
* Accepting responsibility and apologizing to those affected by our mistakes,
24+
and learning from the experience
25+
* Focusing on what is best not just for us as individuals, but for the overall
26+
community
27+
28+
Examples of unacceptable behavior include:
29+
30+
* The use of sexualized language or imagery, and sexual attention or advances of
31+
any kind
32+
* Trolling, insulting or derogatory comments, and personal or political attacks
33+
* Public or private harassment
34+
* Publishing others' private information, such as a physical or email address,
35+
without their explicit permission
36+
* Other conduct which could reasonably be considered inappropriate in a
37+
professional setting
38+
39+
## Enforcement Responsibilities
40+
41+
Community leaders are responsible for clarifying and enforcing our standards of
42+
acceptable behavior and will take appropriate and fair corrective action in
43+
response to any behavior that they deem inappropriate, threatening, offensive,
44+
or harmful.
45+
46+
Community leaders have the right and responsibility to remove, edit, or reject
47+
comments, commits, code, wiki edits, issues, and other contributions that are
48+
not aligned to this Code of Conduct, and will communicate reasons for moderation
49+
decisions when appropriate.
50+
51+
## Scope
52+
53+
This Code of Conduct applies within all community spaces, and also applies when
54+
an individual is officially representing the community in public spaces.
55+
Examples of representing our community include using an official e-mail address,
56+
posting via an official social media account, or acting as an appointed
57+
representative at an online or offline event.
58+
59+
## Enforcement
60+
61+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
62+
reported to the community leaders responsible for enforcement via GitHub issues.
63+
All complaints will be reviewed and investigated promptly and fairly.
64+
65+
All community leaders are obligated to respect the privacy and security of the
66+
reporter of any incident.
67+
68+
## Enforcement Guidelines
69+
70+
Community leaders will follow these Community Impact Guidelines in determining
71+
the consequences for any action they deem in violation of this Code of Conduct:
72+
73+
### 1. Correction
74+
75+
**Community Impact**: Use of inappropriate language or other behavior deemed
76+
unprofessional or unwelcome in the community.
77+
78+
**Consequence**: A private, written warning from community leaders, providing
79+
clarity around the nature of the violation and an explanation of why the
80+
behavior was inappropriate. A public apology may be requested.
81+
82+
### 2. Warning
83+
84+
**Community Impact**: A violation through a single incident or series of
85+
actions.
86+
87+
**Consequence**: A warning with consequences for continued behavior. No
88+
interaction with the people involved, including unsolicited interaction with
89+
those enforcing the Code of Conduct, for a specified period of time. This
90+
includes avoiding interactions in community spaces as well as external channels
91+
like social media. Violating these terms may lead to a temporary or permanent
92+
ban.
93+
94+
### 3. Temporary Ban
95+
96+
**Community Impact**: A serious violation of community standards, including
97+
sustained inappropriate behavior.
98+
99+
**Consequence**: A temporary ban from any sort of interaction or public
100+
communication with the community for a specified period of time. No public or
101+
private interaction with the people involved, including unsolicited interaction
102+
with those enforcing the Code of Conduct, is allowed during this period.
103+
Violating these terms may lead to a permanent ban.
104+
105+
### 4. Permanent Ban
106+
107+
**Community Impact**: Demonstrating a pattern of violation of community
108+
standards, including sustained inappropriate behavior, harassment of an
109+
individual, or aggression toward or disparagement of classes of individuals.
110+
111+
**Consequence**: A permanent ban from any sort of public interaction within the
112+
community.
113+
114+
## Attribution
115+
116+
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
117+
version 2.1, available at
118+
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
119+
120+
Community Impact Guidelines were inspired by
121+
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
122+
123+
For answers to common questions about this code of conduct, see the FAQ at
124+
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
125+
[https://www.contributor-covenant.org/translations][translations].
126+
127+
[homepage]: https://www.contributor-covenant.org
128+
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
129+
[Mozilla CoC]: https://github.com/mozilla/diversity
130+
[FAQ]: https://www.contributor-covenant.org/faq
131+
[translations]: https://www.contributor-covenant.org/translations

0 commit comments

Comments
 (0)