[RFC]: Develop a project test runner

[Hecker165]

Full name

Abhay Punia

University status

Yes

University name

Indian Institute of Technology, Jammu

University program

Mathematics and Computing

Expected graduation

2027

Short biography

I am a second-year student at IIT Jammu, pursuing a B.Tech in Mathematics and Computing. I have good experience in Python, JavaScript, and C++. In my first year, I started learning Web Development alongside Competitive Programming and Data Structures & Algorithms, which has given me a strong grasp of JavaScript, TypeScript, Next.js, React.js, MongoDB, Node.js, and Express.js.

Due to courses such as Numerical Methods taught at my institute, I have implemented several mathematical functions in C++ and have gained significant experience in this area. I also have experience in Artificial Intelligence and Machine Learning, including concepts such as RAGs and AI Agents.

In the future, I am eager to explore more in the AI-ML and Web Development domains to harness the true potential of both technologies.

Timezone

Indian Standard Time ( IST ), UTC+5:30

Contact details

Email:- abhaypunia40@gmail.com, Github:-Hecker165

Platform

Windows

Editor

VS Code, as I have been using it for the past 7-8 years because of its simplicity, efficiency, and extensive extensions that enhance my development workflow. It is my go-to editor for all my projects.

Programming experience

My coding journey began with Python when I used to create small projects after watching tutorials on YouTube. Then, in college, I was taught C++ as part of the DSA course, which helped me become highly proficient in the language. After that, I started learning Web Development and picked up languages such as JavaScript and TypeScript, along with technologies and frameworks like React, Next.js, etc.

Some of my major projects:

Agentic RAG

We took the concept of Agentic RAGs beyond PDFs and applied it to APIs. We developed an Agentic RAG using the Pathway library to create a chatbot that could be hosted on a website. Instead of manually navigating the website, users could ask queries directly to the chatbot, which would determine the appropriate API endpoint to query and return the desired output.
Technologies used: Flask, Pathway, Next.js, Pydantic, along with various AI-related Python libraries.
Our idea was ranked 13th out of 23 IITs at the Inter-IIT Tech Meet Hackathon.

ML Model for MOF Property Prediction

Developed an ML model for property prediction of MOFs and optimizing these properties by redesigning the MOF. It is an ongoing project, expected to be completed by April 2025. The project heavily relies on AI-ML libraries in Python.

Emotion Recognition and Rephraser Web App

As part of the Web-Dev Inter-Branch Hackathon at my institute, we developed a web app for emotion detection in text. The app extracts content from PDFs, integrates NLP models, and rephrases text based on the required target emotion using LLMs.
Technologies used: Hugging Face models, Flask, HTML, CSS, and JavaScript.

Airbnb Clone

While learning Web Development, I also created a clone of Airbnb’s website. It was built using HTML, CSS, JavaScript, Node.js, Express.js, MongoDB, and several Node packages.

I have also participated in global open-source events such as Hacktoberfest 2024 and Social Winter of Code 2025.

JavaScript experience

I started learning JavaScript in my first year, as it is the foundation of Web Development. It is extensively used across Web-Dev technologies, from vanilla JavaScript to modern frameworks like React and Next.js. JavaScript is also one of the easiest languages to understand, given its vast potential. In the projects I have built, JavaScript was used everywhere—from the backend to the frontend—giving me a strong understanding of how the language functions across different stages of the Web Development process.

My favorite feature of JavaScript is the variety of built-in functions it provides for arrays, strings, objects, etc. Functions like indexOf(), filter(), {...obj}, lastIndexOf(), and many more offer great flexibility to programmers while writing code.

However, my least favorite feature of JavaScript is type coercion. While it sometimes speeds up development by handling conversions automatically, it also leads to unexpected and confusing bugs that can take hours to debug.

Node.js experience

In all of my projects, I have used Node.js in some form or another. While I have experience building efficient server-side web applications with JavaScript, I have also worked extensively with various npm and Yarn packages.

I have used Node.js to develop backend services and APIs, handling tasks such as authentication, database connections, and data processing. This has given me a solid understanding of how to build scalable and efficient backend systems.

C/Fortran experience

I have a strong understanding of C++, as it was part of my college coursework. Additionally, I have some experience with C and am confident that I won’t have any issues understanding or referring to code written in it.

I also have some experience with Fortran, thanks to my professors, and I am fairly proficient in understanding Fortran code. Moreover, I am always open to learning any of these languages in greater depth if needed.

Interest in stdlib

I really like stdlib because it gives me a well-organized, high-quality set of tools for JavaScript. It fills in many of the gaps left by the built-in features. Working closely with stdlib has helped me see how its simple, modular design and strong performance improvements make coding easier. Plus, the clear documentation means I can quickly understand, update, and use it in many different projects.
I like how stdlib puts a big focus on performance. It uses low-level tricks, manages memory well, and is built to be lightweight. This means even when handling large amounts of data, everything runs smoothly. In projects where speed mattered a lot, I found stdlib's functions worked better than other options, making it a key part of my workflow.

Version control

Yes

Contributions to stdlib

• Merged PRs
  
• Open PRs

stdlib showcase

In progress

Goals

Details about my implementation plan have been described in the following pdf:

implementation_plan.pdf

The primary goal of this project is to replace the external tape dependency with a minimal, purpose-built test harness internal to stdlib, enhancing maintainability and control over the testing infrastructure.

Main Goals:

• Develop the Core Test Harness: Implement the @stdlib/test/harness package, including the sequential test execution logic and a tape-like API (test(name, t => {...})) familiar to stdlib contributors.

• Implement Essential Assertions: Provide implementations within the harness for the specific subset of assertion methods (ok, strictEqual, end, throws, notEqual, notOk, fail, deepEqual, equal) actively used across stdlib tests.

• Integrate String Interpolation: Leverage @stdlib/string/format to provide built-in, printf-style string interpolation capabilities for assertion messages and comments within the harness.

• Ensure TAP Compliance: Guarantee the harness generates Test Anything Protocol (TAP) version 13 compliant output to standard out, ensuring compatibility with existing TAP consumers.

• Migrate Existing Tests: Successfully migrate all existing unit tests within the stdlib monorepo from using tape to using the new @stdlib/test/harness.

Supporting Goals:

• Develop Migration Tooling: Create an automated script (codemod) using tools like jscodeshift to facilitate the large-scale migration of test files.

• Implement CLI Runner: Develop a command-line interface (stdlib-test) executable for easy test discovery (using glob patterns) and execution across the project.

• Ensure Harness Test Coverage: Write comprehensive unit tests for the @stdlib/test/harness package itself to ensure its correctness and robustness.

• Provide Documentation: Create clear documentation detailing how to use the new test harness and its CLI tool.

• Investigate Coverage Tool Migration: Explore the feasibility (considering Node.js version compatibility) of migrating test coverage reporting from istanbul to nyc or c8 (as a bonus/stretch goal).

The main goals are the priority, with the supporting goals facilitating their successful completion and integration. The ultimate deliverable is the complete removal of the tape dependency and the successful adoption of @stdlib/test/harness across the entire stdlib project.

Why this project?

This project excites me because it’s a chance to build something impactful for stdlib.js. A dedicated in-house test runner will make testing more efficient, maintainable, and tailored to the library’s needs. Instead of relying on external tools, this project ensures full control over the testing process, making stdlib.js more robust.

Developing a test runner involves handling assertions, managing async execution, optimizing performance, and ensuring clear reporting. It’s a deep learning experience, building a tool that developers can rely on.

Beyond the technical challenge, I’m motivated by the opportunity to contribute something meaningful to the open-source community. I’m eager to take on this responsibility, explore the internals of testing frameworks, and build something that adds real value.

Qualifications

I have experience using Tape for unit testing, so I know how test runners work and what makes them effective. Writing and running tests with Tape has shown me what works well and what could be improved, which will help in designing something that fits stdlib.js better.

I’m also familiar with the implementation of @stdlib/bench/harness and its file structure. Looking into its internals has given me a good understanding of how it runs tests and manages execution, which will be useful when building this test runner.

Prior art

This area has been widely explored, not just by libraries like Tape, which use the Test Anything Protocol (TAP) for simple, sequential testing, but also by larger test frameworks like Mocha, Jest, and AVA. Each of these takes a different approach, with Tape standing out for its minimal and straightforward design.

Within stdlib, we already have @stdlib/bench/harness for benchmarking, proving that we can build lightweight, well-integrated tools. By following Tape’s simplicity and using our own benchmarking harness as a guide, we can create a test runner that aligns with stdlib’s design.

Commitment

I will be having a full summer break from 8th March till 28th July. After that, I’ll be managing the project alongside my college. My plan is to focus on completing the project in the last few weeks before my class tests so that I can be fully committed to this project without any clashes.

I am fully committed to dedicating 350 hours for the implementation of this project. As of now, I do not have any other major commitments. The only period where I may be working fewer hours is during a one-week vacation in the initial community bonding period.

Beyond the GSoC, I definitely plan to remain an active member of the Stdlib community. I aim to contribute further by maintaining and improving the project, engaging in discussions, and assisting new contributors. My goal is to ensure that my work continues to be useful and beneficial to the community in the long run.

Schedule

Here is the detailed plan and decided Phases for the implementation of the project:

implementation_plan.pdf

implementation_schedule.pdf

Assuming a 12 week schedule,

• Community Bonding Period: I will start working on the project from this period itself, however, I'm not committing to any specific target in this period. All progress made is just to give me a breathing space for the future deadlines.

• Week 1: Initiate Phase A: Set up the core package structure and implement the basic Harness and singleton management logic. Define the main test() entry point function.

• Week 2: Continue Phase A: Implement the sequential test execution flow (queue management, runNext) and the basic Test Context (t) object with the essential t.end() method.

• Week 3: Continue Phase A: Implement foundational assertions (t.ok, t.strictEqual, t.equal) and the basic TAP reporter for version and ok/not ok lines.

• Week 4: Complete Phase A: Implement remaining simple assertions (t.pass, t.fail). Begin creating tests for the harness itself and refine the core execution logic.

• Week 5: Initiate Phase B: Implement complex assertions (t.throws, t.notEqual, t.notOk, t.pass). Begin integrating @stdlib/string/format for message interpolation in assertions.

• Week 6: (midterm) Mid-Term Evaluation Prep: Focus exclusively on testing all implemented features thoroughly, fixing bugs, cleaning up code, and updating documentation (README, internal comments).

• Week 7: Continue Phase B: Implement the t.deepEqual() and t.notDeepEqual() assertion with careful attention to tape semantics. Implement t.comment() with interpolation support. Enhance TAP failure diagnostics (YAML block).

• Week 8: Complete Phase B: Develop and test the bin/stdlib-test command-line interface for discovering and running test files based on glob patterns. Add usage documentation.

• Week 9: Polish Phase B & Buffer: Finalize all features, add examples, improve harness test coverage, and address any lingering edge cases or minor issues. Use as buffer time if needed.

• Week 10: Initiate Phase C: Develop and rigorously test the automated migration script. Perform a pilot migration on a selected subset of stdlib packages.

• Week 11: Complete Phase C: Execute the full migration script across the entire stdlib repository. Run all tests; focus intensively on debugging the harness and verifying test suite results.

• Week 12: Final Evaluation Prep: Conduct final end-to-end testing of the migrated codebase. Address critical bugs, finalize all documentation, clean up code, and prepare the final GSoC submission.

• Final Week: Proceed for implementing any kind of feedback and finally submitting the project.

Notes:

• The community bonding period is a 3 week period built into GSoC to help you get to know the project community and participate in project discussion. This is an opportunity for you to setup your local development environment, learn how the project's source control works, refine your project plan, read any necessary documentation, and otherwise prepare to execute on your project project proposal.
• Usually, even week 1 deliverables include some code.
• By week 6, you need enough done at this point for your mentor to evaluate your progress and pass you. Usually, you want to be a bit more than halfway done.
• By week 11, you may want to "code freeze" and focus on completing any tests and/or documentation.
• During the final week, you'll be submitting your project.

Related issues

No response

Checklist

• I have read and understood the Code of Conduct.
• I have read and understood the application materials found in this repository.
• I understand that plagiarism will not be tolerated, and I have authored this application in my own words.
• I have read and understood the patch requirement which is necessary for my application to be considered for acceptance.
• I have read and understood the stdlib showcase requirement which is necessary for my application to be considered for acceptance.
• The issue name begins with [RFC]: and succinctly describes your proposal.
• I understand that, in order to apply to be a GSoC contributor, I must submit my final application to https://summerofcode.withgoogle.com/ before the submission deadline.

[kgryte]

@Hecker165 Thanks for opening this RFC. A few comments/questions:

1. We currently rely on a fork of tape. Is there anything from the upstream tape implementation that we should consider for our in-house test runner?
2. Personally, I'd like to consolidate assertions. Namely, we don't need separate equal and strictEqual methods. These are the same in tape, and I'd prefer to just have one method: equal. Ditto for not equal.
3. Personally, I'd also be okay with removing ok and notOk.
4. We'd likely want to think a bit about deep equality. Currently, this relies on Node's deep-equal, but we may want to consider supporting stdlib's various "exotic" types, such as accessor arrays (boolean, complex), ndarrays, etc. Otherwise, maybe we just scrap deepEqual altogether and a dev should be forced to explicitly import stdlib's deep-equal implementation or the specific assertion package they need.
5. I am not terribly optimistic about running a full migration script across the codebase. It may be worthwhile for you to perform a survey of all the different ways that we write test assertion messages so that you have a concrete idea of how many patterns we need to account for. It may be possible to perform a limited migration on a per-namespace basis.