[RFC]: add support for working with arrays backed by memory-mapped files

[PrathamBhamare]

Full name

Pratham Bhamare

University status

Yes

University name

Veermata Jijabai Technological Institute

University program

Bachelors in Information Technology with Minor in Robotics

Expected graduation

2027

Short biography

I am a sophomore at VJTI, Mumbai, pursuing a B.Tech in Information Technology. During my school days, I didn’t have much exposure to technology or computers. However, I later developed an interest in problem-solving and decided to opt for IT. Since childhood, I have been in love with mathematics and was even selected for this university based on my mathematics score.

I am currently comfortable working with C++, Python, and JavaScript. I can also read and understand code in C. I enjoy challenging myself with difficult tasks. My open-source journey started with STDLIB, and my first merged PR was with STDLIB. I am looking forward to working on this project and learning a lot through the process.

Timezone

UTC + 5:30

Contact details

pratham.bhamare15@gmail.com

Platform

Windows

Editor

I use VS Code as I started my programming journey through various tutorials. Most of the tutorials used VS Code as their editor, and eventually, I became comfortable using it.

Programming experience

I started my programming journey a year ago with basic HTML, CSS, and JavaScript, along with Data Structures and Algorithms (DSA) in C++. Later, I began creating projects.

My first project was a student registration portal, followed by Tindog — a dating website for dogs.

My most recent project was a Stock Market Prediction and Sentiment Analysis platform, where I handled the entire backend. I worked on user authentication, database design, and built the backend using Node.js and Express.

Alongside these projects, I’ve also built a strong foundation in DSA, which continues to help me in problem-solving and system design.

JavaScript experience

I started programming in JavaScript for my web projects. The frontend never fascinated me much, so I shifted my focus to backend development using Node.js and Express. As I explored further, I began understanding how servers, APIs, and databases work together to power dynamic applications. My learning journey involved building small modules, debugging extensively, and gradually scaling up to complete full-stack projects. This hands-on approach helped me gain confidence and deepen my understanding of backend systems.

Node.js experience

I have used Node.js extensively for the backend of my projects. From building RESTful APIs to handling authentication and integrating with databases like MongoDB, Node.js has been my go-to choice for scalable and efficient backend development.

C/Fortran experience

I do not have extensive experience in C or Fortran. However, I have learned C to a decent level while researching for this project, and since I have a strong understanding of C++, I believe transitioning to C won’t be very difficult for me.

Interest in stdlib

stdlib is a standard library for JavaScript and includes a wide range of mathematical and scientific computing utilities. I have a strong interest in mathematics, and while exploring organizations, as soon as I came across stdlib, my first instinct was to work with this organization.

Version control

Yes

Contributions to stdlib

I have made a pull request titled "address commit comments for commit bb378ff (issue #6499)" #6565.
This pull request has already been merged.

As I have just started contributing to open source, this is currently my only contribution. However, I am actively exploring the codebase and issues to make more meaningful contributions in the near future.

stdlib showcase

I have created a showcase project to demonstrate the functionality of stdlib.
The GitHub URL for the project is: https://github.com/PrathamBhamare/stdlib-showcase

This is an interactive webpage where users can provide input and see the use cases of various stdlib functions. Currently, it supports a few selected functions, and I plan to expand its functionality to include more in the near future.

Goals

The goal of this project is to add support for typed arrays backed by memory-mapped files in the stdlib library. This will enable efficient handling of large datasets by allowing partial file access without loading the entire file into memory. It will also support inter-process data sharing across environments like Node.js and Python.

The project involves:

Creating typed array constructors (e.g., Float64ArrayMMap, Int32ArrayMMap, etc.) with support for file-based memory mapping.

Ensuring compatibility with stdlib’s accessor protocol for smooth integration.

Establishing C-JavaScript interoperability via native addons for efficient low-level memory access.

Implementing flexible constructor options (file path, mode, offset, length).

This feature will improve memory performance, support shared memory use cases, and extend stdlib’s capabilities for high-performance data processing.

Why this project?

What excites me the most about this project is that it acts as a bridge between C and JavaScript—two languages I’ve been actively working with. While I have a strong foundation in C++ and JavaScript, I’ve recently been deepening my understanding of C, especially while exploring this project. The idea of building memory-mapped typed arrays and integrating them through native Node.js addons taps into low-level systems programming while keeping the accessibility and flexibility of JavaScript intact.

Moreover, many of the core concepts involved—like memory management, file systems, and inter-process communication—align closely with topics from my curriculum, such as Operating Systems and Computer Architecture. This alignment not only makes the project intellectually stimulating but also a great opportunity to apply my academic knowledge to solve real-world problems in an open-source setting.

Qualifications

I believe I’m well-suited to work on this project due to both my academic background and practical experience. I’ve completed a Computer Architecture course in my second semester and an Operating Systems course in my third semester, both of which provided me with a solid foundation in memory management, file systems, and low-level system operations—key concepts that are central to this project.

On the practical side, I’ve been actively working with the languages required for this project, namely C, JavaScript, and Node.js. While my primary experience is in C++, transitioning to C has been smooth, and I’ve already started contributing to the stdlib repository, gaining familiarity with the codebase and development workflow.

This combination of theoretical knowledge and hands-on coding experience gives me the confidence to contribute meaningfully and effectively to the project.

Prior art

Yes, the concept of using memory-mapped files to back arrays has been implemented in other programming languages and libraries. One notable example is NumPy in Python, which provides support for memory-mapped arrays through the numpy.memmap class. This allows large arrays to be stored on disk and accessed as if they were in memory, enabling efficient processing of data that doesn’t fit entirely into RAM.

Similarly, in C and C++, memory mapping is often used in high-performance computing and systems programming, where functions like mmap() (on Unix systems) or CreateFileMapping() (on Windows) are used to map files directly into the address space of a process.

There are also relevant discussions and blog posts on the use of memory-mapped files in JavaScript environments via native Node.js addons, which allow developers to interface with lower-level system APIs. For example, projects like node-mmap provide bindings to Unix's mmap for Node.js.

These existing implementations demonstrate the feasibility and effectiveness of memory-mapped files for working with large datasets and suggest that a similar approach in stdlib could offer powerful capabilities for scientific and numerical computing in JavaScript.

By learning from these prior works, this project aims to bring similar functionality to the JavaScript ecosystem in a clean, consistent, and developer-friendly manner.

Commitment

I plan to commit approximately 20 hours per week to this project throughout the duration of the Google Summer of Code program. I am applying as a part-time contributor, but I am confident that 20 hours per week will be sufficient for me to make steady progress and meet all milestones effectively.

Before GSoC (Community Bonding Period):
I will use this time to deepen my understanding of the stdlib codebase, engage actively with the community, and refine the technical plan for the project. I will continue making small contributions and improving my familiarity with relevant modules and coding practices used in the project.

During GSoC (Coding Period):
I will dedicate 20 hours weekly, distributed across 5-6 days with a mix of coding, testing, documentation, and communication with mentors.

Time Off:
I will be taking a short vacation for 8 days after May 20. During this time, I may have limited access to my development setup and slower internet, so I plan to frontload some of the lighter research or planning work, and will adjust my weekly hours before and after the vacation to ensure my timeline remains on track.

After GSoC:
Even after the official program ends, I intend to stay involved with the stdlib project. I plan to help maintain and expand the feature if needed, fix any issues that arise, and potentially contribute to future improvements and enhancements in related packages.

Schedule

Assuming a 12 week schedule,

• Community Bonding Period:

• Week 1: Prototype C code for mmap file access and basic Node.js binding setup.

• Week 2: Implement Float64 read-only accessor and test simple usage.

• Week 3: Extend accessors, support index-based access, and add wrapper constructors.

• Week 4: Add write support and implement Float32, Int32, and Uint32 typed arrays.

• Week 5: Start on Complex64/Complex128 types and improve testing and benchmarks

• Week 6: (midterm) Submit working core implementations with documentation and tests.

• Week 7: Add remaining typed arrays (Int8, Uint8, Boolean), refactor codebase.

• Week 8: Support offset/mode arguments, improve platform compatibility.

• Week 9: Implement DataViewMMap and inter-process memory sharing demo.

• Week 10: Finalize implementations, clean up bindings, and optimize performance

• Week 11: Code freeze, write full documentation, usage examples, and tests.

• Week 12: Polish all deliverables and prepare for final submission and review.

• Final Week: Submit project, prepare final report/demo, and wrap up with community.

Notes:

• The community bonding period is a 3 week period built into GSoC to help you get to know the project community and participate in project discussion. This is an opportunity for you to setup your local development environment, learn how the project's source control works, refine your project plan, read any necessary documentation, and otherwise prepare to execute on your project project proposal.
• Usually, even week 1 deliverables include some code.
• By week 6, you need enough done at this point for your mentor to evaluate your progress and pass you. Usually, you want to be a bit more than halfway done.
• By week 11, you may want to "code freeze" and focus on completing any tests and/or documentation.
• During the final week, you'll be submitting your project.

Related issues

Yes, your proposal is directly related to the following issue in the main stdlib project repository:

Proposal Idea Issue: Add support for working with arrays backed by memory-mapped files

This issue outlines the core concept, motivation, and expectations for implementing typed arrays backed by memory-mapped files in stdlib. Your project proposal is a direct implementation of this idea.

Checklist

• I have read and understood the Code of Conduct.
• I have read and understood the application materials found in this repository.
• I understand that plagiarism will not be tolerated, and I have authored this application in my own words.
• I have read and understood the patch requirement which is necessary for my application to be considered for acceptance.
• I have read and understood the stdlib showcase requirement which is necessary for my application to be considered for acceptance.
• The issue name begins with [RFC]: and succinctly describes your proposal.
• I understand that, in order to apply to be a GSoC contributor, I must submit my final application to https://summerofcode.withgoogle.com/ before the submission deadline.