Asynchronous Web Server

This project is a high-performance, asynchronous web server built in C for Linux environments. It is designed to efficiently serve both static and dynamic content by leveraging advanced, non-blocking I/O operations. The server is built around a single-threaded, event-driven architecture to handle a large number of concurrent connections with minimal resource overhead.

Features

• Asynchronous I/O: Utilizes epoll for handling multiple client connections concurrently without blocking.
• Efficient Static File Transfer: Employs sendfile for zero-copy transfer of static files, minimizing CPU overhead.
• Asynchronous File I/O: Reads dynamic content from disk using the AIO library to prevent blocking the main event loop.
• Non-Blocking Sockets: All socket operations are non-blocking to ensure the server remains responsive under heavy load.
• HTTP/1.1 Parsing: Integrates a lightweight, callback-based HTTP parser to handle incoming requests.
• Connection State Machine: Manages the lifecycle of each connection through a well-defined state machine.
• Robust Testing Suite: Includes a comprehensive test suite to verify functionality and check for memory leaks.

Core Concepts and Technical Implementation

The server's design is centered around a non-blocking, event-driven architecture. Here are the key technologies and concepts that make this possible:

• Asynchronous I/O with epoll: The server's core is an event loop built around the epoll API. This allows a single thread to monitor a large number of file descriptors (sockets) for I/O events (e.g., a new connection, incoming data, or a socket being ready for writing) without polling each one individually. This is the foundation of the server's asynchronous and highly concurrent nature.

• Zero-Copy Static File Serving with sendfile: For serving static files (from the /static directory), the server uses the sendfile system call. This is a "zero-copy" operation that allows the kernel to transfer data directly from a file descriptor to a socket descriptor. By avoiding the intermediate step of copying data into user-space buffers, sendfile dramatically reduces CPU overhead and improves performance, especially for large files.

• Asynchronous File I/O for Dynamic Content: When serving dynamic content (from the /dynamic directory), the server needs to read the file into memory before sending it. To avoid blocking the main event loop while reading from disk, the server uses the asynchronous I/O (AIO) library. The aio_read function initiates a non-blocking read operation, allowing the event loop to continue processing other connections. The server is notified of the read's completion through the epoll event loop.

• Connection State Machine: To manage the lifecycle of each client connection, the server implements a state machine. Each connection is always in a specific state, such as STATE_READING_REQUEST, STATE_SENDING_HEADER, or STATE_SENDING_FILE. The server transitions the connection between these states based on the I/O events it receives from epoll. This robust model allows the server to keep track of the progress of thousands of concurrent connections and perform the appropriate actions at each stage.

• HTTP Parsing with http-parser: The server uses http-parser, a lightweight, callback-based C library, to parse incoming HTTP requests. As data is read from a socket, it is fed to the parser. The parser then invokes a series of callbacks to notify the server of the request method, URL, headers, and body, allowing the server to construct an appropriate response.

Architecture and Request Lifecycle

The server operates on a single-threaded event loop. Here's a step-by-step breakdown of how a typical request is handled:

1. Initialization: The server creates a listening socket, binds it to a port, and adds it to the epoll instance to monitor for incoming connections.

2. Accepting a Connection: When a new client connects, the listening socket becomes readable. The server accepts the new connection, makes the client socket non-blocking, and adds it to the epoll instance. A new state machine is created for this connection, with its initial state set to STATE_READING_REQUEST.

3. Reading and Parsing the Request: When the client sends data, the socket becomes readable. The server reads the data and feeds it to the http-parser. The parser's callbacks populate a request structure with the parsed information (URL, method, etc.).

4. Serving the Response: Based on the parsed URL, the server decides how to respond:

   • Static Content: If the requested file is in the /static directory, the server opens the file and transitions the connection state to STATE_SENDING_HEADER. After sending the HTTP headers, it moves to STATE_SENDING_FILE and uses sendfile to stream the file directly to the client.
   • Dynamic Content: If the requested file is in the /dynamic directory, the server initiates an asynchronous read using aio_read and transitions the state to STATE_AIO_READING. The server continues to process other events. When the AIO read completes, the server is notified via epoll, sends the HTTP headers, and then sends the file's content (now in memory) to the client.
   • 404 Not Found: If the requested file does not exist, the server sends a "404 Not Found" response.

5. Closing the Connection: Once the response has been completely sent, the server closes the client socket and cleans up all resources associated with the connection's state machine.

Building the Server

1. Clone the repository:

   git clone https://github.com/raresgoidescu/async-web-server.git
   cd async-web-server

2. Compile the source code:

   make -C src

   This will create an executable named aws in the src directory.

Running the Server

To start the server, run the following command from the project's root directory:

./src/aws

The server will start listening for incoming connections on the configured port.

Usage

The web server serves files from two main directories, which should be located in the AWS_DOCUMENT_ROOT path specified in the server's configuration.

• /static: Place files that can be served directly (e.g., HTML, CSS, images) in this directory. These will be served using the efficient sendfile system call.
• /dynamic: Place files that require server-side processing or should be read into memory before sending in this directory. These are read asynchronously.

To request a file, use a web browser or a tool like curl:

# Request a static file
curl http://localhost:8080/static/index.html

# Request a dynamic file
curl http://localhost:8080/dynamic/api/data

Running Tests

To run the comprehensive test suite, navigate to the tests directory and use make:

cd tests
make check

The tests will verify all aspects of the server's functionality, from basic connectivity and file transfers to error handling and memory leak detection.