Test coverage and test results summary

Makefile

@@ -58,6 +58,9 @@ install-reqs:
 	@echo $(H1)Installing HTTPie$(H1END)
 	$(VENV_PIP) install --upgrade --editable .
 
+	@echo $(H1)Installing radon$(H1END)
+	$(VENV_PIP) install --upgrade radon
+
 	@echo
 
 

courseProjectCode/Metrics/Readme.md

@@ -0,0 +1,46 @@
+# Setup and Running Maintainability Metrics
+
+## Setup Instructions
+
+1. Clone the repository
+
+   - git clone your-repo-url
+   - cd cli-swen777
+
+2. Install dependencies and setup environment
+
+    - make all
+
+    The above command will:
+    - Create a virtual environment inside ./venv
+    - Install all dependencies
+    - Install HTTPie in editable mode
+    - Run the test suite once
+
+3. Activate the virtual environment
+
+    - source venv/bin/activate
+
+4. Run the testability metrics script
+
+    - python3 courseProjectCode/Metrics/Testability.py
+
+    The above command will:
+
+    - Run all tests with coverage enabled
+    - Generate a coverage report
+    - Output the total number of test cases and test suites
+
+![](testability_report.png)
+
+5. Run the code structure metrics script
+
+    - ./structure.sh
+
+    The above command will:
+
+    - Scan the repository for source lines of code, comments, and cyclomatic complexity
+    - Calculates comment density (per-repo, not per-file) and average cyclomatic complexity
+    - Output the SLOC, Comments, Comment Density, and Average Cyclomatic Complexity
+
+![](structure_report.png)

courseProjectCode/Metrics/Testability.py

@@ -0,0 +1,27 @@
+import subprocess # Import subprocess to run shell commands from Python
+
+def run_command(command): # function to execute shell command and returns its output as a string
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    return result.stdout.strip()
+
+def count_test_cases(): # Fucntion to count total test cases by collecting pytest functions
+    output = run_command("pytest --collect-only | grep '<Function' | wc -l")
+    return int(output) if output else 0
+
+def count_test_suites():# Fucntion to count total test suites by collecting pytest modules
+    output = run_command("pytest --collect-only | grep '<Module' | wc -l")
+    return int(output) if output else 0
+
+def run_coverage(): # Runs pytest with coverage to measure code coverage and display the report
+    print("\nRunning tests with coverage...")
+    subprocess.run("coverage run --source=httpie -m pytest || true", shell=True)
+    print("\nCoverage Report:")
+    subprocess.run("coverage report", shell=True)
+
+if __name__ == "__main__":
+    print("Collecting Testability Metrics for HTTPie...\n")
+    run_coverage()
+    total_tests = count_test_cases()
+    total_suites = count_test_suites()
+    print(f"Total test cases: {total_tests}")
+    print(f"Total test suites (modules): {total_suites}")

courseProjectCode/Metrics/structure.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+# Calculating SLOC and comment density using radon's raw output. (C % S) is the total comment density for all source lines.
+if ! command -v radon &>/dev/null; then
+    echo "radon is not installed. Install with: pip install radon"
+    exit 1
+fi
+
+echo "Calculating comment density for Python files..."
+radon raw -s ../../ | awk '/SLOC:/ {sloc=$2} /Comments:/ {comments=$2} /\(C % S\)/ {density=$4} END {print "SLOC:", sloc,", Comments:", comments,", Density:", density}'
+
+echo
+echo "Calculating cyclomatic complexity for Python files..."
+PYTHONWARNINGS="ignore" radon cc -s -a ../../ | grep -i average

courseProjectCode/project-proposal.md

@@ -0,0 +1,41 @@
+# Project Proposal: Enhancing HTTPie
+
+## Project Overview
+
+HTTPie is a command-line HTTP client designed as human friendly as possible. It provides an intuitive user interface, JSON support, syntax highlighting, and support for plugins and extensions. HTTPie simplifies testing and interacting with HTTP APIs, making it a popular tool among developers, testers, and system administrators.
+
+This project focuses on contributing enhancements to the HTTPie open-source project while ensuring code quality, maintainability, and usability. The main goals are:
+
+- Familiarize ourselves with HTTPie’s architecture and testing framework.  
+- Create quality metric gathering tools and perform analyses based on these metrics.  
+- Extend or improve existing functionality through bug fixes, new features, or optimizations.  
+- Ensure any modifications align with HTTPie’s philosophy of user-friendliness and developer experience.  
+- Collaborate using GitHub workflows (branches, pull requests, and code reviews).
+
+Deliverables include code contributions, improved documentation, and automated test coverage to validate correctness.
+
+## Key Quality Attributes
+
+The following report summarizes this team’s findings pertaining to quality attributes gathered from HTTPie’s codebase. The objective of this report is to provide insights into the maintainability of the codebase, with a focus on code structure and test robustness. All Quality Attributes were calculated programmatically, aside from test coverage percentage, which was already available through the official repository. This analysis focuses exclusively on Python source files, excluding other files such as, but not limited to, markup, setup scripts, and documentation.
+
+1. Codebase size and structure  
+* *Average LOC per file \= 134.5*  
+* *Total SLOC \= 14028*  
+  The fairly moderate average lines of code metric would suggest that the codebase is well structured, where most files have a strong singular focus. Functionality seems to be well isolated, preventing overly large or complex files.  
+
+2. Documentation  
+* *Total comments \= 794*  
+* *Comment density for source lines only \= 6%*  
+  The relatively low comment density of HHTPie suggests one of two things; either the files are written in such a way that most of the code is self-explanatory, or there is a lack of adequate documentation that may cause confusion for open source developers.  
+
+3. Complexity  
+* *Average cyclomatic complexity \= \~2.75*  
+  This low average cyclomatic complexity, which Radon rates an A, indicates a straight-forward control structure. As a result, functions should be easy to test and maintain, while also benefitting from reduced risk of bugs and ease of refactoring.
+
+4. Testing Quality  
+* *Total number of tests \= 1028*  
+* *Test coverage \= 91%*  
+  For a simple CLI tool, which is not meant for critical operations, this is a strong showing in terms of testing. The high coverage percentage and large number of individual tests indicates an extensive suite of testing that covers nearly all the functionality that HHTPie provides.  
+
+5. Conclusion  
+   Overall, HTTPie’s codebase showcases impressive quality attributes across the areas we are focusing on. We see manageable file sizes with low complexity, a control structure that keeps the codebase maintainable, scalable, and modifiable, and a testing suite that encompasses nearly all the possible functionalities being provided to users. Our only critique would be that the comment density is well below industry averages, which can range anywhere from 15%-25%, depending on the source. This leaves a solid opportunity for our team to come in and improve inline documentation in an effort to shore up the one apparent gap in HTTPie’s code quality, improving the development experience for those who come after.

courseProjectDocs/Setup/readme.md

@@ -0,0 +1,19 @@
+# Setup to run the tests and coverage report
+
+1. Running the tests
+
+* make test
+    -   Runs all the tests present in the repo
+
+![](test_rsults.png)
+
+2. Getting the test coverage
+
+* python3 courseProjectCode/Metrics/Testability.py
+
+    The above command will:
+
+    - Run all tests with coverage enabled
+    - Generate a coverage report
+    - Output the total number of test cases and test suites
+

courseProjectDocs/Setup/report.md

@@ -0,0 +1,41 @@
+# Test suite summary
+
+* Unit tests: These are the vast majority of tests we see in Httpie, these cover base functionality such as authentication/authorization, parsing user input, raising proper exceptions, and connecting to external services. These tests often do not include mocking.
+* Integration tests: These tests validate the behaviour between the CLI against HTTP endpoints to confirm if the features work together correctly. One such example would be testing the authentication which sends requests to the mock up server with credentials and check that the server responds with authenticated access. 
+* UI tests: While there is no "UI" specifically, the terminal acts as a UI as this is where all of the user interaction will take place. The UI tests themselves involve parsing input, displaying proper output, output formatting, and handling utf conversions. These tests often involve a mock environment and dummy URL's
+* Regression: While there are no tests explicitly labeled as regression tests, we found references to regression testing through release documentation and git history. This indicates that certain tests have been implemented that implicitly prevent regression while serving a different explicit purpose 
+
+# Metrics and observations
+
+* Number of tests run - 1028
+* Number of tests passed - 1020
+* Number of tests failed - 3
+* Number of tests skipped - 5
+
+# Branch coverage summary
+
+# Steps to get the branch coverage
+
+* Run the below commands: 
+    - coverage run --branch --source=httpie -m pytest
+    - coverage run -m
+
+* Branch coverage percentage: 89
+
+![](Branch_coverage_report.png)
+
+# Steps to get the Statement coverage
+
+* Run the below commands: 
+    - coverage run --source=httpie -m pytest
+    - coverage run -m
+
+* Statement coverage percentage: 91
+
+![](Statement_coverage_report.png)
+
+# Observations
+
+1. Since the tests are dependent on external services, we observed inconsistency in the test results where we running the tests back to back gave different results and also different results on windows and mac. 
+2. The documentation is designed around a unix based system (bash), which raised errors when running the same on a windows based system (powershell). 
+

courseProjectDocs/requirements-and-oracles.md

@@ -0,0 +1,31 @@
+# Requirements and Test Oracles
+
+## Functional Requirements
+1. The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).
+2. The system shall allow the user to upload/download files.
+3. The system shall allow the user to send request in different data formats (JSON, URL Parameters, HTTP Headers).
+4. The system shall allow the user to configure their terminal and profile with their preferences.
+5. The system shall allow the user to see all responses from the request in printed output to the console.
+6. The system shall allow the user to authenticate via username and password before accessing the resource.
+7. The system shall allow the user to set and send custom request names.
+
+
+## Non-Functional Requirements
+1. The system shall work consistently across Linux, MacOS, windows and FreeBSD systems.
+2. The system shall be accessible on web and mobile devices and desktop CLI.
+3. The system shall allow a maximum of n redirects, where n is defined by the user.
+4. The system shall timeout stale requests after n milliseconds, where n is defined by the user.
+5. The system shall accept no more than one request at a time.
+
+## Test Oracles
+
+| Requirement ID | Requirement Description | Test Oracle (Expected Behavior) |
+|-----------------------|-----------------------------------|---------------------------------------------|
+| FR-1                   | The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).|When the user sends an HTTP DELETE request, the resource under control should no longer contain those elements.|
+| FR-1                   | The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).|When a user sends an HTTP GET request containing Foo:bar, the response must include “Foo”: “bar”.|
+| FR-1                   | The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).|When a user sends an HTTP GET request containing Accept: or User-Agent:, those headers shall be absent from the request.|
+| FR-1                   | The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).|When the user sends an HTTP POST request with a specified file, then it will be added to the resource under control.|
+| NFR-3                  | The system shall allow a maximum of n redirects, where n is defined by the user.|When the redirect count exceeds the limit, the CLI must return ExitStatus.ERROR_TOO_MANY_REDIRECTS.|
+| NFR-4                  |The system shall timeout stale requests after n milliseconds, where n is defined by the user.|When the time to process a request exceeds the timeout limit, the CLI must return ExitStatus.ERROR_TIMEOUT.|
+| FR-5                   |The system shall allow the user to see all responses from the request in printed output to the console.|When a response is received from any request, the response will be printed to the terminal.|
+| FR-7                   |The system shall allow the user to set and send custom request names.|When the user creates a custom HTTP request name, then that name is added to the list of available requests that can successfully be sent.|