diff --git a/Makefile b/Makefile index a2a80a1778..3a534154b4 100644 --- a/Makefile +++ b/Makefile @@ -58,6 +58,9 @@ install-reqs: @echo $(H1)Installing HTTPie$(H1END) $(VENV_PIP) install --upgrade --editable . + @echo $(H1)Installing radon$(H1END) + $(VENV_PIP) install --upgrade radon + @echo diff --git a/courseProjectCode/Metrics/Readme.md b/courseProjectCode/Metrics/Readme.md new file mode 100644 index 0000000000..f937b3bc81 --- /dev/null +++ b/courseProjectCode/Metrics/Readme.md @@ -0,0 +1,46 @@ +# Setup and Running Maintainability Metrics + +## Setup Instructions + +1. Clone the repository + + - git clone your-repo-url + - cd cli-swen777 + +2. Install dependencies and setup environment + + - make all + + The above command will: + - Create a virtual environment inside ./venv + - Install all dependencies + - Install HTTPie in editable mode + - Run the test suite once + +3. Activate the virtual environment + + - source venv/bin/activate + +4. Run the testability metrics script + + - python3 courseProjectCode/Metrics/Testability.py + + The above command will: + + - Run all tests with coverage enabled + - Generate a coverage report + - Output the total number of test cases and test suites + +![](testability_report.png) + +5. Run the code structure metrics script + + - ./structure.sh + + The above command will: + + - Scan the repository for source lines of code, comments, and cyclomatic complexity + - Calculates comment density (per-repo, not per-file) and average cyclomatic complexity + - Output the SLOC, Comments, Comment Density, and Average Cyclomatic Complexity + +![](structure_report.png) \ No newline at end of file diff --git a/courseProjectCode/Metrics/Testability.py b/courseProjectCode/Metrics/Testability.py new file mode 100644 index 0000000000..9b8efb70dc --- /dev/null +++ b/courseProjectCode/Metrics/Testability.py @@ -0,0 +1,27 @@ +import subprocess # Import subprocess to run shell commands from Python + +def run_command(command): # function to execute shell command and returns its output as a string + result = subprocess.run(command, shell=True, capture_output=True, text=True) + return result.stdout.strip() + +def count_test_cases(): # Fucntion to count total test cases by collecting pytest functions + output = run_command("pytest --collect-only | grep '/dev/null; then + echo "radon is not installed. Install with: pip install radon" + exit 1 +fi + +echo "Calculating comment density for Python files..." +radon raw -s ../../ | awk '/SLOC:/ {sloc=$2} /Comments:/ {comments=$2} /\(C % S\)/ {density=$4} END {print "SLOC:", sloc,", Comments:", comments,", Density:", density}' + +echo +echo "Calculating cyclomatic complexity for Python files..." +PYTHONWARNINGS="ignore" radon cc -s -a ../../ | grep -i average diff --git a/courseProjectCode/Metrics/structure_report.png b/courseProjectCode/Metrics/structure_report.png new file mode 100644 index 0000000000..0403c74187 Binary files /dev/null and b/courseProjectCode/Metrics/structure_report.png differ diff --git a/courseProjectCode/Metrics/testability_report.png b/courseProjectCode/Metrics/testability_report.png new file mode 100644 index 0000000000..cd2de240ad Binary files /dev/null and b/courseProjectCode/Metrics/testability_report.png differ diff --git a/courseProjectCode/project-proposal.md b/courseProjectCode/project-proposal.md new file mode 100644 index 0000000000..539d82e550 --- /dev/null +++ b/courseProjectCode/project-proposal.md @@ -0,0 +1,41 @@ +# Project Proposal: Enhancing HTTPie + +## Project Overview + +HTTPie is a command-line HTTP client designed as human friendly as possible. It provides an intuitive user interface, JSON support, syntax highlighting, and support for plugins and extensions. HTTPie simplifies testing and interacting with HTTP APIs, making it a popular tool among developers, testers, and system administrators. + +This project focuses on contributing enhancements to the HTTPie open-source project while ensuring code quality, maintainability, and usability. The main goals are: + +- Familiarize ourselves with HTTPie’s architecture and testing framework. +- Create quality metric gathering tools and perform analyses based on these metrics. +- Extend or improve existing functionality through bug fixes, new features, or optimizations. +- Ensure any modifications align with HTTPie’s philosophy of user-friendliness and developer experience. +- Collaborate using GitHub workflows (branches, pull requests, and code reviews). + +Deliverables include code contributions, improved documentation, and automated test coverage to validate correctness. + +## Key Quality Attributes + +The following report summarizes this team’s findings pertaining to quality attributes gathered from HTTPie’s codebase. The objective of this report is to provide insights into the maintainability of the codebase, with a focus on code structure and test robustness. All Quality Attributes were calculated programmatically, aside from test coverage percentage, which was already available through the official repository. This analysis focuses exclusively on Python source files, excluding other files such as, but not limited to, markup, setup scripts, and documentation. + +1. Codebase size and structure +* *Average LOC per file \= 134.5* +* *Total SLOC \= 14028* + The fairly moderate average lines of code metric would suggest that the codebase is well structured, where most files have a strong singular focus. Functionality seems to be well isolated, preventing overly large or complex files. + +2. Documentation +* *Total comments \= 794* +* *Comment density for source lines only \= 6%* + The relatively low comment density of HHTPie suggests one of two things; either the files are written in such a way that most of the code is self-explanatory, or there is a lack of adequate documentation that may cause confusion for open source developers. + +3. Complexity +* *Average cyclomatic complexity \= \~2.75* + This low average cyclomatic complexity, which Radon rates an A, indicates a straight-forward control structure. As a result, functions should be easy to test and maintain, while also benefitting from reduced risk of bugs and ease of refactoring. + +4. Testing Quality +* *Total number of tests \= 1028* +* *Test coverage \= 91%* + For a simple CLI tool, which is not meant for critical operations, this is a strong showing in terms of testing. The high coverage percentage and large number of individual tests indicates an extensive suite of testing that covers nearly all the functionality that HHTPie provides. + +5. Conclusion + Overall, HTTPie’s codebase showcases impressive quality attributes across the areas we are focusing on. We see manageable file sizes with low complexity, a control structure that keeps the codebase maintainable, scalable, and modifiable, and a testing suite that encompasses nearly all the possible functionalities being provided to users. Our only critique would be that the comment density is well below industry averages, which can range anywhere from 15%-25%, depending on the source. This leaves a solid opportunity for our team to come in and improve inline documentation in an effort to shore up the one apparent gap in HTTPie’s code quality, improving the development experience for those who come after. \ No newline at end of file diff --git a/courseProjectDocs/Setup/Branch_coverage_report.png b/courseProjectDocs/Setup/Branch_coverage_report.png new file mode 100644 index 0000000000..93628e199e Binary files /dev/null and b/courseProjectDocs/Setup/Branch_coverage_report.png differ diff --git a/courseProjectDocs/Setup/Statement_coverage_report.png b/courseProjectDocs/Setup/Statement_coverage_report.png new file mode 100644 index 0000000000..e6c4dce64f Binary files /dev/null and b/courseProjectDocs/Setup/Statement_coverage_report.png differ diff --git a/courseProjectDocs/Setup/Test_results.png b/courseProjectDocs/Setup/Test_results.png new file mode 100644 index 0000000000..28307aa7ca Binary files /dev/null and b/courseProjectDocs/Setup/Test_results.png differ diff --git a/courseProjectDocs/Setup/readme.md b/courseProjectDocs/Setup/readme.md new file mode 100644 index 0000000000..15720be83e --- /dev/null +++ b/courseProjectDocs/Setup/readme.md @@ -0,0 +1,19 @@ +# Setup to run the tests and coverage report + +1. Running the tests + +* make test + - Runs all the tests present in the repo + +![](test_rsults.png) + +2. Getting the test coverage + +* python3 courseProjectCode/Metrics/Testability.py + + The above command will: + + - Run all tests with coverage enabled + - Generate a coverage report + - Output the total number of test cases and test suites + diff --git a/courseProjectDocs/Setup/report.md b/courseProjectDocs/Setup/report.md new file mode 100644 index 0000000000..1932dc2366 --- /dev/null +++ b/courseProjectDocs/Setup/report.md @@ -0,0 +1,41 @@ +# Test suite summary + +* Unit tests: These are the vast majority of tests we see in Httpie, these cover base functionality such as authentication/authorization, parsing user input, raising proper exceptions, and connecting to external services. These tests often do not include mocking. +* Integration tests: These tests validate the behaviour between the CLI against HTTP endpoints to confirm if the features work together correctly. One such example would be testing the authentication which sends requests to the mock up server with credentials and check that the server responds with authenticated access. +* UI tests: While there is no "UI" specifically, the terminal acts as a UI as this is where all of the user interaction will take place. The UI tests themselves involve parsing input, displaying proper output, output formatting, and handling utf conversions. These tests often involve a mock environment and dummy URL's +* Regression: While there are no tests explicitly labeled as regression tests, we found references to regression testing through release documentation and git history. This indicates that certain tests have been implemented that implicitly prevent regression while serving a different explicit purpose + +# Metrics and observations + +* Number of tests run - 1028 +* Number of tests passed - 1020 +* Number of tests failed - 3 +* Number of tests skipped - 5 + +# Branch coverage summary + +# Steps to get the branch coverage + +* Run the below commands: + - coverage run --branch --source=httpie -m pytest + - coverage run -m + +* Branch coverage percentage: 89 + +![](Branch_coverage_report.png) + +# Steps to get the Statement coverage + +* Run the below commands: + - coverage run --source=httpie -m pytest + - coverage run -m + +* Statement coverage percentage: 91 + +![](Statement_coverage_report.png) + +# Observations + +1. Since the tests are dependent on external services, we observed inconsistency in the test results where we running the tests back to back gave different results and also different results on windows and mac. +2. The documentation is designed around a unix based system (bash), which raised errors when running the same on a windows based system (powershell). + diff --git a/courseProjectDocs/requirements-and-oracles.md b/courseProjectDocs/requirements-and-oracles.md new file mode 100644 index 0000000000..ff2c6d05fc --- /dev/null +++ b/courseProjectDocs/requirements-and-oracles.md @@ -0,0 +1,31 @@ +# Requirements and Test Oracles + +## Functional Requirements +1. The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE). +2. The system shall allow the user to upload/download files. +3. The system shall allow the user to send request in different data formats (JSON, URL Parameters, HTTP Headers). +4. The system shall allow the user to configure their terminal and profile with their preferences. +5. The system shall allow the user to see all responses from the request in printed output to the console. +6. The system shall allow the user to authenticate via username and password before accessing the resource. +7. The system shall allow the user to set and send custom request names. + + +## Non-Functional Requirements +1. The system shall work consistently across Linux, MacOS, windows and FreeBSD systems. +2. The system shall be accessible on web and mobile devices and desktop CLI. +3. The system shall allow a maximum of n redirects, where n is defined by the user. +4. The system shall timeout stale requests after n milliseconds, where n is defined by the user. +5. The system shall accept no more than one request at a time. + +## Test Oracles + +| Requirement ID | Requirement Description | Test Oracle (Expected Behavior) | +|-----------------------|-----------------------------------|---------------------------------------------| +| FR-1 | The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).|When the user sends an HTTP DELETE request, the resource under control should no longer contain those elements.| +| FR-1 | The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).|When a user sends an HTTP GET request containing Foo:bar, the response must include “Foo”: “bar”.| +| FR-1 | The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).|When a user sends an HTTP GET request containing Accept: or User-Agent:, those headers shall be absent from the request.| +| FR-1 | The system shall allow the user to send requests with common HTTP methods (POST, PUT, GET, DELETE).|When the user sends an HTTP POST request with a specified file, then it will be added to the resource under control.| +| NFR-3 | The system shall allow a maximum of n redirects, where n is defined by the user.|When the redirect count exceeds the limit, the CLI must return ExitStatus.ERROR_TOO_MANY_REDIRECTS.| +| NFR-4 |The system shall timeout stale requests after n milliseconds, where n is defined by the user.|When the time to process a request exceeds the timeout limit, the CLI must return ExitStatus.ERROR_TIMEOUT.| +| FR-5 |The system shall allow the user to see all responses from the request in printed output to the console.|When a response is received from any request, the response will be printed to the terminal.| +| FR-7 |The system shall allow the user to set and send custom request names.|When the user creates a custom HTTP request name, then that name is added to the list of available requests that can successfully be sent.|