TurboGears/CherryPy Removal; Flask Rest API as Replacement #255
Description
--- Is your feature request related to a problem? Please describe. ---
This feature is to encompass the updated scope of work this previously approved feature request requested.
Python 2 is dead. Let's Modernize the Beaker code by removing the TurboGears/CherryPy dependencies and replacing them with the popular and widely used Flask/SQLAlchemy/Gunicorn dependencies so more people can contribute to and understand the project. This will also be an opportunity to ensure all dependent packages work with Python3.10+ so security patches are enabled through 2026 and the application works with modern OS's like RHEL9.
--- Describe the solution you'd like ---
A simplified Beaker application that uses popular standard dependencies like Flask, SQLAlchemy, and Gunicorn instead of niche dependencies like TurboGears or CherryPy.
API calls that follow RestFUL API conventions to simplify the work Beaker Server primarily deals with (resources management).
Code that is functional on Python 3.10+ so it can be secure and run on RHEL9.
--- Changes that need to occur ---
Remove TurboGears references from the Server & Update Associated Integration Tests;
- TurboGears Functionality to be replaced by Flask/SQLAlchemy
** Database -> SQLAlchemy built-in
** Config -> Flask Config Env Vars (Follow 12 factor app best practices)
** Widgets -> Flask WTF Forms
** Flashes -> Flask Templating - Update Build (setup.py & .spec files)
** Include updated Flask/SQLAlchemy packages
** Remove TurboGears and dependencies
Replace all CherryPy endpoints with Flask Endpoints
- Transform XMLRPC Endpoints into Standard RestFUL API Schema
** Update other services that interacted with via these endpoints
** Update Integration Tests - CherryPy Endpoint Functionality to be Replaced
** Request -> Flask Request
** Response -> Flask Response
** Expose Endpoint to URL -> Flask URL attached to endpoint
** Errors -> Flask Error Handling
** Serve Files -> Flask File Serving
** Grouping API Endpoints -> Flask Blueprints - Replace CherryPy with Gunicorn to host application
--- Describe alternatives you've considered ---
Upgrading to TurboGears2; Upgrading to TurboGears2 requires large changes still and doesn't align the project with the community. Flask is much more popular in the OpenSource Community and has much more documentation. Replacing TurboGears with Flask will encourage contributions and make future changes easier to make down the line.
Keeping XMLRPC commands; Most of the work that Beaker Server does is centered around Resource operations, this is the ideal use case for RestFUL API calls. Using a RestFUL API standard will make the Beaker application easier to understand to the community at large as well. Since we are replacing the CherryPy API with Flask we might as well replace it with an API architecture that makes sense for resource management (Rest).
Keeping CherryPy to run the WSGI server; Beaker already has a Proxy server built in that can be used to handle SSL and HTTPS, this is the main appeal of CherryPy. Using Gunicorn will make all the web servers WSGI tooling standard and allow for greater scalability down the line. GUnicorn is also easy to configure with a config file and is easier to deploy via Docker. The one downside of using GUnicorn compared to CherryPy is that CherryPy works on windows while GUnicorn does not but Beaker is being run on Linux Systems primarily anwyas.
--- Other Considerations ---
While Flask does support XML templates, they are not as common as HTML templates; Upgrading the Beaker project to use HTML Templates that are formatted via Jinja2 instead of using XML Templates formatted via Kid would be another step towards using tools that are well known, documented, and popular in the OpenSource community. This piece of work can be done at a future date because Flask can format XML files so it wont be a blocker.
--- Upgrade Considerations ---
To make the transition from Python 2 systems to Python 3 systems smooth we need to keep support for Python2.7 were possible. This will allow teams to test the code using a Python 2 interpreter and switch on Python 3 down the line with an environment variable feature flag on an appropriate OS.
--- Work Breakdown ---
This work is no small chunk and will be worked on by multiple people. A logical way to divide it is:
- TurboGears Widgets Replaced with Flask WTF Forms (Server and Integration Tests); This should be done first because it will become a blocker for some CherryPy changes that are required below.
- TurboGears Flashes Replaced with Flask templates and template vars (Server and Integration Tests); This should be done second because it could become a blocker for some CherryPy changes that are required below.
- TurboGears Database References Replaced with SQLAlchemy (Server and Integration Tests)
- TurboGears Config References Replaced with Flask Env Vars (Server and Integration Tests)
- Any other TurboGears imports that need to be replaced and tested
- CherryPy XMLRPC Classes Broken down into Flask Blueprints one at a time (Server, Other Comoponents, and Integration Tests)
- For Each XMLRPC Class Broken down into Flask Blueprint translate the API from XMLRPC to RestFUL API's
- Packaging Updates to remove unneeded dependencies like CherryPy and TurboGears
--- Questions ---
- Does the application have to be backward compatible with old versions?
I would hope that it does not as these changes will be significant and Python2.7 is no longer supported by the community. - How do we plan on managing slight changes to the UI? These will occur because of changes made to TurboGears Widgets.
I would assume that slight UI changes are alright as long as they aren't major and don't change the User Journey.