Merge pull request #156 from python-ellar/lazyloading_application_module

Feat - Lazy Loading ApplicationModule

Author: eadwinCode

docs/basics/dynamic-modules.md

@@ -1,6 +1,6 @@
 # **Dynamic Modules**
-We have seen in many example given on how to statically configure a [module](../overview/modules.md){_target='blank'}. 
-In this section we are going to look at different ways to dynamically set up a module.
+We have seen in many examples given on how to statically configure a [module](../overview/modules.md){_target='blank'}. 
+In this section, we are going to look at different ways to dynamically set up a module.
 
 Why is this important? Consider a scenario where a general-purpose module needs to behave differently in different use cases, 
 it may be useful to use a configuration-based approach to allow customization. This is similar to the concept of a "plugin" in many systems, 
@@ -36,7 +36,7 @@ while the `ModuleSetup` instance is used when the module does not require any ad
 `DynamicModule` is a dataclass type that is used to **override** `Module` decorated attributes without having to modify the module code directly.
 In other words, it gives you the flexibility to reconfigure module.
 
-For example: Lets look at the code below:
+For example, Let's look at the code below:
 ```python
 from ellar.common import Module
 from ellar.core import DynamicModule
@@ -79,11 +79,12 @@ It allows you to define the module **dependencies** and allow a **callback facto
 Let's look this `ModuleSetup` example code below with our focus on how we eventually configured `DynamicService` type, 
 how we used `my_module_configuration_factory` to dynamically build `MyModule` module.
 
-```python
+```python linenums="1"
 import typing as t
 from ellar.common import Module, IModuleSetup
 from ellar.di import ProviderConfig
-from ellar.core import DynamicModule, ModuleBase, Config, ModuleSetup, AppFactory
+from ellar.core import DynamicModule, ModuleBase, Config, ModuleSetup
+from ellar.app import AppFactory
 
 
 class Foo:
@@ -123,6 +124,7 @@ app = AppFactory.create_from_app_module(ApplicationModule, config_module=dict(
 ))
 
 dynamic_service = app.injector.get(DynamicService)
+
 assert dynamic_service.param1 == "param1"
 assert dynamic_service.param2 == "param2"
 assert dynamic_service.foo == "foo"
@@ -142,3 +144,96 @@ all the required **parameters** and returned a `DynamicModule` of `MyModule`.
 
 For more example, checkout [Ellar Throttle Module](https://github.com/eadwinCode/ellar-throttler/blob/master/ellar_throttler/module.py){target="_blank"}
 or [Ellar Cache Module](../techniques/caching.md){target="_blank"}
+
+
+## **Lazy Loading Modules**
+Ellar supports loading module decorated classes through a string reference using `LazyModuleImport`.
+For a better application context availability usage in module like, `current_config`,
+`current_app` and `current_injector`, it's advised to go with lazy module import.
+
+For example,
+we can lazy load `CarModule` from our example [here](../overview/modules.md#feature-modules){target="_blank"} into
+`ApplicationModule`
+
+```python title="project_name/root_module.py" linenums="1"
+
+from ellar.common import IExecutionContext, Module, exception_handler
+from ellar.common.responses import JSONResponse, Response
+from ellar.core import ModuleBase, LazyModuleImport as lazyLoad
+from ellar.samples.modules import HomeModule
+
+
+@Module(modules=[HomeModule, lazyLoad('apps.car.module:CarModule')])
+class ApplicationModule(ModuleBase):
+    @exception_handler(404)
+    def exception_404_handler(cls, ctx: IExecutionContext, exc: Exception) -> Response:
+        return JSONResponse({"detail": "Resource not found."}, status_code=404)
+```
+
+In the above illustration, we provided a string reference to `CarModule` into `LazyModuleImport` instance.
+And during `AppFactory` Module bootstrap, `CarModule` will be resolved, validated and registered into the application
+
+### **Properties**
+`LazyModuleImport` attributes,
+
+- `module`: String reference for Module import
+- `setup`: Module setup function name for modules that requires specific function as in case of `DynamicModule` and `ModuleSetup`.
+- `setup_options`: Module setup function parameters
+
+### **Lazy Loading DynamicModules**
+Having the understanding of `DynamicModule` and its registration pattern,
+to lazy load DynamicModule follows the same pattern.
+
+For example, lets lazy load `MyModule` as a `DynamicModule`.
+For that to happen, we need to call `MyModule.setup` with some parameters and in turn returns a `DynamicModule`
+
+```python title="project_name/root_module.py" linenums="1"
+from ellar.common import Module, exception_handler, JSONResponse, IExecutionContext, Response
+from ellar.core import ModuleBase
+from .custom_module import MyModule, Foo
+
+
+@Module(modules=[MyModule.setup(12, 23, Foo())])
+class ApplicationModule(ModuleBase):
+    @exception_handler(404)
+    def exception_404_handler(cls, ctx: IExecutionContext, exc: Exception) -> Response:
+        return JSONResponse({"detail": "Resource not found."}, status_code=404)
+```
+
+Let's rewrite this using `LazyModuleImport`.
+
+```python title="project_name/root_module.py" linenums="1"
+from ellar.common import Module, exception_handler, JSONResponse, IExecutionContext, Response
+from ellar.core import ModuleBase, LazyModuleImport as lazyLoad
+
+
+@Module(modules=[
+    lazyLoad('project_name.custom_module:MyModule', 'setup', param1=12, param2=23, foo=Foo()), 
+])
+class ApplicationModule(ModuleBase):
+    @exception_handler(404)
+    def exception_404_handler(cls, ctx: IExecutionContext, exc: Exception) -> Response:
+        return JSONResponse({"detail": "Resource not found."}, status_code=404)
+
+```
+
+### **Lazy Loading ModuleSetup**
+Just as in `DynamicModule`, `ModuleSetup` can be lazy loaded the same way. 
+Let's take [CacheModule](https://github.com/python-ellar/ellar/blob/main/ellar/cache/module.py) for example.
+
+```python title="project_name/root_module.py" linenums="1"
+from ellar.common import Module, exception_handler, JSONResponse, IExecutionContext, Response
+from ellar.core import ModuleBase, LazyModuleImport as lazyLoad
+
+
+@Module(modules=[
+    lazyLoad('ellar.cache.module:CacheModule', 'register_setup'), 
+])
+class ApplicationModule(ModuleBase):
+    @exception_handler(404)
+    def exception_404_handler(cls, ctx: IExecutionContext, exc: Exception) -> Response:
+        return JSONResponse({"detail": "Resource not found."}, status_code=404)
+
+```
+In the above illustration, we have registered `CacheModule` through `register_setup` function 
+which returns a `ModuleSetup` that configures the `CacheModule` to read its configurations from application config.

docs/overview/modules.md

@@ -16,8 +16,8 @@ Thus, the architecture resulting from most applications will include multiple mo
 Building an application as a group of feature modules bundled together helps to manage complexity, have a maintainable, extendable, and testable code base, and encourage development using SOLID principles.
 
 A typical example of a feature module is the **car** project. The `CarModule` wraps all the services and controller that manages the `car` resource which makes it easy to maintain, extend, and testable.
-```python
-# project_name/apps/car/module.py
+```python title='project_name/apps/car/module.py' linenums="1"
+
 
 from ellar.common import Module
 from ellar.core import ModuleBase
@@ -78,26 +78,24 @@ class BookModule(ModuleBase):
 ### **Module Events**
 Every registered Module receives two event calls during its instantiation and when the application is ready.
 
-```python
+```python linenums="1"
 from ellar.common import Module
-from ellar.core import ModuleBase, Config, App
+from ellar.core import ModuleBase, Config
 
 @Module()
 class ModuleEventSample(ModuleBase):
     @classmethod
     def before_init(cls, config: Config) -> None:
         """Called before creating Module object"""
-    
-    def application_ready(self, app: App) -> None:
-        """Called when application is ready - this is similar to @on_startup event"""
 
 ```
-`before_init` receives current app `Config` as a parameter and `application_ready` function receives `App` instance as parameter.
+`before_init` receives current app `Config` as a parameter for further configurations before `ModuleEventSample` is initiated.
+It's important to note that returned values from `before_init` will be passed to the constructor of `ModuleEventSample` during instantiation.
 
 ### **Module Exceptions**
 Custom exception handlers can be registered through modules.
 
-```python
+```python linenums="1"
 from ellar.common import Module, exception_handler, JSONResponse, Response, IHostContext
 from ellar.core import ModuleBase
 
@@ -114,7 +112,7 @@ We can also define `Jinja2` templating filters in project Modules or any `@Modul
 The defined filters are be passed down to `Jinja2` **environment** instance alongside the `template_folder` 
 value when creating **TemplateLoader**.
 
-```python
+```python linenums="1"
 
 from ellar.common import Module, template_global, template_filter
 from ellar.core import ModuleBase
@@ -137,9 +135,9 @@ class ModuleTemplateFilterSample(ModuleBase):
 ## **Dependency Injection**
 A module class can inject providers as well (e.g., for configuration purposes):
 
-For example, from our sample project, the can inject `Config` to the `CarModule`
+For example, from our sample project, we can inject `Config` to the `CarModule`
 
-```python
+```python linenums="1"
 # project_name/apps/car/module.py
 
 from ellar.common import Module
@@ -169,7 +167,7 @@ Middlewares functions can be defined at Module level with `@middleware()` functi
 
 For example:
 
-```python
+```python linenums="1"
 from ellar.common import Module, middleware, IHostContext, PlainTextResponse
 from ellar.core import ModuleBase
 
@@ -211,7 +209,7 @@ As an added support, you can create or reuse modules from `injector` Modules.
 !!! info
     This type of module is used to configure `injector` **bindings** and **providers** for dependency injection purposes.
 
-```python
+```python linenums="1"
 from ellar.core import ModuleBase
 from ellar.di import Container
 from injector import provider

docs/overview/providers.md

@@ -1,7 +1,7 @@
 # **Providers**
 A provider is any class or object that is **injectable** as a dependency to another class when creating an instance of that class.
 
-Providers are like services, repositories services, factories, etc., classes that manage complex tasks. These providers can be made available to a controller, a route handler, or to another provider as a dependency. 
+Providers are like services, repository services, factories, etc., classes that manage complex tasks. These providers can be made available to a controller, a route handler, or to another provider as a dependency. 
 This concept is commonly known as [Dependency Injection](https://en.wikipedia.org/wiki/Dependency_injection){target="_blank"}
 
 In Ellar, you can easily create a `provider/injectable` class by decorating that class with the `@injectable()` mark and stating the scope.

docs/security/authentication.md

@@ -166,17 +166,16 @@ Let us configure the JWTModule inside AuthModule. 
 from datetime import timedelta
 
 from ellar.common import Module
-from ellar.core import ModuleBase
+from ellar.core import ModuleBase, LazyModuleImport as lazyLoad
 from ellar_jwt import JWTModule
 
 from .controllers import AuthController
-from ..user.module import UserModule
 from .services import AuthService
 
 
 @Module(
     modules=[
-        UserModule,
+        lazyLoad('project_name.users.module:UserModule'),
         JWTModule.setup(
             signing_secret_key="my_poor_secret_key_lol", lifetime=timedelta(minutes=5)
         ),
@@ -231,14 +230,12 @@ To do that, we need to register `AuthModule` to the `ApplicationModule`.
 ```python title="project_name.root_module.py" linenums="1"
 from ellar.common import Module, exception_handler
 from ellar.common import IExecutionContext, JSONResponse, Response
-from ellar.core import ModuleBase
+from ellar.core import ModuleBase, LazyModuleImport as lazyLoad
 from ellar.samples.modules import HomeModule
-from .apps.car.module import CarModule
-from .apps.auth.module import AuthModule
 
 
 @Module(
-    modules=[HomeModule, CarModule, AuthModule],
+    modules=[HomeModule, lazyLoad('project_name.auth.module:AuthModule'),],
 )
 class ApplicationModule(ModuleBase):
     @exception_handler(404)
@@ -451,18 +448,17 @@ First, let us register `AuthGuard` a global guard in `AuthModule`.
 from datetime import timedelta
 
 from ellar.common import GlobalGuard, Module
-from ellar.core import ModuleBase
+from ellar.core import ModuleBase, LazyModuleImport as lazyLoad
 from ellar.di import ProviderConfig
 from ellar_jwt import JWTModule
 
 from .controllers import AuthController
-from ..user.module import UserModule
 from .services import AuthService
 from .guards import AuthGuard
 
 @Module(
     modules=[
-        UserModule,
+        lazyLoad('project_name.users.module:UserModule'),
         JWTModule.setup(
             signing_secret_key="my_poor_secret_key_lol", lifetime=timedelta(minutes=5)
         ),
@@ -658,12 +654,12 @@ Let us make `JWTAuthentication` Handler available for ellar to use as shown belo
 ```python title='project_name.server.py' linenums='1'
 import os
 from ellar.common.constants import ELLAR_CONFIG_MODULE
-from ellar.core.factory import AppFactory
-from .root_module import ApplicationModule
+from ellar.app import AppFactory
+from ellar.core import LazyModuleImport as lazyLoad
 from .auth_scheme import JWTAuthentication
 
 application = AppFactory.create_from_app_module(
-    ApplicationModule,
+    lazyLoad('project_name.root_module:ApplicationModule'),
     config_module=os.environ.get(
         ELLAR_CONFIG_MODULE, "project_name.config:DevelopmentConfig"
     ),
@@ -738,18 +734,17 @@ just like we did in [applying guard globally](#apply-authguard-globally)
 from datetime import timedelta
 from ellar.auth.guard import AuthenticatedRequiredGuard
 from ellar.common import GlobalGuard, Module
-from ellar.core import ModuleBase
+from ellar.core import ModuleBase, LazyModuleImport as lazyLoad
 from ellar.di import ProviderConfig
 from ellar_jwt import JWTModule
 
-from ..users.module import UsersModule
 from .controllers import AuthController
 from .services import AuthService
 
 
 @Module(
     modules=[
-        UsersModule,
+        lazyLoad('project_name.users.module:UserModule'),
         JWTModule.setup(
             signing_secret_key="my_poor_secret_key_lol", lifetime=timedelta(minutes=5)
         ),

ellar/app/factory.py

@@ -76,6 +76,9 @@ def _build_modules(
         :param injector: App Injector instance
         :return: `None`
         """
+        if isinstance(app_module, LazyModuleImport):
+            app_module = app_module.get_module("AppFactory")
+
         assert reflect.get_metadata(
             MODULE_WATERMARK, app_module
         ), "Only Module is allowed"
@@ -129,8 +132,6 @@ def _get_config_kwargs() -> t.Dict:
                 return {"config_module": config_module}
             return dict(config_module)
 
-        assert reflect.get_metadata(MODULE_WATERMARK, module), "Only Module is allowed"
-
         config = Config(app_configured=True, **_get_config_kwargs())
 
         injector = EllarInjector(auto_bind=config.INJECTOR_AUTO_BIND)

examples/01-carapp/carapp/server.py

@@ -2,17 +2,16 @@
 
 from ellar.app import AppFactory
 from ellar.common.constants import ELLAR_CONFIG_MODULE
+from ellar.core import LazyModuleImport as lazyLoad
 from ellar.openapi import (
     OpenAPIDocumentBuilder,
     OpenAPIDocumentModule,
     ReDocsUI,
     SwaggerUI,
 )
 
-from .root_module import ApplicationModule
-
 application = AppFactory.create_from_app_module(
-    ApplicationModule,
+    lazyLoad("carapp.root_module:ApplicationModule"),
     config_module=os.environ.get(
         ELLAR_CONFIG_MODULE, "carapp.config:DevelopmentConfig"
     ),

examples/02-socketio-app/socketio_app/apps/events/gateways.py

@@ -22,7 +22,7 @@ async def my_broadcast_event(self, message: MessageData = WsBody()):
 
     @subscribe_message("join")
     async def join(self, message: MessageRoom = WsBody()):
-        await self.context.server.enter_room(self.context.sid, message.room)
+        self.context.server.enter_room(self.context.sid, message.room)
         await self.context.server.emit(
             "my_response",
             {"data": "Entered room: " + message.room},
@@ -31,7 +31,7 @@ async def join(self, message: MessageRoom = WsBody()):
 
     @subscribe_message("leave")
     async def leave(self, message: MessageRoom = WsBody()):
-        await self.context.server.leave_room(self.context.sid, message.room)
+        self.context.server.leave_room(self.context.sid, message.room)
         await self.context.server.emit(
             "my_response", {"data": "Left room: " + message.room}, room=self.context.sid
         )

examples/02-socketio-app/socketio_app/apps/events/routers.py

@@ -0,0 +1,10 @@
+"""
+Define endpoints routes in python function fashion
+example:
+
+my_router = ModuleRouter("/cats", tag="Cats", description="Cats Resource description")
+
+@my_router.get('/')
+def index(request: Request):
+    return {'detail': 'Welcome to Cats Resource'}
+"""