user can now define a callback fn, use signals, custom save fn. See examples/ for code

Author: eshaan7

README.md

@@ -19,8 +19,8 @@ Inspired by the work of awesome folks over at [msoap/shell2http](https://github.
 - Set a script that runs on a succesful POST request to an endpoint of your choice. See [Example code](examples/run_script.py).
 - Map a base command to an endpoint and pass dynamic arguments to it. See [Example code](examples/basic.py).
 - Can also process multiple uploaded files in one command. See [Example code](examples/multiple_files.py).
-- This is useful for internal docker-to-docker communications if you have lots of different binaries. See [real-life example](https://github.com/intelowlproject/IntelOwl/blob/develop/integrations/peframe/app.py).
-- Currently, all commands are run asynchronously, so result is not available directly. An option would be provided for this in future release.
+- This is useful for internal docker-to-docker communications if you have different binaries distributed in micro-containers. See [real-life example](https://github.com/intelowlproject/IntelOwl/blob/develop/integrations/peframe/app.py).
+- Currently, all commands are run asynchronously (default timeout is 3600 seconds), so result is not available directly. An option _may_ be provided for this in future release.
 
 > Note: This extension is primarily meant for executing long-running
 > shell commands/scripts (like nmap, code-analysis' tools) in background from an HTTP request and getting the result at a later time.
@@ -29,7 +29,7 @@ Inspired by the work of awesome folks over at [msoap/shell2http](https://github.
 
 [![Documentation Status](https://readthedocs.org/projects/flask-shell2http/badge/?version=latest)](https://flask-shell2http.readthedocs.io/en/latest/?badge=latest)
 
-Read the [Quickstart](https://flask-shell2http.readthedocs.io/en/latest/Quickstart.html) 
+Read the [Quickstart](https://flask-shell2http.readthedocs.io/en/stable/Quickstart.html) 
 from the [documentation](https://flask-shell2http.readthedocs.io/) to get started!
 
 ## Why?

docs/source/Examples.md

@@ -1,7 +1,10 @@
 ## Examples
 
-We have created some example python scripts to demonstrate various use-cases. These include extension setup as well as making HTTP calls with python's [requests](https://requests.readthedocs.io/en/master/) module.
+I have created some example python scripts to demonstrate various use-cases. These include extension setup as well as making test HTTP calls with python's [requests](https://requests.readthedocs.io/en/master/) module.
 
 - [run_script.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/run_script.py): Execute a script on a succesful POST request to an endpoint.
-- [basic.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/basic.py): Map a base command to an endpoint and pass dynamic arguments to it.
-- [multiple_files.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/multiple_files.py): Upload multiple files for a single command.
+- [basic.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/basic.py): Map a base command to an endpoint and pass dynamic arguments to it. Can also pass in a timeout.
+- [multiple_files.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/multiple_files.py): Upload multiple files for a single command.
+- [with_callback.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/with_callback.py): Define a callback function that executes on command/process completion.
+- [with_signals.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/with_signals.py): Using [Flask Signals](https://flask.palletsprojects.com/en/1.1.x/signals/) as callback function.
+- [custom_save_fn.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/custom_save_fn.py): There may be cases where the process doesn't print result to standard output but to a file/database. This example shows how to intercept the future object after completion and update it's result attribute.

docs/source/Quickstart.md

@@ -27,7 +27,7 @@ app = Flask(__name__)
 executor = Executor(app)
 shell2http = Shell2HTTP(app=app, executor=executor, base_url_prefix="/commands/")
 
-shell2http.register_command(endpoint="saythis", command_name="echo")
+shell2http.register_command(endpoint="saythis", command_name="echo", callback_fn=None)
 ```
 
 Run the application server with, `$ flask run -p 4000`.
@@ -45,7 +45,8 @@ $ curl -X POST -H 'Content-Type: application/json' -d '{"args": ["Hello", "World
 <details><summary>or using python's requests module,</summary>
 
 ```python
-data = {"args": ["Hello", "World!"]}
+# You can also add a timeout if you want, default value is 3600 seconds
+data = {"args": ["Hello", "World!"], "timeout": 60}
 resp = requests.post("http://localhost:4000/commands/saythis", json=data)
 print("Result:", resp.json())
 ```
@@ -57,29 +58,33 @@ returns JSON,
 ```json
 {
    "key": "ddbe0a94847c",
-   "result_url": "http://localhost:4000/commands/saythis?key=ddbe0a94847c",
+   "result_url": "http://localhost:4000/commands/saythis?key=ddbe0a94",
    "status": "running"
 }
 ```
 
 Then using this `key` you can query for the result or just by going to the `result_url`,
 
 ```bash
-$ curl http://localhost:4000/commands/saythis?key=ddbe0a94847c
+$ curl http://localhost:4000/commands/saythis?key=ddbe0a94
 ```
 
 Returns result in JSON,
 
 ```json
 {
-  "end_time": 1593019807.782958, 
-  "error": "", 
-  "md5": "ddbe0a94847c", 
-  "process_time": 0.00748753547668457, 
-  "report": {
-    "result": "Hello World!\n"
-  }, 
-  "start_time": 1593019807.7754705, 
-  "status": "success"
+  "report": "Hello World!\n",
+  "key": "ddbe0a94",
+  "start_time": 1593019807.7754705,
+  "end_time": 1593019807.782958,
+  "process_time": 0.00748753547668457,
+  "status": "success",
+  "returncode": 0,
+  "error": "",
 }
 ```
+
+##### Bonus
+
+You can also define callback functions or use signals for reactive programming. There may be cases where the process doesn't print result to standard output but to a file/database. In such cases, you may want to intercept the future object and update it's result attribute.
+I request you to take a look at [Examples.md](Examples.md) for such use-cases.

docs/source/conf.py

@@ -26,7 +26,7 @@
 author = "Eshaan Bansal"
 
 # The full version, including alpha/beta/rc tags
-release = "1.2.0"
+release = "1.3.0"
 
 
 # -- General configuration ---------------------------------------------------

examples/basic.py

@@ -1,5 +1,4 @@
 # system imports
-import requests
 import logging
 import sys
 
@@ -24,31 +23,28 @@
 executor = Executor(app)
 shell2http = Shell2HTTP(app, executor, base_url_prefix="/cmd/")
 
-ENDPOINT = "echo"
+ENDPOINT_AND_CMD = "echo"
 
-shell2http.register_command(endpoint=ENDPOINT, command_name="echo")
+shell2http.register_command(endpoint=ENDPOINT_AND_CMD, command_name=ENDPOINT_AND_CMD)
 
 
-@app.route("/")
-def test():
+# Test Runner
+if __name__ == "__main__":
+    app.testing = True
+    c = app.test_client()
     """
     The final executed command becomes:
     ```bash
     $ echo hello world
     ```
     """
-    url = f"http://localhost:4000/cmd/{ENDPOINT}"
-    data = {"args": ["hello", "world"]}
-    resp = requests.post(url, json=data)
-    resp_data = resp.json()
-    print(resp_data)
-    key = resp_data["key"]
-    if key:
-        report = requests.get(f"{url}?key={key}")
-        return report.json()
-    return resp_data
-
-
-# Application Runner
-if __name__ == "__main__":
-    app.run(port=4000)
+    # make new request for a command with arguments
+    uri = f"/cmd/{ENDPOINT_AND_CMD}"
+    # timeout in seconds, default value is 3600
+    data = {"args": ["hello", "world"], "timeout": 60}
+    resp1 = c.post(uri, json=data).get_json()
+    print(resp1)
+    # fetch result
+    result_url = resp1["result_url"]
+    resp2 = c.get(result_url).get_json()
+    print(resp2)

examples/custom_save_fn.py

@@ -0,0 +1,50 @@
+# web imports
+from flask import Flask
+from flask_executor import Executor
+from flask_executor.futures import Future
+from flask_shell2http import Shell2HTTP
+
+# Flask application instance
+app = Flask(__name__)
+
+# application factory
+executor = Executor(app)
+shell2http = Shell2HTTP(app, executor)
+
+ENDPOINT = "echo"
+
+
+def intercept_result(future: Future):
+    """
+    Will be invoked on every process completion
+    """
+    data = None
+    if future.done():
+        with open("/path/to/saved/file") as f:
+            data = f.read()
+        # 1. get current result object
+        res = future.result()
+        # 2. update the report variable,
+        # you may update only these: report,error,status
+        res.report = data
+        res.status = "success"
+        # 3. set new result
+        future._result = res
+
+
+shell2http.register_command(
+    endpoint=ENDPOINT, command_name=ENDPOINT, callback_fn=intercept_result
+)
+
+
+# Test Runner
+if __name__ == "__main__":
+    app.testing = True
+    c = app.test_client()
+    # request new process
+    data = {"args": ["hello", "world"]}
+    r = c.post(f"/{ENDPOINT}", json=data)
+    # get result
+    result_url = r.get_json()["result_url"]
+    resp = c.get(result_url)
+    print(resp.get_json())

examples/with_callback.py

@@ -0,0 +1,40 @@
+# web imports
+from flask import Flask
+from flask_executor import Executor
+from flask_executor.futures import Future
+from flask_shell2http import Shell2HTTP
+
+# Flask application instance
+app = Flask(__name__)
+
+# application factory
+executor = Executor(app)
+shell2http = Shell2HTTP(app, executor, base_url_prefix="/cmd/")
+
+ENDPOINT = "echo"
+
+
+def my_callback_fn(future: Future):
+    """
+    Will be invoked on every process completion
+    """
+    print("[i] Process running ?:", future.running())
+    print("[i] Process completed ?:", future.done())
+    print("[+] Result: ", future.result())
+
+
+shell2http.register_command(
+    endpoint=ENDPOINT, command_name=ENDPOINT, callback_fn=my_callback_fn
+)
+
+
+# Test Runner
+if __name__ == "__main__":
+    app.testing = True
+    c = app.test_client()
+    # request new process
+    data = {"args": ["hello", "world"]}
+    c.post(f"cmd/{ENDPOINT}", json=data)
+    # request another new process
+    data = {"args": ["Hello", "Friend!"]}
+    c.post(f"cmd/{ENDPOINT}", json=data)

examples/with_signals.py

@@ -0,0 +1,47 @@
+# web imports
+from flask import Flask
+from blinker import Namespace  # or from flask.signals import Namespace
+from flask_executor import Executor
+from flask_executor.futures import Future
+from flask_shell2http import Shell2HTTP
+
+# Flask application instance
+app = Flask(__name__)
+
+# application factory
+executor = Executor(app)
+shell2http = Shell2HTTP(app, executor, base_url_prefix="/cmd/")
+
+ENDPOINT = "echo"
+CMD = "echo"
+
+# Signal Handling
+signal_handler = Namespace()
+my_signal = signal_handler.signal(f"on_{CMD}_complete")
+# ..or any other name of your choice,
+
+
+@my_signal.connect
+def my_callback_fn(future: Future):
+    """
+    Will be invoked on every process completion
+    """
+    print("Process completed ?:", future.done())
+    print("Result: ", future.result())
+
+
+shell2http.register_command(
+    endpoint=ENDPOINT, command_name=CMD, callback_fn=my_signal.send
+)
+
+
+# Test Runner
+if __name__ == "__main__":
+    app.testing = True
+    c = app.test_client()
+    # request new process
+    data = {"args": ["Hello", "Friend!"]}
+    c.post(f"cmd/{ENDPOINT}", json=data)
+    # request new process
+    data = {"args": ["Bye", "Friend!"]}
+    c.post(f"cmd/{ENDPOINT}", json=data)