Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ dk-*-credentials.txt

# Docker
docker-compose.yml
docker-compose.yaml
obs-docker-compose.yml

# Logs
testgen_demo.log
Expand Down
32 changes: 7 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,17 +49,11 @@ And it allows you to <b>make fast, safe development changes</b>.

#### Requirements for TestGen & Observability

| Software | Tested Versions | Command to check version |
|-------------------------|-------------------------|-------------------------------|
| [Python](https://www.python.org/downloads/) <br/>- Most Linux and macOS systems have Python pre-installed. <br/>- On Windows machines, you will need to download and install it. | 3.9, 3.10, 3.11, 3.12 | `python3 --version` |
| [Docker](https://docs.docker.com/get-docker/) <br/>[Docker Compose](https://docs.docker.com/compose/install/) | 26.1, 27.5, 28.1 <br/> 2.34, 2.35, 2.36 | `docker -v` <br/> `docker compose version` |
| Software | Tested Versions | Command to check version |
|-------------------------|-----------------------------------------|-------------------------------|
| [Python](https://www.python.org/downloads/) <br/>- Most Linux and macOS systems have Python pre-installed. <br/>- On Windows machines, you will need to download and install it. | 3.9, 3.10, 3.11, 3.12, 3.13 | `python3 --version` |
| [Docker](https://docs.docker.com/get-docker/) <br/>[Docker Compose](https://docs.docker.com/compose/install/) | 26.1, 27.5, 28.5 <br/> 2.38, 2.39, 2.40 | `docker -v` <br/> `docker compose version` |

#### Additional Requirements for Observability only

| Software | Tested Versions | Command to check version |
|-------------------------|-------------------------|-------------------------------|
| [Minikube](https://minikube.sigs.k8s.io/docs/start/) | 1.33, 1.34, 1.35 | `minikube version` |
| [Helm](https://helm.sh/docs/intro/install/) | 3.15, 3.16, 3.17 | `helm version` |

### Download the installer

Expand Down Expand Up @@ -96,17 +90,11 @@ Once the installation completes, verify that you can login to the UI with the UR

### Install the Observability application

The installation downloads the latest Helm charts and Docker images for Observability and deploys the application on a new minikube cluster. The process may take 5~30 minutes depending on your machine and network connection.
The installation downloads the latest Docker images for Observability and deploys the application using Docker. The process may take 5~15 minutes depending on your machine and network connection.
```shell
python3 dk-installer.py obs install
```
#### Bind HTTP ports to host machine

This step is required to access the application when using Docker driver on Mac or Windows. It may also be useful for installations on remote machines to access the UI from a local browser.

```shell
python3 dk-installer.py obs expose
```
The `--port` option may be used to set a custom localhost port for the application (default: 8082).

Verify that you can login to the UI with the URL and credentials provided in the output. Leave this process running, and continue the next steps on another terminal window.
Expand Down Expand Up @@ -162,15 +150,9 @@ Upgrade the app to latest version: `python3 dk-installer.py tg upgrade`

### DataOps Observability

The [minikube](https://minikube.sigs.k8s.io/docs/commands/) and [kubectl](https://kubernetes.io/docs/reference/kubectl/) command line tools can be used to operate the Observability application.

Inspect the pods: `kubectl get pods`

Get pod logs: `kubectl logs <POD ID>`

Stop the app: `minikube stop`
Stop the app: `docker compose -f obs-docker-compose.yml obs down`

Restart the app: `minikube start`
Restart the app: `docker compose -f obs-docker-compose.yml up`

## Remove Demo Data

Expand Down
80 changes: 29 additions & 51 deletions demo/demo/main.py
Original file line number Diff line number Diff line change
@@ -1,67 +1,45 @@
import signal

from demo_helper import Config
from observability_demo import run_obs_demo, delete_obs_demo
from observability_demo import run_obs_demo, delete_obs_demo
from heartbeat_demo import run_heartbeat_demo
from testgen_demo import run_tg_demo, delete_tg_demo
from testgen_demo import run_tg_demo, delete_tg_demo
from argparse import ArgumentParser


def init_args() -> ArgumentParser:
parser = ArgumentParser(
description="""
This is a tool to create a demo of DataOps Observability & TestGen.
""",
)
subparsers = parser.add_subparsers(title="subcommands")
def init_parser() -> ArgumentParser:
parser = ArgumentParser(description="This is a tool to create a demo of DataOps Observability & TestGen.")
subparsers = parser.add_subparsers(title="subcommands", required=True)

obs_run_parser = subparsers.add_parser(
"obs-run-demo",
description="Run the Observability demo",
)
obs_run_parser.set_defaults(action="obs-run-demo")

obs_delete_parser = subparsers.add_parser(
"obs-delete-demo",
description="Delete data created by the Observability demo",
commands = (
("obs-run-demo", "Run the Observability demo", run_obs_demo),
("obs-delete-demo", "Delete data created by the Observability demo", delete_obs_demo),
("obs-heartbeat-demo", "Run the Observability Heartbeat demo", run_heartbeat_demo),
("tg-run-demo", "Run the TestGen demo", run_tg_demo),
("tg-delete-demo", "Delete data created by the TestGen demo", delete_tg_demo),
)
obs_delete_parser.set_defaults(action="obs-delete-demo")

obs_heartbeat_parser = subparsers.add_parser(
"obs-heartbeat-demo",
description="Run the Observability Heartbeat demo",
)
obs_heartbeat_parser.set_defaults(action="obs-heartbeat-demo")
for cmd, desc, func in commands:
sub_parser = subparsers.add_parser(cmd, description=desc)
sub_parser.set_defaults(func=func)

tg_run_parser = subparsers.add_parser(
"tg-run-demo",
description="Run the TestGen demo",
)
tg_run_parser.set_defaults(action="tg-run-demo")

tg_delete_parser = subparsers.add_parser(
"tg-delete-demo",
description="Delete data created by the TestGen demo",
)
tg_delete_parser.set_defaults(action="tg-delete-demo")
return parser


def init_signal_handler():
def _keyboard_interrupt(_signum, _frame):
raise KeyboardInterrupt

# Docker sends SIGTERM on Ctrl-C, so we raise a KeyboardInterrupt
signal.signal(signal.SIGTERM, _keyboard_interrupt)

return parser.parse_args()

def main():
args = init_args()
init_signal_handler()
args = init_parser().parse_args()
config = Config()

if args.action == "obs-run-demo":
run_obs_demo(config)
elif args.action == "obs-delete-demo":
delete_obs_demo(config)
elif args.action == "obs-heartbeat-demo":
run_heartbeat_demo(config)
elif args.action == "tg-run-demo":
run_tg_demo(config)
elif args.action == "tg-delete-demo":
delete_tg_demo(config)
else:
print(f"Command [{args.action}] not recognized.")
args.func(config)


if __name__ == "__main__":
main()
main()
Loading