Skip to content

Commit 21c9a2f

Browse files
nmaludynmaludy
authored
Merge pull request #2 from EncoreTechnologies/feature/actions_and_python_files
Feature/actions and python files
2 parents 5610512 + ed45c82 commit 21c9a2f

27 files changed

+1598
-12
lines changed

‎README.md‎

Lines changed: 98 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,105 @@
1-
# sql Integration Pack
1+
# SQL Integration Pack
2+
Query, Insert, Update, and Delete information from PostgreSQL, SQLite, MsSQL, MySQL, Oracle, Firebird, and Sybase Databases
23

3-
## Configuration
4-
TODO: Describe configuration
4+
## Pre-Requisites
5+
This pack is set up to provide funcationality for the above databases. For MySQL and MsSQL we need to install 2 system packages.
56

7+
#### [MySQL](https://pypi.org/project/mysqlclient/)
8+
``` shell
9+
yum install mysql-devel
10+
```
611

7-
# Sensors
12+
#### [MsSQL](https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-2017)
13+
``` shell
14+
curl https://packages.microsoft.com/config/rhel/7/prod.repo > /etc/yum.repos.d/mssql-release.repo
15+
yum install msodbcsql17
16+
yum install unixODBC-devel
17+
```
818

9-
## Example Sensor
10-
TODO: Describe sensor
19+
## Quick Start
1120

21+
1. Install the pack
1222

13-
# Actions
23+
``` shell
24+
st2 pack install sql
25+
```
1426

15-
## example
16-
TODO: Describe action
27+
2. Execute an action (example: query)
28+
29+
``` shell
30+
st2 run sql.query host=test_serve.domain.tld username=test_user password=test_password database=test_database drivername=postgresql query="select * from test;"
31+
```
32+
33+
## Configuration and Connecting to Databases
34+
Connecting to different types of databases is shown below. Connecting to different databases is done in the same manor except with sqlite where all you need to pass is the path to the database in the database option. This is show below. For more information about connections please refer to [SQLAlchemy Connection Docs](https://docs.sqlalchemy.org/en/latest/core/engines.html)
35+
36+
Copy the example configuration in [sql.yaml.example](./sql.yaml.example)
37+
to `/opt/stackstorm/configs/sql.yaml` and edit as required.
38+
39+
It can contain an array of one or more sets of SQL connection parameters, like this:
40+
41+
``` yaml
42+
---
43+
connections:
44+
postgresql:
45+
host: postgresql_db.domain.tld
46+
username: [email protected]
47+
password: Password
48+
database: TestDatabase
49+
port: 5432
50+
drivername: postgresql
51+
mysql:
52+
host: mysql_db.domain.tld
53+
username: [email protected]
54+
password: NewPassword
55+
database: TestDatabase
56+
drivername: mysql
57+
sqlite:
58+
database: /path/to/db.sqlite
59+
drivername: sqlite
60+
```
61+
62+
Each entry should contain
63+
64+
* ``host`` - Database hostname
65+
* ``username`` - Username to authenticate to DB
66+
* ``password`` - Password for DB authentication
67+
* ``database`` - Database to use
68+
* ``port`` - Port to connect to database on. If Default leave blank
69+
* ``drivername`` - The type of database that is being connected to.
70+
71+
When running actions, you can pass in the name of a connection, e.g.
72+
`st2 run sql.query connection="postgresql" query="select * from test;"`
73+
74+
Alternatively, when running an action, you can pass in the host, username, password, database, port, drivername parameters. These parameters can also be used for overrides if you wish to use the configs as well.
75+
76+
**Note** : When modifying the configuration in `/opt/stackstorm/configs/` please remember to tell StackStorm to load these new values by running `st2ctl reload --register-configs`
77+
78+
# Usage
79+
80+
## Actions
81+
82+
| Action | Description |
83+
|--------|-------------|
84+
| query | Generic query action to get inforamtion from the database. |
85+
| insert | Insert data into a database table. Insert data is passed as an object. |
86+
| insert_bulk | Bulk insert data into a database table. Insert data is passed as an array of objects. |
87+
| update | Update data in a database table. |
88+
| delete | Delete data from a database table. |
89+
90+
## Where statements
91+
92+
The Update and Delete actions give the option to include where data into the query. This only works for AND statements.
93+
94+
Example:
95+
```
96+
where = {
97+
'column_1': 'value_1',
98+
'column_2': 'value_2'
99+
}
100+
101+
Produces the statement:
102+
WHERE column_1 == 'value_1' AND column_2 == 'value_2'
103+
```
104+
105+
For more complicated queries please use the query action.

‎actions/README.md‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎actions/delete.py‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
from lib.base_action import BaseAction
2+
import sqlalchemy
3+
4+
5+
class SQLDeleteAction(BaseAction):
6+
def __init__(self, config):
7+
"""Creates a new BaseAction given a StackStorm config object (kwargs works too)
8+
:param config: StackStorm configuration object for the pack
9+
:returns: a new BaseAction
10+
"""
11+
super(SQLDeleteAction, self).__init__(config)
12+
13+
def run(self, **kwargs):
14+
"""Main entry point for the StackStorm actions to execute the operation.
15+
:returns: the dns adapters and the number of network adapters to be
16+
on the VM.
17+
"""
18+
kwargs_dict = dict(kwargs)
19+
20+
where_dict = self.get_del_arg('where', kwargs_dict)
21+
table = self.get_del_arg('table', kwargs_dict)
22+
23+
with self.db_connection(kwargs_dict) as conn:
24+
# Get the SQL table
25+
sql_table = sqlalchemy.Table(table,
26+
self.meta,
27+
autoload=True,
28+
autoload_with=self.engine)
29+
30+
# Intantiate delete object
31+
delete = sql_table.delete() # pylint: disable-msg=no-value-for-parameter
32+
33+
# Generate Where Statement
34+
if where_dict:
35+
delete, where_dict = self.generate_where_clause(sql_table, delete, where_dict)
36+
37+
# Execute query
38+
result = conn.execute(delete, where_dict)
39+
40+
return {'affected_rows': result.rowcount}

‎actions/delete.yaml‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
name: delete
3+
runner_type: "python-script"
4+
description: "Deletes data from the SQL database"
5+
enabled: true
6+
entry_point: delete.py
7+
parameters:
8+
connection:
9+
type: string
10+
description: "Name of <connection> from this pack's configuration that specifies how to connect to a database server."
11+
required: false
12+
host:
13+
type: string
14+
description: >
15+
Optional override of the database host in <connection> (required if <connection> is not specified). Database server to connect to. If not using a default port add that here. ex. host.domain.tld or host.domain.tld:1234
16+
required: false
17+
username:
18+
type: string
19+
description: "Optional override of the username in <connection> (required if <connection> is not specified). Username for authentication"
20+
required: false
21+
password:
22+
type: string
23+
description: "Optional override of the password in <connection> (required if <connection> is not specified). Password of the specified username"
24+
secret: true
25+
required: false
26+
database:
27+
type: string
28+
description: "Optional override of the database in <connection> (required if <connection> is not specified). Database to connect to, to run querys against."
29+
required: false
30+
port:
31+
description: "Port to connect to database on. If Default leave blank"
32+
type: integer
33+
required: false
34+
drivername:
35+
type: string
36+
description: "Optional override of the database_type in <connection> (required if <connection> is not specified). The type of database that is being connected to."
37+
enum:
38+
- postgresql
39+
- sqlite
40+
- mssql
41+
- mysql
42+
- oracle
43+
- firebird
44+
- sybase
45+
required: false
46+
table:
47+
type: string
48+
description: "Database table to DELETE."
49+
required: true
50+
where:
51+
type: object
52+
description: >
53+
Dictionary of data to be used to create a WHERE clause for the DELETE statement
54+
{
55+
'column_1': 'data_to_match_1',
56+
'column_2': 'data_to_match_2',
57+
'column_3': 'data_to_match_3',
58+
'column_4': 'data_to_match_4',
59+
}
60+
required: false
61+
default: {}

‎actions/generic_query.py‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
from lib.base_action import BaseAction
2+
3+
4+
class SQLQueryAction(BaseAction):
5+
def __init__(self, config):
6+
"""Creates a new BaseAction given a StackStorm config object (kwargs works too)
7+
:param config: StackStorm configuration object for the pack
8+
:returns: a new BaseAction
9+
"""
10+
super(SQLQueryAction, self).__init__(config)
11+
12+
def run(self, **kwargs):
13+
"""Main entry point for the StackStorm actions to execute the operation.
14+
:returns: the dns adapters and the number of network adapters to be
15+
on the VM.
16+
"""
17+
kwargs_dict = dict(kwargs)
18+
19+
query = self.get_del_arg('query', kwargs_dict)
20+
21+
with self.db_connection(kwargs_dict) as conn:
22+
# Execute the query
23+
query_result = conn.execute(query)
24+
25+
return_result = {'affected_rows': query_result.rowcount}
26+
if query_result.returns_rows:
27+
return_result = []
28+
all_results = query_result.fetchall()
29+
for row in all_results:
30+
# Rows are returned as tuples with keys.
31+
# Convert that to a dictionary for return
32+
return_result.append(self.row_to_dict(row))
33+
34+
return return_result

‎actions/insert.py‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
from lib.base_action import BaseAction
2+
import sqlalchemy
3+
4+
5+
class SQLInsertAction(BaseAction):
6+
def __init__(self, config):
7+
"""Creates a new BaseAction given a StackStorm config object (kwargs works too)
8+
:param config: StackStorm configuration object for the pack
9+
:returns: a new BaseAction
10+
"""
11+
super(SQLInsertAction, self).__init__(config)
12+
13+
def run(self, **kwargs):
14+
"""Main entry point for the StackStorm actions to execute the operation.
15+
:returns: the dns adapters and the number of network adapters to be
16+
on the VM.
17+
"""
18+
kwargs_dict = dict(kwargs)
19+
20+
insert_data = self.get_del_arg('data', kwargs_dict)
21+
insert_table = self.get_del_arg('table', kwargs_dict)
22+
23+
if not isinstance(insert_data, list):
24+
insert_data = [insert_data]
25+
26+
with self.db_connection(kwargs_dict) as conn:
27+
# Get the Table to insert data into
28+
sql_table = sqlalchemy.Table(insert_table,
29+
self.meta,
30+
autoload=True,
31+
autoload_with=self.engine)
32+
33+
# Execute the insert query
34+
conn.execute(sql_table.insert(), # pylint: disable-msg=no-value-for-parameter
35+
insert_data)
36+
37+
return True

‎actions/insert.yaml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
---
2+
name: insert
3+
runner_type: "python-script"
4+
description: "Insert data into a SQL database"
5+
enabled: true
6+
entry_point: insert.py
7+
parameters:
8+
connection:
9+
type: string
10+
description: "Name of <connection> from this pack's configuration that specifies how to connect to a database server."
11+
required: false
12+
host:
13+
type: string
14+
description: >
15+
Optional override of the database host in <connection> (required if <connection> is not specified). Database server to connect to. If not using a default port add that here. ex. host.domain.tld or host.domain.tld:1234
16+
required: false
17+
username:
18+
type: string
19+
description: "Optional override of the username in <connection> (required if <connection> is not specified). Username for authentication"
20+
required: false
21+
password:
22+
type: string
23+
description: "Optional override of the password in <connection> (required if <connection> is not specified). Password of the specified username"
24+
secret: true
25+
required: false
26+
database:
27+
type: string
28+
description: "Optional override of the database in <connection> (required if <connection> is not specified). Database to connect to, to run querys against."
29+
required: false
30+
port:
31+
description: "Port to connect to database on. If Default leave blank"
32+
type: integer
33+
required: false
34+
drivername:
35+
type: string
36+
description: "Optional override of the database_type in <connection> (required if <connection> is not specified). The type of database that is being connected to."
37+
enum:
38+
- postgresql
39+
- sqlite
40+
- mssql
41+
- mysql
42+
- oracle
43+
- firebird
44+
- sybase
45+
required: false
46+
table:
47+
type: string
48+
description: "Database table to insert data into."
49+
required: true
50+
data:
51+
type: object
52+
description: >
53+
Dictionary of data to be inserted where the key corresponds to the column of the table
54+
{
55+
'column_1': 'data_to_insert_1',
56+
'column_2': 'data_to_insert_2',
57+
'column_3': 'data_to_insert_3',
58+
'column_4': 'data_to_insert_4',
59+
}
60+
required: true

0 commit comments

Comments
 (0)