Skip to content

Commit 8609d7f

Browse files
JohnGarbuttJohnGarbutt
committed
Add host guages
1 parent d2f22de commit 8609d7f

File tree

2 files changed

+26
-144
lines changed

2 files changed

+26
-144
lines changed

‎README.rst‎

Lines changed: 9 additions & 131 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,6 @@ os-capacity
33

44
This is a prototype tool to extract capacity information.
55

6-
.. note::
7-
8-
This is currently quite specific to Ironic powered OpenStack Nova clouds.
9-
106
Install
117
-------
128

@@ -21,139 +17,21 @@ Now lets get that installed inside a virtual environment:
2117

2218
.. code::
2319
24-
virtualenv .venv-test
25-
source .venv-test/bin/activate
20+
python3 -m virtualenv .venv
21+
source .venv/bin/activate
2622
pip install -U .
2723
2824
Prometheus Exporter
2925
-------------------
3026

31-
Assuming you have clouds.yaml in the right place and OS_CLOUD set:
27+
Assuming you have clouds.yaml in the right place,
28+
you can run the exporter doing something like this:
3229

3330
.. code::
3431
35-
./os_capacity/prometheus.py
36-
openstack_total_capacity_per_flavor{flavor="small"} 1
37-
openstack_capacity_by_hostname{hypervisor="aio",flavor="small"} 1
38-
39-
40-
TODOs we need support for:
41-
42-
* add request filter support for require_tenant_aggregate,
43-
map_az_to_placement_aggregate and compute_status_filter
44-
45-
Configuration
46-
-------------
47-
48-
The easiest way to configure this is to populate a typical OpenStack RC file:
49-
50-
.. code::
51-
52-
cat > .openrc <<EOF
53-
export OS_AUTH_URL=http://keystone.example.com:5000/v3
54-
export OS_PROJECT_ID=
55-
export OS_PROJECT_NAME=
56-
export OS_USER_DOMAIN_NAME=
57-
export OS_PROJECT_DOMAIN_NAME=
58-
export OS_USERNAME=
59-
export OS_REGION_NAME=
60-
export OS_INTERFACE=
61-
export OS_IDENTITY_API_VERSION=3
62-
export OS_AUTH_PLUGIN=v3password
63-
echo "Please enter your OpenStack Password for project $OS_PROJECT_NAME as user $OS_USERNAME: "
64-
read -sr OS_PASSWORD_INPUT
65-
export OS_PASSWORD=$OS_PASSWORD_INPUT
66-
EOF
67-
68-
source .openrc
69-
70-
Some openrc files don't contain the OS_AUTH_PLUGIN and OS_PROJECT_DOMAIN_NAME
71-
variables, but os-capacity requires that those are set.
72-
73-
Usage
74-
-----
75-
76-
When opening a new terminal, first activate the venv and the configuration:
77-
78-
.. code::
79-
80-
source .venv-test/bin/activate
81-
source .openrc
82-
83-
84-
You can do things like list all flavors:
85-
86-
.. code::
87-
88-
(.venv-test) $ os-capacity flavor list
89-
+--------------------------------------+-------------+-------+--------+---------+
90-
| UUID | Name | VCPUs | RAM MB | DISK GB |
91-
+--------------------------------------+-------------+-------+--------+---------+
92-
| 2622d978-7072-484d-8c7a-144a308c2709 | my-flavor-1 | 1 | 512 | 20 |
93-
| 45de641c-950e-434b-9c2e-6f76b120f85c | my-flavor-2 | 2 | 1024 | 40 |
94-
+--------------------------------------+-------------+-------+--------+---------+
95-
96-
If you want to see all the REST API calls made, use the verbose flag, and you
97-
can also get the output in json format by adding the format flag:
98-
99-
.. code::
100-
101-
(.venv-test) $ os-capacity -v flavor list -f json
102-
103-
You can look at all the different types of resources and amount used:
104-
105-
.. code::
106-
107-
(.venv-test) $ os-capacity resources group
108-
+----------------------------------+-------+------+------+-------------+
109-
| Resource Class Groups | Total | Used | Free | Flavors |
110-
+----------------------------------+-------+------+------+-------------+
111-
| VCPU:1,MEMORY_MB:512,DISK_GB:20 | 5 | 1 | 4 | my-flavor-1 |
112-
| VCPU:2,MEMORY_MB:1024,DISK_GB:40 | 2 | 0 | 2 | my-flavor-2 |
113-
+----------------------------------+-------+------+------+-------------+
114-
115-
116-
You can also look at the usage grouped by project or user or total usage:
117-
118-
.. code::
119-
120-
(.venv-test) $ os-capacity usages group user --max-width 70
121-
+----------------------+----------------------+----------------------+
122-
| User | Current Usage | Usage Days |
123-
+----------------------+----------------------+----------------------+
124-
| 1e6abb726dd04d4eb4b8 | Count:4, | Count:410, |
125-
| 94e19c397d5e | DISK_GB:1484, | DISK_GB:152110, |
126-
| | MEMORY_MB:524288, | MEMORY_MB:53739520, |
127-
| | VCPU:256 | VCPU:26240 |
128-
| 4661c3e5f2804696ba26 | Count:1, | Count:3, |
129-
| 56b50dbd0f3d | DISK_GB:371, | DISK_GB:1113, |
130-
| | MEMORY_MB:131072, | MEMORY_MB:393216, |
131-
| | VCPU:64 | VCPU:192 |
132-
+----------------------+----------------------+----------------------+
133-
134-
See the online help for more details:
135-
136-
.. code::
137-
138-
os-capacity help
139-
usage: os-capacity [--version] [-v | -q] [--log-file LOG_FILE] [-h] [--debug]
140-
141-
OS-Capacity (StackHPC) Command Line Interface (CLI)
142-
143-
optional arguments:
144-
--version show program's version number and exit
145-
-v, --verbose Increase verbosity of output. Can be repeated.
146-
-q, --quiet Suppress output except warnings and errors.
147-
--log-file LOG_FILE Specify a file to log output. Disabled by default.
148-
-h, --help Show help message and exit.
149-
--debug Show tracebacks on errors.
32+
export OS_CLIENT_CONFIG_FILE=myappcred.yaml
33+
export OS_CLOUD=openstack
15034
151-
Commands:
152-
complete print bash completion command
153-
flavor list List all the flavors.
154-
help print detailed help for another command
155-
prometheus To be run as node exporter textfile collector
156-
resources all List all resource providers, with their resources and servers.
157-
resources group Lists counts of resource providers with similar inventories.
158-
usages all List all current resource usages.
159-
usages group Group usage by specified key (by user or project).
35+
./os_capacity/prometheus.py &
36+
curl localhost:9000 > mytestrun
37+
cat mytestrun

‎os_capacity/prometheus.py‎

Lines changed: 17 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -253,7 +253,17 @@ def get_project_usage(indentity_client, placement_client, compute_client):
253253
return [project_usage_guage, project_quota_guage]
254254

255255

256-
def print_host_usage(resource_providers, placement_client):
256+
def get_host_usage(resource_providers, placement_client):
257+
usage_guage = prom_core.GaugeMetricFamily(
258+
"openstack_hypervisor_placement_allocated",
259+
"Currently allocated resource for each provider in placement.",
260+
labels=["hypervisor", "resource"],
261+
)
262+
capacity_guage = prom_core.GaugeMetricFamily(
263+
"openstack_hypervisor_placement_allocatable_capacity",
264+
"The total allocatable resource in the placement inventory.",
265+
labels=["hypervisor", "resource"],
266+
)
257267
for name, data in resource_providers.items():
258268
rp_id = data["uuid"]
259269
response = placement_client.get(
@@ -265,10 +275,7 @@ def print_host_usage(resource_providers, placement_client):
265275
resource_providers[name]["usages"] = rp_usages
266276

267277
for resource, amount in rp_usages.items():
268-
print(
269-
f'openstack_hypervisor_usage{{hypervisor="{name}",'
270-
f'resource="{resource}"}} {amount}'
271-
)
278+
usage_guage.add_metric([name, resource], amount)
272279

273280
response = placement_client.get(
274281
f"/resource_providers/{rp_id}/inventories",
@@ -280,11 +287,9 @@ def print_host_usage(resource_providers, placement_client):
280287

281288
for resource, data in inventories.items():
282289
amount = data["total"] - data["reserved"]
283-
print(
284-
f'openstack_hypervisor_allocatable_capacity{{hypervisor="{name}",'
285-
f'resource="{resource}"}} {amount}'
286-
)
290+
capacity_guage.add_metric([name, resource], amount)
287291
# print(json.dumps(resource_providers, indent=2))
292+
return [usage_guage, capacity_guage]
288293

289294

290295
def print_exporter_data(app):
@@ -294,7 +299,7 @@ def print_exporter_data(app):
294299
class OpenStackCapacityCollector(object):
295300
def __init__(self):
296301
self.conn = openstack.connect()
297-
openstack.enable_logging(debug=True)
302+
openstack.enable_logging(debug=False)
298303
print("got openstack connection")
299304
# for some reason this makes the logging work?!
300305
self.conn.compute.flavors()
@@ -306,15 +311,14 @@ def collect(self):
306311
guages = []
307312

308313
conn = openstack.connect()
309-
openstack.enable_logging(debug=True)
314+
openstack.enable_logging(debug=False)
310315
try:
311316
resource_providers, host_guages = get_host_details(
312317
conn.compute, conn.placement
313318
)
314319
guages += host_guages
315-
316320
guages += get_project_usage(conn.identity, conn.placement, conn.compute)
317-
print_host_usage(resource_providers, conn.placement)
321+
guages += get_host_usage(resource_providers, conn.placement)
318322
except Exception as e:
319323
print(f"error {e}")
320324

0 commit comments

Comments
 (0)