Skip to content

Commit 3fe624b

Browse files
paalonsopaalonso
committed
Added the Instance Principal script version
1 parent 01b28aa commit 3fe624b

File tree

3 files changed

+309
-4
lines changed

3 files changed

+309
-4
lines changed

‎manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/README.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
# Using python SDK to create OCI Monitoring custom metric namespace: Services Limit monitoring example use case
33

44
## 1. INTRODUCTION
5+
56
Describes how any user can create an OCI Monitoring ***custom metric namespace*** to being able to extend the default services metric namespaces. For that, we'll support on python SDK to create an script that can be run in an OCI VM (using instance principals authentication), or any other external system (using OCI IAM principals). To cover this educational example, we'll use as an example the creation of a custom metric namespace to monitor the OCI Services Limits usage. With this custom metric namespace, OCI alarms can be created and OCI Notification Service can be used to send the alarm information by different means to allow to create proactively a Service Limit Service Request to increase the limit before causing any disruption in the running services or services to be provisioned.
67

78
## 2. SOLUTION
@@ -39,7 +40,7 @@ Basically the **logic** is:
3940

4041
````
4142
Start
42-
Gather the IAM user connection details from OCI Config
43+
Setup the OCI connection authentication (OCI IAM Config or Instance Principals)
4344
Set compartment_ocid
4445
Initialize the clients for the different API calls (IAM, Monitoring, Service Limits)
4546
Gather the full list of Service Limits Definitions sorted by Service Limit name
@@ -71,7 +72,7 @@ To execute the script:
7172
````
7273
$ python3 serviceLimitsMetricsIP.py
7374
````
74-
* The script is available **TBD**
75+
* The script is available [here](./files/Scripts/postServiceLimitsMetricsIP.py)
7576
7677
7778
## 5. REQUIREMENTS
@@ -87,7 +88,11 @@ We have different requirements depending on the variant of this asset that we wo
8788
* use metrics in tenancy
8889
8990
2. **Instance Principal**
90-
* ***TBD***
91+
* An existing dynamic group identifying the VM where to run the script as member of the group
92+
* A policy granting the dynamic group to:
93+
* inspect resource-availability in tenancy
94+
* inspect limits in tenancy
95+
* use metrics in tenancy
9196
9297
3. ***Common requirements***
9398
The VM where to run the script must have installed python3 with the following required packages installed with pip:

‎manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Scripts/postServiceLimitsMetricsIAM.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@
3737
###
3838
# This is a sample python script that post a custom metric (service_limits) to oci monitoring based on Tenancy Service Limits information.
3939
# Run this script on any host with python with access to your tenancy.
40-
# Command: python3 serviceLimitsMetrics.py
40+
# Command: python3 postServiceLimitsMetricsIAM.py
4141
# Version: 0.1
4242
###
4343

‎manageability-and-operations/observability-and-manageability/oci-monitoring/custom-metrics/custom-metric-python-SDK-services-limit-monitoring/files/Scripts/postServiceLimitsMetricsIP.py

Lines changed: 300 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,300 @@
1+
# Copyright (c) 2023 Oracle and/or its affiliates.
2+
3+
# The Universal Permissive License (UPL), Version 1.0
4+
5+
# Subject to the condition set forth below, permission is hereby granted to any
6+
# person obtaining a copy of this software, associated documentation and/or data
7+
# (collectively the "Software"), free of charge and under any and all copyright
8+
# rights in the Software, and any and all patent rights owned or freely
9+
# licensable by each licensor hereunder covering either (i) the unmodified
10+
# Software as contributed to or provided by such licensor, or (ii) the Larger
11+
# Works (as defined below), to deal in both
12+
13+
# (a) the Software, and
14+
# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
15+
# one is included with the Software (each a "Larger Work" to which the Software
16+
# is contributed by such licensors),
17+
18+
# without restriction, including without limitation the rights to copy, create
19+
# derivative works of, display, perform, and distribute the Software and make,
20+
# use, sell, offer for sale, import, export, have made, and have sold the
21+
# Software and the Larger Work(s), and to sublicense the foregoing rights on
22+
# either these or other terms.
23+
24+
# This license is subject to the following condition:
25+
# The above copyright notice and either this complete permission notice or at
26+
# a minimum a reference to the UPL must be included in all copies or
27+
# substantial portions of the Software.
28+
29+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
30+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
31+
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
32+
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
33+
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
34+
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
35+
# SOFTWARE.
36+
37+
###
38+
# This is a sample python script that post a custom metric (service_limits) to oci monitoring based on Tenancy Service Limits information.
39+
# Run this script on any host with python with access to your tenancy.
40+
# Command: python3 postServiceLimitsMetricsIP.py
41+
# Version: 0.1
42+
###
43+
44+
import oci,datetime,json
45+
from pytz import timezone
46+
47+
# Vars:
48+
# Replace here with your tenancy's root compartment OCID
49+
compartment_ocid = "ocid1.tenancy.oc1....."
50+
51+
# Start:
52+
now = datetime.datetime.now().strftime("%d/%m/%Y %H:%M:%S")
53+
print("[", now,"] Starting OCI Service Metrics limits gathering and customer metrics post...")
54+
55+
# Setup the signer for the instance principal auth method
56+
signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner()
57+
58+
# We gather the list of availability domains in the region
59+
identity_client = oci.identity.IdentityClient(config={}, signer=signer)
60+
list_availability_domains_response = identity_client.list_availability_domains(compartment_id = compartment_ocid)
61+
62+
# Get the data from response
63+
print(list_availability_domains_response.data)
64+
65+
# Initialize service client with default config file
66+
#monitoring_client = oci.monitoring.MonitoringClient(config={}, signer=signer)
67+
monitoring_client = oci.monitoring.MonitoringClient(config={},service_endpoint="https://telemetry-ingestion.eu-frankfurt-1.oraclecloud.com",signer=signer)
68+
69+
# Get Service Limits
70+
71+
# Initialize service client with default config file
72+
limits_client = oci.limits.LimitsClient(config={}, signer=signer)
73+
74+
# Send the request to service, some parameters are not required, see API
75+
# doc for more info
76+
list_limit_definitions_response = limits_client.list_limit_definitions(
77+
compartment_id = compartment_ocid,
78+
sort_by="name",
79+
sort_order="ASC")
80+
81+
# We iterate the list of all the service limits
82+
for x in list_limit_definitions_response.data:
83+
limit = json.loads(str(x))
84+
s_name = limit["service_name"]
85+
l_name = limit["name"]
86+
l_scope = limit["scope_type"]
87+
88+
print ("Service: ", s_name, "Limit: ", l_name, "Scope: ", l_scope)
89+
90+
# If the resource limit has an AD scope, we have to specify the AD or we'll get an API 400 response
91+
if l_scope == "AD" :
92+
# We have AD scope, we've to gather the limit for all the ADs in the region and include the availabilityDomain
93+
for AD in list_availability_domains_response.data:
94+
95+
a_domain = json.loads(str(AD))
96+
print("Availability Domain: ", a_domain)
97+
98+
# We gather the service limit usage
99+
get_resource_availability_response = limits_client.get_resource_availability(
100+
service_name = s_name,
101+
limit_name = l_name,
102+
compartment_id = compartment_ocid,
103+
availability_domain = a_domain["name"])
104+
usage = json.loads(str(get_resource_availability_response.data))
105+
used = usage["used"]
106+
available = usage["available"]
107+
108+
# Get the data from response
109+
print(get_resource_availability_response.data)
110+
111+
# We need to gather the service limit limit
112+
list_limit_values_response = limits_client.list_limit_values(
113+
compartment_id = compartment_ocid,
114+
service_name = s_name,
115+
availability_domain = a_domain["name"],
116+
limit = 1)
117+
limit_limit = json.loads(str(list_limit_values_response.data[0]))
118+
max_limit = limit_limit["value"]
119+
120+
# Get the timestamp for setup the monitoring metric post information
121+
times_stamp = datetime.datetime.now(timezone('UTC'))
122+
123+
# Posting custom metrics to oci monitoring for each of the metrics (max, used, available)
124+
125+
# Max limit
126+
post_metric_data_response = monitoring_client.post_metric_data(
127+
post_metric_data_details=oci.monitoring.models.PostMetricDataDetails(
128+
metric_data=[
129+
oci.monitoring.models.MetricDataDetails(
130+
namespace = "limits_metrics",
131+
compartment_id = compartment_ocid,
132+
name = "max_limit",
133+
dimensions={
134+
'service_name': s_name,
135+
'limit_name': l_name,
136+
'availability_domain': a_domain["name"]
137+
},
138+
datapoints=[
139+
oci.monitoring.models.Datapoint(
140+
timestamp=datetime.datetime.strftime(
141+
times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"),
142+
value = max_limit)]
143+
)]
144+
)
145+
)
146+
print("Max_limit: ", post_metric_data_response.data)
147+
148+
# Used
149+
post_metric_data_response = monitoring_client.post_metric_data(
150+
post_metric_data_details=oci.monitoring.models.PostMetricDataDetails(
151+
metric_data=[
152+
oci.monitoring.models.MetricDataDetails(
153+
namespace = "limits_metrics",
154+
compartment_id = compartment_ocid,
155+
name = "used",
156+
dimensions={
157+
'service_name': s_name,
158+
'limit_name': l_name,
159+
'availability_domain': a_domain["name"]
160+
},
161+
datapoints=[
162+
oci.monitoring.models.Datapoint(
163+
timestamp=datetime.datetime.strftime(
164+
times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"),
165+
value = used)]
166+
)]
167+
)
168+
)
169+
print("Used: ", post_metric_data_response.data)
170+
171+
# Available
172+
post_metric_data_response = monitoring_client.post_metric_data(
173+
post_metric_data_details=oci.monitoring.models.PostMetricDataDetails(
174+
metric_data=[
175+
oci.monitoring.models.MetricDataDetails(
176+
namespace = "limits_metrics",
177+
compartment_id = compartment_ocid,
178+
name = "available",
179+
dimensions={
180+
'service_name': s_name,
181+
'limit_name': l_name,
182+
'availability_domain': a_domain["name"]
183+
},
184+
datapoints=[
185+
oci.monitoring.models.Datapoint(
186+
timestamp=datetime.datetime.strftime(
187+
times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"),
188+
value = available)]
189+
)]
190+
)
191+
)
192+
print("Available: ", post_metric_data_response.data)
193+
194+
else :
195+
# We are in GLOBAL or REGION case
196+
197+
# We gather the service limit usage
198+
get_resource_availability_response = limits_client.get_resource_availability(
199+
service_name = s_name,
200+
limit_name = l_name,
201+
compartment_id = compartment_ocid)
202+
usage = json.loads(str(get_resource_availability_response.data))
203+
used = usage["used"]
204+
available = usage["available"]
205+
206+
# Get the data from response
207+
print(get_resource_availability_response.data)
208+
209+
# We need to gather the service limit limit
210+
list_limit_values_response = limits_client.list_limit_values(
211+
compartment_id = compartment_ocid,
212+
service_name = s_name,
213+
limit = 1)
214+
limit_limit = json.loads(str(list_limit_values_response.data[0]))
215+
max_limit = limit_limit["value"]
216+
217+
print(list_limit_values_response.data)
218+
219+
print("max_limit: ", max_limit)
220+
if max_limit == "null" :
221+
continue
222+
223+
# Get the timestamp for setup the monitoring metric post information
224+
times_stamp = datetime.datetime.now(timezone('UTC'))
225+
226+
# Posting custom metrics to oci monitoring for each of the metrics (max, used, available)
227+
228+
# Max limit
229+
post_metric_data_response = monitoring_client.post_metric_data(
230+
post_metric_data_details=oci.monitoring.models.PostMetricDataDetails(
231+
metric_data=[
232+
oci.monitoring.models.MetricDataDetails(
233+
namespace = "limits_metrics",
234+
compartment_id = compartment_ocid,
235+
name = "max_limit",
236+
dimensions={
237+
'service_name': s_name,
238+
'limit_name': l_name
239+
},
240+
datapoints=[
241+
oci.monitoring.models.Datapoint(
242+
timestamp=datetime.datetime.strftime(
243+
times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"),
244+
value = max_limit)]
245+
)]
246+
)
247+
)
248+
print("Max_limit: ", post_metric_data_response.data)
249+
250+
print("used: ", used)
251+
if used is None :
252+
continue
253+
254+
# Used
255+
post_metric_data_response = monitoring_client.post_metric_data(
256+
post_metric_data_details=oci.monitoring.models.PostMetricDataDetails(
257+
metric_data=[
258+
oci.monitoring.models.MetricDataDetails(
259+
namespace = "limits_metrics",
260+
compartment_id = compartment_ocid,
261+
name = "used",
262+
dimensions={
263+
'service_name': s_name,
264+
'limit_name': l_name
265+
},
266+
datapoints=[
267+
oci.monitoring.models.Datapoint(
268+
timestamp=datetime.datetime.strftime(
269+
times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"),
270+
value = used)]
271+
)]
272+
)
273+
)
274+
print("Used: ", post_metric_data_response.data)
275+
276+
# Available
277+
post_metric_data_response = monitoring_client.post_metric_data(
278+
post_metric_data_details=oci.monitoring.models.PostMetricDataDetails(
279+
metric_data=[
280+
oci.monitoring.models.MetricDataDetails(
281+
namespace = "limits_metrics",
282+
compartment_id = compartment_ocid,
283+
name = "available",
284+
dimensions={
285+
'service_name': s_name,
286+
'limit_name': l_name },
287+
datapoints=[
288+
oci.monitoring.models.Datapoint(
289+
timestamp=datetime.datetime.strftime(
290+
times_stamp,"%Y-%m-%dT%H:%M:%S.%fZ"),
291+
value = available)]
292+
)]
293+
)
294+
)
295+
print("Available: ", post_metric_data_response.data)
296+
297+
298+
# Finish:
299+
now = datetime.datetime.now().strftime("%d/%m/%Y %H:%M:%S")
300+
print("[", now,"] Finish OCI Service Metrics limits gathering and customer metrics post.")

0 commit comments

Comments
 (0)