Skip to content

Commit 749599e

Browse files
author
rbutley
committed
cgrp util release 1.0.1, initial commit
0 parents  commit 749599e

File tree

6 files changed

+280
-0
lines changed

6 files changed

+280
-0
lines changed

‎.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
build/*
2+
cgrputil.egg-info/*
3+
venv/*
4+
dist/*
5+
build.txt
6+
test.py

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2020 Rushikesh Butley
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# cgrputil
2+
[![image](https://img.shields.io/pypi/v/py-package-template.svg)](https://pypi.org/project/cgrputil/)
3+
[![Downloads](https://pepy.tech/badge/cgrputil)](https://pepy.tech/project/cgrputil)
4+
5+
**cgrputil** is python library to calculate cpu utilisation of code over specific time interval inside docker container.
6+
7+
Usage of the library can vary, Eg. Think about the use case where you want to know how much specific function consume a cpu over specific time and on the basis of that you want to bill the client or draw charts to understand the growing trend of function.
8+
9+
I was struggling with the same problem, & searched all over internet. But i didnt get anything solid,
10+
& all of the docs, the repos i viewed, code i read, almost pointed to docker repo which had calculation formula.
11+
So seeing the problem & number of resources available for such typical problem i decided to create the library.
12+
I wanted solution which is very easy to use and focus on giving end result, so i tried it making as simple as i could.
13+
I hope this helps :) .
14+
15+
## Installation
16+
17+
Use the package manager [pip](https://pip.pypa.io/en/stable/) to install cgrputil.
18+
19+
```bash
20+
pip install cgrputil
21+
```
22+
23+
## Usage
24+
25+
```python
26+
import cgrputil
27+
28+
'''
29+
Class instantiation expects default param, which will be return in case of any failure
30+
while reading, calculating cpu usage. We suggest to send this value as maxm number of
31+
cores allocated to container.
32+
This value can be pass in two ways either passing it while class instantiation, which is in
33+
example below i.e 3 or setting it inside env variable `RES_CPU_LIMIT`.
34+
Library will fist check for the value pass and then for the env variable, on of them is mandatory
35+
else exception will be raised.
36+
'''
37+
cpu = cgrputil.cpuutilisation(3)
38+
39+
40+
#From this point, it will calculate the cpu utilisation
41+
cpu.start_time()
42+
43+
activity_you_want_to_calculate_cpu_usage()
44+
45+
#Until this point cpu usage will be calculated
46+
cpu.end_time()
47+
48+
'''
49+
This will return the aprox cpu cores used while executing in between start time and end time.
50+
Return format is float. So if it returns 2.3 that means above function have consumed 2.3 cores
51+
out of all the cores available on the host or you can say 2.3 cores out of 3
52+
cores (3 assuming maximum cores allocated to container).
53+
54+
Function returns two values, first is list of errs or exception occured while calculating cpu usage.
55+
Second the actual usage value. If the errs list is not empty it will most likely to return the
56+
default value passed while initiating the class. Ex. cpu = cgrputil.cpuutilisation(3) which is 3 here.
57+
'''
58+
errs, cores_used = cpu.get_core_utilisation()
59+
if errs is not None:
60+
for err in errs:
61+
print(err)
62+
print('Utilisation : ', cores_used)
63+
else:
64+
print('Utilisation : ', cores_used)
65+
```
66+
67+
## Contributing
68+
Pull requests are welcome. For any changes, please open an issue first to discuss what you would like to change.
69+
70+
Please make sure to update tests as appropriate.

‎cgrputil/__init__.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
'''
2+
Cgrputil is the library for people, who wants to calculate the cpu utilisation
3+
of container for period of time without taking heachache ;).
4+
More in README.md on how to use it effectively.
5+
'''
6+
from .cgrputil import *

‎cgrputil/cgrputil.py‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
import os
2+
import traceback
3+
import time
4+
5+
CGRP_CPU_LIMIT = 'RES_CPU_LIMIT'
6+
7+
class cpuutilisation:
8+
def __init__(self, cpu_limit=None):
9+
'''
10+
Initate the object for calculating cpu utilisation.
11+
If it fails to calculate, in any case it will return value pass
12+
while initiating the class or it will try to get value from env variable
13+
RES_CPU_LIMIT. One of them is mandatory.
14+
'''
15+
self.cgrp_init_cycles, self.cgrp_end_cycles = 0, 0
16+
self.sys_init_cycles, self.sys_end_cycles = 0, 0
17+
self.no_of_host_cores = 0
18+
self.errors = []
19+
if cpu_limit is not None:
20+
self.cgrp_cpu_limit = cpu_limit
21+
else:
22+
try:
23+
self.cgrp_cpu_limit = os.environ[CGRP_CPU_LIMIT]
24+
except Exception as e:
25+
raise Exception('No cpu limit specified while initiating class or set in env variable {}'.format(e))
26+
def get_no_of_cores_host(self):
27+
'''
28+
Get total number of cores on host machine
29+
'''
30+
cpu_info = '/proc/cpuinfo'
31+
try:
32+
with open(cpu_info, 'r') as cpus:
33+
lscpu = cpus.readlines()
34+
cores = 0
35+
for cpu in lscpu:
36+
if 'processor' in cpu: #count as core if processor exists in line
37+
cores += 1
38+
return int(cores)
39+
except Exception as e:
40+
err = 'Unable to read number of Cores from System :- {}'.format(e)
41+
self.errors.append(err)
42+
# print(err)
43+
# traceback.print_exc()
44+
def get_cpu_usage_cgrp(self):
45+
'''
46+
Read the cpu cycles of cgroup,
47+
'''
48+
file = '/sys/fs/cgroup/cpu,cpuacct/cpuacct.usage_percpu'
49+
try:
50+
with open(file, 'r') as usage_percpu:
51+
all_cores = usage_percpu.readlines()[0].split()
52+
total_cgrpcpu_cycles = 0
53+
for core in all_cores: total_cgrpcpu_cycles += int(core)
54+
return total_cgrpcpu_cycles
55+
except Exception as e:
56+
err = 'Exception occured while get per cpu cgrp usage :- {}'.format(e)
57+
self.errors.append(err)
58+
#print(err)
59+
# traceback.print_exc()
60+
61+
def get_cpu_usage_host(self):
62+
'''
63+
Read the cpu cycles from of host.
64+
'''
65+
try:
66+
sys_stat = '/proc/stat'
67+
with open(sys_stat, 'r') as sys:
68+
stat = sys.readlines()[0].split() #system wide cpu usage all cores
69+
system_stat = 0
70+
for x in stat[1:]:
71+
system_stat += int(x)
72+
return int(system_stat)
73+
except Exception as e:
74+
err = 'Exception occured duing getting system usage cycle :- {}'.format(e)
75+
self.errors.append(err)
76+
# print(err)
77+
# traceback.print_exc()
78+
def get_core_utilisation(self):
79+
'''
80+
> If it fails to calculate the usage, it return the default value defined in
81+
passed while initiating the class, core_utilisation else value from env variable 'RES_CPU_LIMIT'.
82+
Calculates the delta of cpu usage between start time & end time.
83+
returns the Float value. If total cores in system are 10 and it returns
84+
2.3 that means, function have used 2.3 cores on an average, which translates
85+
to around 2300m in terms, if one checks in docker stats.
86+
'''
87+
try:
88+
self.no_of_host_cores = self.get_no_of_cores_host()
89+
cores_used = float((self.cgrp_end_cycles - self.cgrp_init_cycles)/(self.sys_end_cycles - self.sys_init_cycles) * self.no_of_host_cores * 100)/1000000000
90+
return None, float('{:.2f}'.format(cores_used))
91+
except Exception as e:
92+
err = 'Issue occured in cpu utilisation calculation, due to previous errors returning default maximum value set:- {}'.format(self.cgrp_cpu_limit)
93+
self.errors.append(err)
94+
# print(err)
95+
# print('Core utilisation Calculation error, values passed :- {}'.format(vars(cpuutilisation(self.cgrp_cpu_limit))))
96+
# traceback.print_exc()
97+
return self.errors, float(self.cgrp_cpu_limit)
98+
99+
def start_time(self):
100+
'''
101+
Get the start of cgroup cycles and system cycles at point, when this function is called.
102+
Example Usage:
103+
Initate the core class, with default limit if it fails to calculate. Ex. 3
104+
core = cpuutilisation(3)
105+
Call this function before the start of function, you want to calculate cpu usage for
106+
core.start_time()
107+
actual_function_call_here_for_which_cpu_util_need_to_be_calculated()
108+
'''
109+
self.cgrp_init_cycles = self.get_cpu_usage_cgrp()
110+
self.sys_init_cycles = self.get_cpu_usage_host()
111+
112+
def end_time(self):
113+
'''
114+
Get the end of cgroup cycles and system cycles at point, when this function is called.
115+
Example Usage:
116+
Call this at the end of
117+
actual_function_call_here_for_which_cpu_util_need_to_be_calculated()
118+
core.end_time()
119+
Initate the core class, with default limit if it fails to calculate. Ex. 3
120+
core = cpuutilisation(3)
121+
Call this function before the start of function, you want to calculate cpu usage for
122+
core.start_time()
123+
actual_function_call_here_for_which_cpu_util_need_to_be_calculated()
124+
core.end_time()
125+
'''
126+
127+
self.cgrp_end_cycles = self.get_cpu_usage_cgrp()
128+
self.sys_end_cycles = self.get_cpu_usage_host()
129+
130+
def main():
131+
core = cpuutilisation(3)
132+
core.start_time()
133+
time.sleep(3)
134+
#execute function here
135+
core.end_time()
136+
errs, cores = core.get_core_utilisation()
137+
if errs is not None:
138+
for err in errs:
139+
print(err)
140+
print('\n Utilisation : ', cores)
141+
else:
142+
print('Utilisation : ', cores)
143+
if __name__ == "__main__":
144+
main()

‎setup.py‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
import setuptools
2+
3+
with open("README.md", "r") as fh:
4+
long_description = fh.read()
5+
6+
setuptools.setup(
7+
name="cgrputil",
8+
version="1.0.1",
9+
author="Rushikesh Butley",
10+
author_email="rushikeshbutley@gmail.com",
11+
description="Package to calculate cpu utilisation in cgroups",
12+
long_description=long_description,
13+
long_description_content_type="text/markdown",
14+
url="https://github.com/rushi47/cgrputil",
15+
packages=setuptools.find_packages(),
16+
license="MIT",
17+
classifiers=[
18+
"Environment :: Console",
19+
"Environment :: Other Environment",
20+
"Environment :: Plugins",
21+
"Intended Audience :: Developers",
22+
"Intended Audience :: System Administrators",
23+
"Operating System :: POSIX :: Linux",
24+
"Operating System :: Unix",
25+
"Programming Language :: Python :: 2.7",
26+
"Programming Language :: Python :: 3",
27+
"Topic :: System",
28+
],
29+
python_requires='>=2.7',
30+
data_files=[('LICENSE', ['LICENSE']),
31+
('README', ['README.md']),
32+
]
33+
)

0 commit comments

Comments
 (0)