Skip to content

Commit 8d122f1

Browse files
authored
Merge pull request #580 from AzureAD/benchmark
Guarding against perf regression for acquire_token_for_client()
2 parents ca713b4 + a3e6017 commit 8d122f1

File tree

5 files changed

+120
-5
lines changed

5 files changed

+120
-5
lines changed

.github/workflows/python-package.yml

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,15 +42,40 @@ jobs:
4242
python -m pip install --upgrade pip
4343
python -m pip install flake8 pytest
4444
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
45+
- name: Test with pytest
46+
run: pytest --benchmark-skip
4547
- name: Lint with flake8
4648
run: |
4749
# stop the build if there are Python syntax errors or undefined names
4850
#flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
4951
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
5052
#flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
51-
- name: Test with pytest
53+
54+
cb:
55+
# Benchmark only after the correctness has been tested by CI,
56+
# and then run benchmark only once (sampling with only one Python version).
57+
needs: ci
58+
runs-on: ubuntu-latest
59+
steps:
60+
- uses: actions/checkout@v2
61+
- name: Set up Python 3.9
62+
uses: actions/setup-python@v2
63+
with:
64+
python-version: 3.9
65+
- name: Install dependencies
5266
run: |
53-
pytest
67+
python -m pip install --upgrade pip
68+
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
69+
- name: Run benchmark
70+
run: pytest --benchmark-only --benchmark-json benchmark.json --log-cli-level INFO tests/test_benchmark.py
71+
- name: Render benchmark result
72+
uses: benchmark-action/github-action-benchmark@v1
73+
with:
74+
tool: 'pytest'
75+
output-file-path: benchmark.json
76+
fail-on-alert: true
77+
- name: Publish Gibhub Pages
78+
run: git push origin gh-pages
5479

5580
cd:
5681
needs: ci

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Microsoft Authentication Library (MSAL) for Python
22

3-
| `dev` branch | Reference Docs | # of Downloads per different platforms | # of Downloads per recent MSAL versions |
4-
|---------------|---------------|----------------------------------------|-----------------------------------------|
5-
[![Build status](https://github.com/AzureAD/microsoft-authentication-library-for-python/actions/workflows/python-package.yml/badge.svg?branch=dev)](https://github.com/AzureAD/microsoft-authentication-library-for-python/actions) | [![Documentation Status](https://readthedocs.org/projects/msal-python/badge/?version=latest)](https://msal-python.readthedocs.io/en/latest/?badge=latest) | [![Downloads](https://pepy.tech/badge/msal)](https://pypistats.org/packages/msal) | [![Download monthly](https://pepy.tech/badge/msal/month)](https://pepy.tech/project/msal)
3+
| `dev` branch | Reference Docs | # of Downloads per different platforms | # of Downloads per recent MSAL versions | Benchmark Diagram |
4+
|:------------:|:--------------:|:--------------------------------------:|:---------------------------------------:|:-----------------:|
5+
[![Build status](https://github.com/AzureAD/microsoft-authentication-library-for-python/actions/workflows/python-package.yml/badge.svg?branch=dev)](https://github.com/AzureAD/microsoft-authentication-library-for-python/actions) | [![Documentation Status](https://readthedocs.org/projects/msal-python/badge/?version=latest)](https://msal-python.readthedocs.io/en/latest/?badge=latest) | [![Downloads](https://static.pepy.tech/badge/msal)](https://pypistats.org/packages/msal) | [![Download monthly](https://static.pepy.tech/badge/msal/month)](https://pepy.tech/project/msal) | [📉](https://azuread.github.io/microsoft-authentication-library-for-python/dev/bench/)
66

77
The Microsoft Authentication Library for Python enables applications to integrate with the [Microsoft identity platform](https://aka.ms/aaddevv2). It allows you to sign in users or apps with Microsoft identities ([Azure AD](https://azure.microsoft.com/services/active-directory/), [Microsoft Accounts](https://account.microsoft.com) and [Azure AD B2C](https://azure.microsoft.com/services/active-directory-b2c/) accounts) and obtain tokens to call Microsoft APIs such as [Microsoft Graph](https://graph.microsoft.io/) or your own APIs registered with the Microsoft identity platform. It is built using industry standard OAuth2 and OpenID Connect protocols
88

requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
.
22
python-dotenv
3+
pytest-benchmark

tests/simulator.py

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
"""Simulator(s) that can be used to create MSAL instance
2+
whose token cache is in a certain state, and remains unchanged.
3+
This generic simulator(s) becomes the test subject for different benchmark tools.
4+
5+
For example, you can install pyperf and then run:
6+
7+
pyperf timeit -s "from tests.simulator import ClientCredentialGrantSimulator as T; t=T(tokens_per_tenant=1, cache_hit=True)" "t.run()"
8+
"""
9+
import json
10+
import logging
11+
import random
12+
from unittest.mock import patch
13+
14+
import msal
15+
from tests.http_client import MinimalResponse
16+
17+
18+
logger = logging.getLogger(__name__)
19+
20+
21+
def _count_access_tokens(app):
22+
return len(app.token_cache._cache[app.token_cache.CredentialType.ACCESS_TOKEN])
23+
24+
25+
class ClientCredentialGrantSimulator(object):
26+
27+
def __init__(self, number_of_tenants=1, tokens_per_tenant=1, cache_hit=False):
28+
logger.info(
29+
"number_of_tenants=%d, tokens_per_tenant=%d, cache_hit=%s",
30+
number_of_tenants, tokens_per_tenant, cache_hit)
31+
with patch.object(msal.authority, "tenant_discovery", return_value={
32+
"authorization_endpoint": "https://contoso.com/placeholder",
33+
"token_endpoint": "https://contoso.com/placeholder",
34+
}) as _: # Otherwise it would fail on OIDC discovery
35+
self.apps = [ # In MSAL Python, each CCA binds to one tenant only
36+
msal.ConfidentialClientApplication(
37+
"client_id", client_credential="foo",
38+
authority="https://login.microsoftonline.com/tenant_%s" % t,
39+
) for t in range(number_of_tenants)
40+
]
41+
for app in self.apps:
42+
for i in range(tokens_per_tenant): # Populate token cache
43+
self.run(app=app, scope="scope_{}".format(i))
44+
assert tokens_per_tenant == _count_access_tokens(app), (
45+
"Token cache did not populate correctly: {}".format(json.dumps(
46+
app.token_cache._cache, indent=4)))
47+
48+
if cache_hit:
49+
self.run(app=app)["access_token"] # Populate 1 token to be hit
50+
expected_tokens = tokens_per_tenant + 1
51+
else:
52+
expected_tokens = tokens_per_tenant
53+
app.token_cache.modify = lambda *args, **kwargs: None # Cache becomes read-only
54+
self.run(app=app)["access_token"]
55+
assert expected_tokens == _count_access_tokens(app), "Cache shall not grow"
56+
57+
def run(self, app=None, scope=None):
58+
# This implementation shall be as concise as possible
59+
app = app or random.choice(self.apps)
60+
#return app.acquire_token_for_client([scope or "scope"], post=_fake)
61+
return app.acquire_token_for_client(
62+
[scope or "scope"],
63+
post=lambda url, **kwargs: MinimalResponse( # Using an inline lambda is as fast as a standalone function
64+
status_code=200, text='''{
65+
"access_token": "AT for %s",
66+
"token_type": "bearer"
67+
}''' % kwargs["data"]["scope"],
68+
))
69+

tests/test_benchmark.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
from tests.simulator import ClientCredentialGrantSimulator as CcaTester
2+
3+
# Here come benchmark test cases, powered by pytest-benchmark
4+
# Func names will become diag names.
5+
def test_cca_1_tenant_with_10_tokens_per_tenant_and_cache_hit(benchmark):
6+
tester = CcaTester(tokens_per_tenant=10, cache_hit=True)
7+
benchmark(tester.run)
8+
9+
def test_cca_many_tenants_with_10_tokens_per_tenant_and_cache_hit(benchmark):
10+
tester = CcaTester(number_of_tenants=1000, tokens_per_tenant=10, cache_hit=True)
11+
benchmark(tester.run)
12+
13+
def test_cca_1_tenant_with_10_tokens_per_tenant_and_cache_miss(benchmark):
14+
tester = CcaTester(tokens_per_tenant=10, cache_hit=False)
15+
benchmark(tester.run)
16+
17+
def test_cca_many_tenants_with_10_tokens_per_tenant_and_cache_miss(benchmark):
18+
tester = CcaTester(number_of_tenants=1000, tokens_per_tenant=10, cache_hit=False)
19+
benchmark(tester.run)
20+

0 commit comments

Comments
 (0)