Skip to content

Commit 356ba58

Browse files
JFU-NAVA-PBCJFU-NAVA-PBC
authored
jimfuqian/BB2-3239-fix-CI-check-selenium-tests (#1271)
* supply the ENV var needed by selenium test case: USE_NEW_PERM_SCREEN * try headless on CI * use webdriver local. * fix linting * unify the webdriver path on local and CI run context * make CI selenium tests testing again. * add docker platform spec., correct artifacts names. * add step to trouble shoot, step sh hanging... * add step to trouble shoot, step sh hanging... * trouble shoot: sh in step is hanging. * replace make with pip install etc. * install newrelic 8.8.0 * remove test step... * satisfy deps that conflict... * satisfy deps that conflict... * CI docker: install python 3.11.9 on top of chromium base. * install sqlite3 on CI docker. * fix the sqlite issue. * continue add missing native libs... * create python link to accomodate some scripts * adjust logging integration tests setup on CI node. * adjust details in steps... * use selenium headless on CI * fix linting. * fix logging integration tests schemas (selenium related) * quote out logit tests to trouble shoot * new screen env var needed for app and user mgmt tests * add pytest -s to see stdout... * increase headless window height. * create ui change to trouble shoot. * cleanup... * cleanup... * start selenium hub in pipeline * start selenium-server-standalone hub in pipeline... * locate the selenium jar * verify hub is at localhost. * temp change hub url to localhost. * customize selenium grid host name for CI context. * set chromium location. * add tracing for trouble shoot * replace selenium docker image that is archived with new one. * try DinD in pipeline. * install dependencies in CI ECR image * comment out python packages install - since it is installed in the ECR CI image - save time?. * revert CI ECR image to p311 * set pipeline selenium chromedriver location. * refactor * fix driver ver check step * trouble shoot rendering timeout at hicn field... * trouble shoot.. * fix add_argu syntax... * size... * rearrange form fields of mock login... * try slsx since all env vars needed are set at the top of the pipeline. * local script cleanup, use mock login on CI * use mock and fix var syntax * dump bb2 log for diagnose info... * diagnose... * diagnose... * diagnose... * diagnose... * start mslsx needed by MSLSX mode... * subshell mslsx start and moving on... * fix url for mslsx calls * fix url for medicare callback * trigger CI check * added missed user info mslsx ENV var in pipeline * fix proto method in user info url * clean up and finalize files naming... * fix pipeline ecr reference etc.
1 parent 89e9075 commit 356ba58

14 files changed

+267
-61
lines changed

‎Dockerfile‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,5 @@ WORKDIR /code
88
RUN python -m venv /tmp/venv
99
RUN . /tmp/venv/bin/activate
1010
ENV PATH="/tmp/venv/bin:${PATH}"
11-
RUN pip install --upgrade pip
12-
RUN pip install --upgrade pip-tools
13-
RUN pip install --upgrade setuptools
14-
RUN pip install -r requirements/requirements.dev.txt --no-index --find-links ./vendor/
11+
RUN pip install --upgrade pip pip-tools setuptools
12+
RUN pip install -r requirements/requirements.dev.txt --no-index --find-links ./vendor/

‎Dockerfile.selenium‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,18 @@
1-
FROM seleniarm/standalone-chromium
1+
FROM selenium/standalone-chromium
22

3-
ENV PYTHONUNBUFFERED 1
43
USER root
5-
#RUN apt-get update ; apt-get install -yq git curl libpq-dev libffi-dev
4+
65
RUN apt-get update ; apt-get install -yq python3 python3-venv
76
RUN ln -s /usr/bin/python3 /usr/local/bin/python
8-
RUN useradd -m -s /bin/bash DEV
9-
USER DEV
7+
8+
# switch to existing seluser from selenium docker
9+
USER seluser
10+
1011
ADD . /code
1112
WORKDIR /code
1213
RUN python -m venv /tmp/venv
1314
RUN . /tmp/venv/bin/activate
1415
ENV PATH="/tmp/venv/bin:${PATH}"
16+
1517
RUN pip3 install --upgrade pip
1618
RUN pip3 install selenium pytest debugpy jsonschema python-dateutil
17-

‎Dockerfiles/Dockerfile.selenium-jenkins-python311-plus-chromedriver‎

Whitespace-only changes.

‎Jenkinsfiles/Jenkinsfile.cbc-run-multi-pr-checks-w-selenium‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,7 @@ pipeline {
9999
steps{
100100
sh """
101101
. venv/bin/activate
102-
pytest ./apps/integration_tests/logging_tests.py::TestLoggings::test_auth_fhir_flows_logging
102+
USE_NEW_PERM_SCREEN=true pytest ./apps/integration_tests/logging_tests.py::TestLoggings::test_auth_fhir_flows_logging
103103
"""
104104
}
105105
}
@@ -134,7 +134,7 @@ pipeline {
134134
sh 'echo "RUN selenium tests - testclient based authorization flow tests and data flow tests"'
135135
sh """
136136
. venv/bin/activate
137-
pytest ./apps/integration_tests/selenium_tests.py
137+
USE_NEW_PERM_SCREEN=true pytest ./apps/integration_tests/selenium_tests.py
138138
"""
139139
}
140140
}

‎Jenkinsfiles/Jenkinsfile.cbc-run-multi-pr-checks-w-selenium-chromium‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
pipeline {
2+
agent {
3+
kubernetes {
4+
defaultContainer "bb2-cbc-build-selenium-python311-chromium"
5+
yamlFile "Jenkinsfiles/cbc-pod-deployment-config-w-selenium-p311-chromium.yaml"
6+
}
7+
}
8+
9+
environment {
10+
DJANGO_LOG_JSON_FORMAT_PRETTY = true
11+
DJANGO_SETTINGS_MODULE = "hhs_oauth_server.settings.logging_it"
12+
OAUTHLIB_INSECURE_TRANSPORT = true
13+
DJANGO_SECURE_SESSION = false
14+
DJANGO_FHIR_CERTSTORE = "./certstore"
15+
// use mock login - safer, faster
16+
USE_MSLSX = true
17+
DJANGO_MEDICARE_SLSX_REDIRECT_URI = "http://localhost:8000/mymedicare/sls-callback"
18+
DJANGO_MEDICARE_SLSX_LOGIN_URI = "http://localhost:8080/sso/authorize?client_id=bb2api"
19+
DJANGO_SLSX_HEALTH_CHECK_ENDPOINT = "http://localhost:8080/health"
20+
DJANGO_SLSX_TOKEN_ENDPOINT = "http://localhost:8080/sso/session"
21+
DJANGO_SLSX_SIGNOUT_ENDPOINT = "http://localhost:8080/sso/signout"
22+
DJANGO_SLSX_USERINFO_ENDPOINT="http://localhost:8080/v1/users"
23+
DJANGO_SLSX_CLIENT_ID = credentials("bb2-selenium-tests-slsx-client-id")
24+
DJANGO_SLSX_CLIENT_SECRET = credentials("bb2-selenium-tests-slsx-client-secret")
25+
DJANGO_USER_ID_ITERATIONS = credentials("bb2-integration-tests-bfd-iterations")
26+
DJANGO_USER_ID_SALT = credentials("bb2-integration-tests-bfd-salt")
27+
FHIR_CERT = credentials("bb2-integration-tests-bfd-cert")
28+
FHIR_KEY = credentials("bb2-integration-tests-bfd-key")
29+
FHIR_URL = "${params.FHIR_URL}"
30+
HOSTNAME_URL = "http://localhost:8000"
31+
}
32+
33+
parameters {
34+
string(
35+
name: 'FHIR_URL',
36+
defaultValue: "https://prod-sbx.bfd.cms.gov",
37+
description: 'The default FHIR URL for the back end BFD service.'
38+
)
39+
booleanParam(
40+
name: 'RUN_SELENIUM_TESTS',
41+
defaultValue: false,
42+
description: 'Set to true, selenium tests will be run as part of integration tests'
43+
)
44+
}
45+
46+
stages {
47+
stage("SETUP FHIR cert and key") {
48+
steps {
49+
writeFile(file: "${env.DJANGO_FHIR_CERTSTORE}/certstore/ca.cert.pem", text: readFile(env.FHIR_CERT))
50+
writeFile(file: "${env.DJANGO_FHIR_CERTSTORE}/certstore/ca.key.nocrypt.pem", text: readFile(env.FHIR_KEY))
51+
}
52+
}
53+
54+
stage("INSTALL Python Packages") {
55+
steps {
56+
sh """
57+
pip3 install --upgrade pip setuptools wheel
58+
pip3 install -r requirements/requirements.dev.txt --no-index --find-links ./vendor/
59+
"""
60+
}
61+
}
62+
63+
stage("CHECK Flake8 Python Lint/Style") {
64+
steps{
65+
sh """
66+
flake8
67+
"""
68+
}
69+
}
70+
71+
stage("START BB2 server in background") {
72+
when {
73+
expression { params.RUN_SELENIUM_TESTS == true }
74+
}
75+
steps{
76+
sh """
77+
mkdir ./docker-compose/tmp/
78+
(python3 ./dev-local/app.py&) &&
79+
python3 manage.py migrate &&
80+
python3 manage.py create_admin_groups &&
81+
python3 manage.py loaddata scopes.json &&
82+
python3 manage.py create_blue_button_scopes &&
83+
python3 manage.py create_test_user_and_application &&
84+
python3 manage.py create_user_identification_label_selection &&
85+
python3 manage.py create_test_feature_switches &&
86+
(if [ ! -d 'bluebutton-css' ] ; then git clone https://github.com/CMSgov/bluebutton-css.git ; else echo 'CSS already installed.' ; fi) &&
87+
echo 'starting bb2...' &&
88+
(export DJANGO_SETTINGS_MODULE=hhs_oauth_server.settings.logging_it && python3 manage.py runserver 0.0.0.0:8000 > ./docker-compose/tmp/bb2_email_to_stdout.log 2>&1 &)
89+
"""
90+
}
91+
}
92+
93+
stage("RUN logging integration tests") {
94+
when {
95+
expression { params.RUN_SELENIUM_TESTS == true }
96+
}
97+
steps{
98+
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') {
99+
sh """
100+
USE_NEW_PERM_SCREEN=true ON_REMOTE_CI=true pytest -s ./apps/integration_tests/logging_tests.py::TestLoggings::test_auth_fhir_flows_logging
101+
"""
102+
}
103+
sh """
104+
echo '======================'
105+
cat ./docker-compose/tmp/bb2_email_to_stdout.log
106+
echo '======================'
107+
"""
108+
}
109+
}
110+
111+
stage("RUN selenium user and apps management tests") {
112+
when {
113+
expression { params.RUN_SELENIUM_TESTS == true }
114+
}
115+
steps{
116+
sh 'echo "RUN selenium tests - user account and app management tests"'
117+
sh """
118+
USE_NEW_PERM_SCREEN=true ON_REMOTE_CI=true pytest -s ./apps/integration_tests/selenium_accounts_tests.py::TestUserAndAppMgmt::testAccountAndAppMgmt
119+
"""
120+
}
121+
}
122+
123+
stage("RUN integration tests") {
124+
steps{
125+
sh """
126+
python3 runtests.py --integration apps.integration_tests.integration_test_fhir_resources.IntegrationTestFhirApiResources
127+
"""
128+
}
129+
}
130+
131+
stage("RUN Django Unit Tests") {
132+
steps{
133+
sh """
134+
python3 runtests.py
135+
"""
136+
}
137+
}
138+
139+
stage("RUN selenium tests") {
140+
when {
141+
expression { params.RUN_SELENIUM_TESTS == true }
142+
}
143+
steps{
144+
sh 'echo "RUN selenium tests - testclient based authorization flow tests and data flow tests"'
145+
sh """
146+
USE_NEW_PERM_SCREEN=true ON_REMOTE_CI=true pytest -s ./apps/integration_tests/selenium_tests.py
147+
"""
148+
}
149+
}
150+
}
151+
}

‎Jenkinsfiles/cbc-pod-deployment-config-w-selenium-p311-chromium.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
apiVersion: v1
2+
kind: Pod
3+
spec:
4+
containers:
5+
- name: bb2-cbc-build-selenium-python311-chromium
6+
image: "public.ecr.aws/f5g8o1y9/bb2-cbc-build-selenium-python311-chromium:latest"
7+
tty: true
8+
command: ["tail", "-f"]
9+
imagePullPolicy: Always
10+
nodeSelector:
11+
Agents: true

‎apps/integration_tests/log_event_schemas.py‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@
4646
"auth_client_id": {"type": "string"},
4747
"auth_app_id": {"type": "string"},
4848
"auth_app_name": {"pattern": "TestApp"},
49-
"auth_app_data_access_type": {"pattern": "RESEARCH_STUDY"},
49+
"auth_app_data_access_type": {"pattern": "THIRTEEN_MONTH"},
5050
"auth_share_demographic_scopes": {"pattern": "^True$"},
5151
"auth_require_demographic_scopes": {"pattern": "^True$"},
5252
},
@@ -94,7 +94,7 @@
9494
"auth_client_id": {"type": "string"},
9595
"auth_app_id": {"type": "string"},
9696
"auth_app_name": {"pattern": "TestApp"},
97-
"auth_app_data_access_type": {"pattern": "RESEARCH_STUDY"},
97+
"auth_app_data_access_type": {"pattern": "THIRTEEN_MONTH"},
9898
"auth_require_demographic_scopes": {"pattern": "^True$"},
9999
"req_qparam_client_id": {"type": "string"},
100100
"req_qparam_lang": {"type": "string"},
@@ -129,7 +129,7 @@
129129
"auth_client_id": {"type": "string"},
130130
"auth_app_id": {"type": "string"},
131131
"auth_app_name": {"pattern": "TestApp"},
132-
"auth_app_data_access_type": {"pattern": "RESEARCH_STUDY"},
132+
"auth_app_data_access_type": {"pattern": "THIRTEEN_MONTH"},
133133
"auth_require_demographic_scopes": {"pattern": "^True$"},
134134
"path": {"pattern": "/mymedicare/login"},
135135
"request_method": {"pattern": "GET"},
@@ -158,7 +158,7 @@
158158
"auth_client_id": {"type": "string"},
159159
"auth_app_id": {"type": "string"},
160160
"auth_app_name": {"pattern": "TestApp"},
161-
"auth_app_data_access_type": {"pattern": "RESEARCH_STUDY"},
161+
"auth_app_data_access_type": {"pattern": "THIRTEEN_MONTH"},
162162
"auth_crosswalk_action": {"enum": ["R", "C"]},
163163
"auth_require_demographic_scopes": {"pattern": "^True$"},
164164
"req_user_id": {"type": "number"},
@@ -195,7 +195,7 @@
195195
"auth_client_id": {"type": "string"},
196196
"auth_app_id": {"type": "string"},
197197
"auth_app_name": {"pattern": "TestApp"},
198-
"auth_app_data_access_type": {"pattern": "RESEARCH_STUDY"},
198+
"auth_app_data_access_type": {"pattern": "THIRTEEN_MONTH"},
199199
"auth_crosswalk_action": {"enum": ["R", "C"]},
200200
"auth_require_demographic_scopes": {"pattern": "^True$"},
201201
"req_qparam_client_id": {"type": "string"},
@@ -235,13 +235,13 @@
235235
"auth_client_id": {"type": "string"},
236236
"auth_app_id": {"type": "string"},
237237
"auth_app_name": {"pattern": "TestApp"},
238-
"auth_app_data_access_type": {"pattern": "RESEARCH_STUDY"},
238+
"auth_app_data_access_type": {"pattern": "THIRTEEN_MONTH"},
239239
"auth_crosswalk_action": {"enum": ["R", "C"]},
240240
"auth_require_demographic_scopes": {"pattern": "^True$"},
241241
"req_redirect_uri": {"type": "string"},
242242
"req_scope": {"type": "string"},
243243
"req_share_demographic_scopes": {"pattern": "^True$"},
244-
"req_allow": {"pattern": "Allow"},
244+
"req_allow": {"pattern": "Connect"},
245245
"req_user_id": {"type": "integer"},
246246
"req_user_username": {"type": "string"},
247247
"req_fhir_id": {"type": "string"},

‎apps/integration_tests/selenium_cases.py‎

Lines changed: 32 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -576,16 +576,17 @@ class Action(Enum):
576576
"params": [20, By.ID, AUTH_SCREEN_ID_LANG, AUTH_SCREEN_EN_TXT]
577577
},
578578
# now check the expiration info section
579-
{
580-
"display": "Check for authorization screen expire info in English",
581-
"action": Action.CONTAIN_TEXT,
582-
"params": [20, By.ID, AUTH_SCREEN_ID_EXPIRE_INFO, AUTH_SCREEN_EN_EXPIRE_INFO_TXT]
583-
},
584-
{
585-
"display": "Check en_US date format and validate",
586-
"action": Action.CHECK_DATE_FORMAT,
587-
"params": [20, By.ID, AUTH_SCREEN_ID_END_DATE, AUTH_SCREEN_EN_DATE_FORMAT, EN_US]
588-
},
579+
# comment out due to PROD TestApp now is RESEARCH_STUDY which does not have expire info
580+
# {
581+
# "display": "Check for authorization screen expire info in English",
582+
# "action": Action.CONTAIN_TEXT,
583+
# "params": [20, By.ID, AUTH_SCREEN_ID_EXPIRE_INFO, AUTH_SCREEN_EN_EXPIRE_INFO_TXT]
584+
# },
585+
# {
586+
# "display": "Check en_US date format and validate",
587+
# "action": Action.CHECK_DATE_FORMAT,
588+
# "params": [20, By.ID, AUTH_SCREEN_ID_END_DATE, AUTH_SCREEN_EN_DATE_FORMAT, EN_US]
589+
# },
589590
# the 'approve' and 'deny' button click not using locale based text
590591
# so it is lang agnostic
591592
CLICK_AGREE_ACCESS
@@ -644,16 +645,17 @@ class Action(Enum):
644645
"params": [20, By.ID, AUTH_SCREEN_ID_LANG, AUTH_SCREEN_ES_TXT]
645646
},
646647
# now check the expiration info section
647-
{
648-
"display": "Check for authorization screen expire info in Spanish",
649-
"action": Action.CONTAIN_TEXT,
650-
"params": [20, By.ID, AUTH_SCREEN_ID_EXPIRE_INFO, AUTH_SCREEN_ES_EXPIRE_INFO_TXT]
651-
},
652-
{
653-
"display": "Check Spanish date format and validate",
654-
"action": Action.CHECK_DATE_FORMAT,
655-
"params": [20, By.ID, AUTH_SCREEN_ID_END_DATE, AUTH_SCREEN_ES_DATE_FORMAT, ES_ES]
656-
},
648+
# comment out due to on PROD TestApp is RESEARCH_STUDY which does not have expire info
649+
# {
650+
# "display": "Check for authorization screen expire info in Spanish",
651+
# "action": Action.CONTAIN_TEXT,
652+
# "params": [20, By.ID, AUTH_SCREEN_ID_EXPIRE_INFO, AUTH_SCREEN_ES_EXPIRE_INFO_TXT]
653+
# },
654+
# {
655+
# "display": "Check Spanish date format and validate",
656+
# "action": Action.CHECK_DATE_FORMAT,
657+
# "params": [20, By.ID, AUTH_SCREEN_ID_END_DATE, AUTH_SCREEN_ES_DATE_FORMAT, ES_ES]
658+
# },
657659
# the 'approve' and 'deny' button click not using locale based text
658660
# so it is lang agnostic
659661
CLICK_AGREE_ACCESS
@@ -672,16 +674,16 @@ class Action(Enum):
672674
"params": [20, By.ID, AUTH_SCREEN_ID_LANG, AUTH_SCREEN_ES_TXT]
673675
},
674676
# now check the expiration info section
675-
{
676-
"display": "Check for authorization screen expire info in Spanish",
677-
"action": Action.CONTAIN_TEXT,
678-
"params": [20, By.ID, AUTH_SCREEN_ID_EXPIRE_INFO, AUTH_SCREEN_ES_EXPIRE_INFO_TXT]
679-
},
680-
{
681-
"display": "Check Spanish date format and validate",
682-
"action": Action.CHECK_DATE_FORMAT,
683-
"params": [20, By.ID, AUTH_SCREEN_ID_END_DATE, AUTH_SCREEN_ES_DATE_FORMAT, ES_ES]
684-
},
677+
# {
678+
# "display": "Check for authorization screen expire info in Spanish",
679+
# "action": Action.CONTAIN_TEXT,
680+
# "params": [20, By.ID, AUTH_SCREEN_ID_EXPIRE_INFO, AUTH_SCREEN_ES_EXPIRE_INFO_TXT]
681+
# },
682+
# {
683+
# "display": "Check Spanish date format and validate",
684+
# "action": Action.CHECK_DATE_FORMAT,
685+
# "params": [20, By.ID, AUTH_SCREEN_ID_END_DATE, AUTH_SCREEN_ES_DATE_FORMAT, ES_ES]
686+
# },
685687
# the 'approve' and 'deny' button click not using locale based text
686688
# so it is lang agnostic
687689
CLICK_AGREE_ACCESS

0 commit comments

Comments
 (0)