Skip to content

Commit 93b63af

Browse files
emmagiffordemmagifford
authored
/complete-your-testing (#125)
* /complete-your-testing * Re-ordered section headings * Links on /using-nhs-notify parent page and amended new section header
1 parent 2901c0f commit 93b63af

File tree

6 files changed

+225
-16
lines changed

6 files changed

+225
-16
lines changed

‎docs/_data/sidebar-section-order.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,4 @@ sections:
77
- "Accessing NHS Notify"
88
- "Writing a message"
99
- "Sending a message"
10+
- "Going live"

‎docs/pages/get-started/onboard-with-notify.md‎

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -168,18 +168,9 @@ If you're sending campaign or event based messages, you need to create and submi
168168

169169
## 5. Complete integration testing
170170

171-
You'll need to complete integration testing to make sure you're ready to start sending messages with NHS Notify.
171+
You'll need to [complete integration testing]({% link pages/using-nhs-notify/complete-your-testing.md %}) to make sure you're ready to start sending messages with NHS Notify.
172172

173-
You'll be responsible for:
174-
175-
- assigning someone to manage and complete your testing
176-
- completing testing in our integration environment
177-
- how long your testing will take
178-
- specific NHS App message testing (if you're using the NHS App)
179-
180-
Your onboarding manager will share advice on how to carry out testing when you reach this step.
181-
182-
You can start testing earlier using our [Sandbox environment](https://digital.nhs.uk/developer/api-catalogue/nhs-notify#overview--environments-and-testing) or by using [free-text inputs with NHS Notify API](https://digital.nhs.uk/developer/api-catalogue/nhs-notify#overview--enable-users-to-write-and-send-messages-from-your-software) or [NHS Notify MESH](https://digital.nhs.uk/developer/api-catalogue/nhs-notify-mesh/set-up-nhs-notify-mesh#write-your-messages).
173+
If you're sending NHS App messages, you'll also need to [complete NHS App testing]({% link pages/using-nhs-notify/complete-your-testing.md %}).
183174

184175
## 6. Apply to go live
185176

@@ -210,7 +201,7 @@ Your onboarding manager will ask for your PROD Mesh Mailbox ID.
210201

211202
Once we’ve received your details, you’ll need to:
212203

213-
- complete smoke testing, with support from NHS Notify and your technical team
204+
- [complete smoke testing]({% link pages/using-nhs-notify/complete-your-testing.md %}), with support from NHS Notify and your technical team
214205
- discuss your rollout or implementation plan with us
215206

216207
We will:

‎docs/pages/using-nhs-notify/approve-your-messages.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@
55
layout: page
66
title: Approve your messages before we send them
77
parent: Using NHS Notify
8-
nav_order: 6
8+
nav_order: 1
99
permalink: /using-nhs-notify/approve-your-messages
10-
section: Sending a message
10+
section: Going live
1111
---
1212

1313
When you're ready to send messages to recipients, you'll need to approve final proofs of your messages. You'll do this every time you provide new templates.

‎docs/pages/using-nhs-notify/complete-your-testing.md‎

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
---
2+
# Feel free to add content and custom Front Matter to this file.
3+
# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
4+
5+
layout: page
6+
title: Complete your testing
7+
parent: Using NHS Notify
8+
nav_order: 2
9+
permalink: /using-nhs-notify/complete-your-testing
10+
section: Going live
11+
---
12+
You'll need to carry out testing to make sure you're ready to start sending messages to recipients with NHS Notify.
13+
14+
You're responsible for:
15+
16+
- leading your testing
17+
- completing integration testing in our integration (INT) environment
18+
- how long testing will take
19+
- deciding when to carry out integration testing
20+
21+
You need to:
22+
23+
- [complete your integration testing](#how-to-carry-out-integration-testing).<!-- markdownlint-disable-line -->
24+
- [complete NHS App testing](#nhs-app-integration-testing) if you're sending NHS App messages
25+
- [complete production smoke testing](#production-smoke-testing) just before you go live
26+
27+
We need specific evidence for your NHS App testing.
28+
29+
To complete integration testing, you need to give us your final templates and routing plans.
30+
31+
You can start testing earlier using:
32+
33+
- our [Sandbox environment](https://digital.nhs.uk/developer/api-catalogue/nhs-notify#overview--environments-and-testing)
34+
- [free-text inputs with NHS Notify API](https://digital.nhs.uk/developer/api-catalogue/nhs-notify#overview--enable-users-to-write-and-send-messages-from-your-software) or [NHS Notify MESH](https://digital.nhs.uk/developer/api-catalogue/nhs-notify-mesh/set-up-nhs-notify-mesh#write-your-messages) - our [routing plan IDs](https://digital.nhs.uk/developer/api-catalogue/nhs-notify#overview--making-your-request-to-send-messages-from-your-software) can be helpful if you've not got your own set up yet
35+
36+
## Before you start
37+
38+
Make sure your team has someone to:
39+
40+
- test (like a test engineer, developer or someone who can send API or MESH requests)
41+
- manage your testing and report on progress (like a test manager or delivery manager)
42+
43+
Make sure you've [prepared your integration](https://notify.nhs.uk/get-started/onboard-with-nhs-notify#2-prepare-your-integration).
44+
45+
## How to carry out integration testing
46+
47+
### 1. Prepare for testing
48+
49+
To prepare for testing, you need to:
50+
51+
- arrange your own Personal Demographics Service (PDS) data - contact your integration manager if you have any questions
52+
- provide us with your [test personalisation data](https://notify.nhs.uk/using-nhs-notify/personalisation#providing-example-data)
53+
- ask your onboarding manager to set up your NHS App test data (if you're using the NHS App)
54+
55+
### 2. Confirm your connection to the integration environment
56+
57+
You need to show you've connected to our INT environment by sending an API request or a MESH message. This will trigger the creation of a message in INT.
58+
59+
You'll know your connection is working when you can successfully run tests in the INT environment.
60+
61+
{% include components/inset-text.html
62+
text='If you’re using free-text inputs and NHS Notify’s dedicated routing plans, you do not need to do any more steps for integration testing.
63+
64+
Next, carry out [NHS App testing](#nhs-app-integration-testing) (if you’re sending NHS App messages) and [production smoke testing](#production-smoke-testing).'
65+
%}
66+
67+
### 3. Approve proofs of your messages
68+
69+
You need to [approve final proofs of your messages]({% link pages/using-nhs-notify/approve-your-messages.md %}). This is to ensure that NHS Notify sends your messages exactly as you expect your recipients to get them.
70+
71+
### 4. Wait for NHS Notify to set up your routing plans
72+
73+
NHS Notify will build your routing plans in the INT environment.
74+
75+
### 5. Check your templates and routing plans work as you expect
76+
77+
You need to check that we've set up your templates and routing plans in line with your needs.
78+
79+
It's up to you to decide how to do this. We recommend making sure that:
80+
81+
- the message channels in your routing plan are in the correct order
82+
- the correct templates are used in the correct routing plans, for example, one plan might be for invitations and one might be for reminders
83+
84+
{% include components/details.html
85+
heading='If you need help getting test evidence'
86+
text='Let your onboarding manager know 2 weeks in advance if you need support to carry out this step.
87+
88+
We can get evidence of your test outputs for emails and text messages if you give us your test data and request IDs. Find out about [NHS App message test outputs](#nhs-app-integration-testing).'
89+
%}
90+
91+
{% include components/details.html
92+
heading='If you need support'
93+
text='Contact your onboarding manager if you need support or have any problems with your testing. Tell us the following details:
94+
95+
- a description of the request and what you expected to happen
96+
- the date and time of the request
97+
- the routing plan ID, also known as the sending group ID
98+
- the request ID that the API batch-message endpoint returned, if you used NHS Notify API
99+
- the request item ID that the API message endpoint returned, if you used NHS Notify API
100+
- the request ID that MESH returned, if you used NHS Notify MESH'
101+
%}
102+
103+
## NHS App integration testing
104+
105+
You'll need to use a special test version of the NHS App and test user accounts to check how your messages will appear in the NHS App.
106+
107+
We need to see evidence of these messages in the test version of the NHS App.
108+
109+
### 1. Provide details for the test app
110+
111+
Give your onboarding manager the following details of the people on your team who need access to the test version of the NHS App:
112+
113+
- their email addresses - these can be personal details if they're testing on personal devices
114+
- whether they have Android or Apple devices
115+
116+
We'll then send you a link to download the app from the [Apple Store](https://www.apple.com/uk/store) or [Google Play](https://play.google.com/store/apps?hl=en).
117+
118+
### 2. We'll send you test user account details
119+
120+
We'll give you sign in details for 3 test user accounts.
121+
122+
### 3. Sign in with test accounts
123+
124+
Sign in with the 3 test user accounts on a mobile device that we approved for the test NHS App,
125+
126+
### 4. Send test messages
127+
128+
Wait 24 hours after you first sign in.
129+
130+
Then send test messages using requests through NHS Notify API or NHS Notify MESH. Use the test users' NHS numbers.
131+
132+
### 5. Receive the notification
133+
134+
The notification will be delivered to the test NHS App on the phone of the person who's signed in to the test accounts.
135+
136+
### 6. Show evidence of your NHS App testing
137+
138+
You'll need to show evidence of the results with a screen recording or on a video call with us. You can send us the screen recordings in one video or in separate videos if this is easier.
139+
140+
We can provide extra evidence from our database if you request this.
141+
142+
Your test results should show certain results for each test patient account:
143+
144+
#### P9 level with NHS App available, identity verified and push notifications enabled
145+
146+
- NHS App receives the message
147+
- The message can be read
148+
- Callback goes to NHS Notify to say that the message was read (if you've set up callbacks)
149+
- Any embedded links work correctly
150+
151+
#### P9 level with NHS App available, identity verified and push notifications disabled
152+
153+
- NHS App receives the message
154+
- The message is treated as if it will not be read
155+
- Rejected callback goes to NHS Notify (if you've set up callbacks)
156+
157+
#### P5 level with identity not verified for NHS App
158+
159+
- NHS App does not receive the message
160+
- Rejected callback goes to NHS Notify (if you've set up callbacks)
161+
- The next available message channel in the routing plan is triggered
162+
163+
## Production smoke testing
164+
165+
Smoke testing helps us to make sure:
166+
167+
- your new integration is working in the production environment
168+
- your production configuration is set up correctly in line with the INT set up
169+
170+
For letters, smoke testing will take place 24 hours before you go live with NHS Notify. For other message channels, it can take place on the same day that you go live.
171+
172+
The output of the smoke tests will be on our test devices. This means you will not see the messages at this stage.
173+
174+
We do not check any links in your message templates during smoke testing. You can check your links work correctly during [proofing]({% link pages/using-nhs-notify/approve-your-messages.md %}) (if you're using templates rather than free-text inputs).
175+
176+
### Before your smoke test
177+
178+
We'll:
179+
180+
- draft a production smoke test plan
181+
- prepare your test data
182+
- arrange a smoke test call with you
183+
184+
### During your smoke test call
185+
186+
We'll share test data with you, including NHS numbers and routing plans.
187+
188+
You'll then use this data to send requests using either NHS Notify API or NHS Notify MESH.
189+
190+
If there are any issues, we'll work through these with you.
191+
192+
We'll record the results of the smoke test and confirm once you're ready to go live and send messages to recipients.
193+
194+
Here's what to expect for each channel:
195+
196+
#### NHS App
197+
198+
We'll check the NHS App has received the notification on the test device. We usually expect it to come through after a few minutes.
199+
200+
#### Email
201+
202+
We'll check our test inbox to make sure the email was received. We usually expect it to come through after a few minutes.
203+
204+
#### Text message
205+
206+
We'll check that the test device received the text message. We usually expect it to come through after a few minutes.
207+
208+
#### Letters
209+
210+
After you've triggered the request, it will go into the letter supplier's overnight queue. The day after your smoke test, we'll confirm that they received it.
211+
212+
Your letters will not be printed and posted at this stage.

docs/pages/using-nhs-notify/nhs-app-for-messages-new-digital-first-strategy.md renamed to docs/pages/using-nhs-notify/nhs-digital-first-strategy.md

File renamed without changes.

‎docs/pages/using-nhs-notify/using-nhs-notify.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,12 @@ This guidance is to help teams understand how to use NHS Notify.
3737
- [Tell recipients who your messages are from]({% link pages/using-nhs-notify/tell-recipients-who-your-messages-are-from.md %})
3838
- [Delivery times]({% link pages/using-nhs-notify/delivery-times.md %})
3939
- [Tell us who you want to message]({% link pages/using-nhs-notify/tell-us-who-you-want-to-message.md %})
40-
- [Message plans]({% link pages/using-nhs-notify/message-plans.md %})
40+
- [Routing plans]({% link pages/using-nhs-notify/message-plans.md %})
4141
- [Recipients you cannot message]({% link pages/using-nhs-notify/recipients-you-cannot-message.md %})
42-
- [Approve your messages before we send them]({% link pages/using-nhs-notify/approve-your-messages.md %})
4342
- [Message, channel and supplier status]({% link pages/using-nhs-notify/message-status.md %})
43+
- [NHS App for messages: new NHS digital-first strategy]({% link pages/using-nhs-notify/nhs-digital-first-strategy.md %})
44+
45+
## Going live
46+
47+
- [Approve your messages before we send them]({% link pages/using-nhs-notify/approve-your-messages.md %})
48+
- [Complete your testing]({% link pages/using-nhs-notify/complete-your-testing.md %})

0 commit comments

Comments
 (0)