Plugin CleverReach
Create your channel like it is described in the output of the Install-AptecoPSFramework command. In the channel settings integration parameter you have the following modes
| Mode | Upload Data | Tag Receivers | Copy Mailing | Trigger Broadcast | Setup |
|---|---|---|---|---|---|
| Upload and Broadcast | x | x | x | x | Upload and Broadcast |
| Prepare | x | x | x | Upload and Broadcast Integration parameter mode=prepare
|
|
| Tagging | x | x | Retrieve Existing List Names=Truemode=taggingOnly
|
||
| Upload only | x | x | Upload only |
The whole process from uploading data up to the automatic trigger of a copied mailing.
During the process the module will create new local attributes to the new/existing list, upsert data to it, give the receivers a new tag, create a filter/segment, copy a mailing and schedule it a few seconds later.
To create a new list, just enter a name you would like:
To use an existing list, just open the dropdown and optionally filter it and choose a list:
With this mode and the prepare mode you are able to use the preview functionalities. So you can interactively enter data and get a rendered personalised email preview back from CleverReach. It looks like this:
Aufzeichnung.2023-07-03.175052-1.mp4
Added support for tagging for all modes. It works in upload and preview and works like in this video:
Aufzeichnung.2023-07-04.113318-1.mp4
The whole process from uploading data up to the preparation of a copied mailing. The difference to the broadcast is the not scheduled mailing. Response data will still be able to be mapped as all IDs are already created and saved for matching.
Set to "Upload and Broadcast"
Integration parameters like settingsFile=D:\Scripts\CleverReach\PSCleverReachModule\settings.json;mode=prepare
The mechanism for lists is the same as in "Upload and Broadcast"
Upload your data and tag your receivers with a specific tag you can choose of. Please make sure you dont setup Upload Only, otherwise the MessageName/Tagname will not be transferred by PeopleStage.
The mechanism for lists is the same as in "Upload and Broadcast"
Please be aware, that you still need to choose a mailing template, but that does not have an effect for the upload.
The mechanism for lists is the same as in "Upload and Broadcast"
Besides the default commands for PeopleStage functionalities you have additional command you can use straight away after you have imported the module with
Import-Module "AptecoPSFramework" -Verbose
Import-Settings -Path "D:\Scripts\CleverReach\PSCleverReachModule\settings.json"There are commands available for
Get-LocalDeactivated
Get-ReceiversWithTag
Get-Tags
Get-Blocklist
Get-Bounces
Get-GlobalDeactivated
Get-CRGroups
Get-GroupSegments
Get-GroupStats
Get-GroupStatsByRuntime
Remove-TagsAtReceiversImport-Module "C:\Users\Florian\Documents\GitHub\AptecoPSModules\InvokeCleverReach\InvokeCleverReach"To change the source code and debug it, start always a new powershell session because modules cannot be overloaded. Do it like if you want to use other credentials
$cred = Get-Credential
start-process powershell.exe -WorkingDirectory "C:\faststats\scripts\cleverreach" -Credential ($cred)To use the same user, just do it like
start-process powershell.exe -WorkingDirectory "C:\faststats\scripts\cleverreach"You can save variables and values into the variable $Script:pluginDebug and output it with Get-PluginDebug.
If you don't have this function, you can just put this file in your Public folder and it will be dot sourced the next time:
function Get-PluginDebug {
[CmdletBinding()]
param ()
process {
$Script:pluginDebug
}
}If you know what you are doing, just proceed with this guide
# Check your executionpolicy: https:/go.microsoft.com/fwlink/?LinkID=135170
Get-ExecutionPolicy
# Either set it to Bypass to generally allow scripts for current user
Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope CurrentUser
# or
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
# Make sure to have PowerShellGet >= 1.6.0
Get-InstalledModule -Name PowerShellGet -MinimumVersion 1.6.0
# Install PowerShellGet with a current version
Install-Module -Name PowerShellGet -force -verbose -allowclobber
# Execute this with elevated rights or with the user you need to execute it with, e.g. the apteco service user
install-script install-dependencies, import-dependencies
install-module writelog
Install-Dependencies -module aptecopsframework
# Import Module and install more dependencies
Import-Module aptecopsframework
Install-AptecoPSFramework -Verbose
#-----------------------------------------------
# Please open another PowerShell window to enforce a reload of that module, recommended with elevated rights, if the plugin has dependencies
# Change to the location where you wish to install the plugin to
Set-Location -Path "C:\Apteco\Scripts\AptecoPSFramework\CleverReach"
# Import the module
Import-Module aptecopsframework -Verbose
# Choose a plugin
$plugin = get-plugins | Select guid, name, version, update, path | Out-GridView -PassThru | Select -first 1
# Install the plugin before loading it (installing dependencies)
Install-Plugin -Guid $plugin.guid
# Import the plugin into this session
import-plugin -Guid $plugin.guid
# Get merged settings for this plugin and change some
$settings = Get-settings
$settings.logfile = ".\file.log"
$tokenFile = ".\cr.token"
$tokenSettings = ".\cr_token_settings.json"
# Set the settings
Set-Settings -PSCustom $settings
# Create a token for cleverreach
Request-Token -SettingsFile $tokenSettings -TokenFile $tokenFile -UseStateToPreventCSRFAttacks
# You are getting asked for a secret, just paste it interactively
# The secret should look like: JQZrcjW2gJtr93bvys4GFVeaUCTecpa
# Save the settings into a file
$settingsFile = ".\settings.json"
Export-Settings -Path $settingsFile
# Register a task for automatic token refreshment
# You will be asked for a user to execute the task with
Register-TokenRefreshTask -SettingsFile $settingsFile
#-----------------------------------------------
# Please open another PowerShell window to enforce a reload of that module, recommended with elevated rights, if the plugin has dependencies
# Import the module and the settings file, which contains the plugin settings, too
Import-Module aptecopsframework -Verbose
Import-Settings -Path ".\settings.json"
# List all commands of this plugin
get-command -module "*CleverReach*"
# Then you can use commands like these
Test-Login
Get-CRGroups
Get-Messages
Get-Tags
Get-Bounces
Get-Blocklist
#-----------------------------------------------
# To manually refresh your token later, just execute
Save-NewToken
Please review DefaultSettings.ps1 in the modules folder AND the plugins folder for more information. Some of the settings are explained here in detail.
- Add more explanations
| Path | Setting | Default | Explanation |
|---|---|---|---|
| / | base | https://rest.cleverreach.com/v3/ | The default API address for CleverReach |
| / | contentType | application/json; charset=utf-8 | Default content type that will be used for API requests |
| / | pageSize | 500 | If paging is used to read information, this is the default pagesize that will be used automatically |
| / | mailinglimit | 999 | No of mailings that will be loaded |
| / | additionalHeaders | additional headers that should automatically be included in the API requests | |
| / | additionalParameters | additional parameters for the Invoke-RestMethod e.g. proxy parameters | |
| / | logAPIrequests | true | Output GET and POST requests in the console window |
| /errorhandling/ | HttpErrorDelay | 200 | Delay for next try in case of an exception |
| /errorhandling/ | RepeatOnHttpErrors | [502] | http errors array that should get the number of retries from MaximumRetriesOnHttpErrorList |
| /errorhandling/ | MaximumRetriesOnHttpErrorList | 3 | Repeats for specific http errors that are catched |
| /errorhandling/ | MaximumRetriesGeneric | 1 | Repeats for generic errors that are not specifically catched |
| /token/ | tokenUsage | consume |
consume or generate, depending on the mode you are wishing |
| /token/ | tokenFilePath | path for the file containing the token that should be consumed or generated | |
| /upload/ | countRowsInputFile | true | Automatically count the number of rows in the input file. This uses streaming and does not parse anything, so it is extremly fast. |
| /upload/ | validateReceivers | true | Uses a CleverReach API call to validate receivers. It removes blacklisted, not active and not in the list contained emails addresses. |
| /upload/ | excludeNotValidReceivers | false | If this is set to true, only active email addresses of the specific list will be used. This does only have an effect when using existing lists. So new contacts will not be uploaded, only existing ones in CleverReach will be used instead. |
| /upload/ | excludeBounces | true | Exclude bounces from upload |
| /upload/ | excludeGlobalDeactivated | false | Exclude deactivated (unsubscribed) receivers from any list (groupid=0) |
| /upload/ | excludeLocalDeactivated | true | Exclude deactivated (unsubscribed) receivers for the chosen list |
| /upload/ | uploadSize | 300 | Max no of rows per batch upload call, max of 1000 |
| /upload/ | tagSource | Apteco | Default tag source that will be used like Apteco.a1qhvh3_20230607201732
|
| /upload/ | useTagForUploadOnly | true | adds a tag to receivers, even if no email gets prepared or send out |
| /upload/ | reservedFields | ["tags"] | field names that should not be used in uploads |
| /upload/ | loadRuntimeStatistics | true | Loads total, active, inactive, bounced receivers of the group after upserting the data. This loads all receivers on the list, so can need a while and cause many api calls |
| /broadcast/ | defaultReleaseOffset | 120 | Seconds offset that will added to the current time when broadcasting a mailing |
| /broadcast/ | addPreheaderAfterBody | true | Adding a default preheader after the <body>
|
| /broadcast/ | preheaderFieldname | AptecoPreheader | The variable/field name that will trigger a preheader insertion/replacement. |
| /broadcast/ | removeNativePreheader | true | Sometimes CleverReach already inserts a preheader into the template. This command removes the native CleverReach Preheader |
| /broadcast/ | defaultContentType | html/text | We cannot read the content type of the mailing, so we are setting it here. Could be "html", "text" or "html/text" |
| /broadcast/ | defaultEditor | eddytor | We cannot read the used editor from the template so we are setting it through this entry. Could be "eddytor", "wizard", "freeform", "advanced", "plaintext" |
| /broadcast/ | defaultOpenTracking | true | We cannot read from the template if the open tracking is active or not. So it will be set through this setting. |
| /broadcast/ | defaultClickTracking | true | We cannot read from the template if the link/click tracking is active or not. So it will be set through this setting. |
| /broadcast/ | defaultLinkTrackingUrl | Could something be like "27.wayne.cleverreach.com" | |
| /broadcast/ | defaultLinkTrackingType | Could be "google", "intelliad", "crconnect" | |
| /broadcast/ | defaultGoogleCampaignName | Something like "My Campaign" for tracking reports in Google Analytics | |
| /broadcast/ | waitUntilFinished | false | PS or Orbit are waiting until mailing is confirmed to be sent off |
| /broadcast/ | maxWaitForFinishedAfterOffset | 120 | Wait for another 120 seconds (or more or less) until it is confirmed of send off |
| /response/ | useSeparateLogfile | false | Change this to true if you want to use a separate logfile |
| /response/ | logfile | .\response.log | The logfile if you decide to use a separate logfile |
| /response/ | urnFieldName | urn | Primary key field name, which should be global and is needed for matching |
| /response/ | communicationKeyAttributeName | communication_key | The local group attribute that will be loaded from the group, not used yet |
| /response/ | filePrefix | responses_ | Prefix for the response files that are generated |
| /response/ | messagePeriod | 60 | How many days do you want to go backwards for loading mailing reports? |
| /response/ | responsePeriod | 120 | How many days to you want to go backwards for response data per message? Normally this numbers is smaller than the messagePeriod |
| /response/ | saveLastTimestamp | true | Save the current timestamp at the end of the response download |
| /response/ | saveLastTimestampFile | .\lastresponsedownload.json | Where to save the current timestamp? |
| /response/ | loadSent | true | Load confirmed email deliveries |
| /response/ | loadOpens | true | Load open reactions (if tracking is allowed in CleverReach) |
| /response/ | loadClicks | true | Load click reactions (if tracking is allowed in CleverReach) |
| /response/ | loadBounces | true | Load confirmed bounced emails |
| /response/ | loadUnsubscribes | true | Load unsubscribed emails based on a mailing/group |
| /response/ | triggerFerge | true | Should ferge get triggered after the response download is finished? |
| /response/ | fergePath | C:\Program Files\Apteco\FastStats Email Response Gatherer x64\EmailResponseGatherer64.exe | Where is the FERGE exe located? |
| /response/ | fergeConfigurationXml | D:\Scripts\CleverReach\PSCleverReachModule\responses.xml | Should be changed for the correct path. This xml file should be created with the FERGE configuration tool |
This plugin has a functionality builtin to load responses with a command.
There are three important requisites:
- Install the
FastStats Email Response Gathererfrom Apteco on the same machine - Create a configuration xml file when executing the configurator at
C:\Program Files\Apteco\FastStats Email Response Gatherer x64\EmailResponseConfig.exe. Please fill out and then save the file where you like it:
- Connection String: Your email response database connection string, usually something like
Data Source=localhost;Initial Catalog=RS_Handel;User Id=serviceuser;Password=password123; - Bulk Insert Folder: A folder that is temporarily needed for inserting the data. This path needs to be accessible by the SQL-Server as if the SQL-Server would enter this path
- PeoplStage connection string: Your PeopleStage database connection string, usually something like
Data Source=localhost;Initial Catalog=PS_Handel;User Id=serviceuser;Password=password123; - Broadcaster:
PowerShell - Username: Could be any dummy value
- Password: Could be any dummy value
- Broadcast Parameters: Please have a look at the following table and check each parameter, especially
FTPURLwhich points to your folder where you are saving your response files:
| Parameter | Value |
|---|---|
| CLICKDATECOLUMNNAME | timestamp |
| CLICKURLCOLUMNNAME | link |
| DELIVERYDATECOLUMNNAME | timestamp |
| EMAILCOLUMNNAME | |
| EVENTTRIGGEREDDATECOLUMNNAME | timestamp |
| TYPECOLUMNNAME | MessageType |
| URNCOLUMNNAME | urn |
| DATEFORMAT | UnixTimeStamp |
| RemoveFiles | true |
| MESSAGENAMECOLUMNNAME | mailingName |
| FILEPATTERN | responses_* |
| FTPURL | File://D:\Scripts\CleverReach\PSCleverReachModule\r |
| DELIMITER | TAB |
| ENCLOSER | DOUBLEQUOTE |
| SENDIDCOLUMNNAME | mailingId |
- Check your settings json file
- So please check your settings json file that you have configured this section. Is it important you have checked at minimum the following settings:
- fergePath: The path to your response gatherer, usually something like
C:\Program Files\Apteco\FastStats Email Response Gatherer x64\EmailResponseGatherer64.exe - fergeConfigurationXml The path to your xml file that you have created in the previous step
- fergePath: The path to your response gatherer, usually something like
Just execute these commands which can also be used for a scheduled task. Please change to the directory where you wish to save the response files to.
Set-Location -Path "D:\Scripts\CleverReach\PSCleverReachModule\r"
Import-Module "AptecoPSFramework" -Verbose
Import-Settings -Path "D:\Scripts\CleverReach\PSCleverReachModule\settings.json"
Get-ResponseAs per default, FERGE should be automatically triggered after downloading and parsing the response data.
Still needs to be implemented here. In the meantime have a look here: https://github.com/Apteco/HelperScripts/tree/master/scripts/cleverreach/check-token
We are using a already existing and prepared apteco app to make this happen. This app has some privileges that you don't get by default. So for debugging purposes you can copy this plugin and change your clientid, clientsecret and redirecturi, but we would recommend to use the Apteco App. Please be aware, we are not getting notified about the app usage and don't have any access on the data. The app is only your kind of gateway to your account through the API.
cleanup of tags cleanup of segments cleanup of lists
It is supported to have as many settings json files as you wish. Just enter a different filename when you do Export-Settings -Path ".\settings_new.json and put the absolute file name into your channel editor integration parameters.
It is easy to combine multiple commands like
Get-Tags | where { $_.origin -eq "Apteco" } | % { Remove-TagsAtReceivers -Source $_.origin -Tag $_.tag }This command gets all tags, filters it by the first part of the tag and removes this tag from all receivers
Here is another example to remove tags that are older than 30 days
get-tags | where-object { $_.origin -eq "Apteco" -and [datetime]::ParseExact($_.tag.substring($_.tag.IndexOf("_")+1),"yyyyMMddHHmmss",$null) -lt [datetime]::now.AddDays(-30) } | % { Remove-TagsAtReceivers -Source $_.origin -Tag $_.tag }This can be easily done with
$segments = Get-Groups | % { Get-GroupSegments -GroupId $_.id }
$segmentsGet-CRGroups | % { $g =$_; Get-ReceiversFromList -ListId $g.id | ConvertTo-Json -Depth
99 | sc -Path ".\$( $g.name ).json" }Yes, this plugin allows the support of Preheaders. This is not possible to gather from CleverReach API at the moment, so we are using regular expressions to cut out existing preheaders and set a personalised one, which can also use Variables. The removal and replacement only takes place, if you define a preheader variable in Apteco with the Name AptecoPreheader (not case sensitive). So you see Apteco on the left hand side and my inbox on the right hand side:
The email on the right hand side with preheader also shows an example above how it looks like without a defined preheader variable.
This behaviour is dependent on the broadcast settings named addPreheaderAfterBody and removeNativePreheader
The preheader html is defined as
<div style="display:none;font-size:1px;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;mso-hide:all;">{APTECOPREHEADER}‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ </div>If you want to empty to your preheader column, please use null as field value in PeopleStage or Orbit.
The settings upload.excludeGlobalDeactivated (default: false) and upload.excludeLocalDeactivated (default: true) are controlling this behaviour.
If excludeGlobalDeactivated is set to true and if there is a receiver deactivated on one list and is active on another list, it will be in the result as deactivated.
There is also a setting in CleverReach that puts a deactivation/unsubscription automatically on a Blocklist. This setting causes that those contacts are automatically excluded from uploads in Apteco but also automatically excluded in CleverReach. So it is a double safety net.
There is another already integrated implementation available that uses an older approach where receivers are getting activated and deactivated before the upload when working on existing lists. New lists will be filled with only active receivers. If you already use this implementation, there are two differences:
- URN field: In the Channel Editor you define the URN field as a parameter. This is not supported yet. So you need to add your URN to the additional variables and give it the label you wish to (like CustomerID).
- Communication Key: Beforehand, the communication key was always created in CleverReach with an underscore in the name and the description like
COMMUNICATION_KEY. Now spaces are allowed and lead to an error in PeopleStage and Orbit that shows:Error[9001]: Failed to sync attributes. In the detailed log file you also see an HTTP409 Conflict, because the communication key should be created but is already there. Since version0.0.9this module will automatically look for a not existing communication key andcommunication keyandcommunication_key, so the parallel use of the existing integration and this framework is possible on the same list (but possibly different impacts as the existing integration deactivates receivers).
It is pretty simple now. Just update your module with
Import-Module AptecoPSFramework -ForceAnd then execute an import and export. Doing this you are ensuring you get all new settings options into the settings.json file. Existing (changed) settings won't be overridden and the existing file will be renamed with an attached timestamp, so it will not be deleted.
Import-Settings .\settings.json
Export-Settings .\settings.json- get lists
- get mailings
- migrate "refresh token with scheduled task" to here
- setup boilerplate (copy files of a subfolder to somewhere else and hints, that this folder needs to be accessed by a e.g. service and hints to the paths for get-messages etc.)
- cleanup job of lists and tags -> cmdlets already implemented
- put token in a separate file (or give the option for it to use multiple settings). Or maybe have a "main" settings file and give an option to export the token and in the other settings file use that one like PeopleStage, too
- check the validations about bounces
- implement and test tags in a field, especially for preview
- Try an overlay in preview to edit mailing
- Try to use regex to identify used variables in mailing html and show in preview
- Umlaute in filenames in debug mode
- debug mode not set in the plugin itself
- additionalParameters and additionalHeaders
- exception for api call
- dependencies
- Differentiate between new lists and existing lists
- test reserverd fields, tags
- check the processid
- manually expire a token and test the stacktrace
- test on multiple table levels and their dependency with URN