Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 87 additions & 78 deletions IntegrationTests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,54 +75,57 @@ Records all mParticle SDK API requests using WireMock for later use in integrati
**Build Artifacts:**
- `temp_artifacts/mParticle_Apple_SDK.xcframework` - Compiled SDK framework (auto-generated, not committed to git)

### `extract_request_body.py` - Extract Request Body from WireMock Mapping
### `sanitize_mapping.py` - Remove API Keys and Rename WireMock Mappings

Extracts JSON request body from a WireMock mapping file for easier editing and maintenance.
Sanitizes WireMock mapping files by replacing API keys in URLs with regex patterns, removing API keys from filenames, and renaming files based on test name.

```bash
# Extract request body without field replacements
python3 extract_request_body.py wiremock-recordings/mappings/mapping-v1-identify.json identify_test

# Extract with field replacements (replaces dynamic fields with ${json-unit.ignore})
python3 extract_request_body.py wiremock-recordings/mappings/mapping-v1-identify.json identify_test --replace
# Sanitize API keys and rename based on test name
python3 sanitize_mapping.py \
wiremock-recordings/mappings/mapping-v1-us1-abc123-identify.json \
--test-name identify
```

**What it does:**
- Extracts the JSON request body from WireMock mapping file
- Optionally replaces known dynamic fields (IDs, timestamps, device info) with `${json-unit.ignore}`
- Saves extracted body to `wiremock-recordings/requests/{test_name}.json`
- Makes it easier to edit and maintain test request bodies

**Dynamic fields replaced with `--replace`:**
`a`, `bid`, `bsv`, `ct`, `das`, `dfs`, `dlc`, `dn`, `dosv`, `est`, `ict`, `id`, `lud`, `sct`, `sid`, `vid`

**Output file format:**
```json
{
"test_name": "identify_test",
"source_mapping": "wiremock-recordings/mappings/mapping-v1-identify.json",
"request_method": "POST",
"request_url": "/v1/identify",
"request_body": { ... }
}
```
- Replaces API keys in URLs with regex pattern `us1-[a-f0-9]+` (matches any mParticle API key)
- Renames mapping file based on test name
- Renames response body file based on test name
- Updates body filename reference in mapping JSON
- Creates clean, sanitized recordings without sensitive information

**Example transformations with `--test-name identify`:**
- URL: `/v2/us1-abc123def456.../events` → `/v2/us1-[a-f0-9]+/events`
- File: `mapping-v1-us1-abc123-identify.json` → `mapping-v1-identify.json`
- Body: `body-v1-us1-abc123-identify.json` → `body-v1-identify.json`

**Example with `--test-name log-event`:**
- URL: `/v2/us1-xyz789.../events` → `/v2/us1-[a-f0-9]+/events`
- File: `mapping-v2-us1-xyz789-events.json` → `mapping-v2-log-event.json`
- Body: `body-v2-us1-xyz789-events.json` → `body-v2-log-event.json`

### `update_mapping_from_extracted.py` - Update WireMock Mapping from Extracted Body
### `transform_mapping_body.py` - Transform Request Bodies in WireMock Mappings

Updates a WireMock mapping file with a modified request body from an extracted JSON file.
Transforms request body JSON in WireMock mappings with multiple operation modes.

```bash
python3 update_mapping_from_extracted.py wiremock-recordings/requests/identify_test.json
# Display request body in readable format
python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json unescape

# Show escaped format for manual editing
python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json escape

# Replace dynamic fields and save
python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json unescape+update
```

**What it does:**
- Reads the extracted request body from JSON file
- Updates the source WireMock mapping file with the modified request body
- Preserves all WireMock configuration (response, headers, etc.)
**Operation modes:**

**Use case:** After extracting and editing a request body, use this script to apply changes back to the mapping.
1. **`unescape`** - Convert equalToJson from escaped string to formatted JSON object
2. **`escape`** - Convert equalToJson from JSON object back to escaped string
3. **`unescape+update`** - Parse, replace dynamic fields with `${json-unit.ignore}`, convert to JSON object, and save

**Note:** This script is automatically called by `run_clean_integration_tests.sh` for all files in `wiremock-recordings/requests/` before starting WireMock, so manual execution is usually not needed during testing.
**Dynamic fields replaced with `${json-unit.ignore}`:**
`a`, `bid`, `bsv`, `ck`, `ct`, `das`, `dfs`, `dlc`, `dn`, `dosv`, `en`, `est`, `ict`, `id`, `lud`, `sct`, `sid`, `vid`

## Troubleshooting

Expand Down Expand Up @@ -181,38 +184,67 @@ If another application is using the ports, terminate it before running the scrip
- Review request URLs, methods, and bodies
- Verify response data in `wiremock-recordings/__files/`

### Editing and Maintaining Test Request Bodies
### Sanitizing and Processing Recorded Mappings

After recording, you should sanitize and process mappings to remove sensitive data and handle dynamic values:

After recording, you should update request bodies to make them more maintainable (e.g., replace dynamic values):
1. **Sanitize and rename mapping file:**
```bash
python3 sanitize_mapping.py \
wiremock-recordings/mappings/mapping-v1-us1-abc123-identify.json \
--test-name identify
```

This automatically:
- Replaces API keys in URLs with regex pattern `us1-[a-f0-9]+`
- Renames the mapping file to `mapping-v1-identify.json` (or based on your test name)
- Renames the response body file to `body-v1-identify.json`
- Updates all references in the mapping JSON

1. **Extract request body from mapping:**
2. **Transform request body (replace dynamic fields):**
```bash
# Extract with automatic field replacement (recommended)
python3 extract_request_body.py \
# Replace dynamic fields and save
python3 transform_mapping_body.py \
wiremock-recordings/mappings/mapping-v1-identify.json \
identify_test \
--replace
unescape+update
```

This creates `wiremock-recordings/requests/identify_test.json` with dynamic fields replaced by `${json-unit.ignore}`
This replaces dynamic fields (timestamps, IDs, device info) with `${json-unit.ignore}`

3. **Verify the changes:**
```bash
# Check that API keys are replaced with regex pattern
grep "us1-\[a-f0-9\]+" wiremock-recordings/mappings/mapping-v1-identify.json

# Should show the regex pattern us1-[a-f0-9]+

# Verify files were renamed correctly
ls -l wiremock-recordings/mappings/mapping-v1-identify.json
ls -l wiremock-recordings/__files/body-v1-identify.json

# View the transformed request body
wiremock-recordings/mappings/mapping-identify.json
```

4. **Commit the sanitized files:**
```bash
git add wiremock-recordings/mappings/mapping-identify.json
git add wiremock-recordings/__files/body-identify.json
git commit -m "Add sanitized identify request mapping"
```

2. **Edit the extracted request body** (optional):
- Open `wiremock-recordings/requests/identify_test.json`
- Modify the `request_body` section as needed
- Add or remove fields, change expected values, etc.
**Alternative workflow - manual editing of request body:**

3. **Update the mapping file with changes:**
If you need to manually edit the request body:

1. **Edit the request body manually:**
```bash
python3 update_mapping_from_extracted.py \
wiremock-recordings/requests/identify_test.json
open wiremock-recordings/mappings/mapping-identify.json
```

This updates the original mapping file with your changes

4. **Commit both files:**
2. **Commit the changes:**
```bash
git add wiremock-recordings/mappings/mapping-v1-identify.json
git add wiremock-recordings/requests/identify_test.json
git add wiremock-recordings/mappings/mapping-identify.json
git commit -m "Update identify request mapping"
```

Expand All @@ -229,34 +261,11 @@ Use the verification script to run full end-to-end integration tests:
1. **Rebuilds SDK:** Compiles mParticle SDK as xcframework for iOS Simulator from latest source code
2. **Regenerates project:** Runs Tuist to regenerate project linked to the new xcframework
3. **Resets environment:** Cleans simulators and builds test app
4. **📝 Applies user-friendly mappings:** Automatically converts all user-friendly request bodies from `wiremock-recordings/requests/` back to WireMock mappings
4. **📝 Prepares mappings:** Escapes request body JSON in WireMock mappings for proper matching
5. **Starts WireMock:** Launches WireMock container in verification mode with updated mappings
6. **Runs tests:** Executes test app in simulator
7. **Verifies results:** Checks that all requests matched mappings and all mappings were invoked
8. **Returns exit code:** Exits with code 1 if any verification fails (CI/CD compatible)

**Note:** The SDK xcframework is built fresh on each run, stored in `temp_artifacts/mParticle_Apple_SDK.xcframework`. This ensures tests always use your latest code changes.

**Automatic mapping application:**

The script automatically looks for user-friendly request bodies in `wiremock-recordings/requests/` and applies them to the corresponding WireMock mappings before starting WireMock. This means you can:

1. Edit request bodies in `wiremock-recordings/requests/*.json`
2. Run `./run_clean_integration_tests.sh`
3. Changes are automatically applied - no manual update step needed!

**Example workflow:**

```bash
# 1. Edit user-friendly mapping
vim wiremock-recordings/requests/<test_name>.json

# 2. Run verification (automatically applies changes)
./run_clean_integration_tests.sh

# 3. If tests pass, commit both files
git add wiremock-recordings/requests/<test_name>.json
git add wiremock-recordings/mappings/<mapping_file>.json
git commit -m "Update request expectations"
```

158 changes: 146 additions & 12 deletions IntegrationTests/Sources/main.swift
Original file line number Diff line number Diff line change
@@ -1,16 +1,142 @@
import Foundation
import mParticle_Apple_SDK

// Listener for tracking upload events
class EventUploadWaiter: NSObject, MPListenerProtocol {
private var uploadCompletedSemaphore: DispatchSemaphore?
var mparticle = MParticle.sharedInstance()

@discardableResult
func wait(timeout: Int = 10) -> Bool {
mparticle.upload()
let semaphore = DispatchSemaphore(value: 0)
uploadCompletedSemaphore = semaphore

let timeoutTime = DispatchTime.now() + .seconds(timeout)
let result = semaphore.wait(timeout: timeoutTime)

uploadCompletedSemaphore = nil

return result == .success
}

func onNetworkRequestFinished(_ type: MPEndpoint,
url: String,
body: NSObject,
responseCode: Int) {
if type == .events {
uploadCompletedSemaphore?.signal()
}
}

func onNetworkRequestStarted(_ type: MPEndpoint, url: String, body: NSObject) {}
}

// Test 1: Simple Event
func testSimpleEvent(mparticle: MParticle, uploadWaiter: EventUploadWaiter) {
mparticle.logEvent("Simple Event Name", eventType: .other, eventInfo: ["SimpleKey": "SimpleValue"])
uploadWaiter.wait()
}

// Test 2: Log Event with Custom Attributes and Custom Flags
// Based on ViewController.m logEvent method (lines 131-147)
func testEventWithCustomAttributesAndFlags(mparticle: MParticle, uploadWaiter: EventUploadWaiter) {
let event = MPEvent(name: "Event Name", type: .transaction)

// Use static date instead of Date() for deterministic testing
let staticDate = Date(timeIntervalSince1970: 1700000000) // Fixed timestamp: 2023-11-14 22:13:20 UTC

// Add custom attributes including string, number, date, and nested dictionary
event?.customAttributes = [
"A_String_Key": "A String Value",
"A Number Key": 42,
"A Date Key": staticDate,
"test Dictionary": [
"test1": "test",
"test2": 2,
"test3": staticDate
]
]

// Custom flags - sent to mParticle but not forwarded to other providers
event?.addCustomFlag("Top Secret", withKey: "Not_forwarded_to_providers")

// Log the event
if let event = event {
mparticle.logEvent(event)
}
uploadWaiter.wait()
}

// Test 3: Log Screen
// Based on ViewController.m logScreen method (lines 149-151)
func testLogScreen(mparticle: MParticle, uploadWaiter: EventUploadWaiter) {
mparticle.logScreen("Home Screen", eventInfo: nil)
uploadWaiter.wait()
}

// Test 4: Log Commerce Event with Product and Transaction
// Based on ViewController.m logCommerceEvent method (lines 153-180)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Based on ViewController.m logCommerceEvent method (lines 153-180)
// Based on ViewController.m logCommerceEvent method

Nit: Line numbers here and other comments with line numbers are highly likely to change separately to the comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks will update

func testCommerceEvent(mparticle: MParticle, uploadWaiter: EventUploadWaiter) {
let product = MPProduct(
name: "Awesome Book",
sku: "1234567890",
quantity: NSNumber(value: 1),
price: NSNumber(value: 9.99)
)
product.brand = "A Publisher"
product.category = "Fiction"
product.couponCode = "XYZ123"
product.position = 1
product["custom key"] = "custom value" // Product may contain custom key/value pairs

// Create a commerce event with purchase action
let commerceEvent = MPCommerceEvent(action: .purchase, product: product)
commerceEvent.checkoutOptions = "Credit Card"
commerceEvent.screenName = "Timeless Books"
commerceEvent.checkoutStep = 4
commerceEvent.customAttributes = ["an_extra_key": "an_extra_value"] // Commerce event may contain custom key/value pairs

// Create transaction attributes
let transactionAttributes = MPTransactionAttributes()
transactionAttributes.affiliation = "Book seller"
transactionAttributes.shipping = NSNumber(value: 1.23)
transactionAttributes.tax = NSNumber(value: 0.87)
transactionAttributes.revenue = NSNumber(value: 12.09)
transactionAttributes.transactionId = "zyx098"
commerceEvent.transactionAttributes = transactionAttributes

// Log the commerce event
mparticle.logEvent(commerceEvent)
uploadWaiter.wait()
}

// Test 5: Rokt Select Overlay Placement
// Based on ViewController.m selectOverlayPlacement method (lines 182-192)
// Tests Rokt SDK integration through mParticle for selecting placements with custom attributes
func testRoktSelectPlacement(mparticle: MParticle, uploadWaiter: EventUploadWaiter) {
let roktAttributes: [String: String] = [
"email": "[email protected]",
"firstname": "Jenny",
"lastname": "Smith",
"sandbox": "true",
"mobile": "(555)867-5309"
]

// Select Rokt placement with identifier and attributes
mparticle.rokt.selectPlacements("RoktLayout", attributes: roktAttributes)
uploadWaiter.wait()
}

var options = MParticleOptions(
key: "", // Put your key
secret: "" // Put your secret
key: "",
secret: ""
)

var identityRequest = MPIdentityApiRequest.withEmptyUser()
identityRequest.email = "[email protected]";
identityRequest.customerId = "123456";
options.identifyRequest = identityRequest;
identityRequest.email = "[email protected]"
identityRequest.customerId = "123456"
options.identifyRequest = identityRequest

options.onIdentifyComplete = { apiResult, error in
if let apiResult {
Expand All @@ -20,17 +146,25 @@ options.onIdentifyComplete = { apiResult, error in
options.logLevel = .verbose

var networkOptions = MPNetworkOptions()
networkOptions.configHost = "127.0.0.1"; // config2.mparticle.com
networkOptions.eventsHost = "127.0.0.1"; // nativesdks.mparticle.com
networkOptions.identityHost = "127.0.0.1"; // identity.mparticle.com
networkOptions.configHost = "127.0.0.1" // config2.mparticle.com
networkOptions.eventsHost = "127.0.0.1" // nativesdks.mparticle.com
networkOptions.identityHost = "127.0.0.1" // identity.mparticle.com
networkOptions.pinningDisabled = true;

options.networkOptions = networkOptions;
options.networkOptions = networkOptions

// Register listener for tracking upload events
let uploadWaiter = EventUploadWaiter()
MPListenerController.sharedInstance().addSdkListener(uploadWaiter)

let mparticle = MParticle.sharedInstance()
mparticle.start(with: options)

sleep(1)

mparticle.logEvent("Simple Event Name", eventType: .other, eventInfo: ["SimpleKey": "SimpleValue"])

sleep(7)
// Run tests
testSimpleEvent(mparticle: mparticle, uploadWaiter: uploadWaiter)
testEventWithCustomAttributesAndFlags(mparticle: mparticle, uploadWaiter: uploadWaiter)
testLogScreen(mparticle: mparticle, uploadWaiter: uploadWaiter)
testCommerceEvent(mparticle: mparticle, uploadWaiter: uploadWaiter)
testRoktSelectPlacement(mparticle: mparticle, uploadWaiter: uploadWaiter)
Loading