diff --git a/IntegrationTests/README.md b/IntegrationTests/README.md index a4155211..5b82ee29 100644 --- a/IntegrationTests/README.md +++ b/IntegrationTests/README.md @@ -75,54 +75,57 @@ Records all mParticle SDK API requests using WireMock for later use in integrati **Build Artifacts:** - `temp_artifacts/mParticle_Apple_SDK.xcframework` - Compiled SDK framework (auto-generated, not committed to git) -### `extract_request_body.py` - Extract Request Body from WireMock Mapping +### `sanitize_mapping.py` - Remove API Keys and Rename WireMock Mappings -Extracts JSON request body from a WireMock mapping file for easier editing and maintenance. +Sanitizes WireMock mapping files by replacing API keys in URLs with regex patterns, removing API keys from filenames, and renaming files based on test name. ```bash -# Extract request body without field replacements -python3 extract_request_body.py wiremock-recordings/mappings/mapping-v1-identify.json identify_test - -# Extract with field replacements (replaces dynamic fields with ${json-unit.ignore}) -python3 extract_request_body.py wiremock-recordings/mappings/mapping-v1-identify.json identify_test --replace +# Sanitize API keys and rename based on test name +python3 sanitize_mapping.py \ + wiremock-recordings/mappings/mapping-v1-us1-abc123-identify.json \ + --test-name identify ``` **What it does:** -- Extracts the JSON request body from WireMock mapping file -- Optionally replaces known dynamic fields (IDs, timestamps, device info) with `${json-unit.ignore}` -- Saves extracted body to `wiremock-recordings/requests/{test_name}.json` -- Makes it easier to edit and maintain test request bodies - -**Dynamic fields replaced with `--replace`:** -`a`, `bid`, `bsv`, `ct`, `das`, `dfs`, `dlc`, `dn`, `dosv`, `est`, `ict`, `id`, `lud`, `sct`, `sid`, `vid` - -**Output file format:** -```json -{ - "test_name": "identify_test", - "source_mapping": "wiremock-recordings/mappings/mapping-v1-identify.json", - "request_method": "POST", - "request_url": "/v1/identify", - "request_body": { ... } -} -``` +- Replaces API keys in URLs with regex pattern `us1-[a-f0-9]+` (matches any mParticle API key) +- Renames mapping file based on test name +- Renames response body file based on test name +- Updates body filename reference in mapping JSON +- Creates clean, sanitized recordings without sensitive information + +**Example transformations with `--test-name identify`:** +- URL: `/v2/us1-abc123def456.../events` → `/v2/us1-[a-f0-9]+/events` +- File: `mapping-v1-us1-abc123-identify.json` → `mapping-v1-identify.json` +- Body: `body-v1-us1-abc123-identify.json` → `body-v1-identify.json` + +**Example with `--test-name log-event`:** +- URL: `/v2/us1-xyz789.../events` → `/v2/us1-[a-f0-9]+/events` +- File: `mapping-v2-us1-xyz789-events.json` → `mapping-v2-log-event.json` +- Body: `body-v2-us1-xyz789-events.json` → `body-v2-log-event.json` -### `update_mapping_from_extracted.py` - Update WireMock Mapping from Extracted Body +### `transform_mapping_body.py` - Transform Request Bodies in WireMock Mappings -Updates a WireMock mapping file with a modified request body from an extracted JSON file. +Transforms request body JSON in WireMock mappings with multiple operation modes. ```bash -python3 update_mapping_from_extracted.py wiremock-recordings/requests/identify_test.json +# Display request body in readable format +python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json unescape + +# Show escaped format for manual editing +python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json escape + +# Replace dynamic fields and save +python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json unescape+update ``` -**What it does:** -- Reads the extracted request body from JSON file -- Updates the source WireMock mapping file with the modified request body -- Preserves all WireMock configuration (response, headers, etc.) +**Operation modes:** -**Use case:** After extracting and editing a request body, use this script to apply changes back to the mapping. +1. **`unescape`** - Convert equalToJson from escaped string to formatted JSON object +2. **`escape`** - Convert equalToJson from JSON object back to escaped string +3. **`unescape+update`** - Parse, replace dynamic fields with `${json-unit.ignore}`, convert to JSON object, and save -**Note:** This script is automatically called by `run_clean_integration_tests.sh` for all files in `wiremock-recordings/requests/` before starting WireMock, so manual execution is usually not needed during testing. +**Dynamic fields replaced with `${json-unit.ignore}`:** +`a`, `bid`, `bsv`, `ck`, `ct`, `das`, `dfs`, `dlc`, `dn`, `dosv`, `en`, `est`, `ict`, `id`, `lud`, `sct`, `sid`, `vid` ## Troubleshooting @@ -181,38 +184,67 @@ If another application is using the ports, terminate it before running the scrip - Review request URLs, methods, and bodies - Verify response data in `wiremock-recordings/__files/` -### Editing and Maintaining Test Request Bodies +### Sanitizing and Processing Recorded Mappings + +After recording, you should sanitize and process mappings to remove sensitive data and handle dynamic values: -After recording, you should update request bodies to make them more maintainable (e.g., replace dynamic values): +1. **Sanitize and rename mapping file:** + ```bash + python3 sanitize_mapping.py \ + wiremock-recordings/mappings/mapping-v1-us1-abc123-identify.json \ + --test-name identify + ``` + + This automatically: + - Replaces API keys in URLs with regex pattern `us1-[a-f0-9]+` + - Renames the mapping file to `mapping-v1-identify.json` (or based on your test name) + - Renames the response body file to `body-v1-identify.json` + - Updates all references in the mapping JSON -1. **Extract request body from mapping:** +2. **Transform request body (replace dynamic fields):** ```bash - # Extract with automatic field replacement (recommended) - python3 extract_request_body.py \ + # Replace dynamic fields and save + python3 transform_mapping_body.py \ wiremock-recordings/mappings/mapping-v1-identify.json \ - identify_test \ - --replace + unescape+update ``` - This creates `wiremock-recordings/requests/identify_test.json` with dynamic fields replaced by `${json-unit.ignore}` + This replaces dynamic fields (timestamps, IDs, device info) with `${json-unit.ignore}` + +3. **Verify the changes:** + ```bash + # Check that API keys are replaced with regex pattern + grep "us1-\[a-f0-9\]+" wiremock-recordings/mappings/mapping-v1-identify.json + + # Should show the regex pattern us1-[a-f0-9]+ + + # Verify files were renamed correctly + ls -l wiremock-recordings/mappings/mapping-v1-identify.json + ls -l wiremock-recordings/__files/body-v1-identify.json + + # View the transformed request body + wiremock-recordings/mappings/mapping-identify.json + ``` + +4. **Commit the sanitized files:** + ```bash + git add wiremock-recordings/mappings/mapping-identify.json + git add wiremock-recordings/__files/body-identify.json + git commit -m "Add sanitized identify request mapping" + ``` -2. **Edit the extracted request body** (optional): - - Open `wiremock-recordings/requests/identify_test.json` - - Modify the `request_body` section as needed - - Add or remove fields, change expected values, etc. +**Alternative workflow - manual editing of request body:** -3. **Update the mapping file with changes:** +If you need to manually edit the request body: + +1. **Edit the request body manually:** ```bash - python3 update_mapping_from_extracted.py \ - wiremock-recordings/requests/identify_test.json + open wiremock-recordings/mappings/mapping-identify.json ``` - - This updates the original mapping file with your changes -4. **Commit both files:** +2. **Commit the changes:** ```bash - git add wiremock-recordings/mappings/mapping-v1-identify.json - git add wiremock-recordings/requests/identify_test.json + git add wiremock-recordings/mappings/mapping-identify.json git commit -m "Update identify request mapping" ``` @@ -229,7 +261,7 @@ Use the verification script to run full end-to-end integration tests: 1. **Rebuilds SDK:** Compiles mParticle SDK as xcframework for iOS Simulator from latest source code 2. **Regenerates project:** Runs Tuist to regenerate project linked to the new xcframework 3. **Resets environment:** Cleans simulators and builds test app -4. **📝 Applies user-friendly mappings:** Automatically converts all user-friendly request bodies from `wiremock-recordings/requests/` back to WireMock mappings +4. **📝 Prepares mappings:** Escapes request body JSON in WireMock mappings for proper matching 5. **Starts WireMock:** Launches WireMock container in verification mode with updated mappings 6. **Runs tests:** Executes test app in simulator 7. **Verifies results:** Checks that all requests matched mappings and all mappings were invoked @@ -237,26 +269,3 @@ Use the verification script to run full end-to-end integration tests: **Note:** The SDK xcframework is built fresh on each run, stored in `temp_artifacts/mParticle_Apple_SDK.xcframework`. This ensures tests always use your latest code changes. -**Automatic mapping application:** - -The script automatically looks for user-friendly request bodies in `wiremock-recordings/requests/` and applies them to the corresponding WireMock mappings before starting WireMock. This means you can: - -1. Edit request bodies in `wiremock-recordings/requests/*.json` -2. Run `./run_clean_integration_tests.sh` -3. Changes are automatically applied - no manual update step needed! - -**Example workflow:** - -```bash -# 1. Edit user-friendly mapping -vim wiremock-recordings/requests/.json - -# 2. Run verification (automatically applies changes) -./run_clean_integration_tests.sh - -# 3. If tests pass, commit both files -git add wiremock-recordings/requests/.json -git add wiremock-recordings/mappings/.json -git commit -m "Update request expectations" -``` - diff --git a/IntegrationTests/Sources/main.swift b/IntegrationTests/Sources/main.swift index 6b6cb5b9..6bcd5243 100644 --- a/IntegrationTests/Sources/main.swift +++ b/IntegrationTests/Sources/main.swift @@ -1,16 +1,142 @@ import Foundation import mParticle_Apple_SDK +// Listener for tracking upload events +class EventUploadWaiter: NSObject, MPListenerProtocol { + private var uploadCompletedSemaphore: DispatchSemaphore? + var mparticle = MParticle.sharedInstance() + + @discardableResult + func wait(timeout: Int = 10) -> Bool { + mparticle.upload() + let semaphore = DispatchSemaphore(value: 0) + uploadCompletedSemaphore = semaphore + + let timeoutTime = DispatchTime.now() + .seconds(timeout) + let result = semaphore.wait(timeout: timeoutTime) + + uploadCompletedSemaphore = nil + + return result == .success + } + + func onNetworkRequestFinished(_ type: MPEndpoint, + url: String, + body: NSObject, + responseCode: Int) { + if type == .events { + uploadCompletedSemaphore?.signal() + } + } + + func onNetworkRequestStarted(_ type: MPEndpoint, url: String, body: NSObject) {} +} + +// Test 1: Simple Event +func testSimpleEvent(mparticle: MParticle, uploadWaiter: EventUploadWaiter) { + mparticle.logEvent("Simple Event Name", eventType: .other, eventInfo: ["SimpleKey": "SimpleValue"]) + uploadWaiter.wait() +} + +// Test 2: Log Event with Custom Attributes and Custom Flags +// Based on ViewController.m logEvent method (lines 131-147) +func testEventWithCustomAttributesAndFlags(mparticle: MParticle, uploadWaiter: EventUploadWaiter) { + let event = MPEvent(name: "Event Name", type: .transaction) + + // Use static date instead of Date() for deterministic testing + let staticDate = Date(timeIntervalSince1970: 1700000000) // Fixed timestamp: 2023-11-14 22:13:20 UTC + + // Add custom attributes including string, number, date, and nested dictionary + event?.customAttributes = [ + "A_String_Key": "A String Value", + "A Number Key": 42, + "A Date Key": staticDate, + "test Dictionary": [ + "test1": "test", + "test2": 2, + "test3": staticDate + ] + ] + + // Custom flags - sent to mParticle but not forwarded to other providers + event?.addCustomFlag("Top Secret", withKey: "Not_forwarded_to_providers") + + // Log the event + if let event = event { + mparticle.logEvent(event) + } + uploadWaiter.wait() +} + +// Test 3: Log Screen +// Based on ViewController.m logScreen method (lines 149-151) +func testLogScreen(mparticle: MParticle, uploadWaiter: EventUploadWaiter) { + mparticle.logScreen("Home Screen", eventInfo: nil) + uploadWaiter.wait() +} + +// Test 4: Log Commerce Event with Product and Transaction +// Based on ViewController.m logCommerceEvent method (lines 153-180) +func testCommerceEvent(mparticle: MParticle, uploadWaiter: EventUploadWaiter) { + let product = MPProduct( + name: "Awesome Book", + sku: "1234567890", + quantity: NSNumber(value: 1), + price: NSNumber(value: 9.99) + ) + product.brand = "A Publisher" + product.category = "Fiction" + product.couponCode = "XYZ123" + product.position = 1 + product["custom key"] = "custom value" // Product may contain custom key/value pairs + + // Create a commerce event with purchase action + let commerceEvent = MPCommerceEvent(action: .purchase, product: product) + commerceEvent.checkoutOptions = "Credit Card" + commerceEvent.screenName = "Timeless Books" + commerceEvent.checkoutStep = 4 + commerceEvent.customAttributes = ["an_extra_key": "an_extra_value"] // Commerce event may contain custom key/value pairs + + // Create transaction attributes + let transactionAttributes = MPTransactionAttributes() + transactionAttributes.affiliation = "Book seller" + transactionAttributes.shipping = NSNumber(value: 1.23) + transactionAttributes.tax = NSNumber(value: 0.87) + transactionAttributes.revenue = NSNumber(value: 12.09) + transactionAttributes.transactionId = "zyx098" + commerceEvent.transactionAttributes = transactionAttributes + + // Log the commerce event + mparticle.logEvent(commerceEvent) + uploadWaiter.wait() +} + +// Test 5: Rokt Select Overlay Placement +// Based on ViewController.m selectOverlayPlacement method (lines 182-192) +// Tests Rokt SDK integration through mParticle for selecting placements with custom attributes +func testRoktSelectPlacement(mparticle: MParticle, uploadWaiter: EventUploadWaiter) { + let roktAttributes: [String: String] = [ + "email": "j.smit@example.com", + "firstname": "Jenny", + "lastname": "Smith", + "sandbox": "true", + "mobile": "(555)867-5309" + ] + + // Select Rokt placement with identifier and attributes + mparticle.rokt.selectPlacements("RoktLayout", attributes: roktAttributes) + uploadWaiter.wait() +} var options = MParticleOptions( - key: "", // Put your key - secret: "" // Put your secret + key: "", + secret: "" ) var identityRequest = MPIdentityApiRequest.withEmptyUser() -identityRequest.email = "foo@example.com"; -identityRequest.customerId = "123456"; -options.identifyRequest = identityRequest; +identityRequest.email = "foo@example.com" +identityRequest.customerId = "123456" +options.identifyRequest = identityRequest options.onIdentifyComplete = { apiResult, error in if let apiResult { @@ -20,17 +146,25 @@ options.onIdentifyComplete = { apiResult, error in options.logLevel = .verbose var networkOptions = MPNetworkOptions() -networkOptions.configHost = "127.0.0.1"; // config2.mparticle.com -networkOptions.eventsHost = "127.0.0.1"; // nativesdks.mparticle.com -networkOptions.identityHost = "127.0.0.1"; // identity.mparticle.com +networkOptions.configHost = "127.0.0.1" // config2.mparticle.com +networkOptions.eventsHost = "127.0.0.1" // nativesdks.mparticle.com +networkOptions.identityHost = "127.0.0.1" // identity.mparticle.com networkOptions.pinningDisabled = true; -options.networkOptions = networkOptions; +options.networkOptions = networkOptions + +// Register listener for tracking upload events +let uploadWaiter = EventUploadWaiter() +MPListenerController.sharedInstance().addSdkListener(uploadWaiter) + let mparticle = MParticle.sharedInstance() mparticle.start(with: options) sleep(1) -mparticle.logEvent("Simple Event Name", eventType: .other, eventInfo: ["SimpleKey": "SimpleValue"]) - -sleep(7) +// Run tests +testSimpleEvent(mparticle: mparticle, uploadWaiter: uploadWaiter) +testEventWithCustomAttributesAndFlags(mparticle: mparticle, uploadWaiter: uploadWaiter) +testLogScreen(mparticle: mparticle, uploadWaiter: uploadWaiter) +testCommerceEvent(mparticle: mparticle, uploadWaiter: uploadWaiter) +testRoktSelectPlacement(mparticle: mparticle, uploadWaiter: uploadWaiter) \ No newline at end of file diff --git a/IntegrationTests/common.sh b/IntegrationTests/common.sh index 1203edab..685ade8d 100755 --- a/IntegrationTests/common.sh +++ b/IntegrationTests/common.sh @@ -243,8 +243,22 @@ start_wiremock() { eval $docker_cmd } +show_wiremock_logs() { + echo "" + echo "📋 WireMock container logs:" + echo "════════════════════════════════════════════════════════════════" + docker logs ${CONTAINER_NAME} 2>&1 || echo "❌ Could not retrieve container logs" + echo "════════════════════════════════════════════════════════════════" + echo "" +} + stop_wiremock() { docker stop ${CONTAINER_NAME} 2>/dev/null || true docker rm ${CONTAINER_NAME} 2>/dev/null || true } +stop_wiremock_with_logs() { + show_wiremock_logs + stop_wiremock +} + diff --git a/IntegrationTests/extract_request_body.py b/IntegrationTests/extract_request_body.py deleted file mode 100755 index 2b6f9a1c..00000000 --- a/IntegrationTests/extract_request_body.py +++ /dev/null @@ -1,235 +0,0 @@ -#!/usr/bin/env python3 -""" -Script for extracting JSON request bodies from WireMock mappings. - -Usage: - python3 extract_request_body.py [--replace] - -Example: - python3 extract_request_body.py wiremock-recordings/mappings/mapping-v1-identify.json identify_test - python3 extract_request_body.py wiremock-recordings/mappings/mapping-v1-identify.json identify_test --replace -""" - -import json -import sys -import os -import argparse -from pathlib import Path -from typing import Dict, Any, List, Union - - -# Default list of fields to replace with ${json-unit.ignore} -# Based on existing WireMock mappings that contain dynamic/timestamp values -DEFAULT_REPLACE_FIELDS = [ - 'a', # App ID - 'bid', # Bundle ID / Build ID - 'bsv', # Build System Version - 'ct', # Creation Time / Current Time - 'das', # Device Application Stamp - 'dfs', # Device Fingerprint String - 'dlc', # Device Locale - 'dn', # Device Name - 'dosv', # Device OS Version - 'est', # Event Start Time - 'ict', # Init Config Time - 'id', # ID (various message/event IDs) - 'lud', # Last Update Date - 'sct', # Session Creation Time - 'sid', # Session ID - 'vid', # Vendor ID -] - - -def replace_field_value(data: Union[Dict, List, Any], field_name: str, replacement_value: str) -> Union[Dict, List, Any]: - """ - Recursively replaces the value of a specified field in a JSON structure. - - Args: - data: JSON data (dict, list, or primitive value) - field_name: Name of the field to replace - replacement_value: New value to set for the field - - Returns: - Modified data structure with replaced field values - - Example: - data = {"id": "123", "name": "test", "nested": {"id": "456"}} - result = replace_field_value(data, "id", "${json-unit.ignore}") - # result = {"id": "${json-unit.ignore}", "name": "test", "nested": {"id": "${json-unit.ignore}"}} - """ - if isinstance(data, dict): - # For dictionaries, check each key - result = {} - for key, value in data.items(): - if key == field_name: - # Replace the value for this field - result[key] = replacement_value - else: - # Recursively process the value - result[key] = replace_field_value(value, field_name, replacement_value) - return result - elif isinstance(data, list): - # For lists, recursively process each item - return [replace_field_value(item, field_name, replacement_value) for item in data] - else: - # For primitive values, return as is - return data - - -def replace_fields_from_list(data: Union[Dict, List, Any], field_names: List[str], replacement_value: str = "${json-unit.ignore}") -> Union[Dict, List, Any]: - """ - Replaces values of multiple fields in a JSON structure with a specified value. - - Args: - data: JSON data (dict, list, or primitive value) - field_names: List of field names to replace - replacement_value: Value to use for replacement (default: "${json-unit.ignore}") - - Returns: - Modified data structure with all specified fields replaced - - Example: - data = {"id": "123", "ct": "1234567890", "name": "test", "nested": {"id": "456", "ct": "0987654321"}} - result = replace_fields_from_list(data, ["id", "ct"]) - # result = {"id": "${json-unit.ignore}", "ct": "${json-unit.ignore}", "name": "test", - # "nested": {"id": "${json-unit.ignore}", "ct": "${json-unit.ignore}"}} - - # Or with custom replacement value: - result = replace_fields_from_list(data, ["id", "ct"], "IGNORED") - # result = {"id": "IGNORED", "ct": "IGNORED", "name": "test", "nested": {"id": "IGNORED", "ct": "IGNORED"}} - """ - result = data - - # Apply replacement for each field in the list - for field_name in field_names: - result = replace_field_value(result, field_name, replacement_value) - - return result - -def extract_request_body(mapping_file: str, test_name: str, replace_fields: bool = False) -> None: - """ - Extracts JSON body from WireMock mapping and saves it to a separate file. - - Args: - mapping_file: Path to mapping file - test_name: Test name - replace_fields: If True, replaces known dynamic fields with ${json-unit.ignore} - """ - # Check if mapping file exists - mapping_path = Path(mapping_file) - if not mapping_path.exists(): - print(f"❌ Error: mapping file not found: {mapping_file}") - sys.exit(1) - - # Read mapping file - try: - with open(mapping_path, 'r', encoding='utf-8') as f: - mapping_data = json.load(f) - except json.JSONDecodeError as e: - print(f"❌ Error: failed to parse JSON from mapping file: {e}") - sys.exit(1) - except Exception as e: - print(f"❌ Error reading mapping file: {e}") - sys.exit(1) - - # Extract request information - try: - request_data = mapping_data.get('request', {}) - method = request_data.get('method', 'UNKNOWN') - # Support both 'url' and 'urlPattern' fields - url = request_data.get('url') or request_data.get('urlPattern', 'UNKNOWN') - - body_patterns = request_data.get('bodyPatterns', []) - - # Check for body presence - if not body_patterns: - # This might be a GET request or another method without body - print(f"⚠️ Warning: bodyPatterns not found in mapping") - print(f" Request method: {method}") - print(f" URL: {url}") - print(" (GET requests usually don't have a body)") - sys.exit(1) - - # Get escaped JSON string - equal_to_json = body_patterns[0].get('equalToJson') - if equal_to_json is None: - print("❌ Error: equalToJson not found in bodyPatterns") - sys.exit(1) - - # Parse escaped JSON string to get the actual JSON object - # (unescape) - request_body = json.loads(equal_to_json) - - # Apply field replacements if requested - if replace_fields: - print(f"🔄 Replacing {len(DEFAULT_REPLACE_FIELDS)} known dynamic fields with ${{json-unit.ignore}}") - request_body = replace_fields_from_list(request_body, DEFAULT_REPLACE_FIELDS) - - except json.JSONDecodeError as e: - print(f"❌ Error: failed to parse JSON from equalToJson: {e}") - sys.exit(1) - except Exception as e: - print(f"❌ Error extracting body from mapping: {e}") - sys.exit(1) - - # Form output structure - output_data = { - "test_name": test_name, - "source_mapping": str(mapping_path), - "request_method": method, - "request_url": url, - "request_body": request_body - } - - # Create directory for saving extracted bodies - output_dir = Path("wiremock-recordings/requests") - output_dir.mkdir(parents=True, exist_ok=True) - - # Form output filename - output_file = output_dir / f"{test_name}.json" - - # Save to file with pretty-print for readability - try: - with open(output_file, 'w', encoding='utf-8') as f: - json.dump(output_data, f, indent=2, ensure_ascii=False) - - print(f"✅ JSON body successfully extracted and saved to: {output_file}") - print(f"📝 Test name: {test_name}") - print(f"🔗 Source mapping: {mapping_path}") - - except Exception as e: - print(f"❌ Error saving file: {e}") - sys.exit(1) - - -def main(): - # Set up command line argument parser - parser = argparse.ArgumentParser( - description='Extract JSON request bodies from WireMock mappings', - formatter_class=argparse.RawDescriptionHelpFormatter - ) - - parser.add_argument( - 'mapping_file', - help='Path to WireMock mapping file' - ) - - parser.add_argument( - 'test_name', - help='Test name for the output file' - ) - - parser.add_argument( - '--replace', - action='store_true', - help='Replace known dynamic fields with ${json-unit.ignore}' - ) - - args = parser.parse_args() - - extract_request_body(args.mapping_file, args.test_name, replace_fields=args.replace) - - -if __name__ == "__main__": - main() - diff --git a/IntegrationTests/run_clean_integration_tests.sh b/IntegrationTests/run_clean_integration_tests.sh index bc0bcfca..b087a507 100755 --- a/IntegrationTests/run_clean_integration_tests.sh +++ b/IntegrationTests/run_clean_integration_tests.sh @@ -19,24 +19,34 @@ tuist generate --no-open # === Script-specific configuration === CONTAINER_NAME="wiremock-verify" -apply_user_friendly_mappings() { - echo "🔄 Applying user-friendly mappings to WireMock..." - local REQUESTS_DIR="${MAPPINGS_DIR}/requests" + +escape_mapping_bodies() { + echo "🔄 Converting mapping bodies to escaped format (WireMock-compatible)..." + local MAPPINGS_FILES="${MAPPINGS_DIR}/mappings" - if [ -d "$REQUESTS_DIR" ] && [ "$(ls -A $REQUESTS_DIR/*.json 2>/dev/null)" ]; then - local APPLIED_COUNT=0 - for request_file in "$REQUESTS_DIR"/*.json; do - if [ -f "$request_file" ]; then - echo " 📝 Applying: $(basename $request_file)" - python3 update_mapping_from_extracted.py "$request_file" 2>/dev/null || { - echo " ⚠️ Failed to apply $(basename $request_file)" + if [ -d "$MAPPINGS_FILES" ] && [ "$(ls -A $MAPPINGS_FILES/*.json 2>/dev/null)" ]; then + for mapping_file in "$MAPPINGS_FILES"/*.json; do + if [ -f "$mapping_file" ]; then + python3 transform_mapping_body.py "$mapping_file" escape > /dev/null 2>&1 || { + echo "⚠️ Failed to escape $(basename $mapping_file)" + } + fi + done + fi +} + +unescape_mapping_bodies() { + echo "🔄 Converting mapping bodies back to unescaped format (readable)..." + local MAPPINGS_FILES="${MAPPINGS_DIR}/mappings" + + if [ -d "$MAPPINGS_FILES" ] && [ "$(ls -A $MAPPINGS_FILES/*.json 2>/dev/null)" ]; then + for mapping_file in "$MAPPINGS_FILES"/*.json; do + if [ -f "$mapping_file" ]; then + python3 transform_mapping_body.py "$mapping_file" unescape > /dev/null 2>&1 || { + echo "⚠️ Failed to unescape $(basename $mapping_file)" } - APPLIED_COUNT=$((APPLIED_COUNT + 1)) fi done - echo "✅ Applied $APPLIED_COUNT user-friendly mapping(s)" - else - echo "⚠️ No user-friendly mappings found in $REQUESTS_DIR (skipping)" fi } @@ -65,6 +75,8 @@ verify_wiremock_results() { echo "❌ Found requests that did not match any mappings:" curl -s http://localhost:${WIREMOCK_PORT}/__admin/requests/unmatched | \ jq -r '.requests[] | " [\(.method)] \(.url)"' + echo "" + show_wiremock_logs stop_wiremock exit 1 else @@ -128,8 +140,26 @@ verify_wiremock_results() { echo "🎉 Verification completed successfully!" } -# Trap to ensure cleanup on exit -trap stop_wiremock EXIT INT TERM +# Cleanup function to restore mappings and stop WireMock +cleanup() { + unescape_mapping_bodies + stop_wiremock +} + +# Error handler that shows logs before cleanup +error_handler() { + local exit_code=$? + echo "" + echo "❌ Script failed with exit code: $exit_code" + show_wiremock_logs + unescape_mapping_bodies + stop_wiremock + exit $exit_code +} + +# Trap to ensure cleanup on exit or error +trap cleanup EXIT INT TERM +trap error_handler ERR # === Main execution flow === build_application @@ -137,7 +167,7 @@ find_app_path reset_simulators find_available_device find_device -apply_user_friendly_mappings +escape_mapping_bodies start_wiremock "verify" wait_for_wiremock echo "📝 WireMock is running in verification mode" @@ -149,4 +179,5 @@ install_application launch_application wait_for_app_completion verify_wiremock_results +unescape_mapping_bodies stop_wiremock diff --git a/IntegrationTests/sanitize_mapping.py b/IntegrationTests/sanitize_mapping.py new file mode 100755 index 00000000..0f205c02 --- /dev/null +++ b/IntegrationTests/sanitize_mapping.py @@ -0,0 +1,339 @@ +#!/usr/bin/env python3 +""" +Script for sanitizing WireMock mapping files by: +- Replacing API keys in URLs with regex pattern: us1-[a-f0-9]+ +- Removing API keys from filenames +- Renaming files based on test name + +Usage: + python3 sanitize_mapping.py --test-name + +Example: + python3 sanitize_mapping.py wiremock-recordings/mappings/mapping-v1-us1-abc123-identify.json --test-name identify + + Before: /v2/us1-abc123def456.../events → After: /v2/us1-[a-f0-9]+/events +""" + +import json +import sys +import argparse +import re +from pathlib import Path +from typing import Dict, Any, Tuple + + +def sanitize_url(url: str) -> str: + """ + Replaces API key in URL with a regex pattern for WireMock. + + Matches patterns like: + - /v2/us1-abc123.../events -> /v2/us1-[a-f0-9]+/events + - /v1/us1-xyz789.../identify -> /v1/us1-[a-f0-9]+/identify + + Args: + url: Original URL that may contain API key + + Returns: + Sanitized URL with API key replaced by regex pattern us1-[a-f0-9]+ + """ + # Pattern to match API keys like "us1-{32 hex characters}" + api_key_pattern = r'us1-[a-f0-9]{32}' + + # Replace API key with regex pattern for WireMock + sanitized = re.sub(api_key_pattern, 'us1-[a-f0-9]+', url) + + return sanitized + + +def sanitize_body_filename(filename: str, test_name: str = None) -> Tuple[str, str]: + """ + Removes API key from body filename and optionally renames based on test name. + + Args: + filename: Original filename that may contain API key + test_name: Optional test name to use for renaming + + Returns: + Tuple of (old_filename, new_filename) + """ + if not filename: + return (filename, filename) + + # If test_name is provided, use it for the new filename + if test_name: + # Get file extension + ext = Path(filename).suffix + new_filename = f"body-{test_name}{ext}" + return (filename, new_filename) + + # Otherwise, just remove API key from filename + # Pattern to match API keys in filename + api_key_pattern = r'us1-[a-f0-9]{32}' + + # Find API key in filename + match = re.search(api_key_pattern, filename) + if not match: + # No API key found, return as is + return (filename, filename) + + # Replace API key pattern with generic name + # Example: body-v2-us1-XXX-events-YYY.json -> body-v2-events-YYY.json + new_filename = re.sub(r'-us1-[a-f0-9]{32}', '', filename) + + return (filename, new_filename) + + +def rename_body_file(old_filename: str, new_filename: str, base_path: Path) -> bool: + """ + Renames body file if names are different. + + Args: + old_filename: Current filename + new_filename: New filename to rename to + base_path: Base path to the mapping file directory + + Returns: + True if file was renamed, False otherwise + """ + if old_filename == new_filename: + return False + + # Construct paths relative to the mapping file location + files_dir = base_path.parent.parent / "__files" + old_path = files_dir / old_filename + new_path = files_dir / new_filename + + if not old_path.exists(): + print(f"⚠️ Warning: Body file not found: {old_path}") + return False + + if new_path.exists(): + print(f"⚠️ Warning: Target body file already exists: {new_path}") + return False + + try: + old_path.rename(new_path) + print(f"📝 Renamed body file: {old_filename} -> {new_filename}") + return True + except Exception as e: + print(f"❌ Error renaming body file: {e}") + return False + + +def load_mapping_file(mapping_file: str) -> Tuple[Path, Dict[str, Any]]: + """ + Loads and parses mapping file. + + Args: + mapping_file: Path to mapping file + + Returns: + Tuple (Path to file, mapping data from JSON) + """ + # Check if mapping file exists + mapping_path = Path(mapping_file) + if not mapping_path.exists(): + print(f"❌ Error: mapping file not found: {mapping_file}") + sys.exit(1) + + # Read mapping file + try: + with open(mapping_path, 'r', encoding='utf-8') as f: + mapping_data = json.load(f) + return (mapping_path, mapping_data) + except json.JSONDecodeError as e: + print(f"❌ Error: failed to parse JSON from mapping file: {e}") + sys.exit(1) + except Exception as e: + print(f"❌ Error reading mapping file: {e}") + sys.exit(1) + + +def rename_mapping_file(mapping_path: Path, test_name: str = None) -> Tuple[Path, bool]: + """ + Consistently renames mapping file if it contains API key or test name is provided. + + Args: + mapping_path: Path to mapping file + test_name: Optional test name to use for renaming + + Returns: + Tuple (Path to renamed file or original, whether file was renamed) + """ + filename = mapping_path.name + + # If test_name is provided, use it for the new filename + if test_name: + ext = mapping_path.suffix + new_filename = f"mapping-{test_name}{ext}" + new_path = mapping_path.parent / new_filename + + if new_path.exists() and new_path != mapping_path: + print(f"⚠️ Warning: Target mapping file already exists: {new_path}") + return (mapping_path, False) + + if new_filename != filename: + try: + mapping_path.rename(new_path) + print(f"📝 Renamed mapping file: {filename} -> {new_filename}") + return (new_path, True) + except Exception as e: + print(f"❌ Error renaming mapping file: {e}") + return (mapping_path, False) + + return (mapping_path, False) + + # Otherwise, just remove API key from filename + api_key_pattern = r'-us1-[a-f0-9]{32}' + if re.search(api_key_pattern, filename): + new_filename = re.sub(api_key_pattern, '', filename) + new_path = mapping_path.parent / new_filename + + if new_path.exists(): + print(f"⚠️ Warning: Target mapping file already exists: {new_path}") + return (mapping_path, False) + + try: + mapping_path.rename(new_path) + print(f"📝 Renamed mapping file: {filename} -> {new_filename}") + return (new_path, True) + except Exception as e: + print(f"❌ Error renaming mapping file: {e}") + return (mapping_path, False) + + return (mapping_path, False) + + +def sanitize_mapping_data(mapping_data: Dict[str, Any], mapping_path: Path, test_name: str = None) -> Tuple[Dict[str, Any], bool]: + """ + Sanitizes mapping data by: + 1. Removing API key from URL + 2. Renaming response body file and updating reference + + Args: + mapping_data: Mapping data from JSON + mapping_path: Path to mapping file (for locating body files) + test_name: Optional test name to use for renaming body file + + Returns: + Tuple (updated mapping data, whether any changes were made) + """ + changes_made = False + + # Sanitize URL + request_data = mapping_data.get('request', {}) + original_url = request_data.get('url') or request_data.get('urlPattern', '') + + if original_url: + sanitized_url = sanitize_url(original_url) + + if original_url != sanitized_url: + print(f"🔐 Sanitized API key from URL:") + print(f" Before: {original_url}") + print(f" After: {sanitized_url}") + + # After sanitization, URL contains regex pattern, so use urlPattern + # Remove 'url' if it exists and use 'urlPattern' instead + if 'url' in request_data: + del mapping_data['request']['url'] + print(f" Changed 'url' to 'urlPattern' (contains regex)") + mapping_data['request']['urlPattern'] = sanitized_url + + changes_made = True + + # Sanitize response body filename + response_data = mapping_data.get('response', {}) + original_body_filename = response_data.get('bodyFileName', '') + + if original_body_filename: + old_body_filename, new_body_filename = sanitize_body_filename(original_body_filename, test_name) + + if old_body_filename != new_body_filename: + # Rename the file + if rename_body_file(old_body_filename, new_body_filename, mapping_path): + # Update mapping data with new filename + mapping_data['response']['bodyFileName'] = new_body_filename + print(f"🔐 Updated body filename reference in mapping:") + print(f" Before: {old_body_filename}") + print(f" After: {new_body_filename}") + changes_made = True + + return (mapping_data, changes_made) + + +def save_mapping_file(mapping_data: Dict[str, Any], mapping_path: Path) -> None: + """ + Saves mapping data to file. + + Args: + mapping_data: Mapping data to save + mapping_path: Path where to save the file + """ + try: + with open(mapping_path, 'w', encoding='utf-8') as f: + json.dump(mapping_data, f, indent=2, ensure_ascii=False) + print(f"✅ Saved updated mapping file: {mapping_path}") + except Exception as e: + print(f"❌ Error saving mapping file: {e}") + sys.exit(1) + + +def main(): + # Set up command line argument parser + parser = argparse.ArgumentParser( + description='Sanitize WireMock mapping files by removing API keys and renaming based on test name', + formatter_class=argparse.RawDescriptionHelpFormatter, + epilog=""" +Examples: + # Sanitize API keys and rename files + python3 sanitize_mapping.py wiremock-recordings/mappings/mapping-v1-us1-abc123-identify.json --test-name identify + python3 sanitize_mapping.py wiremock-recordings/mappings/mapping-v2-us1-abc123-events.json --test-name log-event + """ + ) + + parser.add_argument( + 'mapping_file', + help='Path to WireMock mapping file' + ) + + parser.add_argument( + '--test-name', + dest='test_name', + required=True, + help='Test name to use for renaming mapping and body files (e.g., "log-event", "identify", "transaction-complex-attrs")' + ) + + args = parser.parse_args() + + print(f"🔍 Processing mapping file: {args.mapping_file}") + print(f"📝 Test name: {args.test_name}") + print() + + # Load mapping file + mapping_path, mapping_data = load_mapping_file(args.mapping_file) + + # Sanitize mapping data (URL and body filename) + mapping_data, data_changes_made = sanitize_mapping_data(mapping_data, mapping_path, args.test_name) + + # Rename mapping file itself + mapping_path, file_was_renamed = rename_mapping_file(mapping_path, args.test_name) + + # Save mapping file if any changes were made + if data_changes_made: + save_mapping_file(mapping_data, mapping_path) + + # Print summary + print() + if data_changes_made or file_was_renamed: + print("✅ Sanitization complete!") + if file_was_renamed: + print(" - Mapping file renamed") + if data_changes_made: + print(" - Mapping data updated") + else: + print("ℹ️ No changes needed - mapping is already sanitized") + + +if __name__ == "__main__": + main() + diff --git a/IntegrationTests/transform_mapping_body.py b/IntegrationTests/transform_mapping_body.py new file mode 100644 index 00000000..15e454e9 --- /dev/null +++ b/IntegrationTests/transform_mapping_body.py @@ -0,0 +1,357 @@ +#!/usr/bin/env python3 +""" +Script for transforming request body in WireMock mappings. + +This script can: +- Unescape JSON body from string to formatted JSON object +- Escape JSON body from object back to string format +- Update dynamic fields with ${json-unit.ignore} placeholder + +Usage: + python3 transform_mapping_body.py + +Modes: + unescape - Convert equalToJson from escaped string to formatted JSON object in file + escape - Convert equalToJson from JSON object back to escaped string in file + unescape+update - Parse, replace dynamic fields with ${json-unit.ignore}, convert to JSON object, and save + +Examples: + python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json unescape + python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json escape + python3 transform_mapping_body.py wiremock-recordings/mappings/mapping-v1-identify.json unescape+update +""" + +import json +import sys +import argparse +from pathlib import Path +from typing import Dict, Any, List, Union + + +# Default list of fields to replace with ${json-unit.ignore} +# Based on existing WireMock mappings that contain dynamic/timestamp values +DEFAULT_REPLACE_FIELDS = [ + 'a', # App ID + 'bid', # Bundle ID / Build ID + 'bsv', # Build System Version + 'ck', # Cookies (appears after first API response) + 'ct', # Creation Time / Current Time + 'das', # Device Application Stamp + 'dfs', # Device Fingerprint String + 'dlc', # Device Locale + 'dn', # Device Name + 'dosv', # Device OS Version + 'en', # Event Number (position in session, e.g., 0, 1, 2...) + 'est', # Event Start Time + 'ict', # Init Config Time + 'id', # ID (various message/event IDs) + 'lud', # Last Update Date + 'sct', # Session Creation Time + 'sid', # Session ID + 'vid', # Vendor ID +] + + +def replace_field_value(data: Union[Dict, List, Any], field_name: str, replacement_value: str) -> Union[Dict, List, Any]: + """ + Recursively replaces the value of a specified field in a JSON structure. + + Args: + data: JSON data (dict, list, or primitive value) + field_name: Name of the field to replace + replacement_value: New value to set for the field + + Returns: + Modified data structure with replaced field values + """ + if isinstance(data, dict): + result = {} + for key, value in data.items(): + if key == field_name: + result[key] = replacement_value + else: + result[key] = replace_field_value(value, field_name, replacement_value) + return result + elif isinstance(data, list): + return [replace_field_value(item, field_name, replacement_value) for item in data] + else: + return data + + +def replace_fields_from_list(data: Union[Dict, List, Any], field_names: List[str], replacement_value: str = "${json-unit.ignore}") -> Union[Dict, List, Any]: + """ + Replaces values of multiple fields in a JSON structure with a specified value. + + Args: + data: JSON data (dict, list, or primitive value) + field_names: List of field names to replace + replacement_value: Value to use for replacement (default: "${json-unit.ignore}") + + Returns: + Modified data structure with all specified fields replaced + """ + result = data + for field_name in field_names: + result = replace_field_value(result, field_name, replacement_value) + return result + + +def load_mapping_file(mapping_file: str) -> tuple[Path, Dict[str, Any]]: + """ + Loads and parses mapping file. + + Args: + mapping_file: Path to mapping file + + Returns: + Tuple (Path to file, mapping data from JSON) + """ + mapping_path = Path(mapping_file) + if not mapping_path.exists(): + print(f"❌ Error: mapping file not found: {mapping_file}") + sys.exit(1) + + try: + with open(mapping_path, 'r', encoding='utf-8') as f: + mapping_data = json.load(f) + return (mapping_path, mapping_data) + except json.JSONDecodeError as e: + print(f"❌ Error: failed to parse JSON from mapping file: {e}") + sys.exit(1) + except Exception as e: + print(f"❌ Error reading mapping file: {e}") + sys.exit(1) + + +def get_request_body_from_mapping(mapping_data: Dict[str, Any]) -> Dict[str, Any]: + """ + Extracts and unescapes request body from mapping data. + + Args: + mapping_data: Mapping data from JSON + + Returns: + Parsed request body as JSON object + """ + try: + request_data = mapping_data.get('request', {}) + body_patterns = request_data.get('bodyPatterns', []) + + if not body_patterns: + print("❌ Error: bodyPatterns not found in mapping") + sys.exit(1) + + equal_to_json = body_patterns[0].get('equalToJson') + if equal_to_json is None: + print("❌ Error: equalToJson not found in bodyPatterns") + sys.exit(1) + + # If equalToJson is already a dict/list (JSON object), return it directly + if isinstance(equal_to_json, (dict, list)): + return equal_to_json + + # Otherwise, parse escaped JSON string to actual JSON object + request_body = json.loads(equal_to_json) + return request_body + + except json.JSONDecodeError as e: + print(f"❌ Error: failed to parse JSON from equalToJson: {e}") + sys.exit(1) + except Exception as e: + print(f"❌ Error extracting body from mapping: {e}") + sys.exit(1) + + +def set_request_body_in_mapping(mapping_data: Dict[str, Any], request_body: Dict[str, Any], as_string: bool = True) -> Dict[str, Any]: + """ + Sets request body in mapping data. + + Args: + mapping_data: Mapping data from JSON + request_body: Request body as JSON object + as_string: If True, convert to escaped string. If False, keep as JSON object. + + Returns: + Updated mapping data + """ + try: + # Update mapping data + if 'request' not in mapping_data: + mapping_data['request'] = {} + if 'bodyPatterns' not in mapping_data['request']: + mapping_data['request']['bodyPatterns'] = [{}] + + if as_string: + # Convert JSON object to escaped string (for escape mode) + escaped_json = json.dumps(request_body, ensure_ascii=False) + mapping_data['request']['bodyPatterns'][0]['equalToJson'] = escaped_json + else: + # Keep as JSON object (for unescape mode) + mapping_data['request']['bodyPatterns'][0]['equalToJson'] = request_body + + mapping_data['request']['bodyPatterns'][0]['ignoreExtraElements'] = False + + return mapping_data + + except Exception as e: + print(f"❌ Error setting body in mapping: {e}") + sys.exit(1) + + +def mode_unescape(mapping_data: Dict[str, Any], mapping_path: Path) -> None: + """ + Mode: unescape + Converts equalToJson from escaped string to formatted JSON object and saves to file. + + Args: + mapping_data: Mapping data from JSON + mapping_path: Path to mapping file + """ + request_body = get_request_body_from_mapping(mapping_data) + + print(f"📄 Mapping file: {mapping_path}") + print(f"🔄 Converting equalToJson from string to formatted JSON object...") + + # Replace escaped string with actual JSON object + if 'request' in mapping_data and 'bodyPatterns' in mapping_data['request']: + mapping_data['request']['bodyPatterns'][0]['equalToJson'] = request_body + mapping_data['request']['bodyPatterns'][0]['ignoreExtraElements'] = False + + # Save updated mapping back to file + try: + # Save updated mapping + with open(mapping_path, 'w', encoding='utf-8') as f: + json.dump(mapping_data, f, indent=2, ensure_ascii=False) + + print(f"✅ Mapping file updated successfully!") + print(f"📝 equalToJson is now a formatted JSON object:\n") + print(json.dumps(request_body, indent=2, ensure_ascii=False)) + + except Exception as e: + print(f"❌ Error saving updated mapping file: {e}") + sys.exit(1) + + +def mode_escape(mapping_data: Dict[str, Any], mapping_path: Path) -> None: + """ + Mode: escape + Converts equalToJson from JSON object to escaped string and saves to file. + + Args: + mapping_data: Mapping data from JSON + mapping_path: Path to mapping file + """ + # Check if equalToJson is already a string (nothing to do) + try: + equal_to_json = mapping_data['request']['bodyPatterns'][0]['equalToJson'] + + # If it's already a string, nothing to escape + if isinstance(equal_to_json, str): + print(f"📄 Mapping file: {mapping_path}") + print(f"ℹ️ equalToJson is already a string (escaped format)") + print(f"✅ No action needed") + return + + # It's a JSON object, convert it to string + print(f"📄 Mapping file: {mapping_path}") + print(f"🔄 Converting equalToJson from JSON object to escaped string...") + + escaped_json = json.dumps(equal_to_json, ensure_ascii=False) + mapping_data['request']['bodyPatterns'][0]['equalToJson'] = escaped_json + mapping_data['request']['bodyPatterns'][0]['ignoreExtraElements'] = False + + # Save updated mapping back to file + with open(mapping_path, 'w', encoding='utf-8') as f: + json.dump(mapping_data, f, indent=2, ensure_ascii=False) + + print(f"✅ Mapping file updated successfully!") + print(f"📝 equalToJson is now an escaped string") + + except KeyError: + print(f"❌ Error: equalToJson not found in mapping") + sys.exit(1) + except Exception as e: + print(f"❌ Error: {e}") + sys.exit(1) + + +def mode_unescape_update(mapping_data: Dict[str, Any], mapping_path: Path) -> None: + """ + Mode: unescape+update + Unescapes body, replaces dynamic fields, converts to JSON object, and saves back to mapping file. + + Args: + mapping_data: Mapping data from JSON + mapping_path: Path to mapping file + """ + request_body = get_request_body_from_mapping(mapping_data) + + print(f"📄 Mapping file: {mapping_path}") + print(f"🔄 Replacing {len(DEFAULT_REPLACE_FIELDS)} dynamic fields with ${{json-unit.ignore}}...") + + # Replace dynamic fields + updated_body = replace_fields_from_list(request_body, DEFAULT_REPLACE_FIELDS) + + # Update mapping data with modified body as JSON object (not string) + mapping_data = set_request_body_in_mapping(mapping_data, updated_body, as_string=False) + + # Save updated mapping back to file + try: + # Save updated mapping + with open(mapping_path, 'w', encoding='utf-8') as f: + json.dump(mapping_data, f, indent=2, ensure_ascii=False) + + print(f"✅ Mapping file updated successfully!") + print(f"📝 Updated body with replaced fields:\n") + print(json.dumps(updated_body, indent=2, ensure_ascii=False)) + + except Exception as e: + print(f"❌ Error saving updated mapping file: {e}") + sys.exit(1) + + +def main(): + parser = argparse.ArgumentParser( + description='Transform request body in WireMock mappings', + formatter_class=argparse.RawDescriptionHelpFormatter, + epilog=""" +Modes: + unescape Convert equalToJson from escaped string to formatted JSON object + escape Convert equalToJson from JSON object back to escaped string + unescape+update Parse, replace dynamic fields, convert to JSON object, and save + +Examples: + python3 transform_mapping_body.py mappings/mapping-v1-identify.json unescape + python3 transform_mapping_body.py mappings/mapping-v1-identify.json escape + python3 transform_mapping_body.py mappings/mapping-v1-identify.json unescape+update + """ + ) + + parser.add_argument( + 'mapping_file', + help='Path to WireMock mapping file' + ) + + parser.add_argument( + 'mode', + choices=['unescape', 'escape', 'unescape+update'], + help='Operation mode' + ) + + args = parser.parse_args() + + # Load mapping file + mapping_path, mapping_data = load_mapping_file(args.mapping_file) + + # Execute based on mode + if args.mode == 'unescape': + mode_unescape(mapping_data, mapping_path) + elif args.mode == 'escape': + mode_escape(mapping_data, mapping_path) + elif args.mode == 'unescape+update': + mode_unescape_update(mapping_data, mapping_path) + + +if __name__ == "__main__": + main() + diff --git a/IntegrationTests/update_mapping_from_extracted.py b/IntegrationTests/update_mapping_from_extracted.py deleted file mode 100644 index 73982b07..00000000 --- a/IntegrationTests/update_mapping_from_extracted.py +++ /dev/null @@ -1,145 +0,0 @@ -#!/usr/bin/env python3 -""" -Script to update WireMock mapping from extracted request body file. - -Usage: - python3 update_mapping_from_extracted.py - -Example: - python3 update_mapping_from_extracted.py wiremock-recordings/requests/identify_test.json -""" - -import json -import sys -from pathlib import Path -from typing import Dict, Any - - -def update_mapping(extracted_file: str) -> None: - """ - Updates WireMock mapping from extracted request body file. - - Args: - extracted_file: Path to extracted file - """ - # Check if file exists - extracted_path = Path(extracted_file) - if not extracted_path.exists(): - print(f"❌ Error: extracted file not found: {extracted_file}") - sys.exit(1) - - # Read extracted file - try: - with open(extracted_path, 'r', encoding='utf-8') as f: - extracted_data = json.load(f) - except json.JSONDecodeError as e: - print(f"❌ Error: failed to parse JSON from extracted file: {e}") - sys.exit(1) - except Exception as e: - print(f"❌ Error reading extracted file: {e}") - sys.exit(1) - - # Extract data - try: - test_name = extracted_data.get('test_name') - source_mapping = extracted_data.get('source_mapping') - request_body = extracted_data.get('request_body') - - if not source_mapping: - print("❌ Error: source_mapping not found in extracted file") - sys.exit(1) - - if request_body is None: - print("❌ Error: request_body not found in extracted file") - sys.exit(1) - - except Exception as e: - print(f"❌ Error extracting data from extracted file: {e}") - sys.exit(1) - - # Check if source mapping exists - mapping_path = Path(source_mapping) - if not mapping_path.exists(): - print(f"❌ Error: source mapping not found: {source_mapping}") - sys.exit(1) - - # Read source mapping - try: - with open(mapping_path, 'r', encoding='utf-8') as f: - mapping_data = json.load(f) - except json.JSONDecodeError as e: - print(f"❌ Error: failed to parse JSON from mapping: {e}") - sys.exit(1) - except Exception as e: - print(f"❌ Error reading mapping: {e}") - sys.exit(1) - - # Update request body in mapping - try: - # Convert request_body back to escaped JSON string - escaped_json = json.dumps(request_body, ensure_ascii=False, separators=(',', ':')) - - # Update bodyPatterns - if 'request' not in mapping_data: - mapping_data['request'] = {} - - if 'bodyPatterns' not in mapping_data['request']: - mapping_data['request']['bodyPatterns'] = [{}] - - mapping_data['request']['bodyPatterns'][0]['equalToJson'] = escaped_json - - # Update URL from extracted data (supports both url and urlPattern) - request_url = extracted_data.get('request_url') - if request_url: - # If URL contains regex pattern, use urlPattern, otherwise use url - if '[' in request_url or '\\' in request_url: - mapping_data['request']['urlPattern'] = request_url - # Remove old 'url' field if it exists - mapping_data['request'].pop('url', None) - else: - mapping_data['request']['url'] = request_url - # Remove old 'urlPattern' field if it exists - mapping_data['request'].pop('urlPattern', None) - - except Exception as e: - print(f"❌ Error updating mapping: {e}") - sys.exit(1) - - # Save updated mapping - try: - with open(mapping_path, 'w', encoding='utf-8') as f: - json.dump(mapping_data, f, indent=2, ensure_ascii=False) - - print(f"✅ Mapping successfully updated: {mapping_path}") - print(f"📝 Test: {test_name}") - - except Exception as e: - print(f"❌ Error saving mapping: {e}") - sys.exit(1) - - -def main(): - """Main function.""" - # Check command line arguments - if len(sys.argv) != 2: - print("❌ Error: invalid number of arguments") - print() - print("Usage:") - print(f" {sys.argv[0]} ") - print() - print("Example:") - print(f" {sys.argv[0]} wiremock-recordings/requests/identify_test.json") - sys.exit(1) - - extracted_file = sys.argv[1] - - # Warning - print("⚠️ WARNING: This script will overwrite the existing mapping!") - print() - - update_mapping(extracted_file) - - -if __name__ == "__main__": - main() - diff --git a/IntegrationTests/wiremock-recordings/__files/body-commerce-event-purchase.json b/IntegrationTests/wiremock-recordings/__files/body-commerce-event-purchase.json new file mode 100644 index 00000000..cfc11fc9 --- /dev/null +++ b/IntegrationTests/wiremock-recordings/__files/body-commerce-event-purchase.json @@ -0,0 +1 @@ +{"dt":"rh","id":"672e5a89-0570-4858-b266-9e329e179603","ct":1763666321288,"msgs":[],"ci":{"mpid":6504934091054997508,"ck":{"uid":{"c":"g=19f202d6-e284-4631-aebf-ad9547740d2f","e":"2035-11-04T19:37:06.7933933Z"}},"das":"19f202d6-e284-4631-aebf-ad9547740d2f"}} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/__files/body-log-event-with-flags.json b/IntegrationTests/wiremock-recordings/__files/body-log-event-with-flags.json new file mode 100644 index 00000000..98263ef5 --- /dev/null +++ b/IntegrationTests/wiremock-recordings/__files/body-log-event-with-flags.json @@ -0,0 +1 @@ +{"dt":"rh","id":"3516ca06-20df-4e0a-b1ab-e6ca01c863a0","ct":1763665080833,"msgs":[],"ci":{"mpid":6504934091054997508,"ck":{"uid":{"c":"g=6a8af173-8a25-4920-9069-b565e53d287e","e":"2035-11-04T19:37:06.7933933Z"}},"das":"6a8af173-8a25-4920-9069-b565e53d287e"}} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/__files/body-log-screen.json b/IntegrationTests/wiremock-recordings/__files/body-log-screen.json new file mode 100644 index 00000000..4b7713f9 --- /dev/null +++ b/IntegrationTests/wiremock-recordings/__files/body-log-screen.json @@ -0,0 +1 @@ +{"dt":"rh","id":"0ec49fd1-a90c-4dcf-87dd-815b2811c20a","ct":1763665684829,"msgs":[],"ci":{"mpid":6504934091054997508,"ck":{"uid":{"c":"g=05c4b166-dafa-46c9-8d7d-395d7a3c007f","e":"2035-11-04T19:37:06.7933933Z"}},"das":"05c4b166-dafa-46c9-8d7d-395d7a3c007f"}} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/__files/body-rokt-select-placement.json b/IntegrationTests/wiremock-recordings/__files/body-rokt-select-placement.json new file mode 100644 index 00000000..02c2a832 --- /dev/null +++ b/IntegrationTests/wiremock-recordings/__files/body-rokt-select-placement.json @@ -0,0 +1 @@ +{"dt":"rh","id":"ff15dc58-7690-44b6-9856-2829be512f1d","ct":1763667082278,"msgs":[],"ci":{"mpid":6504934091054997508,"ck":{"uid":{"c":"g=07f1d026-4175-4bec-a77f-f3389dbe048e","e":"2035-11-04T19:37:06.7933933Z"}},"das":"07f1d026-4175-4bec-a77f-f3389dbe048e"}} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/mappings/mapping-commerce-event-purchase.json b/IntegrationTests/wiremock-recordings/mappings/mapping-commerce-event-purchase.json new file mode 100644 index 00000000..beed453e --- /dev/null +++ b/IntegrationTests/wiremock-recordings/mappings/mapping-commerce-event-purchase.json @@ -0,0 +1,137 @@ +{ + "id": "6fd492b0-abdb-39f1-976e-8e7fb6f7ed00", + "request": { + "method": "POST", + "bodyPatterns": [ + { + "equalToJson": { + "id": "${json-unit.ignore}", + "ltv": 0, + "msgs": [ + { + "sct": "${json-unit.ignore}", + "pd": { + "tr": "12.09", + "pl": [ + { + "ca": "Fiction", + "nm": "Awesome Book", + "qt": "1", + "id": "${json-unit.ignore}", + "tpa": "9.99", + "pr": "9.99", + "attrs": { + "custom key": "custom value" + }, + "br": "A Publisher", + "cc": "XYZ123", + "ps": "1" + } + ], + "an": "purchase", + "co": "Credit Card", + "ti": "zyx098", + "ts": "1.23", + "ta": "Book seller", + "tt": "0.87", + "cs": "4" + }, + "ct": "${json-unit.ignore}", + "dt": "cm", + "id": "${json-unit.ignore}", + "sn": "Timeless Books", + "sid": "${json-unit.ignore}", + "attrs": { + "an_extra_key": "an_extra_value" + }, + "mt": false, + "vc": "off_thread" + } + ], + "dt": "h", + "a": "${json-unit.ignore}", + "ai": { + "av": "1.0", + "sideloaded_kits_count": 0, + "bid": "${json-unit.ignore}", + "pir": false, + "fi": false, + "lud": "${json-unit.ignore}", + "arc": "arm64", + "tsv": "90000", + "apn": "com.mparticle.IntegrationTests", + "env": 1, + "abn": "1", + "bsv": "${json-unit.ignore}", + "ict": "${json-unit.ignore}" + }, + "ct": "${json-unit.ignore}", + "ck": "${json-unit.ignore}", + "das": "${json-unit.ignore}", + "mpid": 6504934091054997508, + "ui": [ + { + "i": "123456", + "n": 1 + }, + { + "i": "foo@example.com", + "n": 7 + } + ], + "uitl": 60, + "oo": false, + "sdk": "8.40.0", + "di": { + "b": "arm64", + "p": "arm64", + "tz": "-5", + "dn": "${json-unit.ignore}", + "dma": "Apple", + "dlc": "${json-unit.ignore}", + "dsh": "1440", + "dll": "en", + "idst": false, + "vid": "${json-unit.ignore}", + "dp": "iOS", + "tzn": "America/New_York", + "dsw": "960", + "dosv": "${json-unit.ignore}", + "it": false, + "dmdl": "arm64", + "dr": "None", + "jb": { + "cydia": false + }, + "lat": false, + "arc": "arm64e", + "bid": "${json-unit.ignore}" + }, + "stl": 60 + }, + "ignoreArrayOrder": true, + "ignoreExtraElements": false + } + ], + "urlPattern": "/v2/us1-[a-f0-9]+/events" + }, + "response": { + "status": 202, + "bodyFileName": "body-commerce-event-purchase.json", + "headers": { + "X-Cache": "MISS, MISS", + "X-MP-Trace-Id": "691f69910145954350c1981a987e0d31", + "Server": "Kestrel", + "X-Origin-Name": "fastlyshield--shield_ssl_cache_iad_kiad7000043_IAD", + "Date": "Thu, 20 Nov 2025 19:18:41 GMT", + "X-Timer": "S1763666321.275657,VS0,VE17", + "Via": "1.1 varnish, 1.1 varnish", + "Accept-Ranges": "bytes", + "X-Served-By": "cache-iad-kiad7000043-IAD, cache-lga21934-LGA", + "Vary": "Accept-Encoding", + "X-Cache-Hits": "0, 0", + "Content-Type": "application/json" + } + }, + "uuid": "6fd492b0-abdb-39f1-976e-8e7fb6f7ed00" +} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/mappings/mapping-log-event-with-flags.json b/IntegrationTests/wiremock-recordings/mappings/mapping-log-event-with-flags.json new file mode 100644 index 00000000..d0095edc --- /dev/null +++ b/IntegrationTests/wiremock-recordings/mappings/mapping-log-event-with-flags.json @@ -0,0 +1,123 @@ +{ + "id": "122cf764-f4d5-3671-9259-e3e45cad2a24", + "request": { + "method": "POST", + "bodyPatterns": [ + { + "equalToJson": { + "id": "${json-unit.ignore}", + "ltv": 0, + "msgs": [ + { + "et": "Transaction", + "id": "${json-unit.ignore}", + "vc": "off_thread", + "dt": "e", + "flags": { + "Not_forwarded_to_providers": [ + "Top Secret" + ] + }, + "ct": "${json-unit.ignore}", + "el": 0, + "attrs": { + "test Dictionary": "{\n test1 = test;\n test2 = 2;\n test3 = \"2023-11-14 22:13:20 +0000\";\n}", + "A Date Key": "2023-11-14T22:13:20+0000", + "A Number Key": "42", + "A_String_Key": "A String Value" + }, + "sct": "${json-unit.ignore}", + "sid": "${json-unit.ignore}", + "en": "${json-unit.ignore}", + "est": "${json-unit.ignore}", + "n": "Event Name", + "mt": false + } + ], + "dt": "h", + "a": "${json-unit.ignore}", + "ai": { + "av": "1.0", + "sideloaded_kits_count": 0, + "bid": "${json-unit.ignore}", + "pir": false, + "fi": false, + "lud": "${json-unit.ignore}", + "arc": "arm64", + "tsv": "90000", + "apn": "com.mparticle.IntegrationTests", + "env": 1, + "abn": "1", + "bsv": "${json-unit.ignore}", + "ict": "${json-unit.ignore}" + }, + "ct": "${json-unit.ignore}", + "ck": "${json-unit.ignore}", + "das": "${json-unit.ignore}", + "mpid": 6504934091054997508, + "ui": [ + { + "i": "123456", + "n": 1 + }, + { + "i": "foo@example.com", + "n": 7 + } + ], + "uitl": 60, + "oo": false, + "sdk": "8.40.0", + "di": { + "tz": "-5", + "p": "arm64", + "b": "arm64", + "dn": "${json-unit.ignore}", + "dll": "en", + "dsh": "1440", + "idst": false, + "dlc": "${json-unit.ignore}", + "vid": "${json-unit.ignore}", + "dma": "Apple", + "dp": "iOS", + "tzn": "America/New_York", + "dsw": "960", + "dosv": "${json-unit.ignore}", + "it": false, + "dr": "None", + "dmdl": "arm64", + "jb": { + "cydia": false + }, + "lat": false, + "arc": "arm64e", + "bid": "${json-unit.ignore}" + }, + "stl": 60 + }, + "ignoreArrayOrder": true, + "ignoreExtraElements": false + } + ], + "urlPattern": "/v2/us1-[a-f0-9]+/events" + }, + "response": { + "status": 202, + "bodyFileName": "body-log-event-with-flags.json", + "headers": { + "X-Cache": "MISS, MISS", + "X-MP-Trace-Id": "691f64b86c4734235e4f789de20cf675", + "Server": "Kestrel", + "X-Origin-Name": "fastlyshield--shield_ssl_cache_iad_kiad7000142_IAD", + "Date": "Thu, 20 Nov 2025 18:58:00 GMT", + "X-Timer": "S1763665081.826916,VS0,VE11", + "Via": "1.1 varnish, 1.1 varnish", + "Accept-Ranges": "bytes", + "X-Served-By": "cache-iad-kiad7000142-IAD, cache-lga21976-LGA", + "Vary": "Accept-Encoding", + "X-Cache-Hits": "0, 0", + "Content-Type": "application/json" + } + }, + "uuid": "122cf764-f4d5-3671-9259-e3e45cad2a24" +} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/mappings/mapping-log-screen.json b/IntegrationTests/wiremock-recordings/mappings/mapping-log-screen.json new file mode 100644 index 00000000..2eb9977f --- /dev/null +++ b/IntegrationTests/wiremock-recordings/mappings/mapping-log-screen.json @@ -0,0 +1,110 @@ +{ + "id": "57ecf4ed-9d3c-3d1d-b0aa-0a337e10ffbe", + "request": { + "method": "POST", + "bodyPatterns": [ + { + "equalToJson": { + "id": "${json-unit.ignore}", + "ltv": 0, + "msgs": [ + { + "vc": "off_thread", + "sct": "${json-unit.ignore}", + "dt": "v", + "ct": "${json-unit.ignore}", + "id": "${json-unit.ignore}", + "sid": "${json-unit.ignore}", + "n": "Home Screen", + "mt": false, + "est": "${json-unit.ignore}", + "el": 0 + } + ], + "dt": "h", + "a": "${json-unit.ignore}", + "ai": { + "av": "1.0", + "sideloaded_kits_count": 0, + "bid": "${json-unit.ignore}", + "pir": false, + "fi": false, + "lud": "${json-unit.ignore}", + "arc": "arm64", + "tsv": "90000", + "apn": "com.mparticle.IntegrationTests", + "env": 1, + "abn": "1", + "bsv": "${json-unit.ignore}", + "ict": "${json-unit.ignore}" + }, + "ct": "${json-unit.ignore}", + "ck": "${json-unit.ignore}", + "das": "${json-unit.ignore}", + "mpid": 6504934091054997508, + "ui": [ + { + "i": "123456", + "n": 1 + }, + { + "i": "foo@example.com", + "n": 7 + } + ], + "uitl": 60, + "oo": false, + "sdk": "8.40.0", + "di": { + "tz": "-5", + "bid": "${json-unit.ignore}", + "p": "arm64", + "dn": "${json-unit.ignore}", + "dma": "Apple", + "dsh": "1440", + "idst": false, + "dlc": "${json-unit.ignore}", + "vid": "${json-unit.ignore}", + "dll": "en", + "tzn": "America/New_York", + "dp": "iOS", + "dsw": "960", + "dosv": "${json-unit.ignore}", + "it": false, + "dr": "None", + "dmdl": "arm64", + "jb": { + "cydia": false + }, + "lat": false, + "arc": "arm64e", + "b": "arm64" + }, + "stl": 60 + }, + "ignoreArrayOrder": true, + "ignoreExtraElements": false + } + ], + "urlPattern": "/v2/us1-[a-f0-9]+/events" + }, + "response": { + "status": 202, + "bodyFileName": "body-log-screen.json", + "headers": { + "X-Cache": "MISS, MISS", + "X-MP-Trace-Id": "691f67140c4ce7e4e718720bc5eca924", + "Server": "Kestrel", + "X-Origin-Name": "fastlyshield--shield_ssl_cache_iad_kiad7000054_IAD", + "Date": "Thu, 20 Nov 2025 19:08:04 GMT", + "X-Timer": "S1763665685.817972,VS0,VE16", + "Via": "1.1 varnish, 1.1 varnish", + "Accept-Ranges": "bytes", + "X-Served-By": "cache-iad-kiad7000054-IAD, cache-lga21948-LGA", + "Vary": "Accept-Encoding", + "X-Cache-Hits": "0, 0", + "Content-Type": "application/json" + } + }, + "uuid": "57ecf4ed-9d3c-3d1d-b0aa-0a337e10ffbe" +} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/mappings/mapping-rokt-select-placement.json b/IntegrationTests/wiremock-recordings/mappings/mapping-rokt-select-placement.json new file mode 100644 index 00000000..ef4fc23a --- /dev/null +++ b/IntegrationTests/wiremock-recordings/mappings/mapping-rokt-select-placement.json @@ -0,0 +1,119 @@ +{ + "id": "c2644c1c-86a7-3f6d-aa7f-071eea68634e", + "request": { + "method": "POST", + "bodyPatterns": [ + { + "equalToJson": { + "id": "${json-unit.ignore}", + "ltv": 0, + "msgs": [ + { + "sct": "${json-unit.ignore}", + "oi": { + "dfs": "${json-unit.ignore}", + "i": "foo@example.com", + "n": 7, + "f": false + }, + "ct": "${json-unit.ignore}", + "ni": { + "dfs": "${json-unit.ignore}", + "i": "j.smit@example.com", + "n": 7, + "f": false + }, + "dt": "uic", + "id": "${json-unit.ignore}", + "sid": "${json-unit.ignore}", + "mt": false, + "vc": "off_thread" + } + ], + "dt": "h", + "a": "${json-unit.ignore}", + "ai": { + "av": "1.0", + "sideloaded_kits_count": 0, + "bid": "${json-unit.ignore}", + "pir": false, + "fi": false, + "lud": "${json-unit.ignore}", + "arc": "arm64", + "tsv": "90000", + "apn": "com.mparticle.IntegrationTests", + "env": 1, + "abn": "1", + "bsv": "${json-unit.ignore}", + "ict": "${json-unit.ignore}" + }, + "ct": "${json-unit.ignore}", + "ck": "${json-unit.ignore}", + "das": "${json-unit.ignore}", + "mpid": 6504934091054997508, + "ui": [ + { + "i": "123456", + "n": 1 + }, + { + "i": "j.smit@example.com", + "n": 7 + } + ], + "uitl": 60, + "oo": false, + "sdk": "8.40.0", + "di": { + "p": "arm64", + "tz": "-5", + "bid": "${json-unit.ignore}", + "dn": "${json-unit.ignore}", + "dll": "en", + "dsh": "1440", + "dlc": "${json-unit.ignore}", + "dma": "Apple", + "vid": "${json-unit.ignore}", + "idst": false, + "dp": "iOS", + "tzn": "America/New_York", + "dsw": "960", + "dosv": "${json-unit.ignore}", + "it": false, + "dmdl": "arm64", + "dr": "None", + "jb": { + "cydia": false + }, + "lat": false, + "arc": "arm64e", + "b": "arm64" + }, + "stl": 60 + }, + "ignoreArrayOrder": true, + "ignoreExtraElements": false + } + ], + "urlPattern": "/v2/us1-[a-f0-9]+/events" + }, + "response": { + "status": 202, + "bodyFileName": "body-rokt-select-placement.json", + "headers": { + "X-Cache": "MISS, MISS", + "X-MP-Trace-Id": "691f6c8a67782ff119fc374a12526891", + "Server": "Kestrel", + "X-Origin-Name": "fastlyshield--shield_ssl_cache_iad_kcgs7200068_IAD", + "Date": "Thu, 20 Nov 2025 19:31:22 GMT", + "X-Timer": "S1763667082.266146,VS0,VE18", + "Via": "1.1 varnish, 1.1 varnish", + "Accept-Ranges": "bytes", + "X-Served-By": "cache-iad-kcgs7200068-IAD, cache-lga21927-LGA", + "Vary": "Accept-Encoding", + "X-Cache-Hits": "0, 0", + "Content-Type": "application/json" + } + }, + "uuid": "c2644c1c-86a7-3f6d-aa7f-071eea68634e" +} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/mappings/mapping-v1-identify.json b/IntegrationTests/wiremock-recordings/mappings/mapping-v1-identify.json index f2e86f48..d7994c11 100644 --- a/IntegrationTests/wiremock-recordings/mappings/mapping-v1-identify.json +++ b/IntegrationTests/wiremock-recordings/mappings/mapping-v1-identify.json @@ -5,9 +5,24 @@ "method": "POST", "bodyPatterns": [ { - "equalToJson": "{\"client_sdk\":{\"platform\":\"ios\",\"sdk_version\":\"8.40.0\",\"sdk_vendor\":\"mparticle\"},\"environment\":\"development\",\"request_timestamp_ms\":\"${json-unit.ignore}\",\"request_id\":\"${json-unit.ignore}\",\"known_identities\":{\"email\":\"foo@example.com\",\"customerid\":\"123456\",\"ios_idfv\":\"${json-unit.ignore}\",\"device_application_stamp\":\"${json-unit.ignore}\"}}", + "equalToJson": { + "client_sdk": { + "platform": "ios", + "sdk_version": "8.40.0", + "sdk_vendor": "mparticle" + }, + "environment": "development", + "request_timestamp_ms": "${json-unit.ignore}", + "request_id": "${json-unit.ignore}", + "known_identities": { + "email": "foo@example.com", + "customerid": "123456", + "ios_idfv": "${json-unit.ignore}", + "device_application_stamp": "${json-unit.ignore}" + } + }, "ignoreArrayOrder": true, - "ignoreExtraElements": true + "ignoreExtraElements": false } ] }, diff --git a/IntegrationTests/wiremock-recordings/mappings/mapping-v2-events-log-event.json b/IntegrationTests/wiremock-recordings/mappings/mapping-v2-events-log-event.json index 032b1997..1630dace 100644 --- a/IntegrationTests/wiremock-recordings/mappings/mapping-v2-events-log-event.json +++ b/IntegrationTests/wiremock-recordings/mappings/mapping-v2-events-log-event.json @@ -5,9 +5,151 @@ "method": "POST", "bodyPatterns": [ { - "equalToJson": "{\"id\":\"${json-unit.ignore}\",\"ltv\":0,\"msgs\":[{\"vc\":\"off_thread\",\"ct\":\"${json-unit.ignore}\",\"psl\":0,\"id\":\"${json-unit.ignore}\",\"dt\":\"ss\",\"mt\":false},{\"sct\":\"${json-unit.ignore}\",\"id\":\"${json-unit.ignore}\",\"ct\":\"${json-unit.ignore}\",\"dt\":\"fr\",\"sid\":\"${json-unit.ignore}\",\"mt\":false,\"vc\":\"off_thread\"},{\"sct\":\"${json-unit.ignore}\",\"dt\":\"uic\",\"ct\":\"${json-unit.ignore}\",\"ni\":{\"dfs\":\"${json-unit.ignore}\",\"i\":\"123456\",\"n\":1,\"f\":true},\"id\":\"${json-unit.ignore}\",\"sid\":\"${json-unit.ignore}\",\"mt\":false,\"vc\":\"off_thread\"},{\"sct\":\"${json-unit.ignore}\",\"dt\":\"uic\",\"ct\":\"${json-unit.ignore}\",\"ni\":{\"dfs\":\"${json-unit.ignore}\",\"i\":\"foo@example.com\",\"n\":7,\"f\":true},\"id\":\"${json-unit.ignore}\",\"sid\":\"${json-unit.ignore}\",\"mt\":false,\"vc\":\"off_thread\"},{\"id\":\"${json-unit.ignore}\",\"vc\":\"off_thread\",\"sf\":true,\"dt\":\"ast\",\"ct\":\"${json-unit.ignore}\",\"sct\":\"${json-unit.ignore}\",\"sc\":true,\"sid\":\"${json-unit.ignore}\",\"t\":\"app_init\",\"ifr\":true,\"nsi\":0,\"mt\":false},{\"et\":\"Other\",\"id\":\"${json-unit.ignore}\",\"vc\":\"off_thread\",\"dt\":\"e\",\"el\":0,\"ct\":\"${json-unit.ignore}\",\"attrs\":{\"SimpleKey\":\"SimpleValue\"},\"sct\":\"${json-unit.ignore}\",\"sid\":\"${json-unit.ignore}\",\"en\":0,\"n\":\"Simple Event Name\",\"est\":\"${json-unit.ignore}\",\"mt\":false}],\"dt\":\"h\",\"a\":\"${json-unit.ignore}\",\"ai\":{\"av\":\"1.0\",\"sideloaded_kits_count\":0,\"bid\":\"${json-unit.ignore}\",\"pir\":false,\"fi\":false,\"lud\":\"${json-unit.ignore}\",\"arc\":\"arm64\",\"tsv\":\"90000\",\"apn\":\"com.mparticle.IntegrationTests\",\"env\":1,\"abn\":\"1\",\"bsv\":\"${json-unit.ignore}\",\"ict\":\"${json-unit.ignore}\"},\"ct\":\"${json-unit.ignore}\",\"das\":\"${json-unit.ignore}\",\"mpid\":6504934091054997508,\"ui\":[{\"i\":\"123456\",\"n\":1},{\"i\":\"foo@example.com\",\"n\":7}],\"uitl\":60,\"oo\":false,\"sdk\":\"8.40.0\",\"di\":{\"p\":\"arm64\",\"bid\":\"${json-unit.ignore}\",\"tz\":\"-5\",\"dn\":\"${json-unit.ignore}\",\"dll\":\"en\",\"dlc\":\"${json-unit.ignore}\",\"dsh\":\"1440\",\"dma\":\"Apple\",\"vid\":\"${json-unit.ignore}\",\"idst\":false,\"tzn\":\"America/New_York\",\"dp\":\"iOS\",\"dsw\":\"960\",\"dosv\":\"${json-unit.ignore}\",\"it\":false,\"dmdl\":\"arm64\",\"dr\":\"None\",\"jb\":{\"cydia\":false},\"lat\":false,\"arc\":\"arm64e\",\"b\":\"arm64\"},\"stl\":60}", + "equalToJson": { + "id": "${json-unit.ignore}", + "ltv": 0, + "msgs": [ + { + "vc": "off_thread", + "ct": "${json-unit.ignore}", + "psl": 0, + "id": "${json-unit.ignore}", + "dt": "ss", + "mt": false + }, + { + "sct": "${json-unit.ignore}", + "id": "${json-unit.ignore}", + "ct": "${json-unit.ignore}", + "dt": "fr", + "sid": "${json-unit.ignore}", + "mt": false, + "vc": "off_thread" + }, + { + "sct": "${json-unit.ignore}", + "dt": "uic", + "ct": "${json-unit.ignore}", + "ni": { + "dfs": "${json-unit.ignore}", + "i": "123456", + "n": 1, + "f": true + }, + "id": "${json-unit.ignore}", + "sid": "${json-unit.ignore}", + "mt": false, + "vc": "off_thread" + }, + { + "sct": "${json-unit.ignore}", + "dt": "uic", + "ct": "${json-unit.ignore}", + "ni": { + "dfs": "${json-unit.ignore}", + "i": "foo@example.com", + "n": 7, + "f": true + }, + "id": "${json-unit.ignore}", + "sid": "${json-unit.ignore}", + "mt": false, + "vc": "off_thread" + }, + { + "id": "${json-unit.ignore}", + "vc": "off_thread", + "sf": true, + "dt": "ast", + "ct": "${json-unit.ignore}", + "sct": "${json-unit.ignore}", + "sc": true, + "sid": "${json-unit.ignore}", + "t": "app_init", + "ifr": true, + "nsi": 0, + "mt": false + }, + { + "et": "Other", + "id": "${json-unit.ignore}", + "vc": "off_thread", + "dt": "e", + "el": 0, + "ct": "${json-unit.ignore}", + "attrs": { + "SimpleKey": "SimpleValue" + }, + "sct": "${json-unit.ignore}", + "sid": "${json-unit.ignore}", + "en": 0, + "n": "Simple Event Name", + "est": "${json-unit.ignore}", + "mt": false + } + ], + "dt": "h", + "a": "${json-unit.ignore}", + "ai": { + "av": "1.0", + "sideloaded_kits_count": 0, + "bid": "${json-unit.ignore}", + "pir": false, + "fi": false, + "lud": "${json-unit.ignore}", + "arc": "arm64", + "tsv": "90000", + "apn": "com.mparticle.IntegrationTests", + "env": 1, + "abn": "1", + "bsv": "${json-unit.ignore}", + "ict": "${json-unit.ignore}" + }, + "ct": "${json-unit.ignore}", + "das": "${json-unit.ignore}", + "mpid": 6504934091054997508, + "ui": [ + { + "i": "123456", + "n": 1 + }, + { + "i": "foo@example.com", + "n": 7 + } + ], + "uitl": 60, + "oo": false, + "sdk": "8.40.0", + "di": { + "p": "arm64", + "bid": "${json-unit.ignore}", + "tz": "-5", + "dn": "${json-unit.ignore}", + "dll": "en", + "dlc": "${json-unit.ignore}", + "dsh": "1440", + "dma": "Apple", + "vid": "${json-unit.ignore}", + "idst": false, + "tzn": "America/New_York", + "dp": "iOS", + "dsw": "960", + "dosv": "${json-unit.ignore}", + "it": false, + "dmdl": "arm64", + "dr": "None", + "jb": { + "cydia": false + }, + "lat": false, + "arc": "arm64e", + "b": "arm64" + }, + "stl": 60 + }, "ignoreArrayOrder": true, - "ignoreExtraElements": true + "ignoreExtraElements": false } ] }, diff --git a/IntegrationTests/wiremock-recordings/requests/identify.json b/IntegrationTests/wiremock-recordings/requests/identify.json deleted file mode 100644 index ba7300f8..00000000 --- a/IntegrationTests/wiremock-recordings/requests/identify.json +++ /dev/null @@ -1,22 +0,0 @@ -{ - "test_name": "identify", - "source_mapping": "wiremock-recordings/mappings/mapping-v1-identify.json", - "request_method": "POST", - "request_url": "/v1/identify", - "request_body": { - "client_sdk": { - "platform": "ios", - "sdk_version": "8.40.0", - "sdk_vendor": "mparticle" - }, - "environment": "development", - "request_timestamp_ms": "${json-unit.ignore}", - "request_id": "${json-unit.ignore}", - "known_identities": { - "email": "foo@example.com", - "customerid": "123456", - "ios_idfv": "${json-unit.ignore}", - "device_application_stamp": "${json-unit.ignore}" - } - } -} \ No newline at end of file diff --git a/IntegrationTests/wiremock-recordings/requests/log_event.json b/IntegrationTests/wiremock-recordings/requests/log_event.json deleted file mode 100644 index 3758cf0a..00000000 --- a/IntegrationTests/wiremock-recordings/requests/log_event.json +++ /dev/null @@ -1,149 +0,0 @@ -{ - "test_name": "log_event", - "source_mapping": "wiremock-recordings/mappings/mapping-v2-events-log-event.json", - "request_method": "POST", - "request_url": "/v2/us1-[a-f0-9]+/events", - "request_body": { - "id": "${json-unit.ignore}", - "ltv": 0, - "msgs": [ - { - "vc": "off_thread", - "ct": "${json-unit.ignore}", - "psl": 0, - "id": "${json-unit.ignore}", - "dt": "ss", - "mt": false - }, - { - "sct": "${json-unit.ignore}", - "id": "${json-unit.ignore}", - "ct": "${json-unit.ignore}", - "dt": "fr", - "sid": "${json-unit.ignore}", - "mt": false, - "vc": "off_thread" - }, - { - "sct": "${json-unit.ignore}", - "dt": "uic", - "ct": "${json-unit.ignore}", - "ni": { - "dfs": "${json-unit.ignore}", - "i": "123456", - "n": 1, - "f": true - }, - "id": "${json-unit.ignore}", - "sid": "${json-unit.ignore}", - "mt": false, - "vc": "off_thread" - }, - { - "sct": "${json-unit.ignore}", - "dt": "uic", - "ct": "${json-unit.ignore}", - "ni": { - "dfs": "${json-unit.ignore}", - "i": "foo@example.com", - "n": 7, - "f": true - }, - "id": "${json-unit.ignore}", - "sid": "${json-unit.ignore}", - "mt": false, - "vc": "off_thread" - }, - { - "id": "${json-unit.ignore}", - "vc": "off_thread", - "sf": true, - "dt": "ast", - "ct": "${json-unit.ignore}", - "sct": "${json-unit.ignore}", - "sc": true, - "sid": "${json-unit.ignore}", - "t": "app_init", - "ifr": true, - "nsi": 0, - "mt": false - }, - { - "et": "Other", - "id": "${json-unit.ignore}", - "vc": "off_thread", - "dt": "e", - "el": 0, - "ct": "${json-unit.ignore}", - "attrs": { - "SimpleKey": "SimpleValue" - }, - "sct": "${json-unit.ignore}", - "sid": "${json-unit.ignore}", - "en": 0, - "n": "Simple Event Name", - "est": "${json-unit.ignore}", - "mt": false - } - ], - "dt": "h", - "a": "${json-unit.ignore}", - "ai": { - "av": "1.0", - "sideloaded_kits_count": 0, - "bid": "${json-unit.ignore}", - "pir": false, - "fi": false, - "lud": "${json-unit.ignore}", - "arc": "arm64", - "tsv": "90000", - "apn": "com.mparticle.IntegrationTests", - "env": 1, - "abn": "1", - "bsv": "${json-unit.ignore}", - "ict": "${json-unit.ignore}" - }, - "ct": "${json-unit.ignore}", - "das": "${json-unit.ignore}", - "mpid": 6504934091054997508, - "ui": [ - { - "i": "123456", - "n": 1 - }, - { - "i": "foo@example.com", - "n": 7 - } - ], - "uitl": 60, - "oo": false, - "sdk": "8.40.0", - "di": { - "p": "arm64", - "bid": "${json-unit.ignore}", - "tz": "-5", - "dn": "${json-unit.ignore}", - "dll": "en", - "dlc": "${json-unit.ignore}", - "dsh": "1440", - "dma": "Apple", - "vid": "${json-unit.ignore}", - "idst": false, - "tzn": "America/New_York", - "dp": "iOS", - "dsw": "960", - "dosv": "${json-unit.ignore}", - "it": false, - "dmdl": "arm64", - "dr": "None", - "jb": { - "cydia": false - }, - "lat": false, - "arc": "arm64e", - "b": "arm64" - }, - "stl": 60 - } -} \ No newline at end of file