Download payloads?

[ianwremmel]

I know I have the option to copy the body of each received message, but is there any way to download the full request? I'd like to be able to set up a services that eventually delivers webhooks to my application, poke around in the ui that those services are connected to, then commit each webhook (headers and all) into my testsuite so I can play them back in every test without needing to go through any third-party service.

[JasonEtco]

👋 There's not currently a way to download payloads. However, it seems like you're more asking about viewing the headers than anything else. If that's correct, we can rename this issue to Show headers in the payload UI or something?

[ianwremmel]

Hi @JasonEtco it's less that I'm interested in viewing things as I am in downloading a fully machine-readable representation of the event (for example, as a JSON blob or a har). Basically, I'd like to download a request, commit it into the repo, and then do something like

import fetch from 'cross-fetch';
const {body, headers} = JSON.parse(fs.readFileSync('fixture.json'));
const result = await fetch('localhost:3000/webhook-receiver', {body, headers});
expect(result.status).toBe(202);

That way, I can write a unit test against real data, rather than trying to author a fixture by hand.

Even better, but probably much more work, it would be cool to do something like

import smee from 'smee-client';
const result = smee.replay('./fixture.json', 'localhost:3000/webhook-receiver');
expect(result.status).toBe(202);

[JasonEtco]

Ah gotcha. Well, the copy feature is supposed to do just that - so it sounds like it's missing information. I'm not opposed to adding a second "download" feature that includes the headers then 👍 If you're open to a PR that'd be great!

[ianwremmel]

Yea, i can probably look into a PR. Would that just be a button next to copy that triggers a download?

one issue: it looks like the copy button only includes the request body, not any of the headers:

{
    "payload": {
      REDACTED
    }
}

[ianwremmel]

is it the smee webapp discarding them or the service?

[JasonEtco]

Nope, it's all available in the UI already, just not shown. Here's where we send the event to clients:

smee.io/lib/server.js

Lines 100 to 105 in 79e4087

	payload: {
	...req.headers,
	body: req.body,
	query: req.query,
	timestamp: Date.now()
	}

And then in the UI, we render it:

smee.io/src/components/ListItem.js

Lines 49 to 51 in 79e4087

	const event = item['x-github-event']
	const payload = item.body
	const id = item['x-github-delivery'] || item.timestamp

We could do something like to get the headers off of item:

const { body, timestamp, query, ...headers } = item

[ianwremmel]

Wouldn't that imply that I should have seen a body field in my payload?

here's a less redacted version of the copy results:

{
    "payload": {
        "_metadata": {
            "nodeVersion": "12.16.1"
        },
        "channel": "server",
        "context": {
            "groupId": "group-7e2fc219-73cf-4c06-900e-405416c4fcf6",
            "library": {
                "name": "analytics-node",
                "version": "3.4.0-beta.1"
            }
        },
        "event": "REDACTED",
        "integrations": {},
        "messageId": "node-62bcdd13a7489c031ab33e29ce6d080e-ed7d2c6e-2704-4987-9748-683acc8f8172",
        "originalTimestamp": "2020-04-12T02:25:53.908Z",
        "projectId": "zphc3xAxVl",
        "receivedAt": "2020-04-12T02:26:04.070Z",
        "sentAt": "2020-04-12T02:26:03.911Z",
        "timestamp": "2020-04-12T02:25:54.067Z",
        "type": "track",
        "userId": "group-7e2fc219-73cf-4c06-900e-405416c4fcf6",
        "version": 2,
        "writeKey": "REDACTED"
    }
}

it looks like payload is body rather than containing body and I don't believe headers have been spread across payload.

I should note that this is when I return to the page, so the data has been loaded from localStorage.

In any case, I'll see about putting together a PR soonish and maybe I'll be less confused when I actually look at the code 😄.

[ianwremmel]

ok, #53 adds a download button. The payload shape is a little surprising though. It looks like query and body are their own keys, but they're just mixed in with all the headers; headers are not contained in a headers object.