Skip to content

Update singlestore destination doc #7833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 20, 2025
Merged

Update singlestore destination doc #7833

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f1af0b2
changed doc info
psharma-1909 Jul 31, 2025
3106730
Access nested data
psharma-1909 Aug 13, 2025
a2a15e8
asked changes
psharma-1909 Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 43 additions & 17 deletions src/connections/destinations/catalog/actions-singlestore/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,37 +27,63 @@ This destination is maintained by SingleStore. For any issues with the destinati
{% include components/actions-fields.html %}

### Finding your SingleStore connection settings
To find your SingleStore connection settings, head to the [SingleStore Portal](https://portal.singlestore.com){:target="_blank”} and complete the following steps:
1. Select **Deployments**.
2. Choose your Workspace and Database within the list of Deployments
3. From the Connect dropdown, select **Connect to your own app**. SingleStore will display the the key settings you need to connect your SingleStore database to Segment.
To connect Segment to SingleStore, use the SingleStore Data API (typically on port 443). Follow these steps to enable and locate your Data API connection settings:

1. Head to the [SingleStore Portal](https://portal.singlestore.com){:target="_blank"}.
2. Select **Deployments**.
3. Choose your Workspace and Database within the list of Deployments
4. From the Connect dropdown, select **Connect to your own app**. SingleStore will display the the key settings you need to connect your SingleStore database to Segment.

**Note:**
- The Data API is enabled by default for all SingleStore Cloud workspaces.
- Segment always uses the Data API (typically on port 443).
- If you are using a self-hosted or development SingleStore deployment (such as the SingleStore-dev image), the Data API may run on a different port. Refer to your deployment’s documentation or settings to confirm the correct port.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- If you are using a self-hosted or development SingleStore deployment (such as the SingleStore-dev image), the Data API may run on a different port. Refer to your deployment’s documentation or settings to confirm the correct port.
- If you use a self-hosted or development SingleStore deployment (such as the SingleStore-dev image), the Data API may run on a different port. Refer to your deployment’s documentation or settings to confirm the correct port.


For more details, see the [SingleStore Data API documentation](https://docs.singlestore.com/cloud/reference/data-api/){:target="_blank"}.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For more details, see the [SingleStore Data API documentation](https://docs.singlestore.com/cloud/reference/data-api/){:target="_blank"}.
For more information, see the [SingleStore Data API documentation](https://docs.singlestore.com/cloud/reference/data-api/){:target="_blank"}.


## Database structure
Segment creates a table called `segment_raw_data` and writes data to your SingleStore database using the following schema:
Segment writes data to your specified table in SingleStore (by default, this is `segment_data`) using the following schema:

| Column | Type | Description |
| -------- | ------ | ----------- |
| `message` | JSON (utf8_bin) | The entire message received from Segment, in JSON format |
| `timestamp` | datetime | The timestamp of when the event was generated |
| `event` | VARCHAR(255) | The event name (for Track events) |
| `messageId` | VARCHAR(255) | The unique identifier of the event to ensure there is no duplication |
| `type` | VARCHAR(255) | The type of the event (for example, Identify, Track, Page, Group) |
| `messageId` | TEXT | A unique identifier for the event to ensure there is no duplication. |
| `timestamp` | Datetime(6) | The timestamp of when the event was generated |
| `type` | TEXT | The type of the event (e.g., "track", "identify", "page", "screen", "group", "alias"). |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `type` | TEXT | The type of the event (e.g., "track", "identify", "page", "screen", "group", "alias"). |
| `type` | TEXT | The type of the event (for example, Track, Identify, Page, Screen, Group, or Alias). |

| `event` | TEXT | The name of the event. Only required for "track" events. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `event` | TEXT | The name of the event. Only required for "track" events. |
| `event` | TEXT | The name of the event. Only required for Track events. |

| `name` | TEXT | The name of the page or screen. |
| `properties` | JSON | The properties of the track, page or screen event. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `properties` | JSON | The properties of the track, page or screen event. |
| `properties` | JSON | The properties of the Track, Page or Screen event. |

| `userId` | TEXT | The user ID associated with the event. |
| `anonymousId` | TEXT | The anonymous ID associated with the event. |
| `groupId` | TEXT | The group ID associated with the event. |
| `traits` | JSON | The traits of the user associated with the event. |
| `context` | JSON | The context of the event. Contains user environment information. |


### Accessing nested data
To query specific data from the Segment event within SingleStore, you can de-reference the JSON pointer within the message column. For example:
To query nested information from the `traits`, `context`, or `properties` JSON columns, use the `::` operator to access nested properties within the JSON column. For example:

To extract a property1 from the `properties` column:
```sql
SELECT message::properties FROM segment_raw_data;
SELECT properties::property1 AS property_1 FROM <your_table_name>;
```

This query retrieves the properties object from the JSON message, allowing you to work with nested event data.
This query allows you to extract specific nested properties from the JSON column for analysis.

## Troubleshooting

> **Note:**
> The SingleStore Data API has a limit of 1MB per request. If a batch of events sent by Segment exceeds this limit, you may see an error similar to:
>
> ```
> Error reading request body: http: request body too large
> ```
>
> If you encounter this error, reduce the batch size in your Segment destination settings.


### Connection Errors
If you're unable to connect to the SingleStore database:
* Verify that the Host and Port are correct.
* Verify that the credentials are correct.
* Ensure that your SingleStore database is accessible from Segment’s servers.
* Check firewall settings and network configurations.

Expand All @@ -67,20 +93,20 @@ If you encounter authentication errors when Segment attempts to connect:
* Ensure that the user has the necessary permissions to write to the database.

### Data Not Appearing in SingleStore
If events are not recorded in the `segment_raw_data` table:
If events are not recorded in the specified table:
* Verify that your sources are correctly sending data to Segment.
* Check the event types to ensure they are supported.
* Review your SingleStore database logs for any errors.

## Frequently Asked Questions
### Can I customize the schema used in SingleStore?

By default, the mappings store the complete raw Segment events in the `segment_raw_data` table. If you prefer, within the mapping, you can choose to selectively include or exclude specific fields to be sent and written into SingleStore.
By default, the mapping stores all fields from the Segment event in separate columns in the `segment_data` table. If you prefer, you can customize the mapping to selectively include or exclude specific fields to be sent and written into SingleStore.

### How does SingleStore handle data types from Segment?

All event data is stored natively as JSON in the message column. This allows for flexible schema management and easy access to nested properties using SQL queries. SingleStore's ability to dynamically and quickly parse the JSON allows all types of complex events to be queried or used in notebooks.

### Is the data ingestion process real-time?

Yes, Segment forwards data to SingleStore in real-time, enabling immediate analysis and action on your customer data. Generally data is available in the SingleStore database within a few seconds of Segment sending the event.
Yes, Segment forwards data to SingleStore in near real-time, enabling immediate analysis and action on your customer data. Generally, data is available in the SingleStore database within approximately 30 seconds of Segment sending the event.