Network Next writes data to bigquery so you can run data science and analytics queries.
For example, once every 10 seconds network performance data such as accelerated and non-accelerated latency (RTT), jitter, packet loss, bandwidth usage and out order packets for every session is written to bigquery.
At the end of each session a summary data entry is written, which makes it much faster and cheaper to query data on a per-session basis.
There is also data written each time a client pings relays at the start of each session, so you can look at direct ping results from clients to relays, and data from each server and relay in your fleet is sent so you can track their performance and uptime.
Data in bigquery is retained for 90 days by default to comply with GDPR. Within this dataset the "user hash" is considered pseudonymized data within the GDPR, and is personal data only if you have the ability to identify the player from the user hash plus other data that you have for this player. The client IP address and port are also considered personal data.
Schemas for all this data are described below.
Session updates contain network performance data once every 10 seconds for a session. This is the primary network performance data, including everything shown in the portal for a session and much more.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the session update occurred |
| session_id | INT64 | Unique identifier for this session |
| slice_number | INT64 | Slices are 10 second periods starting from slice number 0 at the start of the session |
| real_packet_loss | FLOAT64 | Packet loss between the client and the server measured from game packets (%) |
| real_jitter | FLOAT64 | Jitter between the client and the server measured from game packets (milliseconds) |
| real_out_of_order | FLOAT64 | Percentage of packets that arrive out of order between the client and the server (%) |
| session_events | INT64 | Custom set of 64bit event flags. Optional. NULL if no flags are set |
| internal_events | INT64 | Internal SDK event flags. Optional. NULL if no flags are set |
| direct_rtt | FLOAT64 | Latency between client and server as measured by direct pings (unaccelerated path). Milliseconds. IMPORTANT: Will be 0.0 on slice 0 always because it is not known yet |
| direct_jitter | FLOAT64 | Jitter between client and server as measured by direct pings (unaccelerated path). Milliseconds. IMPORTANT: Will be 0.0 on slice 0 always because it is not known yet |
| direct_packet_loss | FLOAT64 | Packet loss between client and server as measured by direct pings (unaccelerated path). Percent. Generally this is inaccurate and higher that real value, because direct pings are sent infrequently, any packet loss results in an outsized % of packet loss in a 10 second period. |
| direct_kbps_up | INT64 | Bandwidth in the client to server direction along the direct path (unaccelerated). Kilobits per-second |
| direct_kbps_down | INT64 | Bandwidth in the server to client direction along the direct path (unaccelerated). Kilobits per-second |
| next | BOOL | True if this slice is being accelerated over network next |
| next_rtt | FLOAT64 | Latency between client and server as measured by next pings (accelerated path). Milliseconds. NULL if not on network next |
| next_jitter | FLOAT64 | Jitter between client and server as measured by next pings (accelerated path). Milliseconds. NULL if not on network next |
| next_packet_loss | FLOAT64 | Packet loss between client and server as measured by next pings (accelerated path). Percent. NULL if not on network next. Generally inaccurate and higher than real packet loss due to infrequent sending of ping packets |
| next_kbps_up | FLOAT64 | Bandwidth in the client to server direction along the next path (accelerated). Kilobits per-second |
| next_kbps_down | FLOAT64 | Bandwidth in the server to client direction along the next path (accelerated). Kilobits per-second |
| next_predicted_rtt | FLOAT64 | Conservative predicted latency between client and server from the control plane. Milliseconds. NULL if not on network next |
| next_route_relays | []INT64 | Array of relay ids for the network next path (accelerated). NULL if not on network next |
| fallback_to_direct | BOOL | True if the SDK has encountered a fatal error and cannot continue acceleration. Typically this only happens when the system is misconfigured or overloaded. |
| reported | BOOL | True if this session was reported by the player |
| latency_reduction | BOOL | True if this session took network next this slice to reduce latency |
| packet_loss_reduction | BOOL | True if this session took network next this slice to reduce packet loss |
| force_next | BOOL | True if this session took network next this slice because it was forced to |
| long_session_update | BOOL | True if the processing for this slice on the server backend took a long time. This may indicate that the server backend is overloaded. |
| veto | BOOL | True if the routing logic decided that this session should no longer be accelerated for some reason. |
| disabled | BOOL | True if the buyer is disabled. Disabled buyers don't perform any acceleration or analytics on network next. |
| not_selected | BOOL | If the route shader selection % is any value other than 100%, then this is true for sessions that were not selected for acceleration. |
| a | BOOL | This session was part of an AB test, and is in the A group. (potentially accelerated) |
| b | BOOL | This session was part of an AB test, and is in the B group. (never accelerated) |
A session summary is written at the end of each session, with the intent that if you want per-session data you can query it here, instead of needing to process all the 10 second slices belonging to that session to get the data you want.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the session summary was generated (at the end of the session). |
| session_id | INT64 | Unique identifier for this session |
| datacenter_id | INT64 | The datacenter the server is in |
| buyer_id | INT64 | The buyer this session belongs to |
| user_hash | INT64 | Pseudonymized hash of a unique user id passed up from the SDK |
| latitude | FLOAT64 | Approximate latitude of the player from ip2location |
| longitude | FLOAT64 | Approximate longitude of the player from ip2location |
| client_address | STRING | Anonymized client address |
| server_address | STRING | Server address and port number |
| connection_type | INT64 | Connection type: 0 = unknown, 1 = wired, 2 = wifi, 3 = cellular |
| platform_type | INT64 | Platform type: 0 = unknown, 1 = windows, 2 = mac, 3 = linux, 4 = switch, 5 = ps4, 6 = ios, 7 = xbox one, 8 = xbox series x, 9 = ps5 |
| sdk_version_major | INT64 | The major SDK version on the server |
| sdk_version_minor | INT64 | The minor SDK version on the server |
| sdk_version_patch | INT64 | The patch SDK version on the server |
| client_to_server_packets_sent | INT64 | The total number of game packets sent from client to server in this session |
| server_to_client_packets_sent | INT64 | The total number of game packets sent from server to client in this session |
| client_to_server_packets_lost | INT64 | The total number of game packets lost from client to server in this session |
| server_to_client_packets_lost | INT64 | The total number of game packets lost from server to client in this session |
| client_to_server_packets_out_of_order | INT64 | The total number of game packets received out of order from client to server in this session |
| server_to_client_packets_out_of_order | INT64 | The total number of game packets received out of order from server to client in this session |
| total_next_envelope_bytes_up | INT64 | The total number of envelope bytes sent across network next in the client to server direction for this session |
| total_next_envelope_bytes_down | INT64 | The total number of envelope bytes sent across network next in the server to client direction for this session |
| duration_on_next | INT64 | Total time spent on network next in this session (time accelerated). Seconds |
| session_duration | INT64 | Length of this session in seconds |
| start_timestamp | TIMESTAMP | The time when this session started |
| error | INT64 | Error flags to diagnose what's happening with a session. Look up SessionError_* in the codebase for a list of errors. 0 if no error has occurred. |
| reported | BOOL | True if this session was reported by the player |
| latency_reduction | BOOL | True if this session took network next to reduce latency |
| packet_loss_reduction | BOOL | True if this session took network next to reduce packet loss |
| force_next | BOOL | True if this session took network next because it was forced to |
| long_session_update | BOOL | True if the processing for any slices in this session took a long time. This may indicate that the server backend is overloaded. |
| veto | BOOL | True if the routing logic decided that this session should no longer be accelerated for some reason. |
| disabled | BOOL | True if the buyer is disabled. Disabled buyers don't perform any acceleration or analytics on network next. |
| not_selected | BOOL | If the route shader selection % is any value other than 100%, then this is true for sessions that were not selected for acceleration. |
| a | BOOL | This session was part of an AB test, and is in the A group (potentially accelerated) |
| b | BOOL | This session was part of an AB test, and is in the B group (never accelerated) |
Server init entries are added once when the server first connects with the Network Next backend. They are intended to let you quickly look up the datacenter name for servers, which is not included in the regular update.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the server init occurred |
| sdk_version_major | INT64 | The major SDK version number on the server |
| sdk_version_minor | INT64 | The minor SDK version number on the server |
| sdk_version_patch | INT64 | The patch SDK version number on the server |
| buyer_id | INT64 | The buyer this server belongs to |
| datacenter_id | INT64 | The datacenter this server is in |
| datacenter_name | STRING | The name of the datacenter, for example: 'google.iowa.1' |
| server_address | STRING | The address and port of the server, for example: '123.254.10.5:40000' |
This entry is updated once every 10 seconds while a server is running. The only value that changes is the number of sessions connected to the server, which you can use to track server utilization.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the server init occurred |
| sdk_version_major | INT64 | The major SDK version number on the server |
| sdk_version_minor | INT64 | The minor SDK version number on the server |
| sdk_version_patch | INT64 | The patch SDK version number on the server |
| buyer_id | INT64 | The buyer this server belongs to |
| datacenter_id | INT64 | The datacenter this server is in |
| server_address | STRING | The address and port of the server, for example: '123.254.10.5:40000' |
| num_sessions | INT64 | The number of client sessions currently connected to the server |
This data is updated once every 10 seconds per-relay. It is useful for tracking the activity of your relay fleet, and identifying high and low performing relays.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the relay update occurred |
| relay_id | INT64 | Unique relay id. The fnv1a hash of the relay address + port as a string |
| session_count | INT64 | The number of sessions currently going through this relay |
| max_sessions | INT64 | The maximum number of sessions allowed through this relay (optional: NULL if not specified) |
| envelope_bandwidth_up_kbps | INT64 | The current amount of envelope bandwidth in the client to server direction through this relay |
| envelope_bandwidth_down_kbps | INT64 | The current amount of envelope bandwidth in the server to client direction through this relay |
| packets_sent_per_second | FLOAT64 | The number of packets sent per-second by this relay |
| packets_received_per_second | FLOAT64 | The number of packets received per-second by this relay |
| bandwidth_sent_kbps | FLOAT64 | The amount of bandwidth sent by this relay in kilobits per-second |
| bandwidth_received_kbps | FLOAT64 | The amount of bandwidth received by this relay in kilobits per-second |
| client_pings_per_second | FLOAT64 | The number of client relay pings received by this relay per-second |
| server_pings_per_second | FLOAT64 | The number of server relay pings received by this relay per-second |
| relay_pings_per_second | FLOAT64 | The number of relay pings sent from other relays received by this relay per-second |
| relay_flags | INT64 | The current value of the relay flags. See RelayFlags_* in the source code |
| num_routable | INT64 | The number of other relays this relay can route to |
| num_unroutable | INT64 | The number of other relays this relay cannot route to |
| start_time | INT64 | The start time of the relay as a unix timestamp according to the clock on the relay |
| current_time | INT64 | The start time of the relay as a unix timestamp according to the clock on the relay. Together with start_time and timestamp this can be used to determine relay uptime, and clock desynchronization between the relay and the backend. |
| relay_counters | []INT64 | Array of counters used to diagnose what is going on with a relay. Search for RELAY_COUNTER_ in the codebase for counter names |
These entries are written to bigquery at the start of each session when relays are pinged by the client.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the client relay ping occurred |
| buyer_id | INT64 | The buyer this player belongs to |
| session_id | INT64 | Unique id for the session |
| user_hash | INT64 | Pseudonymized hash of a user id passed up from the SDK |
| latitude | FLOAT64 | Approximate latitude of the player from ip2location |
| longitude | FLOAT64 | Approximate longitude of the player from ip2location |
| client_address | STRING | Anonymized client address |
| connection_type | INT64 | Connection type: 0 = unknown, 1 = wired, 2 = wifi, 3 = cellular |
| platform_type | INT64 | Platform type: 0 = unknown, 1 = windows, 2 = mac, 3 = linux, 4 = switch, 5 = ps4, 6 = ios, 7 = xbox one, 8 = xbox series x, 9 = ps5 |
| client_relay_id | INT64 | Relay id being pinged by the client |
| client_relay_rtt | INT64 | Round trip time ping between the client and the relay (milliseconds) |
| client_relay_jitter | INT64 | Jitter between the client and the relay (milliseconds) |
| client_relay_packet_loss | FLOAT64 | Packet loss between the client and the relay (%). Generally inaccurate and higher than true value because client relay pings are sent infrequently. |
These entries are written to bigquery at the start of each session contining pings between the server and destination relays in the same datacenter.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the server relay ping occurred |
| buyer_id | INT64 | The buyer this player belongs to |
| session_id | INT64 | Unique id for the session |
| datacenter_id | INT64 | Unique id for the datacenter |
| server_address | STRING | Server address and port number |
| server_relay_id | INT64 | Relay id being pinged by the server |
| server_relay_rtt | INT64 | Round trip time ping between the server and the relay (milliseconds) |
| server_relay_jitter | INT64 | Jitter between the server and the relay (milliseconds) |
| server_relay_packet_loss | FLOAT64 | Packet loss between the server and the relay (%). Generally inaccurate and higher than true value because server relay pings are sent infrequently. |
Updated once per-second with each route matrix updated. The route matrix is the core data structure used for route planning across relays. You can use this data to track the results of route optimization over time.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the route matrix update occurred |
| cost_matrix_size | INT64 | The size of the cost matrix in bytes |
| route_matrix_size | INT64 | The size of the route matrix in bytes |
| optimize_time | INT64 | Time it took produce this route matrix from the cost matrix (milliseconds) |
| num_relays | INT64 | The number of relays in the route matrix |
| num_active_relays | INT64 | The number of active relays in the relay manager |
| num_dest_relays | INT64 | The number of destination relays in the route matrix |
| num_datacenters | INT64 | The number of datacenters in the route matrix |
| total_routes | INT64 | The total number of routes in the route matrix |
| average_num_routes | FLOAT64 | The average number of routes between any two relays |
| average_route_length | FLOAT64 | The average number of relays per-route |
| no_route_percent | FLOAT64 | The percent of relay pairs that have no route between them |
| one_route_percent | FLOAT64 | The percent of relay pairs with only one route between them |
| no_direct_route_percent | FLOAT64 | The percent of relay pairs with no direct route between them |
| rtt_bucket_no_improvement | FLOAT64 | The percent of relay pairs with no improvement |
| rtt_bucket_0_5ms | FLOAT64 | The percent of relay pairs with 0-5ms reduction in latency |
| rtt_bucket_5_10ms | FLOAT64 | The percent of relay pairs with 5-10ms reduction in latency |
| rtt_bucket_10_15ms | FLOAT64 | The percent of relay pairs with 10-15ms reduction in latency |
| rtt_bucket_15_20ms | FLOAT64 | The percent of relay pairs with 15-20ms reduction in latency |
| rtt_bucket_20_25ms | FLOAT64 | The percent of relay pairs with 20-25ms reduction in latency |
| rtt_bucket_25_30ms | FLOAT64 | The percent of relay pairs with 25-30ms reduction in latency |
| rtt_bucket_30_35ms | FLOAT64 | The percent of relay pairs with 30-35ms reduction in latency |
| rtt_bucket_35_40ms | FLOAT64 | The percent of relay pairs with 35-40ms reduction in latency |
| rtt_bucket_40_45ms | FLOAT64 | The percent of relay pairs with 40-45ms reduction in latency |
| rtt_bucket_45_50ms | FLOAT64 | The percent of relay pairs with 45-50ms reduction in latency |
| rtt_bucket_50ms_plus | FLOAT64 | The percent of relay pairs with 50ms+ reduction in latency |
This data is not uploaded by default because at 1000 relays the number of rows inserted is ~1 million per-second. However, in smaller relay fleets or dev the system can be modified to upload this data, which provides good visibility into intra-relay performance that you can analyze via bigquery data.
| Field | Type | Description |
|---|---|---|
| timestamp | TIMESTAMP | The timestamp when the relay ping occurred |
| source_relay_id | INT64 | The id of the source relay |
| destination_relay_id | INT64 | The id of the destination relay |
| rtt | INT64 | Round trip latency between the two relays (milliseconds) |
| jitter | INT64 | Time variance in latency between the two relays (milliseconds) |
| packet_loss | FLOAT64 | The packet loss between the two relays (%) |