Twilio Message Export

Export the last 30 days of SMS/MMS message records from your Twilio account using the Bulk Export API.

Quick Start

# Clone the repo
git clone https://github.com/sent-dm/twilio-message-export.git
cd twilio-message-export

# Install dependencies
pip install -r requirements.txt

# Set your Twilio credentials
export TWILIO_ACCOUNT_SID="ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
export TWILIO_AUTH_TOKEN="your_auth_token"

# Run the export
python twilio_message_export.py

Requirements

• Python 3.7+
• Twilio Account SID and Auth Token (find them here)

What It Does

1. Creates a Bulk Export job for the last 30 days of messages
2. Polls Twilio until the export completes (typically 5-30 minutes)
3. Downloads each day's data and combines into a single file
4. Outputs a .jsonl file with one message per line

Output Format

The script produces a JSON Lines file:

messages_export_2024-11-28_to_2024-12-28.jsonl

Each line is a JSON object containing all fields from the Twilio Bulk Export API:

{
  "sid": "SMxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "account_sid": "ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "messaging_service_sid": "MGxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "date_created": "2024-12-15T14:32:01Z",
  "date_sent": "2024-12-15T14:32:02Z",
  "date_updated": "2024-12-15T14:32:05Z",
  "from": "+15551234567",
  "to": "+15559876543",
  "body": "Your message content here",
  "status": "delivered",
  "direction": "outbound-api",
  "num_segments": 1,
  "num_media": 0,
  "price": "-0.00750",
  "error_code": null
}

Available Fields

Field	Description
sid	Unique message identifier (SM...)
account_sid	Your Twilio Account SID (AC...)
messaging_service_sid	Messaging Service SID if used (MG...)
date_created	When the message was created (UTC, ISO 8601)
date_sent	When the message was sent (UTC, ISO 8601)
date_updated	When the message was last updated (UTC, ISO 8601)
from	Sender phone number
to	Recipient phone number
body	Message content
status	Delivery status: queued, sending, sent, delivered, undelivered, failed
direction	inbound, outbound-api, outbound-call, outbound-reply
num_segments	Number of SMS segments
num_media	Number of MMS media attachments
price	Cost of the message (may not be present on all records)
error_code	Error code if failed (null if successful)

Note: Bulk Export excludes price_units, api_version, error_message, uri, and subresource_uris fields. The price field may not be present on all records.

Configuration

Environment Variable	Required	Description
TWILIO_ACCOUNT_SID	Yes	Your Twilio Account SID (starts with AC)
TWILIO_AUTH_TOKEN	Yes	Your Twilio Auth Token

Notes

• Date Range: Exports cover 30 days ending 2 days ago (Twilio API requirement)
• Processing Time: Large accounts may take 10-30 minutes
• Rate Limits: Maximum 366 days of exports per UTC day
• File Retention: Twilio deletes export files after 7 days

Troubleshooting

"Missing Twilio credentials"

• Ensure both TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN are set
• Check for typos or extra whitespace

"Authentication failed"

• Verify credentials at console.twilio.com
• Ensure you're using the main Account SID, not a subaccount

"Rate limited"

• You've exceeded 366 days of exports in a UTC day
• Wait until midnight UTC and try again

Export job stuck

• The script will timeout after 2 hours
• Check job status at console.twilio.com

License

MIT