Skip to content

Conversation

Copilot
Copy link

@Copilot Copilot AI commented Aug 22, 2025

This PR implements comprehensive Bulk Operations support for the Shopify Admin API, enabling asynchronous processing of large GraphQL queries and mutations without rate limiting concerns.

Key Features Added

Ruby Module API

  • ShopifyToolkit::BulkOperations module with complete functionality:
    • run_bulk_query(query, group_objects: false) - Submit bulk GraphQL queries
    • run_bulk_mutation(mutation, variables_data, ...) - Submit bulk mutations with automatic staged uploads
    • current_bulk_operation(type: nil) - Check operation status
    • cancel_bulk_operation(operation_id) - Cancel running operations
    • download_results(operation_or_url, parse_results: true) - Download and parse JSONL results
    • poll_until_complete(operation_id, ...) - Poll with progress callbacks

CLI Commands

Five new Thor commands integrated into existing CLI:

  • shopify-toolkit bulk_query QUERY_FILE - Submit bulk queries
  • shopify-toolkit bulk_mutation MUTATION_FILE VARIABLES_FILE - Submit bulk mutations
  • shopify-toolkit bulk_status [OPERATION_ID] - Check operation status
  • shopify-toolkit bulk_cancel OPERATION_ID - Cancel operations
  • shopify-toolkit bulk_results OPERATION_ID_OR_URL - Download results

Advanced Features

  • Automatic staged file uploads for bulk mutations using stagedUploadsCreate
  • JSONL streaming parser for efficient handling of large result files
  • Comprehensive error handling with custom exception classes:
    • BulkOperationError - General bulk operation errors
    • OperationInProgressError - When attempting to start while another operation runs
  • Progress polling with customizable intervals and timeouts
  • Result management supporting both parsed JSON and raw JSONL output

Example Usage

class MyService
  include ShopifyToolkit::BulkOperations
  
  def export_products
    query = "{ products { edges { node { id title } } } }"
    operation = run_bulk_query(query)
    
    completed = poll_until_complete(operation["bulkOperation"]["id"]) do |status|
      puts "Progress: #{status['objectCount']} objects processed"
    end
    
    return download_results(completed) if completed["status"] == "COMPLETED"
  end
end
# CLI usage
shopify-toolkit bulk_query examples/products.graphql --poll --output results.json
shopify-toolkit bulk_mutation examples/create_products.graphql examples/variables.json

Implementation Details

  • Follows existing patterns: Reuses AdminClient module and Thor CLI infrastructure
  • Comprehensive testing: 25 test cases covering all functionality with proper mocking
  • Documentation: Extensive README section with examples and usage patterns
  • Example files: Includes GraphQL queries, mutations, and variable files for testing
  • Error resilience: Handles network failures, parsing errors, and API limitations

The implementation supports all bulk operation workflows described in the Shopify documentation, including proper handling of JSONL formats, staged uploads for mutations, and webhook-style polling for completion status.

Fixes #22.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copy link
Author

Copilot AI commented Aug 22, 2025

@mtylty 👋 This repository doesn't have Copilot instructions. With Copilot instructions, I can understand the repository better, work faster and produce higher quality PRs.

I can generate a .github/copilot-instructions.md file for you automatically. Click here to open a pre-filled issue and assign it to me. I'll write the instructions, and then tag you for review.

@Copilot Copilot AI changed the title [WIP] Implement Bulk Operations support (Shopify Admin API) Implement complete Bulk Operations support for Shopify Admin API Aug 22, 2025
@Copilot Copilot AI requested a review from mtylty August 22, 2025 13:47
Copilot finished work on behalf of mtylty August 22, 2025 13:47
@mtylty mtylty requested a review from nerfologist August 22, 2025 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Bulk Operations support (Shopify Admin API)
2 participants