Skip to content

export_project_configuration() causes server crash/timeout on large projects #46

@markpeo

Description

@markpeo

Description

When calling export_project_configuration() (which uses the GET /config/v1/project?content=serialize endpoint) on large projects, the KEPServerEX runtime returns a 504 Gateway Timeout error & causes the server - Runtime (32 bit) process to crash

This makes the full-export approach unusable for large-scale projects.

Environment

  • KEPServerEX Version: V6.18.318.0
  • SDK Version: 1.4.2
  • Python Version: 3.10
  • OS: Windows 10/11

Project Size

  • Channels: ~20
  • Devices: ~1,500
  • Total Tags: ~926,000

Steps to Reproduce

from kepconfig.connection import server

kep = server('192.168.1.30', 57412, 'Administrator', 'password')

# This call fails on large projects
project_data = kep.export_project_configuration()

Expected Behavior

Either:

  • The export completes successfully (with appropriate timeout), or
  • The SDK/API returns a clear error indicating the project is too large, or
  • Documentation specifies the recommended maximum project size for this endpoint

Actual Behavior

  • 504 Gateway Timeout after ~30 seconds
  • server - Runtime (32 bit) crashes and must be restarted

Questions for PTC

  1. Is there a documented limit for the number of tags/devices that ?content=serialize can handle?

  2. Are there server-side timeout settings that can be adjusted to allow larger exports?

  3. Is this a known limitation? If so, what is the recommended approach for programmatically exporting large projects?

  4. Would chunked/streaming export be considered as a future enhancement? (e.g., export by channel or device)

Workaround

We've implemented an iterative approach that fetches channels, devices, and tags individually via separate API calls. This works reliably even for 900k+ tags but requires many HTTP requests and causes event log flooding of user demotion & promotion when configuration tool is open.

For reference, our approach:

  • Fetches /project/channels
  • For each channel, fetches /devices
  • For each device, fetches /tags and /tag_groups recursively
  • Streams results to SQLite rather than holding in memory

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions