Skip to content

challenge_rules.md

Latest commit

 

History

History
67 lines (52 loc) · 5.54 KB

challenge_rules.md

File metadata and controls

67 lines (52 loc) · 5.54 KB

AI for Industry Challenge: Rules and Technical Specifications

1. Competition Spirit

The AI for Industry Challenge (AIC) is designed to foster creativity, technical excellence, and collaboration in solving real-world robotic manipulation tasks. We encourage participants to develop innovative solutions that demonstrate the capabilities of AI in industrial automation.

Participants are expected to honor the spirit of the competition by developing solutions that use only the officially provided interfaces (i.e., ROS 2 topics, services, and message types defined in the AIC Interfaces documentation). The goal is to build AI models that are generalizable, robust, and compliant with the constraints of the physical system and its simulation.

2. Prohibition of Cheating

a. Use of Provided Interfaces Only

Participants must not attempt to bypass or circumvent the provided interfaces to gain an unfair advantage. Prohibited actions include:

  • Direct State Manipulation: Manually altering the robot, task board, or simulation environment (e.g., teleporting components or forcing cable insertions).
  • Backend & System Interference: Bypassing official APIs to communicate with ROS or Gazebo backends. This includes and not limited to unauthorized interactions with:
    • Configuration & Lifecycle: Modifying ROS 2 parameters or changing node lifecycle states for evaluation nodes.
    • Simulation Control: All topics/services in the /scoring, /gazebo, and /gz_server namespaces.
    • Entity Management: Services used to spawn, despawn, or delete models and entities.
    • Environment States: Data related to /clock, /model, /world_stats, physics control (/pause_physics), or simulation resets.
  • Exploitative Hardcoding: Hardcoding sensor data or environment configurations to exploit knowledge of the specific simulation setup.
  • Process Manipulation: Interfering with the AIC Engine, the scoring system, or the logging infrastructure.

b. Unauthorized Access

Participants are not permitted to reverse-engineer, exploit, or tamper with the cloud-based evaluation infrastructure. This includes:

  • Using unauthorized tools to access or modify the evaluation environment.
  • Submitting containers or models containing malicious code, backdoors, or other mechanisms to defeat the competition architecture.
  • Bypassing the submission portal or registry to inject or manipulate data.

c. External Data and Information Leaks

Participants must not use information from the evaluation environment that is not accessible through official interfaces.

  • State Leaking: Reusing internal state information from the simulation or backend systems during evaluation. During training, participants may use all internal state information, including ground truth data available over the /tf topic.
  • Data Misuse: Training models using data generated by the evaluation infrastructure that is not part of the toolkit.

3. Enforcement and Reporting

Evaluation Integrity

The evaluation platform includes automated access control and integrity checks. In addition, we will manually review the submissions of top-performing teams to verify compliance through:

  1. Container Audits: Reviewing submitted container images to ensure they only interact with the system via defined interfaces.
  2. Behavioral Verification: Confirming model behavior during live evaluations adheres to the rules.
  3. Metric Analysis: Cross-checking performance against expected baselines to detect anomalies.

Note: Teams found in violation of these rules face penalties, including disqualification and revocation of prizes.

Reporting Violations

If you suspect another participant of violating these rules, please report it to the organizers through the official channels. Reports are treated confidentially.

4. Technical Specification: aic_model

  • The submission container must start a process with a ROS 2 Lifecycle node with name aic_model.
  • The aic_model node must exhibit the following behavior:
    • Start in unconfigured lifecycle state.
      • In unconfigured state, no topics should be published by node, especially to command the robot.
      • A transition to configured state must succeed within a timeout of 60s. Loading of any models should occur within this window.
        • In configured state, no topics should be published by node, especially to command the robot.
          • In configured state, the node should reject any goals requests sent to /insert_cable Action Server.
      • A transition to active state must succeed within a timeout of 60s.
        • In active state, the node should accept any goal request sent to /insert_cable and the goals should be cancellable.
        • Goal requests should complete within the time_limit field of the requested Task. When running evaluation, this field will be set according to the time_limit specified in the config passed to the aic_engine.
      • A deactivate transition request should transition the node back to configured state successfully within a timeout of 60s.
      • A cleanup transition request should transition the node back to unconfigured state successfully within a timeout of 60s.
      • A shutdown transition request must succeed within a timeout of 60s.
        • In shutdown state, no topics should be published by node, especially to command the robot.
        • In the shutdown state, no robot command publishers should be present in the graph.