Offline/Online LP for CIM scenario 📚

[riccardopoiani]

Description

Add Offline and Online Linear programming agent for CIM scenario

• Link: paper 1 paper 2
• Screenshot:

[Jinyu-W]

Hi @riccardopoiani , thank you for your attention to our work. Currently, we don't have the plan to transfer all these methods used in the paper to MARO platform. If you have some questions about the implementation, we can have a discussion. Thanks again!

[Ahmedest61]

Hi @Jinyu-W, how about the methods proposed in these papers. Does the current MARO contains the implementation of those? In "Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations paper, it is written that source code is attached in 'code' folder but I couldn't find it. Can you help me, please. Thank you very much :)

[Jinyu-W]

Hi @Jinyu-W, how about the methods proposed in these papers. Does the current MARO contains the implementation of those? In "Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations paper, it is written that source code is attached in 'code' folder but I couldn't find it. Can you help me, please. Thank you very much :)

@wesley-stone could you provide more details about the methods in this paper?

[Ahmedest61]

Hi @Jinyu-W, thank you for the response. I would be grateful to @wesley-stone for sharing code and related info. Many thanks again! :)

[riccardopoiani]

Hi @Jinyu-W thanks for your response and availability.
I am trying to replicate results for Offline and Online LP in different configurations.
Currently, I am able to achieve 0% and 1% shortage in toy domains with Offline and Online LP respectively.
However, when I try to run the program on global trade configurations, I obtain 4% with Offline and 33% with the Online one (which are significantly higher than the ones in the papers).

So, first of all, I have seen that there is an open pull request to adjust the topology for CIM and I was wondering if this was the cause. However, no issue describes the change made there. So, I would start from here, and then maybe we can build a discussion on how to implement correctly the method.

Also, have you got any method that is able to reach high performances with the current version of the simulator? (any sort of method)

[Jinyu-W]

Hi @Jinyu-W thanks for your response and availability.
I am trying to replicate results for Offline and Online LP in different configurations.
Currently, I am able to achieve 0% and 1% shortage in toy domains with Offline and Online LP respectively.
However, when I try to run the program on global trade configurations, I obtain 4% with Offline and 33% with the Online one (which are significantly higher than the ones in the papers).

So, first of all, I have seen that there is an open pull request to adjust the topology for CIM and I was wondering if this was the cause. However, no issue describes the change made there. So, I would start from here, and then maybe we can build a discussion on how to implement correctly the method.

Also, have you got any method that is able to reach high performances with the current version of the simulator? (any sort of method)

Hi @riccardopoiani, I could share more information about the LP methods we used before.

• The Online LP formulated based on the demand(orders) predicted with a simple MA model.
• Besides the reward for the order fulfillment, we also added the reward for safety inventory in the objective function. For example, for each tick, if we have empty containers remained after fulfilling all the orders of this tick, an additional reward would be given: safety_inventory_reward_factor * min(port capacity * safety inventory upper ratio, #remaining empty containers).
• The main difference between the offline LP and the online LP is that the actual demand(orders) is given instead of the prediction.

We may open source a version of Online LP in the future, but it will not happen immediately, it will take some time.

[Jinyu-W]

More information about the paper, the author @wesley-stone will have more say. :)

[riccardopoiani]

@Jinyu-W thanks for you feedback. I will try for sure to integrate the safety reward factor.
For what concerns the Online LP, are the methods in paper 1 and paper 2 differents? Indeed I have some conflicting information about Operation Research baselines.
In paper 1 they state that "the exact future order information to replace the forecasted future demand in the LP model so as to eliminate the effects of external factors leading to uncertain forecasts" (section 5.3 - Online LP).
Indeed, when I talked to the first author, he said that the main difference between Online and Offline was that the plan (in the offline one) was not fed to the simulator, and so, "illegal" actions were not corrected to make them feasible.

Moreover, it seems to me that the crucial point to have good performances (when feeding the plan to the simulator) is that you need estimates that are as accurate as possible (otherwise the plan will soon diverge from reality).
The crucial point in this step is that you want to estimate the future laden of containers at your best (from the laden you can recover estimates of supplies and vessels capacity for empty containers) . How do you do that? Is there anything specific I should pay attention to?

[riccardopoiani]

Also, @Ahmedest61 I have seen something similar to what you are looking for in temp_gnn branch, under the folder examples/cim/ac_gnn, but I have not tried it. (I do not know if that's the code they used to have the results they claim in the paper)