Saving only certain steps in a cycle---If I have 10000 steps in one cycle

[Hanyu-B]

My input current profiles have around 10k–20k steps (the current is different for each second) in one cycle, and I am trying to figure out whether I can only save certain steps within this super long cycle to reduce the RAM requirements (as currently I can only solve 600–1500 steps due to limited RAM). I thought about several solutions for this issue:

1. only save certain steps
2. break the long cycle into shorter cycles and deliver the previous solution as the starting solution of the cycle simulation
3. using the only save at cycle function and change the cycling profiles of the "repeated cycles"

Solution 1 appears to be the most promising, but I'm not sure if it's feasible. I really appreciate any further suggestions regarding how to solve my current issues.

[manjunathnilugal]

Hi, Facing similar issue.
Memory consumption is too high. I have 6000 data points within current drive cycle. The RAM consumption is more than 46 Gb at 5000th data point and simulation is sluggish. Is there any workaround to fix this issue.
I don't have repetitive current cycle, or else save_at_cycles =[ ] would have worked here.

[brosaplanella]

Can't you break your drive cycle into smaller bits so you can do save_at_cycles?

[manjunathnilugal]

Sir, I don't have repetitive current value.,
Or may be i am not aware of splitting the drive cycle.
This is sample code I am using.
Here I have 11 current values, but in my drive cycle I am having 6000 data points.

import pybamm
import numpy as np

model = pybamm.lithium_ion.SPM()
param = pybamm.ParameterValues("Chen2020")

drive_cycles = np.array([2, 1, -2, -1, -3, -1, -2, 3, 2, 5, 4])

Vmin = 2.5
Vmax = 4.3

param['Lower voltage cut-off [V]'] = Vmin
param['Upper voltage cut-off [V]'] = Vmax

solution = None

for i, I_app in enumerate(drive_cycles):
    if I_app < 0:
        instr = "Discharge"
        Vlim = Vmin
    else:
        instr = "Charge"
        Vlim = Vmax
    I_mag = np.abs(I_app)
    try:
        op_cond = f"{instr} at {I_mag}A for 1 hour or until {Vlim}V"
        experiment = pybamm.Experiment(operating_conditions=[op_cond])
        sim = pybamm.Simulation(model=model,
                                parameter_values=param,
                                experiment=experiment)
        solution = sim.solve(starting_solution=solution)
        time_after_solve = solution["Time [h]"].entries[-1]
    except:
        time_after_solve = i
    complete_time = i+1
    time_remaining = np.around(complete_time - time_after_solve, 5)
    if time_remaining > 0:
        print("Cycle:", i , "Hold time:", time_remaining, "hours")
        op_cond = f"Rest for {time_remaining} hours"
        op_cond = f"Hold at {Vlim}V for {time_remaining} hours"
        experiment = pybamm.Experiment(operating_conditions=[op_cond])
        sim = pybamm.Simulation(model=model,
                                parameter_values=param,
                                experiment=experiment)
        solution = sim.solve(starting_solution=solution)
sim.plot()

[valentinsulzer]

Try using starting_solution=solution.last_state instead, so it only starts from the last state each time instead of keeping the whole solution in memory. Then, you will also need to separately save and construct the variables you want (e.g. voltage) in each step, i.e.

# before for loop
voltage = np.array([])
...
# in for loop
voltage = np.concatenate([voltage, solution["Terminal voltage [V]"].data])

By the end, you will have access to only the variables you saved (the whole simulation will not have been kept in memory)

[Mrzhang-hub]

Try using starting_solution=solution.last_state instead, so it only starts from the last state each time instead of keeping the whole solution in memory. Then, you will also need to separately save and construct the variables you want (e.g. voltage) in each step, i.e.

# before for loop
voltage = np.array([])
...
# in for loop
voltage = np.concatenate([voltage, solution["Terminal voltage [V]"].data])

By the end, you will have access to only the variables you saved (the whole simulation will not have been kept in memory)

It seems that solution.last_state can only be used in simulation.step() method. If I use solution.last_state in simulation.solve() method, it will give the following error:

[brosaplanella]

Not sure if there is a better way around, but you could manually split your drive cycle into smaller ones and then each cycle would run one of those chunks so you don't have to store the whole simulation.

[manjunathnilugal]

I am not able to manage the pybamm.t (time (h)) when I split the drive cycle in to two parts...!
The next drive cycle starts from 0 hrs.

[Hanyu-B]

Thanks for all the suggestions! really appreciated! I have tried different ways to resolve this issues with having non-repetitive current profiles experiments, as listed below,

1. use the clustering method to cluster the operating profiles (current profiles) to get the typical profiles with their occurrences. Then, write the experiments using typical scenarios (for example, one day operating profiles) and only save at specific cycles.
2. do an FFT on the input current profile and try to reduce the sampling rate of the raw data to reduce the computational burden (fewer data points).
3. change the current profiles into functions, but I did not get good results, so I gave up on this method.

I hope that may help a bit.

[manjunathnilugal]

I tried your suggestion, solution.last_state but I am getting this error

import pybamm
import numpy as np

model = pybamm.lithium_ion.SPM()
param = pybamm.ParameterValues("Chen2020")

drive_cycles = np.array([2.35, 1.255, -2.164, -1.154, -3.231, -1.148, -2.235, 3.146, 2.23, 5.2, -1.235 ])

Vmin = 2.5
Vmax = 4.2

param['Lower voltage cut-off [V]'] = Vmin
param['Upper voltage cut-off [V]'] = Vmax
solution = None

for i, I_app in enumerate(drive_cycles):
    if I_app < 0:
        instr = "Charge"
        Vlim = Vmax
    else:
        instr = "Discharge"
        Vlim = Vmin
    I_mag = np.abs(I_app)
    try:
        op_cond = f"{instr} at {I_mag}A for 1 hour or until {Vlim}V"
        experiment = pybamm.Experiment(operating_conditions=[op_cond])
        sim = pybamm.Simulation(model=model,  parameter_values=param,  experiment=experiment)
        solution = sim.solve(starting_solution=solution.last_state)
        time_after_solve = solution["Time [h]"].entries[-1]
    except:
        time_after_solve = i
        print(time_after_solve)
    complete_time = i+1
    time_remaining = np.around(complete_time - time_after_solve, 5)
    print(time_remaining)
    if time_remaining > 0:
        print("Cycle:", i ,"current is", I_app, "Hold time:", time_remaining, "hours")
        #op_cond = f"Rest for {time_remaining} hours"
        op_cond = f"Hold at {Vlim}V for {time_remaining} hours"
        experiment = pybamm.Experiment(operating_conditions=[op_cond])
        sim = pybamm.Simulation(model=model,  parameter_values=param, experiment=experiment)
        solution = sim.solve(starting_solution=solution.last_state)

error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_10472/3116575818.py in <module>
     45                                 parameter_values=param,
     46                                 experiment=experiment)
---> 47         solution = sim.solve(starting_solution=solution.last_state)
     48 
     49 sim.plot()

AttributeError: 'NoneType' object has no attribute 'last_state'

[brosaplanella]

That's because the first time you go into the loop solution is None. Instead, define a variable last_state=None and after you have solved an iteration define last_state = solution.last_state.

[manjunathnilugal]

Sir, I tried the solution mentioned, but getting different error, ( I have the recent pybamm version)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_15068\1177678318.py in <module>
     42         experiment = pybamm.Experiment(operating_conditions=[op_cond])
     43         sim = pybamm.Simulation(model=model, parameter_values=param, experiment=experiment)
---> 44         solution = sim.solve(starting_solution = solution.last_state)
     45     last_state = solution.last_state
     46 

NameError: name 'solution' is not defined

[brosaplanella]

You need to set solution = sim.solve(starting_solution = last_state)

[manjunathnilugal]

Getting error if I am setting solution = sim.solve(starting_solution = last_state), is this because of the try and except in the code.
Error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_16736\4133516929.py in <module>
     42         experiment = pybamm.Experiment(operating_conditions=[op_cond])
     43         sim = pybamm.Simulation(model=model, parameter_values=param, experiment=experiment)
---> 44         solution = sim.solve(starting_solution = last_state)
     45     last_state = solution.last_state
     46 

~\AppData\Roaming\Python\Python39\site-packages\pybamm\simulation.py in solve(self, t_eval, solver, check_model, save_at_cycles, calc_esoh, starting_solution, initial_soc, callbacks, **kwargs)
    742                     cycle_first_state,
    743                 ) = pybamm.make_cycle_solution(
--> 744                     starting_solution.steps,
    745                     esoh_solver=esoh_solver,
    746                     save_this_cycle=True,

AttributeError: 'Solution' object has no attribute 'steps'

[brosaplanella]

Please let's follow the discussion here. Can you share a minimum working example of your code?

[manjunathnilugal]

Yes, Here is the code, Kindly let me know where i am going wrong.

import pybamm
import numpy as np

model = pybamm.lithium_ion.SPM()
param = pybamm.ParameterValues("Chen2020")

drive_cycles = np.array([2, 1,-2,-5, -1, -3, -1, -2, 3, 2, 5, 4])

Vmin = 2.5
Vmax = 4.1

param['Lower voltage cut-off [V]'] = Vmin
param['Upper voltage cut-off [V]'] = Vmax

last_state = None

for i, I_app in enumerate(drive_cycles):
    if I_app < 0:
        instr = "Charge"
        Vlim = Vmax
    else:
        instr = "Discharge"
        Vlim = Vmin
    I_mag = np.abs(I_app)
    
    try:
        op_cond = f"{instr} at {I_mag}A for 1 hour or until {Vlim} V"
        experiment = pybamm.Experiment(operating_conditions = [op_cond])
        sim = pybamm.Simulation(model = model, parameter_values = param, experiment = experiment)
        solution = sim.solve(starting_solution = last_state)
        time_after_solve = solution["Time [h]"].entries[-1]
        
    except:
        time_after_solve = i
    
    complete_time = i+1
    time_remaining = np.around(complete_time - time_after_solve, 5)
    
    if time_remaining > 0:
        print("Cycle:", i , "Hold time:", time_remaining, "hours")
        op_cond = f"Hold at {Vlim}V for {time_remaining} hours"
        experiment = pybamm.Experiment(operating_conditions = [op_cond])
        sim = pybamm.Simulation(model = model, parameter_values = param, experiment = experiment)
        solution = sim.solve(starting_solution = last_state)
        
    last_state = solution.last_state
    
time_taken = solution.solve_time
print(time_taken)

Error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_16564\901650437.py in <module>
     42         experiment = pybamm.Experiment(operating_conditions = [op_cond])
     43         sim = pybamm.Simulation(model = model, parameter_values = param, experiment = experiment)
---> 44         solution = sim.solve(starting_solution = last_state)
     45 
     46     last_state = solution.last_state

C:\ProgramData\Anaconda3\lib\site-packages\pybamm\simulation.py in solve(self, t_eval, solver, check_model, save_at_cycles, calc_esoh, starting_solution, initial_soc, callbacks, **kwargs)
    661                     cycle_first_state,
    662                 ) = pybamm.make_cycle_solution(
--> 663                     starting_solution.steps,
    664                     esoh_solver=esoh_solver,
    665                     save_this_cycle=True,

AttributeError: 'Solution' object has no attribute 'steps'

[brosaplanella]

The last_state does not work with starting_solution, not sure if that is intended or is it a bug (I have opened issue #2788). In any case, not sure still why you use this syntax to run the drive cycle. Can't you use a drive cycle within the experiment as explain in this example? Then you could easily break your simulation down into cycles and use save at cycles. This should also help with the speed of the code.

[manjunathnilugal]

If the battery reaches the maximum or minimum voltage then the simulation throws an error, and drive cycle don't have repetitive current values.
Even I am facing the trouble with the syntax due to high ram consumption and computation timing, but I am not able to find the better option.