The mapping functions are handled in the same function? #761
-
|
Lithops with AWS Lambda backend handles user map function requests by using multiprocess lib to enable parallel processing. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Independently of the memory size, AWS Lambda provides the parallelism by starting a separate container for each activation of a For memory sizes > 1769MB, where you get more than one cpu per container, you can use the |
Beta Was this translation helpful? Give feedback.
Independently of the memory size, AWS Lambda provides the parallelism by starting a separate container for each activation of a
map(). By default, each started container processes 1 element of theiterable. For example, if theiterableof amap(func, itrerable)contains 1000 elements, Lithops + AWS lambda will start 1000 separate containers, each one processing one element of theiterable.For memory sizes > 1769MB, where you get more than one cpu per container, you can use the
worker_processesconfig key to send multiple activation to the same worker, this way you will leverage all the CPUs in a container.