Eloquent query to plain objects Collection

[eusonlito]

Hello! I am working with a project that need a high performance backend working with millions of rows and I have a small problem with Eloquent.

This query is returning about 5 millions of rows:

ExchangeModel::byProductId($productId)
  ->afterDate(date('Y-m-d H:i:s', strtotime('-1 week')))
  ->orderById()
  ->get();

This query returns a Collection of Models, but I need a Collection of plain objects.

I can't do ->get()->toArray() because ->get() method implies to have millions of models in memory to convert to array after that.

There are some method to get a Collection (or array) of plain objects from a Model without an intermediate step?

Maybe this is the right way to do it?

ExchangeModel::byProductId($productId)
  ->afterDate(date('Y-m-d H:i:s', strtotime('-1 week')))
  ->orderById()
  ->getQuery()
  ->get();

Thanks!

[denisdulici]

Have you tried Query Builder instead? https://laravel.com/docs/queries

[eusonlito]

Have you tried Query Builder instead? https://laravel.com/docs/queries

I know Query Builder, but I want to use always Models (query automations, query scopes, etc...).

But finally get result as obects collection instead modells collection.

Using getQuery works but I don't know if is the right way to do it :)

[vigneshgurusamy]

Use LazyCollections with toBase().

ExchangeModel::byProductId($productId)
  ->afterDate(date('Y-m-d H:i:s', strtotime('-1 week')))
  ->orderById()
  ->toBase()
  ->cursor();

[eusonlito]

I know LazyCollections., but I need to have all rows in memory at same time to do calculations (average, min, max) and partial filters inside other collections.

[vigneshgurusamy]

Then use toBase() with get().

ExchangeModel::byProductId($productId)
  ->afterDate(date('Y-m-d H:i:s', strtotime('-1 week')))
  ->orderById()
  ->toBase()
  ->get();

[robert-moore96]

i would suggest doing your calculations within the db query rather than pulling all the data into php objects and calculating in php

it will normally be faster, you avoid these problems of initializing lots of data in memory, and depending on your data it can avoid floating point errors

[eusonlito]

@rs-sliske is not so easy 😅 I need data to share with other processes, is not only an input and an output from database records.

Using plain objects I can get process in a acceptable CPU/Time/Memory range, but with Models all is going out of control.

Using a Collection of Models:

• SQL Queries: 312
• SQL Time: 8.86s
• Memory Usage: 114M
• Models: 3.546.975
• Total Time: 26.93s

Same code using a Collection of plain objects (toBase()->get()):

• SQL Queries: 312
• SQL Time: 8.43s
• Memory Usage: 20M
• Models: 0
• Total Time: 10.78s

Finally, the best solution is to use toBase/getQuery to avoid Models on Eloquent result.