You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Writing command line apps it's very common to work with streams of data and to group them to be processed in chunks.
The native PHP's array_chunk in that case is not an efficient function, as it needs the whole input array present and it is generating the full result array in chunks as well. This is at that moment more than doubling the memory overhead.
My proposal is to add the following method to ArrayUtils naming can be changed as needed.
This will allow stream data processing and data grouping without huge memory problems.
/** * Generate chunks of a given size from an iterable list. * * @param mixed $iterator * Items to iterate (array, generator, iterator). * @param int $size * Chunk size to group items. * * @return \generator * Generated items are arrays with up-to $size elements each. */publicstaticfunctionchunksGenerator($iterator, $size = 1000) {
$count = 0;
$buffer = [];
foreach ($iteratoras$item) {
$buffer[] = $item;
++$count;
if (!($count % $size)) {
yield$buffer;
$buffer = [];
$count = 0;
}
}
if ($buffer) {
yield$buffer;
}
}
The text was updated successfully, but these errors were encountered:
Writing command line apps it's very common to work with streams of data and to group them to be processed in chunks.
The native PHP's array_chunk in that case is not an efficient function, as it needs the whole input array present and it is generating the full result array in chunks as well. This is at that moment more than doubling the memory overhead.
My proposal is to add the following method to ArrayUtils naming can be changed as needed.
This will allow stream data processing and data grouping without huge memory problems.
The text was updated successfully, but these errors were encountered: