Drush entity:delete command (EntityCommands.php) should use batch processing since it runs out of memory if deleting thousands of entities about drush

[weitzman]

This has been possible for 3 years #4583. See --chunks option.

[codebymikey]

Hi @weitzman,

The --chunk option doesn't actually mitigate this specific issue.

In that issue, the results of the $result = $query->execute(); call returns an array of the 1,000+ IDs, it just doesn't load or process them into full entities yet. However, if there are more IDs in the database than can be feasibly loaded in memory (e.g. millions), then the command simply doesn't work - which is why handling it within a batch context is the more robust way of handling it.

And in terms of implementation, I was thinking something along the lines of this (but we can't use n.nid > :nid type logic since the entity might not be numeric), this or this.

[weitzman]

Thanks for clarifying. Can you really run out of memory just from a few million integers? That doesnt seem like a lot of memory consumption. What is your CLI memory_limit? Can it be bumped?

[codebymikey]

Hmm, thought about it a bit more after reviewing how many entities I had to delete (720,000), and you were right, and that seems like a feasible amount of entities IDs to be loaded in PHP. I had a memory limit of 256M on the CLI.

$ php -i | grep mem
memory_limit => 256M => 256M

And was able to resolve the issue by temporarily increasing the memory limit in my settings.php with the following:

if (PHP_SAPI === 'cli') {
  # 512M still wasn't enough.
  ini_set('memory_limit', '-1');
}

But I guess it'd just be nice to have the code optimized to not require as much memory if possible.