Asynchronous and Batch Process Tasks

Asynchronous Process Task Runner

Now that we’ve created a synchronous version of our clearing logs task, it’s only a few steps to turn that into an asynchronous task. From within our ClearCacheController, change the CommandTaskRunner import to use the ProcessTaskRunner:

use Concrete\Core\Command\Task\Runner\ProcessTaskRunner;

Then, change the getTaskRunner command to reference this new task runner.

public function getTaskRunner(TaskInterface $task, InputInterface $input): TaskRunnerInterface
{
    $command = new ClearLogCommand();
    return new ProcessTaskRunner($task, $command, $input, t('Clearing the Concrete log...'));
}

There are a couple things to keep in mind here.

1. The ProcessTaskRunner class takes $input as one of its parameters; make sure you add it.
2. The message here is triggered when the command is added to the queue – it's not necessarily when the command has completed. That means we should update the message to reflect that we’re starting the clearing process, not necessarily that it has completed.

Now that we’ve made the changes, let’s run the command from the Dashboard:

And once it starts running, here’s how it looks:

Note a couple differences here:

1. We are redirected to the Tasks Activity Dashboard page. That’s because our task hasn’t officially completed when the response comes back to the browser.
2. We do get our starting text letting the user know the task has started.
3. We see “Clear Log” running in the Currently Running section. When the Dashboard polls for currently running tasks again the task will be moved into the process history.

Anatomy of the Batch Process Task Runner

The last task type is the Batch process task type. Batch processes are excellent for tasks that operate on a large number of objects, but do the same thing to each object. Examples of these types of operations include

• Rescan Files
• Reindex Page Content
• Process Emails
• Send Emails to Users
• Deactivate Users Based on Criteria

In all of these cases, a batch is created beforehand, and converted into a large list of queued commands. As the queue is processed, each unit of work occurs.

Let’s take a look at how a batch processing task works by checking out the “Generate Thumbnails” task. The Generate Thumbnails task is meant to be run on sites that generate thumbnails asynchronously, in case any thumbnails failed to generate. It loops through all files in the system, gets all thumbnails for that file, and creates a command for each one of these objects to regenerate that particular thumbnail. Here’s what the getTaskRunner method looks like for the GenerateThumbnailsController class:

public function getTaskRunner(TaskInterface $task, InputInterface $input): TaskRunnerInterface
{
    $fileList = new FileList();

    $thumbnailTypes = Type::getVersionList();
    $batch = Batch::create();

    foreach ($fileList->getResults() as $file) {
        if ($file instanceof File) {
            foreach ($file->getFileVersions() as $fileVersion) {
                foreach ($thumbnailTypes as $thumbnailType) {
                    if ($fileVersion->getTypeObject()->supportsThumbnails()) {
                        $batch->add(new GeneratedThumbnailCommand((int)$file->getFileID(), (int)$fileVersion->getFileVersionID(), $thumbnailType->getHandle()));
                    }
                }
            }
        }
    }

    return new BatchProcessTaskRunner($task, $batch, $input, t('Thumbnail generation beginning...'));
}

Let’s look at what’s happening here. The first couple lines are just for the business of retrieving files and thumbnail types. The first line that really matters for batch processing is

Batch::create();

This is the static method create found within Concrete\Core\Command\Batch\Batch. This method creates a Concrete\Core\Command\Batch\Batch object, which is used with the BatchProcessTaskRunner. Check out the class – you can use different methods within it to name or set other properties of the object.

Once you have a Batch object, you add messages or commands to it using the $batch->add() method. Here we’re adding GeneratedThumbnailCommand objects, which take file IDs, file version IDs, and thumbnail type handles. The object matters for this particular command, but for batches it could be any message. All that matters is that it can be handled by a command handler.

Once you have a batch with messages/commands in it, you add it to the BatchProcessTaskRunner, along with the task, input object from the command, and the message you want to display to the user when processing begins. Once you create a batch process task with proper handlers, Concrete CMS tasking takes care of everything else: