September 11, 2022

Working with Google Cloud Tasks and Symfony

An important software development best practice is that long-running tasks need to be offloaded from the main thread. This ensures the user experience will not be slowed down, even for a few seconds, when the backend is performing a long-running task.

A solution is to use tasks. Tasks are a simple, efficient method. My walkthrough will use Google Cloud Tasks, the package "google/cloud-tasks" and the Symfony framework (specifically 5.4).

Design

From a service within an app, I create a Task that calls an endpoint to perform the long-running task (off the main thread).

A lesson learned is I now put my TaskService in the main app and it writes a task to Google Cloud. In a non-development deployment, Google Cloud Tasks would then call an HTTP endpoint of my choosing.

During early development, it was easier to create local endpoints and simulate a call via Postman. (For example main Symfony app was http://localhost:8000/ and my endpoint was http://localhost:8000/.internal/endpoint.

Implementation - Google Cloud

First, make sure to enable Cloud Tasks API.

Second, to authenticate from the client side a private key is required. Pull this down also from Google Cloud through IAM & Admin > Service Accounts. (If you do not see any Service Accounts, go to Compute Engine API to ensure it's enabled.). Download the JSON file and move to the Symfony project.

Implementation - Symfony

Save the private key somewhere accessible to the Symfony project. Load up the required package from Google using composer.

google/cloud-tasks Package

composer require google/cloud-tasks

TaskService

I often abstract my service classes from underlying service classes called framework-specific functions. This allows me to interface with a "Create Task()" function and not care about what is going on under the hood. In the future, if I change to another Cloud Task provider, the change is unknown and doesn't affect the rest of the stack.

    public function createTask(
        string $queue,
        array $payload,
        string $endpoint,
    ): Task
    {
        return $this->googleCloudTaskService
            ->createTask($queue, $payload, $endpoint);
    }

GoogleCloudTaskService

Above TaskService calls the below functions with a class I call "GoogleCloudTaskService".

    public function createClient(): CloudTasksClient
    {
        return new CloudTasksClient([
            'credentials' => json_decode(
                file_get_contents(
                    $_ENV['PROJECT_ROOT_PATH'] . $_ENV['TASK_KEYFILE']
                ),
                true
            )
        ]);
    }

    public function createQueue(string $queueName): void
    {
        $client = $this->createClient();
        $project = $_ENV['TASK_PROJECT'];
        $location = $_ENV['TASK_LOCATION'];
        $locationName = $client::locationName($project, $location);
        $queue = new Queue([
            'name' => $queueName
        ]);
        $queue->setName($queueName);
        $client->createQueue($locationName, $queue);
    }

    public function createTask(
        string $queueName,
        array $payload,
        string $endpoint
    ): Task
    {
        try {
            $client = $this->createClient();
            $project = $_ENV['TASK_PROJECT'];
            $location = $_ENV['TASK_LOCATION'];
            $queue = $client::queueName($project, $location, $queueName);
            //
            $httpRequest = new HttpRequest();
            $httpRequest->setUrl($endpoint);
            $httpRequest->setHeaders(['Content-type'=>'application/json']);
            $httpRequest->setHttpMethod(HttpMethod::POST);
            //
            $httpRequest->setBody(json_encode($payload));
            $task = new Task();
            $task->setHttpRequest($httpRequest);
            return $client->createTask($queue, $task);
        } catch (ValidationException $e) {
            throw new Exception($e->getMessage(), 500);
        }
    }

Creating Task

From a function that will call a long-running process, I call the TaskService or BusinessObject->createTask() and, within it, put the logic as such. Three key elements - queue, payload (in JSON), and HTTP endpoint for the task to call.

        $queue = 'name-of-queue';
        $payload = 'something';
        $endpoint = "https://host/action";
        $result = $this->taskService->createTask(
            $queue,
            $payload,
            $endpoint
        );

Receiving Task / Executing Task

In most cases, this isn't anything other than a standard HTTP endpoint. Google Cloud Tasks does offer calls to different endpoints, such as App Engine. But to keep things simple, HTTP endpoints and JSON work best.