Simple Protocol Buffers (Protobuf) PHP Example

Introduction

I must have heard ThePrimeagen banging on about "protobuffs" for ages, referencing them in multiple of his videos. I didn't really know what they were, and had not bothered tofind out until now. After doing a little digging, I found out they are far simpler to implement and get started with than I had thought. When I had started my research, I ended up going down this path to gRPC that I hit roadblocks with (hopefully will fix at some point in the future), but you do not need gRPC to use protobuffs, and it is just a separate technology that gRPC makes use of.

Demo Codebase

I created a codebase on GitHub to provide a simple working example of using Protobuffs in PHP. There are a few steps one needs to go through, such as installing and running the protobuf compiler, which generates the files for your types, from the .proto file in the codebase. These steps are outlined in the README file.

Really, the codebase boils down to this very simple index.php file shown below:

<?php

use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;

require_once(__DIR__ . '/vendor/autoload.php');
require_once(__DIR__ . '/GPBMetadata/MyApplication.php');
require_once(__DIR__ . '/Types/User.php');
require_once(__DIR__ . '/Types/UserList.php');

$app = Slim\Factory\AppFactory::create();
$app->addErrorMiddleware($displayErrorDetails=true, $logErrors=true, $logErrorDetails=true);

$app->get('/', function (ServerRequestInterface $request, ResponseInterface $response) {

    // Create a user.
    $user1 = new \Types\User();
    $user1->setId(1);
    $user1->setName("User1");
    $user1->setEmail("user.1@programster.org");

    // Create a second user.
    $user2 = new \Types\User();
    $user2->setId(2);
    $user2->setName("User2");
    $user2->setEmail("user.2@programster.org");

    // Create the list of users that form our UserList response message.
    $users = [$user1, $user2];
    $userList = new \Types\UserList();
    $userList->setUsers($users);

    if (true)
    {
        // output in the binary protobuf form for efficiency.
        $responseBody = $userList->serializeToString();
    }
    else
    {
        // Alternatively, you can flip the switch to output as normal JSON.
        $responseBody = $userList->serializeToJsonString();
    }

    $response->getBody()->write($responseBody);
    return $response;
});

$app->run();

As you can see, there is nothing complicated or hard about this. It is a really easy way to make your responses more efficient, and if you change your mind and wish to switch back to JSON, this can be done at the flip of a switch.

When To Use Protocol Buffers

Protocol buffers have the following key advantages:

• Generate types for client code in almost any language. E.g. your server may be written in PHP, but your Python client can easily digest the messages from generating the types from the proto file.
• Faster and more efficient serialization and deserialization than JSON, using this binary form.
• More efficient use of bandwidth as message sizes are smaller than using JSON.
• The previous two points can dramatically reduce response times, causing your website to "run" much faster.

Disadvantages

Developer Knowledge

Developers in the team have yet another technology to learn and know about. E.g. how to maintain the .proto file and use it to generate the types etc.

Maintaining Types

The advantage of protocol buffers it the types, but this also has the cost of maintaining those types. If these types change on the server, then the applications need to recieve the updates in order to be able to process the messages as well. With loosly coupled JSON, this may not be as much of an issue. e.g. if the client code just runs a json decode operation and looks for certain keys, this code will likely still work if a key got added, rather than deleted or renamed.

Sending Large Messages

I've only just started with protocol buffers, but it does appear that there are real tradeoffs. I had hoped that it would be a great way to stream large amounts of data between services, but it turns out that protocol buffers need to read and validate the entire message in memory (still more efficient than doing this with JSON, which has to do the same). Thus if passing large amounts of data around, one may actually be better off sending something like a CSV file, which can be handled by parsing the input line-by-line. This way, the application can be programmed to digest an infinite file size within a constrained amount of memory. Alternatively, one can send multiple messages in a response. However, if doing this then one needs to keep track of where each protobuf message begins and ends as described here. Alternatively, one may wish to look into FlatBuffers instead (package).