Getting Started with MongoDB GridFS And PHP

By default, the MongoDB database can only store documents up to 16MB, the BSON-document size limit. This is perfectly fine for most cases, but what if you want to do something crazy, and use MongoDB as a filesystem? 16MB would not be enough for storing large media content, such as movies. GridFS allows you to store data up to any size limit. The file could even be larger than a single server could physically store, made possible through sharding. This tutorial will get you started with using GridFS to store and retrieve files through PHP.

Steps

If you haven't already, install MongoDB 3.0+ on a server and install the PHP driver on whichever server is going to be holding your PHP application (may or may not be the same server). In this tutorial, I am using the same Virtualbox server.

For testing purposes, I am going to use the system to store the Big Buck Bunny video which is 888.9 MiB in size. You can download the torrent which is listed on the download page.

Create A Directory.

First create a directory to store all of our source code within.

mkdir src

Create A Settings File.

All of our scripts are going to need to know how to connect to the mongo database, so create a Settings.php file within your new directory with the following contents:

<?php

# Note: PHP 7 will allow us to just have one define using an array
define("MONGO_USERNAME", "[username]");
define("MONGO_PASSWORD", "[password]");
define("MONGO_DATABASE", "[database name]");
define("MONGO_HOST",     "[server IP or hostname]");

Store A File

The following script allows you store a file. For this tutorial, I called the file StoreFile.php.

<?php

require_once(__DIR__ . '/Settings.php');

if (!isset($argv[1]))
{
    die("Please pass the fileapth to the file you wish to store.");
}

$filepath = $argv[1];

if (!file_exists($filepath) || is_dir($filepath))
{
    die("Invalid filepath provided.");
}


function mongoConnect($username, $password, $database, $host) 
{
    $con = new Mongo("mongodb://{$username}:{$password}@{$host}"); // Connect to Mongo Server
    $db = $con->selectDB($database); // Connect to Database
    return $db;
}

$db = mongoConnect(
    MONGO_USERNAME,
    MONGO_PASSWORD,
    MONGO_DATABASE,
    MONGO_HOST
);

$grid = $db->getGridFS();

# Stick any metadata here. E.g. upload date or the owners id etc.
$metadata = array("date" => new MongoDate());
$filepath = $filepath;
$grid->storeFile($filepath, array("metadata" => $metadata));

If your developing a website rather than a CLI tool, then you may want to use the storeUpload() method rather than storeFile()

You can execute the script with:

php StoreFile.php "Big_Buck_Bunny_1080p_surround_FrostWire.com.avi"

Using relative or full paths matters. The filename stored in MongoDB will be the exact string that was passed in. Thus if you passed a full path such as /path/to/file.mp4, then the filename will be /path/to/file.mp4 and not file.mp4

Calling the script multiple times on the same file will not fail. Instead you will waste storage resources storing the same file twice. It is up to you to implement this logic if you desire.

List Files

Great, so we have stored a lot of files within the system, but how do we tell what files we have stored? The following script will list your files for you, including their file size.

<?php
require_once(__DIR__ . '/Settings.php');

function mongoConnect($username, $password, $database, $host) 
{
    $con = new Mongo("mongodb://{$username}:{$password}@{$host}"); // Connect to Mongo Server
    $db = $con->selectDB($database); // Connect to Database
    return $db;
}

$db = mongoConnect(
    MONGO_USERNAME,
    MONGO_PASSWORD,
    MONGO_DATABASE,
    MONGO_HOST
);

$grid = $db->getGridFS();

# Loop over the files and output their names and file sizes
$files = $grid->find();

while (($file = $files->getNext()) != null)
{
    print $file->getFilename() . "\t" . $file->getSize() . PHP_EOL;
}

Retrieve Files

What use is storing and listing files if I can't fetch them back again? The following script allows you to do just that.

<?php

require_once(__DIR__ . '/Settings.php');

if (!isset($argv[1]))
{
    die("Please pass the fileapth to the file you wish to store.");
}

$filepath = $argv[1];

if (!file_exists($filepath) || is_dir($filepath))
{
    die("Invalid filepath provided.");
}

function mongoConnect($username, $password, $database, $host) 
{
    $con = new Mongo("mongodb://{$username}:{$password}@{$host}"); // Connect to Mongo Server
    $db = $con->selectDB($database); // Connect to Database
    return $db;
}

$db = mongoConnect(
    MONGO_USERNAME,
    MONGO_PASSWORD,
    MONGO_DATABASE,
    MONGO_HOST
);

$grid = $db->getGridFS();

# We are going to search for the filepath passed in as first argument
$searchParams = array("filename" => $filepath);

if (false)
{
    # If you used absolute paths when storing files, then you could use
    # the following to download a folder's contents.
    $folder = '/path/to/folder';
    
    # Refer to https://secure.php.net/manual/en/class.mongoregex.php
    $filename = new MongoRegex("/^$folder");

    $searchParams = array(
        'filename' => $filename
    );
}

# Alternatively use findOne($searchParams) if you
# expect there to only be one file with the provided name.
$files = $grid->find($searchParams);

while (($file = $files->getNext()) != null)
{
    # Use a random string in case there is a file with the same name
    # in the current directory.
    $randString = substr(str_shuffle(MD5(microtime())), 0, 10);
    $outputFilepath = __DIR__ . '/' . $randString . "_" . basename($file->getFilename());
    $file->write($outputFilepath);
    print "Retrieved: " . $outputFilepath . PHP_EOL;
}

Use the script like so:

php RetrieveFiles.php "my_filename.mp4"

The script shows you how you can use regular expressions to fetch files within a "folder".

References

• MongoDB GridFS
• PHP Manual - GridFS Classes