This tutorial is going to show you how to deploy a Ceph cluster. A Ceph cluster can currently provide object or block storage over a network, and will offer a mountable filesystem in future. In layman's terms, this is equivalent to deploying your own Amazon simple storage solution (S3) or elastic block storage (EBS). This is particularly useful if you want to:
- host your own data due to performance/security/costs reasons.
- increase the services available in your cloud offering.
- provide a value added service to your dedicated server business.
We're going to deploy a Ceph cluster across 3 virtual machines, with the help of 4 Virtualbox instances. The cluster will constist of 2 storage nodes and a single monitoring node. The "extra" virtual machine being used that is not part of the cluster is just a node we use to deploy the cluster in the first place.
First, install ubuntu 14.04 server on a single Virtual Machine and ensure it is fully up-to-date, before cloning it 3 times so that you have 4 virtual box instances. We will refer to these new clones as
ceph osd1, and
We need to configure each of the nodes with a static IP and update our DNS server to point to each of these nodes. If you do not have a DNS server, then you will have to rely on their IPs, but I recommend deploying a simple DNS server in 6 easy steps with the help of Docker, which is what I do.
Configure Ceph Deployer
Use the commands below to install the
ceph-deploy tool (1.5.22trusty) on the deployment/admin node. This tool will help us to turn the other virtual machines into a Ceph cluster.
wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add - echo deb http://download.ceph.com/debian-jewel/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list sudo apt update sudo apt install ceph-deploy
jewel in the script above with the new release's name.
sudo apt install ceph-deploy instead of the commands above, then you will end up retrieving ceph-deploy 1.4.0 instead.
All Nodes - NTP & OpenSSH Server
All nodes need NTP and openssh server so that they can be connected to by the deployment tool, and so that the Ceph cluster doesn't get 'confused' due to time differences between nodes in the cluster.
sudo apt-get install ntp openssh-server -y
Configuring The Ceph User
The admin node must have password-less SSH access to all the other Ceph nodes, with sudo privileges. This is because it needs to be able to install software and configuration files without prompting for passwords. We need to create a Ceph user on ALL Ceph nodes in the cluster. A uniform user name across the cluster may improve ease of use but we don't want to use obvious user names to protect agatinst brute force hacks.
Run the following procedure on every node, substituting
[username] for the user name you define, describes how to create a user with passwordless sudo. I recommend putting in something like
USERNAME="[username]" sudo useradd -d /home/$USERNAME -m $USERNAME sudo passwd $USERNAME
Now that we have created the user on the server, we need to allow that user to execute sudo commands without being prompted for a password. We do this by running
sudo visudo -f /etc/sudoers, and adding a line as such:
[USERNAME] ALL=(ALL) NOPASSWD:ALL
Note: if you prefer to use vim over nano, then you may want to set your default editor.
Once you have created the ceph user on every node, log into the ceph user on the deployment/admin node and generate an ssh key, making sure not to set a passphrase.
Now add yourself to each of the other nodes:
ssh-copy-id [ceph user]@[hostname]
Congratulations, you've finished the first stage of deploying your cluster. You will now be able to continue to part 2 to finish deploying the cluster.