Install Python Packages Hosted On Private GitHub

Unfortunately, GitHub does not provide the ability to create a token that is restricted to a specific respository. This is unlike GitLab where one could create a project access token.

Thus, for this tutorial, we will make use of "deploy keys" that are SSH keys we have added to GitHub for specific packages.

Prerequisites / Assumptions

1. We are assuming you already have a private GitHub respository for a python package.
2. Your repository needs to be set up correctly as a python package, with its setup.py at the root/top level of your GitHub repository. You cannot have it within a subfolder.
3. You need the ability to create GitHub deploy keys on the repository.

Extra Requirements If Using Docker

If trying to do this within a Docker container, you will need to have installed the openssh-client, and git packages.

You will also need to have started your SSH agent and created your identity:

eval `ssh-agent -s`

You also need to add the remote host to your known hosts file:

mkdir "$HOME/.ssh"
touch "$HOME/.ssh/known_hosts"
ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts 

If your repository makes use of LFS then you will also need to install the git-lfs package, as well as run:

git lfs install --skip-repo

Steps

Create a GitHub deploy key for your package.

Add the SSH key to your SSH agent, so that it will automatically be used when applicable.

ssh-add my-deploy-key

If you need to do this non-interactively with a passphrase (such as within a Docker container), you can do it like so:

ssh-add /root/ml-deploy-key.pem <<< "$DEPLOY_KEY_PASSPHRASE"

The following examples are lines you would add to your requirements.txt file, or passed to a pip command. Each of them require the #egg=my-package-name where my-package-name is the name of your package.

Specific Hash

git+ssh://git@github.com/github-username/repo-name.git@41b95ec#egg=my-package-name

Specific Branch

Here we use a branch called my-branch-name

git+ssh://git@github.com/github-username/repo-name.git@my-branch-name#egg=my-package-name

Tag

Here we use use a tagged version where the tag is called my-tag-name

git+ssh://git@github.com/github-username/repo-name.git@my-tag-name#egg=egg=my-package-name

Install Command

Once you have the relevant line you can install through a requirements file with:

pip install --requirement requirements.txt

... or you can install directly with:

pip3 install $LOCATION

References

• Fernando Freitas Alves - Pip Installing a Package From a Private Repository
• Stack Overflow - How to state in requirements.txt a direct github source
• Adam Johnson - 'pip install' From a Git Repository
• Stack Overflow - How to install python package from git repo that has git-lfs content with pip?