How to Set Up and Use a Private Docker Registry?

vIn the present times, Containerization, Virtualization, Docker, etc are certain terms which are used by programmers and developers on a daily basis. The terms might be foreign to you as a novice who has just entered the world of programming, but it is vital to be familiar with the terminology related to your field. Among programmers, Docker is known for its excellent containerization service. A beginner may be confused by some of the terminology in the container world, such as pods, clusters, images, etc. Among them is the term Container image.

An image is a file which contains tools, codes, libraries, and dependencies, etc to run an application or perform a function which is commonly used by the team. It can run in an isolated environment. Pulling an image and using it saves the time of writing the code every time you need to use the function.

It resides on a public Docker registry so it is accessible to every team member who needs it.

Docker Hub is one such Docker registry which saves the docker images. But what if you are writing a program or developing an application on your own, and don’t need to share the container image? In that case you can create a private docker registry.

Now you must be wondering about how to set up and use a private docker registry. This guide will provide you detailed instructions on how to set up a private docker registry.

Prerequisites For Setting Up A Private Docker registry

Before telling you how to set up and use a private docker registry, you must be aware of the basic requirements to set up and use a private docker registry.

Docker and Docker-Compose installed on both client and server systems.
Access to a sudo non-root user.
Firewall
Nginx installed on the server.
A domain to connect to the host.

How To Set Up A Private Docker Registry?

Before you dive into the steps, you must ensure that you have Docker and Docker Compose installed on your server and client.

Docker works best for smaller functions such as starting and testing containers but when you are working with multiple containers running simultaneously, Docker Compose is what you need. Docker-Compose .yml file contains configuration for all the containers, making communication between them fast and smooth.

To get started first check whether your server has Docker and Docker Compose installed and what version is installed by executing the following commands.

docker version

This command will check the Docker Version.

docker-compose version

This command will check Docker Compose Version.

You must ensure that Docker service is all set and running. You can start and enable it with the following command.

sudo systemctl start docker

sudo systemctl enable docker

Step 1: Install and Configure The Docker registry

When you have ensured that Docker and Docker Compose are set and running, write a docker-compose.yml file to manage Docker registry. On the main server, you need a Docker Registry to store the configuration.

You can create it by writing the following command-

mkdir – / docker-registry

Then follow with-

cd – / docker-registry

Create a sub-directory for the registry to store images.

mkdir data

Create a .yml file-

nano docker-compose.yml

Create a basic instance of Docker Registry by writing-

~/docker-registry/docker-compose.yml

version: ‘3’

services:

registry:

image: registry:2

ports:

– “5000:5000”

environment:

REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data

volumes:

– ./data:/data

Save the file and close it.

Use docker compose to start configuration-

docker-compose up

The output will look something like this-

Output

Creating network “docker-registry_default” with the default driver

Pulling registry (registry:2)…

2: Pulling from library/registry

e95f33c60a64: Pull complete

4d7f2300f040: Pull complete

35a7b7da3905: Pull complete

d656466e1fe8: Pull complete

b6cb731e4f93: Pull complete

Digest: sha256:da946ca03fca0aade04a73aa94b54ff0dc614216bdd1d47585f97b4c1bdaa0e2

Status: Downloaded newer image for registry:2

Creating docker-registry_registry_1 … done

Attaching to docker-registry_registry_1

registry_1 | time=”2021-03-18T12:32:59.587157744Z” level=warning msg=”No HTTP secret provided – generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable.” go.version=go1.11.2 instance.id=119fe50b-2bb6-4a8d-902d-dfa2db63fc2f service=registry version=v2.7.1

registry_1 | time=”2021-03-18T12:32:59.587912733Z” level=info msg=”redis not configured” go.version=go1.11.2 instance.id=119fe50b-2bb6-4a8d-902d-dfa2db63fc2f service=registry version=v2.7.1

registry_1 | time=”2021-03-18T12:32:59.598496488Z” level=info msg=”using inmemory blob descriptor cache” go.version=go1.11.2 instance.id=119fe50b-2bb6-4a8d-902d-dfa2db63fc2f service=registry version=v2.7.1

registry_1 | time=”2021-03-18T12:32:59.601503005Z” level=info msg=”listening on [::]:5000″ go.version=go1.11.2 instance.id=119fe50b-2bb6-4a8d-902d-dfa2db63fc2f service=registry version=v2.7.1

…

Now you have a docker-compose configuration to start Docker-Registry that will be listening on port 5000.

Step 2: Nginx Port Forwarding

Now, you need to set up Nginx port forwarding to expose your Docker Registry. It will help you direct traffic to the registry container.

Previously, you set up the “/etc/nginx/sites-available/your_domain” file.

Edit it by executing-

sudo nano /etc/nginx/sites-available/your_domain

Look for the existing “location” block-

/etc/nginx/sites-available/your_domain

…

location / {

…

}

…

Port 5000 will listen to the traffic and appended header will provide the additional information about the request. You need to write codes to ensure that the contents of the existing “location” block.

/etc/nginx/sites-available/your_domain

…

location / {

# Do not allow connections from docker 1.5 and earlier

# docker pre-1.6.0 did not properly set the user agent on ping, catch “Go *” user agents

if ($http_user_agent ~ “^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$” ) {

return 404;

}

proxy_pass http://localhost:5000;

proxy_set_header Host $http_host; # required for docker client’s sake

proxy_set_header X-Real-IP $remote_addr; # pass on real client’s IP

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-Forwarded-Proto $scheme;

proxy_read_timeout 900;

}

…

Save the file and close it.
Restart the Nginx to incorporate the changes.

sudo systemctl restart nginx

To ensure that it is working properly, check by running-

docker-compose up

Now open a browser window, go to your domain and open “v2” endpoint with

https:// your_domain/v2

The output will be:

{ }

The output in your terminal will look like-

Output

registry_1 | time=”2018-11-07T17:57:42Z” level=info msg=”response completed” go.version=go1.7.6 http.request.host=cornellappdev.com http.request.id=a8f5984e-15e3-4946-9c40-d71f8557652f http.request.method=GET http.request.remoteaddr=128.84.125.58 http.request.uri=”/v2/” http.request.useragent=”Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7″ http.response.contenttype=”application/json; charset=utf-8″ http.response.duration=2.125995ms http.response.status=200 http.response.written=2 instance.id=3093e5ab-5715-42bc-808e-73f310848860 version=v2.6.2

registry_1 | 172.18.0.1 – – [07/Nov/2018:17:57:42 +0000] “GET /v2/ HTTP/1.0” 200 2 “” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7”

It confirms that your request was successfully handled by the container.
Now, you can stop the execution using “Ctrl+C”.
The registry has been created and you must secure it now.

Step 3: Set Up Authentication

Use HTTPS in Nginx to set up authentication for your docker registry.

It can be done by creating an authentication file which will help you set up a username and password for a private secure access.
“htpasswd” helps you create the authentication file. Install the “apache2-utils” package to obtain htpasswd utility.
Run the following command-

sudo apt install apache2-utils -y

Store the authentication file under ”~/docker-registry/auth” along with credentials by running-

mkdir ~/docker-registry/auth

Follow with-

cd ~/docker-registry/auth

Create your username using-

htpasswd -Bc registry.password username

Replace the “username” with whatever username you want to create.
Then enter the password when asked. These credentials will be saved to Docker-Registry.
Now, you need to inform Docker about the authentication credentials. Hence, you need to edit docker-compose.yml.
Edit it by executing-

nano ~/docker-registry/docker-compose.yml

Follow by adding the highlighted text-

~/docker-registry/docker-compose.yml

version: ‘3’

services:

registry:

image: registry:2

ports:

– “5000:5000”

environment:

REGISTRY_AUTH: htpasswd

REGISTRY_AUTH_HTPASSWD_REALM: Registry

REGISTRY_AUTH_HTPASSWD_PATH: /auth/registry.password

REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data

volumes:

– ./auth:/auth

– ./data:/data

With this, you have informed Docker to use the “htpasswd” file which carries authentication credentials.

It’s time to verify that your authentication set up is successful.

Go to main directory-

cd ~/docker-registry

Execute-

docker-compose up

Refresh the domain page in your browser and enter the credentials asked.
As you enter the details you can see-

{ }

The empty JSON object signifies that your authentication was successful.
Press “Ctrl+C” to exit.
After securing your registry, you are now set to start Docker Registry as a Service.

Step 4: Start Docker Registry As A Service

Instruct the Docker Compose to always keep the registry container running, so that it starts every time on system boot up and crash.
Open the “docker-compose.yml”.

nano docker-compose.yml

Follow with-

docker-compose.yml

…

registry:

restart: always

…

Here you have set it to “restart”, so every time the system reboots or crashes, the container will start.
Save the file and close it.
Set up your registry to run it as a background process by executing-

docker-compose up -d

The registry will now run in background and you can close SSH sessions without affecting the Docker registry.

Step 5: Increase The Docker Registry Size

Docker images vary in size. Some are small while others are large.Your registry should have enough space to handle the large files as well. Increase the size for nginx to ensure that.

Nginx by default size is “1m” which is very meagre. You can increase it by editing the Nginx Config file whose location is-

/etc/nginx/nginx.conf

Open the file.

sudo nano /etc/nginx/nginx.conf

In the http section, type the highlighted line-

/etc/nginx/nginx.conf

…

http {

client_max_body_size 15486m;

…

}

…

The size has now been increased to 15486 mb which is almost 15 GB.
Save the file and close it.
To incorporate the change, restart Nginx.

sudo systemctl restart nginx

With this you have set up the Docker Registry. But it answers only half of the question of how to set up and use a private Docker Registry.

Using A Private Docker Registry

Pushing An Image To Docker Registry

Your Docker Registry is ready to use. You can start by pushing a Docker image. If you have a Docker image available, it’s good that you can try using it by pushing the “Ubuntu” image which can be found in the local Docker Hub.

Go to your client server and use the following command to download the image, run it, and obtain access.

docker run -t -i ubuntu /bin/bash

-t and -i are used to obtain the container shell access.

You can customize your container by creating a file called “Strong”, which will be used to check whether you are running the same container or not.
Run the following code to create the file-

root@f7e13d5464d1:/# touch /SUCCESS

Execute the following to exit the shell-

root@f7e13d5464d1:/# exit

You can create a new image from the customized container by executing-

docker commit $(docker ps -lq) test-image

Now, let’s try pushing it to the container. but first, login using command:

docker login https://your_domain

Instead of “your_domain”, enter your domain name.

Enter the credential and you’ll see-

Output

…

Login Succeeded

Now, rename the image you created by running-

docker tag test-image your_domain/test-image

Push this image to registry with-

docker push your_domain/test-image

The output will look something like-

Output

The push refers to a repository [your_domain/test-image]

420fa2a9b12e: Pushed

c20d459170d8: Pushed

db978cae6a05: Pushed

aeb3f02e9374: Pushed

latest: digest: sha256:88e782b3a2844a8d9f0819dc33f825dde45846b1c5f9eb4870016f2944fe6717 size: 1150

So, you have set up a registry, you have configured authentication, and you have learned how to set up and push an image to the Docker registry. Therefore, it is time to learn how to pull an image from your private Docker registry.

Pulling An Image From Docker Registry

Log in after entering your credentials as you did in the previous step.
Now pull the image from Docker Registry with-

docker pull your_domain/test-image

It will download the image from container and you can run it with the following command-

docker run -it your_domain/test-image /bin/bash

Run the following command to list the files-

It will list down the image STRONG you created earlier, confirming that you are doing it right.

STRONG bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var

Now exit the container shell by executing-

exit

That’s it. This is all about how to set up and use a private Docker registry.

Conclusion

The process of setting up a Private Docker Registry involves various steps and lots of commands. These commands are very simple, so even if you don’t know how to use command-line tools, you shouldn’t have a problem following them. If you are only starting with Docker, you can write them as it is and it will solve your problem of how to set up and use a private Docker registry.

The guide starts with ensuring that you have the required tools ready to set up the registry. It contains directions and commands on how to set up a docker registry, secure it, push and pull images, etc. It contains a comprehensive guide and a detailed answer on how to set up and use your own Docker registry.