Mastering Containerization: A Guide to Creating Docker-Like Environments without Docker | by Dimitris Poulopoulos | Feb, 2023
Unlocking the power of containerization: A step-by-step tutorial on building container-like environments.
Containers have revolutionized how we deploy and manage applications, providing unparalleled levels of portability, scalability, and consistency.
However, you shouldn’t be intimidated by Docker’s sleek exterior — it’s time to delve into the mechanics that make containerization possible. By understanding the inner workings of Docker, you’ll gain a deeper appreciation for the technology and a broader understanding of your operating system.
The last three articles in this series pave the way. We discussed Namespace, Control Groups (cgroups), and overlay file systems. These are the building blocks we use today to create our own container-like environment.
This educational blog post will guide you through building lightweight, isolated environments without relying on Docker. You won’t be able to replace Docker just yet! This is only for educational purposes. Docker offers much more than creating containers. However, the journey is what matters.
Are you ready to unlock the secrets of your OS and take your understanding of containerization to the next level? Buckle up, grab a cup of coffee, and dive into the exciting world of containerization without Docker!
Learning Rate is a newsletter for those who are curious about the world of MLOps. MLOps is a broad field that strives to bring ML models to production in an efficient and rerpoducible way. Containers play a crucial role in the pipeline. If you want to learn more about topics like this subscribe here. You’ll hear from me on the last Saturday of every month with updates and thoughts on the latest MLOps news and articles!
In this article, we’ll dive headfirst into the exciting world of containers. In five minutes, you should be able to run your own Linux distribution in a container-like environment you’ve built yourself.
As we said in the foreword, we have already examined the primitives we’ll use today. But for completeness’ sake, here’s a quick recap:
- Namespaces: Namespaces are kernel features that allow you to create isolated environments within a single Linux system. Each namespace has its own view of the system, meaning that processes within a namespace are unaware of the processes running in other namespaces.
- Control Groups: Linux control groups, or cgroups, is a kernel feature that allows an administrator to allocate resources such as CPU, memory, and I/O bandwidth to groups of processes.
- Overlay File Systems: Overlay file systems enable multiple lower layers to be stacked on top of each other, creating a unified view of the data. In the context of Linux containers, an overlay file system is used to layer the changes made by a container on top of a base image while preserving the original image intact.
So, now that we know what each of the tools we’ll use do, let’s put them to work.
Coming in somewhere between 1 and 5 Mb in on-disk size (depending on the variant), BusyBox is a very good ingredient to craft space-efficient distributions.
To create a BusyBox container, you’d normally run the following command:
docker run -it --rm busybox
This command would give you a shell inside a BusyBox container. We will try to achieve something similar today without running the docker run command.
So, first things first, let’s download the image. For this, we’ll use a tool called Skopeo. This series of articles is all about containers; thus, let’s use their might to download the image we want without installing Skopeo.
Create a directory named busybox-image anywhere you like, and run the following command:
docker run -v /home/vagrant/projects/container-example/busybox-image:/busybox-image quay.io/skopeo/stable copy docker://docker.io/library/busybox:latest dir:/busybox-image
This is the output you should see:
This command will give the directory you created as a volume in the Skopeo container. Then, it will instruct the container to download the BusyBox image in the directory. Thus, if you now run an ls command, you will be able to see the downloaded image in the directory you created:
ls -la busybox-image/
You’ll see a bunch of files there. What we care about is the largest file in size. Let’s untar it in a new directory. First, create a new directory and cd into it:
mkdir busybox && cd busybox
Now untar the image inside the busybox directory:
tar xf ../busybox-image/205dae5015e78dd8c4d302e3db4eb31576fac715b46d099fe09680ba28093a7a
Run thels command again; you’ll see the root file system of the BusyBox image:
Return to the parent directory and create three new ones: an upper directory, a workdir, and a root. We have covered what these are in the post about overlay file systems:
mkdir upper workdir root
Now, let’s create our overlay file system:
sudo mount -t overlay -o lowerdir=busybox,upperdir=upper,workdir=workdir none root
Great! Now, if you run ls -la root, you should be able to see the contents of the BusyBox image inside the root directory you created. As we saw in the overlay file system tutorial, the root directory provides a unified view of the lower and upper directories. However, the lower directory remains read-only, and any changes you make will be recorded in the upper directory. This will keep our busybox base image intact.
Finally, let’s create our container-like environment using the following commands:
unshare -mipunUrf chroot ROOTFS /bin/bash
That’s it! You did it! Let’s run uname -r to verify that we’re in a BusyBox container:
You can even run a well-known command like ping and observe that you just called the BusyBox version of it:
Now, there are a lot of things missing. For example, you have no access to the internet. Running ping or wget will have no result. But that’s for another time. As we said, this won’t be a production-ready environment but an effort to demystify a few things that Docker does.
If you’d like to constrain the resource consumption of your container toy, look into the Control Groups article linked in the introduction. It’s easier than you think.
In conclusion, creating a container-like environment without Docker is a valuable skill for any developer. Whether you’re looking to explore alternative technologies, address compatibility issues, or simply broaden your understanding of containerization, the techniques and tools covered in this article will help you achieve your goals.
By following this step-by-step tutorial, you can build your own lightweight, isolated environments confidently and easily. Will it work? Not yet! Hang in there until the next article!
Remember, containerization is a constantly evolving field, and there’s always more to learn. So, keep exploring, experimenting, and pushing the limits of what’s possible!
My name is Dimitris Poulopoulos, and I’m a machine learning engineer working for Arrikto. I have designed and implemented AI and software solutions for major clients such as the European Commission, Eurostat, IMF, the European Central Bank, OECD, and IKEA.
If you are interested in reading more posts about Machine Learning, Deep Learning, Data Science, and DataOps, follow me on Medium, LinkedIn, or @james2pl on Twitter.
Opinions expressed are solely my own and do not express the views or opinions of my employer.
Unlocking the power of containerization: A step-by-step tutorial on building container-like environments.
Containers have revolutionized how we deploy and manage applications, providing unparalleled levels of portability, scalability, and consistency.
However, you shouldn’t be intimidated by Docker’s sleek exterior — it’s time to delve into the mechanics that make containerization possible. By understanding the inner workings of Docker, you’ll gain a deeper appreciation for the technology and a broader understanding of your operating system.
The last three articles in this series pave the way. We discussed Namespace, Control Groups (cgroups), and overlay file systems. These are the building blocks we use today to create our own container-like environment.
This educational blog post will guide you through building lightweight, isolated environments without relying on Docker. You won’t be able to replace Docker just yet! This is only for educational purposes. Docker offers much more than creating containers. However, the journey is what matters.
Are you ready to unlock the secrets of your OS and take your understanding of containerization to the next level? Buckle up, grab a cup of coffee, and dive into the exciting world of containerization without Docker!
Learning Rate is a newsletter for those who are curious about the world of MLOps. MLOps is a broad field that strives to bring ML models to production in an efficient and rerpoducible way. Containers play a crucial role in the pipeline. If you want to learn more about topics like this subscribe here. You’ll hear from me on the last Saturday of every month with updates and thoughts on the latest MLOps news and articles!
In this article, we’ll dive headfirst into the exciting world of containers. In five minutes, you should be able to run your own Linux distribution in a container-like environment you’ve built yourself.
As we said in the foreword, we have already examined the primitives we’ll use today. But for completeness’ sake, here’s a quick recap:
- Namespaces: Namespaces are kernel features that allow you to create isolated environments within a single Linux system. Each namespace has its own view of the system, meaning that processes within a namespace are unaware of the processes running in other namespaces.
- Control Groups: Linux control groups, or cgroups, is a kernel feature that allows an administrator to allocate resources such as CPU, memory, and I/O bandwidth to groups of processes.
- Overlay File Systems: Overlay file systems enable multiple lower layers to be stacked on top of each other, creating a unified view of the data. In the context of Linux containers, an overlay file system is used to layer the changes made by a container on top of a base image while preserving the original image intact.
So, now that we know what each of the tools we’ll use do, let’s put them to work.
Coming in somewhere between 1 and 5 Mb in on-disk size (depending on the variant), BusyBox is a very good ingredient to craft space-efficient distributions.
To create a BusyBox container, you’d normally run the following command:
docker run -it --rm busybox
This command would give you a shell inside a BusyBox container. We will try to achieve something similar today without running the docker run command.
So, first things first, let’s download the image. For this, we’ll use a tool called Skopeo. This series of articles is all about containers; thus, let’s use their might to download the image we want without installing Skopeo.
Create a directory named busybox-image anywhere you like, and run the following command:
docker run -v /home/vagrant/projects/container-example/busybox-image:/busybox-image quay.io/skopeo/stable copy docker://docker.io/library/busybox:latest dir:/busybox-image
This is the output you should see:
This command will give the directory you created as a volume in the Skopeo container. Then, it will instruct the container to download the BusyBox image in the directory. Thus, if you now run an ls command, you will be able to see the downloaded image in the directory you created:
ls -la busybox-image/
You’ll see a bunch of files there. What we care about is the largest file in size. Let’s untar it in a new directory. First, create a new directory and cd into it:
mkdir busybox && cd busybox
Now untar the image inside the busybox directory:
tar xf ../busybox-image/205dae5015e78dd8c4d302e3db4eb31576fac715b46d099fe09680ba28093a7a
Run thels command again; you’ll see the root file system of the BusyBox image:
Return to the parent directory and create three new ones: an upper directory, a workdir, and a root. We have covered what these are in the post about overlay file systems:
mkdir upper workdir root
Now, let’s create our overlay file system:
sudo mount -t overlay -o lowerdir=busybox,upperdir=upper,workdir=workdir none root
Great! Now, if you run ls -la root, you should be able to see the contents of the BusyBox image inside the root directory you created. As we saw in the overlay file system tutorial, the root directory provides a unified view of the lower and upper directories. However, the lower directory remains read-only, and any changes you make will be recorded in the upper directory. This will keep our busybox base image intact.
Finally, let’s create our container-like environment using the following commands:
unshare -mipunUrf chroot ROOTFS /bin/bash
That’s it! You did it! Let’s run uname -r to verify that we’re in a BusyBox container:
You can even run a well-known command like ping and observe that you just called the BusyBox version of it:
Now, there are a lot of things missing. For example, you have no access to the internet. Running ping or wget will have no result. But that’s for another time. As we said, this won’t be a production-ready environment but an effort to demystify a few things that Docker does.
If you’d like to constrain the resource consumption of your container toy, look into the Control Groups article linked in the introduction. It’s easier than you think.
In conclusion, creating a container-like environment without Docker is a valuable skill for any developer. Whether you’re looking to explore alternative technologies, address compatibility issues, or simply broaden your understanding of containerization, the techniques and tools covered in this article will help you achieve your goals.
By following this step-by-step tutorial, you can build your own lightweight, isolated environments confidently and easily. Will it work? Not yet! Hang in there until the next article!
Remember, containerization is a constantly evolving field, and there’s always more to learn. So, keep exploring, experimenting, and pushing the limits of what’s possible!
My name is Dimitris Poulopoulos, and I’m a machine learning engineer working for Arrikto. I have designed and implemented AI and software solutions for major clients such as the European Commission, Eurostat, IMF, the European Central Bank, OECD, and IKEA.
If you are interested in reading more posts about Machine Learning, Deep Learning, Data Science, and DataOps, follow me on Medium, LinkedIn, or @james2pl on Twitter.
Opinions expressed are solely my own and do not express the views or opinions of my employer.
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.