Let’s make a bet my dear engineer. I’ll ask you a question and if I can guess your answer before you say it aloud, you’ll be obliged to read the rest of this article. If I’m wrong, feel free to carry on your merry way. Ready?
What is the first thing you think of when I say “containers”? Chances are, the first thing that came to mind was a little blue whale and the word Docker. Was I right? I think for most of us, the concept of containerization sits hand in hand with Docker, a group of platform as a service products that enable users to bundle software in lightweight packages called containers.
Tripping Through Time
One of the first implementations of container technology was introduced as early as 1979, during the development of Unix V7. It was known as the chroot system call, an operation that could change the root directory of a process, along with its children, to a new location. This marked the advent of process isolation. However, there was a serious limitation with the chroot mechanism; fragile security. Any “chrooted” program can take advantage of the fact that chroot contexts do not stack properly, allowing it to break out of its isolated environment and potentially gain root permissions.
In 2000, Jails were introduced through FreeBSD. This was one of the first instances of operating-system-level virtualization and it enabled system administrators to compartmentalize their FreeBSD computer into several, independent miniature systems known as “jails”. Processes that run within a jail cannot access any of the resources outside of it, addressing the security flaws of the chroot mechanism.
The groundwork for containerization was established during a time where technology as a whole was still very much in its infancy, leaving these innovations underappreciated for many years. Fast forward to 2013 when a superstar is born and containerization finally enters the limelight. Enter Docker, one of the most popular containerization platforms around. Taking a few cues from its predecessors, Docker utilizes OS-level virtualization to package software in isolated environments known as containers. Much like jails, each container is separate from one another and they all share a single operating system kernel. What distinguishes Docker from the rest of the pack is its Docker Engine, the industry-standard container runtime. What makes Docker Engine so special is its ability to allow containerized applications to run on any infrastructure, completely circumventing the problem of “well, it works on my computer!”.
What’s the Deal with Logs?
With the ability to run multiple applications within standalone domains comes the task of actively trying to monitor the health of each one. Applications hosted within a Docker container can emit something called a log that can help identify any kind of issue that may arise during the lifespan of a container’s use. Docker logs are essentially a containerized application’s way of telling us how it’s day is going. If its day is going well, a log will be sent to the stdout output stream. If it’s not feeling so hot, a log will be sent to the stderr stream instead. If it feels like accepting input from others, it will submit those inputs to its stdin stream. These logs will tell us everything we need to know about how our application is doing along with a myriad of meaningful metrics (timestamps, process-relevant messages, process IDs, image-names…. the list goes on). By observing these logs and the metrics they provide, it is possible to identify when and where something goes wrong, which is vital to the debugging process. For instance, by looking for logs passed through the stderr stream, we can identify which containers are acting up and how frequently by drilling into the container’s id and timestamps to address them accordingly. Analyzing these logs are critical to diagnosing the current state of our Dockerized applications.
Want to read this story later? Save it in Journal.
Just One Problem…
As it stands currently, analyzing Docker logs is quite the challenge. Troubleshooting applications contained within Docker proves to be much more difficult than troubleshooting traditional apps that run on a single node. Traditional applications can be diagnosed by looking at availability, latency, and errors per second. This simple approach does not apply with Docker as the root causes of an issue generally exist outside the scope of these simple metrics. To get the full story, we need to parse through our logs. The problem with this approach is that containers produce large amounts of output and digesting that data becomes exponentially more difficult as the number of logs grows. Identifying any kind of error would be akin to finding a needle in a haystack. To address this issue, companies that are reliant on Docker either have to invest in paid tools to manage their container logs or build their own. As it currently stands, there are very few open-source options to assuage the economic implications involved with solving this problem. Docker does provide a desktop application to aid in container management that comes equipped with log management capabilities, called Docker Desktop; however, it is not very robust and functionality is limited.
These are just the logs from a single container, but what if we have multiple containers running at the same time that all are emitting issues at varying levels of complexity? Each container can produce thousands of logs, and research shows that an average business is running not one, but EIGHT containers at a time; having an individualized display like the picture above is not exactly helpful. If only we could aggregate all incoming logs in one dashboard enabled with filter and search capability for specific criteria, effectively streamlining the debugging process and saving us the trouble of opening up each container to manually read through a giant wall of text …
If Only There Were a Better Way…
Thankfully, there is a solution to all of these problems. Introducing Dockter, a low overhead, open-source Docker log management tool. It keys in on three core features:
- Real-time log aggregation: Dockter can aggregate an immense amount of logs in a short period of time and provide them to the user in a comprehensive, interactive dashboard
- Storage: Dockter is an Electron app that provides users with the ability to store logs on either their machine’s local memory or local disk storage by way of a noSQL database.
- Search: Dockter provides users the ability to filter and search in tandem with live incoming data streams across seven criteria (Container ID, Container Name, Container Image, Time of Log, Public Port, Private Port, and Stream Type)
Dockter is best suited for small to mid-size companies looking for an open-source solution to their log management woes. As more and more companies continue to adopt Docker into their tech stacks, demand for proper log management will grow concurrently. As companies scour the internet in search of solutions, they will notice a gap between what they want to achieve and what they can actually afford. Nestled neatly within this gap rests Dockter, the latest and greatest addition to the Docker ecosystem. The team over at Dockter hopes that by bridging this gap we can continue the democratization of technology.
If you use Docker and love open source, please consider giving us a star on GitHub and checking out our website!
More from Journal
There are many Black creators doing incredible work in Tech. This collection of resources shines a light on some of us:
