April 29, 2026 • 19 mins
In this lesson, you’ll learn about: Docker Compose workflows, API versions, and real-world microservices orchestration1. Essential Docker Compose Commands & WorkflowUsing Docker Compose, you can manage your entire application lifecycle with a few commands:🔹 Core Commands
You can listen and download our episodes for free on more than 10 different platforms:
https://linktr.ee/cybercode_academy
- docker-compose up → Start services
- docker-compose build → Build images
- docker-compose stop → Stop containers
- docker-compose ps → List running containers
- docker-compose logs → View logs
- Builds images
- Pulls dependencies
- Starts everything in detached mode
- Runs multiple instances of a service
- Useful for:
- Load balancing
- Testing distributed systems
- Overrides CMD from Dockerfile
- Lets you reuse the same image for:
- Web server
- Background worker
- Scheduler
- Docker Compose started as:
- Fig (community project)
- Compatible with modern Docker
- Required for newer features
- Flask → frontend
- Node.js → API layer
- .NET → worker service
- Redis → queue/cache
- PostgreSQL → database
- Services are split into:
- Front-tier → user-facing
- Back-tier → internal services
- Security
- Isolation
- Traffic control
- Use host-mounted volumes
- Enables:
- Live code updates
- No rebuild needed
- Requires:
- Rebuilding the image after changes
- You’d manually configure:
- 5+ containers
- Networks
- Dependencies
- Prevents naming conflicts
- Keeps projects isolated
- Docker Compose simplifies multi-container orchestration
- up --build -d = real-world workflow shortcut
- Version 3 is the modern standard
- Supports:
- Scaling
- Networking
- Volume management
- Essential for microservices architectures
- Build images
- Run multi-service apps
- Manage dependencies
- Scale and debug systems
You can listen and download our episodes for free on more than 10 different platforms:
https://linktr.ee/cybercode_academy
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
So imagine you've just been handed the keys to this
completely massive web application. I mean it's a sprawling piece
of engineering.
Speaker 2 (00:09):
Oh yeah, that kind that makes you sweat a little
when you first look at the code.
Speaker 1 (00:12):
Exactly. You've got like a Python frontend for the users,
a NOJS real time results page, some dot net backend
worker crunching data, a Redtis queue for traffic spikes, and
a postcress database holding it all together.
Speaker 2 (00:27):
Right, a very standard but very complex modern stack.
Speaker 1 (00:30):
And starting all this manually, I mean booting up every
single container one by one, linking all those networks, mapping ports,
getting the boot sequenced just right.
Speaker 2 (00:39):
It's a nightmare. You're talking hours of configuration just to
get a local dev environment running, right.
Speaker 1 (00:45):
But what if what if you could spin up this
entire universe, fully networked and ready to develop against with
just a single line of text.
Speaker 2 (00:52):
Well that is the exact capability we are exploring today.
We're looking at the ability to take a sprawling, multi
tiered architecture and just bring it to life instantly and.
Speaker 1 (01:02):
Completely bypass that manual configuration phase. So welcome to the
deep dive. Our mission today is to really master Docker compose.
Speaker 2 (01:11):
Yeah, moving from running single isolated containers from the command
line to orchestrating complex real world apps.
Speaker 1 (01:18):
Exactly, we're going to paint a picture exactly what's happening
under the hood when you're in the terminal and your
code editor. Okay, so let's jump right in. If you
look at the terminal and type Docker, dash composed, dash
dash help, you get this massive list of commands, a.
Speaker 2 (01:33):
Lot of which you know map directly to standard doctor
CLI stuff like build, pull up. But the foundational concept
here is that compose requires.
Speaker 1 (01:41):
A blueprint right set of instructions.
Speaker 2 (01:43):
Exactly. By default, it scans your current directory for a
specific file Dash composed, dot e e mL. That YAML
file is your architectural blueprint.
Speaker 1 (01:52):
Well, what if I have like a setup strictly for
local development and a completely different one for staging.
Speaker 2 (01:57):
Oh, then you use the dash flop to point this
CLI to a custom file name. But standard practice assumes
you're operating out of the project root folder.
Speaker 1 (02:06):
Okay, that makes sense. So looking at those core commands,
Build scans that yamal file looks for any service with
a build property pointing to a Docker file, and compiles
the image.
Speaker 2 (02:18):
Yeah, and pull just grabs any precompiled external images you
might need from Docker hub, you know, like a fresh
Rettis or Postgress.
Speaker 1 (02:24):
Image can then there's UP.
Speaker 2 (02:26):
Yeah. UP is where the orchestration engine really kicks into
high gear. It doesn't just start a bunch of isolated containers.
Speaker 1 (02:32):
It's doing a lot more behind the scenes, right.
Speaker 2 (02:34):
Oh. Absolutely. It parses your YAML file to create the
necessary virtual networks so the containers can actually talk to
each other by name.
Speaker 1 (02:42):
Oh wow.
Speaker 2 (02:42):
Yeah, and it provisions named volumes so your database doesn't
just wipe itself clean every time you restart, and then
it boots the entire stack.
Speaker 1 (02:49):
Wait, so if I can just use Docker run for everything,
why am I bothering with the doctor Compose up. What
is the actual difference.
Speaker 2 (02:56):
Well, with Jocker run, you the developer, are responsible for
for all the network strings tying the containers together. Compose
just automates that glue. When upspins your containers into existence,
it automatically prefixes their names with the directory of your
project to prevent collisions.
Speaker 1 (03:14):
So if my folder is called analytics app, the database
container natively becomes analytics app underscore dB.
Speaker 2 (03:22):
Exactly, and it also appends an underscore and a number
to the end like Underscore one buy the number that
suffix is critical. Compose is inherently designed with scaling in mind.
If your traffic spikes and you suddenly need three instances
of your web server, Compose handles the horizontal scaling for you,
so it.
Speaker 1 (03:41):
Spins up Underscore one, Underscore two, and Underscore three.
Speaker 2 (03:43):
Right now, you do have to architect your yamil file
carefully if you plan to scale. I mean, you can't
hardcode a binding to host port eighty for a web
service because you can't bind three containers to the same
port at the same time without a load balancer.
Speaker 1 (03:56):
Yeah, that would definitely cause a conflict. Yeah, but that
Underscore one naming conventions signals that the engine is built
for a multi instance reality exactly. Still, typing out build
pulling up sequentially every time you sit down to work
isn't super efficient.
Speaker 2 (04:09):
No, it's not. The standard power user combo is Docker
Compose up, dash Dash Dash build right, So.
Speaker 1 (04:15):
The up initializes everything, the dash dash build forces Composed
to check your source code and rebuild any stale images,
and the dash d.
Speaker 2 (04:23):
Flag runs the entire stack in detached mode. It hands
your terminal prompt right back to you.
Speaker 1 (04:28):
Which is great, but it introduces visibility issue. I mean,
if the app is running in the background, you aren't
seeing the logs. You won't know if your node server
is through an exception.
Speaker 2 (04:37):
That's true. So to regain that visibility you use Docker
Composed Logs.
Speaker 1 (04:41):
Dash F and the DASH stands for follow, right.
Speaker 2 (04:44):
Yeah, tailing the logs live. And the cool part is
Compose aggregates the output from every single service into one
unified stream and color CODs it so.
Speaker 1 (04:54):
Your postcress logs might print in blue, your web server
in green.
Speaker 2 (04:58):
Exactly. It allows you to monitor the health of the
entire system at a glance.
Speaker 1 (05:02):
Okay, but what about when you're done monitoring you hit
seter all ceco to stop the log tail. Yeah, and
we run into that well known.
Speaker 2 (05:09):
Quirk, oh, the cuttle sea bug.
Speaker 1 (05:11):
Yeah. Sometimes, instead of just detaching, Compose intercepts the signal
and tries to shut down the containers, but it aborts
with an error message. It just leads the project in
this weird half alive state.
Speaker 2 (05:24):
With a bunch of orphan processes.
Speaker 1 (05:26):
Right, and running Docker kill feels super aggressive. So is
there a cleaner way to force composed to clean up
its own mess?
Speaker 2 (05:33):
You definitely want to avoid randomly killing processes when that
bug hits. The best recovery is running Docker composed stop.
Speaker 1 (05:40):
Oh, so just bypass the UZLC chaos entirely exactly.
Speaker 2 (05:44):
It sends a proper, graceful shut down signal to every
container in your project. And if you want to completely
wipe the stopped containers from your system, you just follow
that up with doctor composed ARM.
Speaker 1 (05:54):
Let's clarify one more CLI interaction that honestly trips up
a lot of developers. Exec versus is run. They look
so similar, but they do totally completely, completely completely completely
completely completely completely completely different things.
Speaker 2 (06:06):
They serve completely completely completely completely completely completely completely completely
different architectural purposes.
Speaker 1 (06:12):
Right, So, doctor composed exec is designed for interacting with
an active environment. If the web server is currently running
and you need to inspect a config file, exec show
opens a shell directly inside that already running container.
Speaker 2 (06:25):
I like to think of it like walking through an
open door into a room where people are actively working.
Speaker 1 (06:31):
That's a great analogy.
Speaker 2 (06:32):
But Docker composed run, on the other hand, is for
isolated one off tasks. It spins up a brand new
temporary container, runs a command, and then immediately exits.
Speaker 1 (06:42):
So if you need to quickly check the specific version
of the reddess image you pulled, you don't boot up
your entire five tier architecture.
Speaker 2 (06:49):
No, you just use run to start a temporary redtus container,
check the version, and let it shut down. And crucially,
Run does not bind any ports to your host machine
by default, which ensures.
Speaker 1 (07:00):
Your temporary task doesn't collide with anything else running on
your system.
Speaker 2 (07:04):
Exactly, you know.
Speaker 1 (07:05):
Managing even these optimized commands, stringing together networks and volumes
from the CLI is exactly what frustrated the open source
community years ago.
Speaker 2 (07:13):
Oh, absolutely, the friction was real.
Speaker 1 (07:15):
It's actually what led a group of independent developers to
build a tool called FIG, which, fun fact, was the
original incarnation of Docker compose. Yeah.
Speaker 2 (07:24):
Fig was born purely out of developer frustration. The community
realized that managing multi container setups manually was just unsustainable, so.
Speaker 1 (07:33):
FIG provided a declarative way to define infrastructure in a
single text file.
Speaker 2 (07:38):
And it became so universally adopted that Docker actually absorbed
the open source project directly into their official tool set
and just rebranded it as Docker Compose.
Speaker 1 (07:47):
It's wild to think about how it started and when
you look at the evolution of that YAML file since
the fig days. The very first line usually declares the version.
Speaker 2 (07:56):
Right, which dictates the composed API version, and that is
tightly coupled to the version of the Docker engine running
on the host machine.
Speaker 1 (08:03):
So version one is the legacy format. I think you
can spot it because it actually lacks a version declaration
at the top entirely.
Speaker 2 (08:09):
Yeah, and you should really avoid version one unless you
are strictly maintaining legacy systems. It lacks all the modern
abstractions required for complex architectures.
Speaker 1 (08:18):
Like it doesn't understand the concept of named services right.
Speaker 2 (08:21):
Nope, no name services, no support for custom virtual networks,
and it lacks volume name spaces.
Speaker 1 (08:27):
Wow. It's essentially like trying to organize a modern smartphone
using a nineteen nineties metal filing cabinet. It just doesn't
have the internal compartments you need for how apps red
data today that's spot on.
Speaker 2 (08:41):
Version two introduced those compartments, making it great for complex
single server deployments. But version three is the modern standard.
Speaker 1 (08:49):
What is version three?
Speaker 2 (08:50):
Add It introduced swarm properties, allowing you to take that
exact same yamal file and spread your containers across a
cluster of multiple physical machines.
Speaker 1 (08:58):
Oh okay, but even for local dev on a single laptop,
Version three is still the recommended default.
Speaker 2 (09:04):
Right yeah, because it future proofs your infrastructure blueprint without
adding any local overhead.
Speaker 1 (09:08):
Okay, let's actually put this blueprint into practice with a
real world scenario. Let's look at that complex microservices voting
app we mentioned earlier.
Speaker 2 (09:15):
Ah, yes, the classic cats versus Dogs voting app.
Speaker 1 (09:19):
Or if you dig into the Python variables and some
versions of this app ear, hair versus nosehair.
Speaker 2 (09:24):
I always prefer the cats and Dogs version honestly.
Speaker 1 (09:27):
Fair enough, So let's trace the data flow here. The
user opens their browser to port five thousand, hitting the
frontend Vote app right.
Speaker 2 (09:36):
That front end is built in Python using the Flask framework.
The moment the user clicks cats, the Flask app receives
the htt II request. But a key architectural decision here
is that the Flask app does not write that vote
to a database.
Speaker 1 (09:49):
Wait really, why not?
Speaker 2 (09:51):
Because databases introduce latency and the front end needs to
respond instantly. So instead, the Python app pushes the vote
payload directly into a queue, which in this case is reddis.
Speaker 1 (10:01):
Ah because reddis is an in memory data store exactly.
Speaker 2 (10:04):
It handles read and write operations at blistering speeds, so
the vote is logged in milliseconds and the Python app's
job is completely finished, so.
Speaker 1 (10:12):
It's super fast for the user. Then, sitting behind the scenes,
completely disconnected from the web traffic, is the Worker yep.
Speaker 2 (10:19):
The Worker is a dot net application written in c sharp.
It has no web interface, exposes no ports. It just
runs in a continuous loop quietly monitoring reddis and.
Speaker 1 (10:28):
The moment it detects a new vote in the queue,
it pulls the data.
Speaker 2 (10:31):
Right the c sharp worker processes the payload and securely
rates the final official vote into the permanent database, which
is Postgress.
Speaker 1 (10:39):
And finally we have the Results app.
Speaker 2 (10:41):
Which is a separate NOJS application using Express running on
port five thousand and one.
Speaker 1 (10:45):
So it maintains a persisting connection to the Postgress database
and streams the live results to a daftboard using web
sockets exactly.
Speaker 2 (10:53):
The user clicks a button on port five thousand and
instantly the progress bar updates on port five thousand and one.
Speaker 1 (10:59):
That is five distinct technologies Python, reddis, c Shark, Postgress
and no js all running together.
Speaker 2 (11:07):
And the isolation here demonstrates the sheer power of micro
services decoupling. I mean, the Python frontend is completely oblivious
to the existence of the no dis application, and.
Speaker 1 (11:16):
The postgress database doesn't care what language generated the data exactly.
Speaker 2 (11:20):
If the engineering team decides to deprecate the Python frontend
tomorrow and rewrite it in rubyond rails, they only touch
one container, the reddest que the Sea Shark worker. Everything
else remains perfectly intact.
Speaker 1 (11:31):
Okay, But practically speaking, if I just type up on
a stack this complex, how does composed no not to
boot the dot networker before the reddest que is actually
ready to accept connections?
Speaker 2 (11:41):
Oh that's a great point. If the worker boots first
and tries to connect to a queue that doesn't exist yet,
the application crashes. So that requires structural logic within the
ammal file itself, specifically using the depends underscore on property.
Speaker 1 (11:55):
How does that work?
Speaker 2 (11:56):
Under your sea sharp worker service definition? You add a
depends underscore on array and list rettis and postcress. This
literally instructs the composed engines boot sequence. It guarantees the
database and q containers are initiated before the worker even
attempts to start.
Speaker 1 (12:11):
Okay, that handles the boot order. But what about credentials.
I mean, if this is a shareable blueprint that gets
committed to version control, I absolutely do not want my
postgress admin password hard coded in plaintext inside the Docker
dash composed dot MLL file.
Speaker 2 (12:26):
Yeah, definitely not. Security best practices dictate using environment variables
instead of hard coding the password. You use variable substitution
in the YAMIL like a dollar sign and curly braces
around posties underscore password.
Speaker 1 (12:39):
And where does it actually get the password from?
Speaker 2 (12:41):
Compose automatically looks for a hidden dot en file on
your local machine, reads the secret from there, and securely
injects it into the container at runtime.
Speaker 1 (12:51):
Oh so the dot en file stays locally on your
machine and is never committed to like GitHub precisely nice.
And if I'm reading the YAML arc texture right, We're
also utilizing custom network tiers to secure the infrastructure. The
database gets tagged to a back dash tier network. Well,
the front end gets tagged to a front dash tier network.
Speaker 2 (13:11):
Yeah. Network isolation is huge.
Speaker 1 (13:12):
Well, how does Compose actually enforce that isolation at the
OS level?
Speaker 2 (13:16):
So under the hood, Docker uses Linux Bridge networks and
modifies host iptable's rules. When you declare front tier and
back tier and compose the engine creates two entirely separate
virtual subnets.
Speaker 1 (13:28):
So the postcress database, the reds Q, and the dot
networker are assigned exclusively to the back tier.
Speaker 2 (13:33):
Literally, yes, they do not have IP addresses on the
public facing network at all.
Speaker 1 (13:38):
But the front end apps, the Python vote app and
the nogs results app, they need to talk to the
user and.
Speaker 2 (13:46):
The queue right, so they are attached to both the
front tier and the back tier networks. They have a
virtual network interface in both subnets, so.
Speaker 1 (13:53):
They accept web traffic on the public tier and they
route data to reddis on the private tier exactly.
Speaker 2 (14:00):
But because the Python app is only aware of reddis,
it physically cannot route a network packet directly to postgress.
Speaker 1 (14:06):
Wow.
Speaker 2 (14:06):
The network topology natively enforces your security boundaries, preventing any
public facing compromise from easily reaching your data layer.
Speaker 1 (14:14):
So it's effectively a firewall rule built natively into the
deployment script that solves a massive infrastructure headache.
Speaker 2 (14:21):
It really does.
Speaker 1 (14:22):
But as a developer actually working inside that fortress, how
do you handle local development without tearing the walls down?
Every time you change a single line of code.
Speaker 2 (14:32):
That is where live development workflows come into play, primarily
utilizing overrides and volumes. Overrides are incredibly useful when you
want to run multiple services from the exact same codebase.
Speaker 1 (14:43):
Oh, let's say we have a Python Jango application. One
service needs to run the guncorn web server to handle requests,
but another service needs to run a sellary background worker
for ACYNC tasks.
Speaker 2 (14:54):
Right like sending emails or something. You don't want to
maintain two separate docro files for the exact same source code.
Speaker 1 (15:00):
Sounds like a maintenance nightmare.
Speaker 2 (15:01):
It is, so in your composed yammel, you define two
distinct services that both build from the same directory. But
for the worker service you use the command property.
Speaker 1 (15:10):
Ah, so that property overrides the default startup instruction in
the docker file exactly.
Speaker 2 (15:16):
The web service executes the web server command while the
worker service executes the sealery command. You get two entirely
different functional containers derived from a single image.
Speaker 1 (15:26):
That eliminates so much tredundancy. But what about editing the
code itself? If I want to change the text on
the voting button from cats to ear hair, rebuilding a
five tier architecture just to test a typo is totally unworkable.
Speaker 2 (15:39):
Oh yeah, nobody has time for that.
Speaker 1 (15:41):
The yamil uses volume mapping for the source code. But
what is docer actually doing there? Is it like constantly
copying the local files into the container every time I
hit save?
Speaker 2 (15:50):
It doesn't copy the files at all. Actually, it creates
a bridge. When you use a bind mount volume in composed,
you are instructing the Docker engine to map a directory
on your host st operating system directly into the container's
isolated file system.
Speaker 1 (16:04):
So it acts similarly to a simlink.
Speaker 2 (16:06):
Yes, when you hit save on your laptop MVS code,
the container instantly reads that exact same file modification from
the host disc and.
Speaker 1 (16:14):
For interpreted languages, that triggers an instant update. Like no
JS uses a tool called nodemen to watch the filesystem.
Speaker 2 (16:21):
Right and the moment the mapped volume registers a save change,
noamen internally restarts the node server inside the container, and
your browser reflects the change in milliseconds.
Speaker 1 (16:32):
It creates a totally seamless live reloading environment. But the
critical distinction here is how compiled languages behave right. Because
your dot networker is written in c shark, which has
to be translated into machine code before it can execute.
Speaker 2 (16:45):
Exactly if you openprogram dot cs for the Dot networker
and change a console log message. Saving the file updates
the text via the volume map, but the running application doesn't.
Speaker 1 (16:56):
Change because the binary executing inside the cant hainer hasn't
actually been recompiled.
Speaker 2 (17:02):
Right. If you modify compiled code, you have to switch
back to your terminal, stop the environment, and execute Docker
compose up Dash dash build.
Speaker 1 (17:10):
So the dash dash build flag is mandatory.
Speaker 2 (17:12):
There it is. It instructs Docker to pull the new
source code, run the c sharp compiler, package the new
binary into a fresh image layer, and restart the back end.
Speaker 1 (17:21):
Honestly, just deducing the difference between how interpreted and compiled
languages interact with Docker volumes saves developers hours of debugging.
Speaker 2 (17:28):
Oh absolutely, you get the instant gratification of live reloading
for your web front ends while recognizing the necessary recompilation
steps for strict compiled back ends.
Speaker 1 (17:38):
If we zoom out and look at what we've covered today,
we started by analyzing the command center right mapping CLI
commands like up build and the detachment flags.
Speaker 2 (17:47):
We traced the architectural history of FIG evolving into the
modern API version three, and.
Speaker 1 (17:53):
We dissected a complete micro services architecture, examining the internal
boot sequence using depends on them, securing credits with environment variables,
and setting up isolated virtual subnets.
Speaker 2 (18:04):
We also connected all those architectural concepts to practical daily workflows,
you know, utilizing command over rides to maximize image efficiency
and establishing bind mounts for rapid local development.
Speaker 1 (18:16):
It's a massive upgrade to the developer experience. But you know,
knowledge is most valuable when you critically apply it to
your own work. So here's your review exercise for this
deep dive. I want you to mentally map out an
application you rely on every.
Speaker 2 (18:29):
Day, maybe a streaming service or a financial dashboard.
Speaker 1 (18:32):
Yeah, if you had to architect that platform using just
three micro services, a public facing front end, an internal
queue to buffer heavy traffic spikes, and a secure back
end worker to process the core logic, how would you
separate those concerns.
Speaker 2 (18:47):
What distinct languages or frameworks would be best suited for each.
Speaker 1 (18:50):
Role, and crucially, how would you design the doctor composed
virtual networks to ensure the frontend never has direct routing
access to the back end.
Speaker 2 (18:59):
Data, visualize the YAML structure and the network topology for
a familiar system is honestly the fastest way to cement
these orchestration concepts.
Speaker 1 (19:07):
Definitely. We began this deep dive by imagining you were
handed the keys to a complex multi service universe, and
we showed how to orchestrate that universe on your local
machine with just a single configuration file. But consider this
final thought. We've seen how effectively Doctor Composed manages a
symphony of containers on a single host processor.
Speaker 2 (19:26):
But what happens when that single machine hits its hardware limits?
Speaker 1 (19:29):
Right? What happens when millions of users attempt to vote
on your platform simultaneously and a single server CPU maxes
out at one hundred percent? The structural logic you just learned,
the declarative yammel, the decoupled services, the virtual networking. That
is the exact foundational blueprint required for scaling into distributed
cloud clusters and cubinetes. The orchestration journey doesn't end on
(19:51):
your laptop. That is precisely where it begins
So imagine you've just been handed the keys to this
completely massive web application. I mean it's a sprawling piece
of engineering.
Speaker 2 (00:09):
Oh yeah, that kind that makes you sweat a little
when you first look at the code.
Speaker 1 (00:12):
Exactly. You've got like a Python frontend for the users,
a NOJS real time results page, some dot net backend
worker crunching data, a Redtis queue for traffic spikes, and
a postcress database holding it all together.
Speaker 2 (00:27):
Right, a very standard but very complex modern stack.
Speaker 1 (00:30):
And starting all this manually, I mean booting up every
single container one by one, linking all those networks, mapping ports,
getting the boot sequenced just right.
Speaker 2 (00:39):
It's a nightmare. You're talking hours of configuration just to
get a local dev environment running, right.
Speaker 1 (00:45):
But what if what if you could spin up this
entire universe, fully networked and ready to develop against with
just a single line of text.
Speaker 2 (00:52):
Well that is the exact capability we are exploring today.
We're looking at the ability to take a sprawling, multi
tiered architecture and just bring it to life instantly and.
Speaker 1 (01:02):
Completely bypass that manual configuration phase. So welcome to the
deep dive. Our mission today is to really master Docker compose.
Speaker 2 (01:11):
Yeah, moving from running single isolated containers from the command
line to orchestrating complex real world apps.
Speaker 1 (01:18):
Exactly, we're going to paint a picture exactly what's happening
under the hood when you're in the terminal and your
code editor. Okay, so let's jump right in. If you
look at the terminal and type Docker, dash composed, dash
dash help, you get this massive list of commands, a.
Speaker 2 (01:33):
Lot of which you know map directly to standard doctor
CLI stuff like build, pull up. But the foundational concept
here is that compose requires.
Speaker 1 (01:41):
A blueprint right set of instructions.
Speaker 2 (01:43):
Exactly. By default, it scans your current directory for a
specific file Dash composed, dot e e mL. That YAML
file is your architectural blueprint.
Speaker 1 (01:52):
Well, what if I have like a setup strictly for
local development and a completely different one for staging.
Speaker 2 (01:57):
Oh, then you use the dash flop to point this
CLI to a custom file name. But standard practice assumes
you're operating out of the project root folder.
Speaker 1 (02:06):
Okay, that makes sense. So looking at those core commands,
Build scans that yamal file looks for any service with
a build property pointing to a Docker file, and compiles
the image.
Speaker 2 (02:18):
Yeah, and pull just grabs any precompiled external images you
might need from Docker hub, you know, like a fresh
Rettis or Postgress.
Speaker 1 (02:24):
Image can then there's UP.
Speaker 2 (02:26):
Yeah. UP is where the orchestration engine really kicks into
high gear. It doesn't just start a bunch of isolated containers.
Speaker 1 (02:32):
It's doing a lot more behind the scenes, right.
Speaker 2 (02:34):
Oh. Absolutely. It parses your YAML file to create the
necessary virtual networks so the containers can actually talk to
each other by name.
Speaker 1 (02:42):
Oh wow.
Speaker 2 (02:42):
Yeah, and it provisions named volumes so your database doesn't
just wipe itself clean every time you restart, and then
it boots the entire stack.
Speaker 1 (02:49):
Wait, so if I can just use Docker run for everything,
why am I bothering with the doctor Compose up. What
is the actual difference.
Speaker 2 (02:56):
Well, with Jocker run, you the developer, are responsible for
for all the network strings tying the containers together. Compose
just automates that glue. When upspins your containers into existence,
it automatically prefixes their names with the directory of your
project to prevent collisions.
Speaker 1 (03:14):
So if my folder is called analytics app, the database
container natively becomes analytics app underscore dB.
Speaker 2 (03:22):
Exactly, and it also appends an underscore and a number
to the end like Underscore one buy the number that
suffix is critical. Compose is inherently designed with scaling in mind.
If your traffic spikes and you suddenly need three instances
of your web server, Compose handles the horizontal scaling for you,
so it.
Speaker 1 (03:41):
Spins up Underscore one, Underscore two, and Underscore three.
Speaker 2 (03:43):
Right now, you do have to architect your yamil file
carefully if you plan to scale. I mean, you can't
hardcode a binding to host port eighty for a web
service because you can't bind three containers to the same
port at the same time without a load balancer.
Speaker 1 (03:56):
Yeah, that would definitely cause a conflict. Yeah, but that
Underscore one naming conventions signals that the engine is built
for a multi instance reality exactly. Still, typing out build
pulling up sequentially every time you sit down to work
isn't super efficient.
Speaker 2 (04:09):
No, it's not. The standard power user combo is Docker
Compose up, dash Dash Dash build right, So.
Speaker 1 (04:15):
The up initializes everything, the dash dash build forces Composed
to check your source code and rebuild any stale images,
and the dash d.
Speaker 2 (04:23):
Flag runs the entire stack in detached mode. It hands
your terminal prompt right back to you.
Speaker 1 (04:28):
Which is great, but it introduces visibility issue. I mean,
if the app is running in the background, you aren't
seeing the logs. You won't know if your node server
is through an exception.
Speaker 2 (04:37):
That's true. So to regain that visibility you use Docker
Composed Logs.
Speaker 1 (04:41):
Dash F and the DASH stands for follow, right.
Speaker 2 (04:44):
Yeah, tailing the logs live. And the cool part is
Compose aggregates the output from every single service into one
unified stream and color CODs it so.
Speaker 1 (04:54):
Your postcress logs might print in blue, your web server
in green.
Speaker 2 (04:58):
Exactly. It allows you to monitor the health of the
entire system at a glance.
Speaker 1 (05:02):
Okay, but what about when you're done monitoring you hit
seter all ceco to stop the log tail. Yeah, and
we run into that well known.
Speaker 2 (05:09):
Quirk, oh, the cuttle sea bug.
Speaker 1 (05:11):
Yeah. Sometimes, instead of just detaching, Compose intercepts the signal
and tries to shut down the containers, but it aborts
with an error message. It just leads the project in
this weird half alive state.
Speaker 2 (05:24):
With a bunch of orphan processes.
Speaker 1 (05:26):
Right, and running Docker kill feels super aggressive. So is
there a cleaner way to force composed to clean up
its own mess?
Speaker 2 (05:33):
You definitely want to avoid randomly killing processes when that
bug hits. The best recovery is running Docker composed stop.
Speaker 1 (05:40):
Oh, so just bypass the UZLC chaos entirely exactly.
Speaker 2 (05:44):
It sends a proper, graceful shut down signal to every
container in your project. And if you want to completely
wipe the stopped containers from your system, you just follow
that up with doctor composed ARM.
Speaker 1 (05:54):
Let's clarify one more CLI interaction that honestly trips up
a lot of developers. Exec versus is run. They look
so similar, but they do totally completely, completely completely completely
completely completely completely completely different things.
Speaker 2 (06:06):
They serve completely completely completely completely completely completely completely completely
different architectural purposes.
Speaker 1 (06:12):
Right, So, doctor composed exec is designed for interacting with
an active environment. If the web server is currently running
and you need to inspect a config file, exec show
opens a shell directly inside that already running container.
Speaker 2 (06:25):
I like to think of it like walking through an
open door into a room where people are actively working.
Speaker 1 (06:31):
That's a great analogy.
Speaker 2 (06:32):
But Docker composed run, on the other hand, is for
isolated one off tasks. It spins up a brand new
temporary container, runs a command, and then immediately exits.
Speaker 1 (06:42):
So if you need to quickly check the specific version
of the reddess image you pulled, you don't boot up
your entire five tier architecture.
Speaker 2 (06:49):
No, you just use run to start a temporary redtus container,
check the version, and let it shut down. And crucially,
Run does not bind any ports to your host machine
by default, which ensures.
Speaker 1 (07:00):
Your temporary task doesn't collide with anything else running on
your system.
Speaker 2 (07:04):
Exactly, you know.
Speaker 1 (07:05):
Managing even these optimized commands, stringing together networks and volumes
from the CLI is exactly what frustrated the open source
community years ago.
Speaker 2 (07:13):
Oh, absolutely, the friction was real.
Speaker 1 (07:15):
It's actually what led a group of independent developers to
build a tool called FIG, which, fun fact, was the
original incarnation of Docker compose. Yeah.
Speaker 2 (07:24):
Fig was born purely out of developer frustration. The community
realized that managing multi container setups manually was just unsustainable, so.
Speaker 1 (07:33):
FIG provided a declarative way to define infrastructure in a
single text file.
Speaker 2 (07:38):
And it became so universally adopted that Docker actually absorbed
the open source project directly into their official tool set
and just rebranded it as Docker Compose.
Speaker 1 (07:47):
It's wild to think about how it started and when
you look at the evolution of that YAML file since
the fig days. The very first line usually declares the version.
Speaker 2 (07:56):
Right, which dictates the composed API version, and that is
tightly coupled to the version of the Docker engine running
on the host machine.
Speaker 1 (08:03):
So version one is the legacy format. I think you
can spot it because it actually lacks a version declaration
at the top entirely.
Speaker 2 (08:09):
Yeah, and you should really avoid version one unless you
are strictly maintaining legacy systems. It lacks all the modern
abstractions required for complex architectures.
Speaker 1 (08:18):
Like it doesn't understand the concept of named services right.
Speaker 2 (08:21):
Nope, no name services, no support for custom virtual networks,
and it lacks volume name spaces.
Speaker 1 (08:27):
Wow. It's essentially like trying to organize a modern smartphone
using a nineteen nineties metal filing cabinet. It just doesn't
have the internal compartments you need for how apps red
data today that's spot on.
Speaker 2 (08:41):
Version two introduced those compartments, making it great for complex
single server deployments. But version three is the modern standard.
Speaker 1 (08:49):
What is version three?
Speaker 2 (08:50):
Add It introduced swarm properties, allowing you to take that
exact same yamal file and spread your containers across a
cluster of multiple physical machines.
Speaker 1 (08:58):
Oh okay, but even for local dev on a single laptop,
Version three is still the recommended default.
Speaker 2 (09:04):
Right yeah, because it future proofs your infrastructure blueprint without
adding any local overhead.
Speaker 1 (09:08):
Okay, let's actually put this blueprint into practice with a
real world scenario. Let's look at that complex microservices voting
app we mentioned earlier.
Speaker 2 (09:15):
Ah, yes, the classic cats versus Dogs voting app.
Speaker 1 (09:19):
Or if you dig into the Python variables and some
versions of this app ear, hair versus nosehair.
Speaker 2 (09:24):
I always prefer the cats and Dogs version honestly.
Speaker 1 (09:27):
Fair enough, So let's trace the data flow here. The
user opens their browser to port five thousand, hitting the
frontend Vote app right.
Speaker 2 (09:36):
That front end is built in Python using the Flask framework.
The moment the user clicks cats, the Flask app receives
the htt II request. But a key architectural decision here
is that the Flask app does not write that vote
to a database.
Speaker 1 (09:49):
Wait really, why not?
Speaker 2 (09:51):
Because databases introduce latency and the front end needs to
respond instantly. So instead, the Python app pushes the vote
payload directly into a queue, which in this case is reddis.
Speaker 1 (10:01):
Ah because reddis is an in memory data store exactly.
Speaker 2 (10:04):
It handles read and write operations at blistering speeds, so
the vote is logged in milliseconds and the Python app's
job is completely finished, so.
Speaker 1 (10:12):
It's super fast for the user. Then, sitting behind the scenes,
completely disconnected from the web traffic, is the Worker yep.
Speaker 2 (10:19):
The Worker is a dot net application written in c sharp.
It has no web interface, exposes no ports. It just
runs in a continuous loop quietly monitoring reddis and.
Speaker 1 (10:28):
The moment it detects a new vote in the queue,
it pulls the data.
Speaker 2 (10:31):
Right the c sharp worker processes the payload and securely
rates the final official vote into the permanent database, which
is Postgress.
Speaker 1 (10:39):
And finally we have the Results app.
Speaker 2 (10:41):
Which is a separate NOJS application using Express running on
port five thousand and one.
Speaker 1 (10:45):
So it maintains a persisting connection to the Postgress database
and streams the live results to a daftboard using web
sockets exactly.
Speaker 2 (10:53):
The user clicks a button on port five thousand and
instantly the progress bar updates on port five thousand and one.
Speaker 1 (10:59):
That is five distinct technologies Python, reddis, c Shark, Postgress
and no js all running together.
Speaker 2 (11:07):
And the isolation here demonstrates the sheer power of micro
services decoupling. I mean, the Python frontend is completely oblivious
to the existence of the no dis application, and.
Speaker 1 (11:16):
The postgress database doesn't care what language generated the data exactly.
Speaker 2 (11:20):
If the engineering team decides to deprecate the Python frontend
tomorrow and rewrite it in rubyond rails, they only touch
one container, the reddest que the Sea Shark worker. Everything
else remains perfectly intact.
Speaker 1 (11:31):
Okay, But practically speaking, if I just type up on
a stack this complex, how does composed no not to
boot the dot networker before the reddest que is actually
ready to accept connections?
Speaker 2 (11:41):
Oh that's a great point. If the worker boots first
and tries to connect to a queue that doesn't exist yet,
the application crashes. So that requires structural logic within the
ammal file itself, specifically using the depends underscore on property.
Speaker 1 (11:55):
How does that work?
Speaker 2 (11:56):
Under your sea sharp worker service definition? You add a
depends underscore on array and list rettis and postcress. This
literally instructs the composed engines boot sequence. It guarantees the
database and q containers are initiated before the worker even
attempts to start.
Speaker 1 (12:11):
Okay, that handles the boot order. But what about credentials.
I mean, if this is a shareable blueprint that gets
committed to version control, I absolutely do not want my
postgress admin password hard coded in plaintext inside the Docker
dash composed dot MLL file.
Speaker 2 (12:26):
Yeah, definitely not. Security best practices dictate using environment variables
instead of hard coding the password. You use variable substitution
in the YAMIL like a dollar sign and curly braces
around posties underscore password.
Speaker 1 (12:39):
And where does it actually get the password from?
Speaker 2 (12:41):
Compose automatically looks for a hidden dot en file on
your local machine, reads the secret from there, and securely
injects it into the container at runtime.
Speaker 1 (12:51):
Oh so the dot en file stays locally on your
machine and is never committed to like GitHub precisely nice.
And if I'm reading the YAML arc texture right, We're
also utilizing custom network tiers to secure the infrastructure. The
database gets tagged to a back dash tier network. Well,
the front end gets tagged to a front dash tier network.
Speaker 2 (13:11):
Yeah. Network isolation is huge.
Speaker 1 (13:12):
Well, how does Compose actually enforce that isolation at the
OS level?
Speaker 2 (13:16):
So under the hood, Docker uses Linux Bridge networks and
modifies host iptable's rules. When you declare front tier and
back tier and compose the engine creates two entirely separate
virtual subnets.
Speaker 1 (13:28):
So the postcress database, the reds Q, and the dot
networker are assigned exclusively to the back tier.
Speaker 2 (13:33):
Literally, yes, they do not have IP addresses on the
public facing network at all.
Speaker 1 (13:38):
But the front end apps, the Python vote app and
the nogs results app, they need to talk to the
user and.
Speaker 2 (13:46):
The queue right, so they are attached to both the
front tier and the back tier networks. They have a
virtual network interface in both subnets, so.
Speaker 1 (13:53):
They accept web traffic on the public tier and they
route data to reddis on the private tier exactly.
Speaker 2 (14:00):
But because the Python app is only aware of reddis,
it physically cannot route a network packet directly to postgress.
Speaker 1 (14:06):
Wow.
Speaker 2 (14:06):
The network topology natively enforces your security boundaries, preventing any
public facing compromise from easily reaching your data layer.
Speaker 1 (14:14):
So it's effectively a firewall rule built natively into the
deployment script that solves a massive infrastructure headache.
Speaker 2 (14:21):
It really does.
Speaker 1 (14:22):
But as a developer actually working inside that fortress, how
do you handle local development without tearing the walls down?
Every time you change a single line of code.
Speaker 2 (14:32):
That is where live development workflows come into play, primarily
utilizing overrides and volumes. Overrides are incredibly useful when you
want to run multiple services from the exact same codebase.
Speaker 1 (14:43):
Oh, let's say we have a Python Jango application. One
service needs to run the guncorn web server to handle requests,
but another service needs to run a sellary background worker
for ACYNC tasks.
Speaker 2 (14:54):
Right like sending emails or something. You don't want to
maintain two separate docro files for the exact same source code.
Speaker 1 (15:00):
Sounds like a maintenance nightmare.
Speaker 2 (15:01):
It is, so in your composed yammel, you define two
distinct services that both build from the same directory. But
for the worker service you use the command property.
Speaker 1 (15:10):
Ah, so that property overrides the default startup instruction in
the docker file exactly.
Speaker 2 (15:16):
The web service executes the web server command while the
worker service executes the sealery command. You get two entirely
different functional containers derived from a single image.
Speaker 1 (15:26):
That eliminates so much tredundancy. But what about editing the
code itself? If I want to change the text on
the voting button from cats to ear hair, rebuilding a
five tier architecture just to test a typo is totally unworkable.
Speaker 2 (15:39):
Oh yeah, nobody has time for that.
Speaker 1 (15:41):
The yamil uses volume mapping for the source code. But
what is docer actually doing there? Is it like constantly
copying the local files into the container every time I
hit save?
Speaker 2 (15:50):
It doesn't copy the files at all. Actually, it creates
a bridge. When you use a bind mount volume in composed,
you are instructing the Docker engine to map a directory
on your host st operating system directly into the container's
isolated file system.
Speaker 1 (16:04):
So it acts similarly to a simlink.
Speaker 2 (16:06):
Yes, when you hit save on your laptop MVS code,
the container instantly reads that exact same file modification from
the host disc and.
Speaker 1 (16:14):
For interpreted languages, that triggers an instant update. Like no
JS uses a tool called nodemen to watch the filesystem.
Speaker 2 (16:21):
Right and the moment the mapped volume registers a save change,
noamen internally restarts the node server inside the container, and
your browser reflects the change in milliseconds.
Speaker 1 (16:32):
It creates a totally seamless live reloading environment. But the
critical distinction here is how compiled languages behave right. Because
your dot networker is written in c shark, which has
to be translated into machine code before it can execute.
Speaker 2 (16:45):
Exactly if you openprogram dot cs for the Dot networker
and change a console log message. Saving the file updates
the text via the volume map, but the running application doesn't.
Speaker 1 (16:56):
Change because the binary executing inside the cant hainer hasn't
actually been recompiled.
Speaker 2 (17:02):
Right. If you modify compiled code, you have to switch
back to your terminal, stop the environment, and execute Docker
compose up Dash dash build.
Speaker 1 (17:10):
So the dash dash build flag is mandatory.
Speaker 2 (17:12):
There it is. It instructs Docker to pull the new
source code, run the c sharp compiler, package the new
binary into a fresh image layer, and restart the back end.
Speaker 1 (17:21):
Honestly, just deducing the difference between how interpreted and compiled
languages interact with Docker volumes saves developers hours of debugging.
Speaker 2 (17:28):
Oh absolutely, you get the instant gratification of live reloading
for your web front ends while recognizing the necessary recompilation
steps for strict compiled back ends.
Speaker 1 (17:38):
If we zoom out and look at what we've covered today,
we started by analyzing the command center right mapping CLI
commands like up build and the detachment flags.
Speaker 2 (17:47):
We traced the architectural history of FIG evolving into the
modern API version three, and.
Speaker 1 (17:53):
We dissected a complete micro services architecture, examining the internal
boot sequence using depends on them, securing credits with environment variables,
and setting up isolated virtual subnets.
Speaker 2 (18:04):
We also connected all those architectural concepts to practical daily workflows,
you know, utilizing command over rides to maximize image efficiency
and establishing bind mounts for rapid local development.
Speaker 1 (18:16):
It's a massive upgrade to the developer experience. But you know,
knowledge is most valuable when you critically apply it to
your own work. So here's your review exercise for this
deep dive. I want you to mentally map out an
application you rely on every.
Speaker 2 (18:29):
Day, maybe a streaming service or a financial dashboard.
Speaker 1 (18:32):
Yeah, if you had to architect that platform using just
three micro services, a public facing front end, an internal
queue to buffer heavy traffic spikes, and a secure back
end worker to process the core logic, how would you
separate those concerns.
Speaker 2 (18:47):
What distinct languages or frameworks would be best suited for each.
Speaker 1 (18:50):
Role, and crucially, how would you design the doctor composed
virtual networks to ensure the frontend never has direct routing
access to the back end.
Speaker 2 (18:59):
Data, visualize the YAML structure and the network topology for
a familiar system is honestly the fastest way to cement
these orchestration concepts.
Speaker 1 (19:07):
Definitely. We began this deep dive by imagining you were
handed the keys to a complex multi service universe, and
we showed how to orchestrate that universe on your local
machine with just a single configuration file. But consider this
final thought. We've seen how effectively Doctor Composed manages a
symphony of containers on a single host processor.
Speaker 2 (19:26):
But what happens when that single machine hits its hardware limits?
Speaker 1 (19:29):
Right? What happens when millions of users attempt to vote
on your platform simultaneously and a single server CPU maxes
out at one hundred percent? The structural logic you just learned,
the declarative yammel, the decoupled services, the virtual networking. That
is the exact foundational blueprint required for scaling into distributed
cloud clusters and cubinetes. The orchestration journey doesn't end on
(19:51):
your laptop. That is precisely where it begins
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy, Jess Hilarious, And Charlamagne Tha God!
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.