Container Camp
San Francisco, California
2015-04-17
At DramaFever, the largest streaming video site for international content, we’ve been running Docker in production since October 2013 (well before it even went 1.0). Cutting (maybe bleeding) edge? Sounds fun! But important technology stack decisions are not made by running a Markov text generator against the front page of Hacker News. So, why are we using Docker? Simply put, it makes our development more consistent and our deployment more repeatable.
Because all the developers are developing locally using the same containers, integration is much easier when their code moves on to their EC2-based personal dev environment, the shared dev environment, QA, staging, and production. (Our previous Vagrant-based process didn’t keep us consistently all using the same environment, as production wasn’t under config management, and setting up local copies of the MySQL database with all the fixtures took just this side of eternity.) Because a production instance is serving code from a container, every new autoscaled instance that has any code at all is going to have the correct code. (And our previous “check out code from GitHub and bake an AMI” deployment process was not what you’d call speedy.)
Docker provides just enough in the way of training wheels for Linux containers that everyone can use it (for rapidly increasing values of everyone). In this talk, I detail how DramaFever implemented Docker for our entire development pipeline from laptops to production, covering the pain points and failure scenarios we’ve encountered and how we’ve worked through them.
My SysAdvent post for 2014 covers this topic in more detail.
@bridgetkromhout.. Awesome meeting you today at @containercamp #ContainerCamp ... Looking forward to your talk. :)
— Lamia Youseff (@lyouseff) April 17, 2015
Super excited to hear from @bridgetkromhout. She's up next here at @containercamp #ContainerCamp
— Ross Kukulinski (@rosskukulinski) April 17, 2015
I couldn't live tweet the end of the last #containercamp talk but I look forward to @bridgetkromhout's talk: streaming video with @docker!
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
Some serious AWS autoscaling at @dramafever /@bridgetkromhout @containercamp pic.twitter.com/VPwi4WGdzy
— Ross Kukulinski (@rosskukulinski) April 17, 2015
🐳@dramafever is streaming international content since 2009, and running @docker in production since October 2013. #ContainerCamp
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
#ContainerCamp pic.twitter.com/oyzn65YQdy
— Sebastian Batalla (@sh0b_) April 17, 2015
🐳@dramafever Peek load: tens of thousands of req/s; traffic can vary up to 20x during the week. #containercamp inb4 autoscaling pic
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
🐳@dramafever stack: @Python+@djangoproject for main site, many @golang microservices (analytics, ingest, realtime processing, bookmarking…)
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
"Which @docker storage driver should I use?" @crosbymichael says "Overlay", crap, I must update my talk about that topic 😱 #containercamp
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
Nice autoscaling you have there! @bridgetkromhout describing @dramafever #containercamp #previoustweet pic.twitter.com/ubftg78ryC
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
The brilliant @bridgetkromhout talking about streaming video with Docker @dramafever pic.twitter.com/wvJ2DIGGKU
— Container Camp (@containercamp) April 17, 2015
🐳@bridgetkromhout explains that @dramafever uses a private @docker registry based registry, because it scales. #ContainerCamp
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
Damn, my duplex ability sucks but I think @bridgetkromhout just explained that they were running one registry instance on each node?
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
✨@jpetazzo @bridgetkromhout a lot of this is discussed in detail in the #hangops she and tim did https://t.co/uwaK3qPybN :)
— brandon burton (perma-afk) (@solarce) April 17, 2015
I totally agree with @bridgetkromhout on self-hosting docker registry. They use S3 backing, we use @Rackspace cloud files for dockerized reg
— Ross Kukulinski (@rosskukulinski) April 17, 2015
@bridgetkromhout Re:“scar tissue” about the broken registry image: you can pull by digest with Docker 1.6.0! But yeah, sorry for this ;-)
— Arnaud Porterie (@icecrime) April 17, 2015
@bridgetkromhout for your docker builds, does Jenkins use cached images? Or do you do a fresh build every time?
— Ross Kukulinski (@rosskukulinski) April 17, 2015
“Yeah, device mapper is basically sadness as a service” @bridgetkromhout at #containercamp
— Arnaud Porterie (@icecrime) April 17, 2015
@bridgetkromhout of @dramafever, awesome hacks on running @docker at scale in production @containercamp. pic.twitter.com/Wjv1Ifln1K
— Lamia Youseff (@lyouseff) April 17, 2015
.@bridgetkromhout wins it with "I run FreeBSD at home" #ContainerCamp
— Casey Bisson (@misterbisson) April 17, 2015
Sadness As A Service@bridgetkromhout killing it at #containercamp
— FRANCESC @ FOSDEM (@francesc) April 17, 2015
@DanaKE @bridgetkromhout Agreed! That talk is super legit, next time ppl ask about @docker in production I want to send them to those slides
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
Sounds like @bridgetkromhout is kicking ass at #containercamp.
— Paddy (@paddycarver) April 17, 2015
Ops team has made great strides in the last year and a half at @dramafever.
"Obligatory picture of container disaster with ship capsizing—protip: does not represent reality" @bridgetkromhout at #containercamp 😄
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
"Disasters? What disasters? No. @docker has been pretty solid for us." @bridgetkromhout at #containercamp
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
"Had some quirks w/ boot2docker clock drifting when your machine suspends, b/c S3 requires accurate clocks" @bridgetkromhout #containercamp
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
Really enjoying @bridgetkromhout's presentation at #ContainerCamp -- it drips with hard-won production Docker wisdom
— Bryan Cantrill (@bcantrill) April 17, 2015
Really enjoying the "Pragmatic Docker" presentation from @bridgetkromhout #ContainerCamp
— Ekate K (@technekate) April 17, 2015
"Containerizing everything is great, except when it's not; but @frazelledazzell will show you otherwise" @bridgetkromhout at #containercamp😆
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
An excellent talk from @bridgetkromhout! Great to see some real-use cases of docker in production! #ContainerCamp
— YLD (@YLDio) April 17, 2015
🐳 @bridgetkromhout's talk about @docker in production at @dramafever is a gold mine of tips, tricks, and best practices. #ContainerCamp
— Jérôme Petazzoni (@jpetazzo) April 17, 2015
#golang + #docker = tiny containers!!
— FRANCESC @ FOSDEM (@francesc) April 17, 2015
That's ~10MB! Can I haz the Dockerfile?@bridgetkromhout at #ContainerCamp pic.twitter.com/TjWU3x6uDJ
Most real talk about @docker in production by @bridgetkromhout ❤️❤️❤️ so nice to hear things they found how they fixed and why they love it!
— Jessie Frazelle (@jessfraz) April 17, 2015
Great talk by @bridgetkromhout at #ContainerCamp, and Minneapolis shoutout, at least 2 of us here.
— Kevan Ahlquist (@KevanAhlquist) April 17, 2015
Terrific talk by @bridgetkromhout at #ContainerSummit. DevOpsDays MN is going to be a can't miss event in her hands http://t.co/yEBLXKrGUD
— Steven Tuck (@sdtuck) April 18, 2015
My #containercamp slides are at http://t.co/dzkUDdAv5P - thanks so much to @lalamaguire for asking me to join all these awesome speakers. :)
— Bridget Kromhout (@bridgetkromhout) April 18, 2015