[quip id=”wf7iAPJZCVb8″ ttl=0]
[quip id=”lfQsAsTTEsF1″ ttl=7200]
A big Thank You to the WordPress Community for putting on this year's WordCamp Conference at Cal State Los Angeles. On Sunday, Cloud Posse presented how we architected and implemented WordPress for massive scale. The slides are below.
Last week we had the pleasure of listening to David Huie present at the DevOps Mastermind at WeWork Promenade. David is an infrastructure engineer at Dollar Shave Club, where he’s helping DSC shave the world using Kubernetes. He presented how they've achieved the Holy Grail of QA automation: running “Unlimited Staging Environments with Kubernetes.”
In modern micro-services architectures, there is a serious need for ad-hoc staging environments since it's often infeasible for developers to run the entire stack on their laptops. At the same time, static staging environments can be difficult to scale as an organization's infrastructure and engineering team grow.
To counter this effect, Dollar Shave Club created a Kubernetes-based system to enable an unlimited number of environments, bounded only by the capacity of the underlying Kubernetes cluster running some 38 nodes! At its core, is an Open Source project called Furan which rapidly builds Docker containers in Docker (DnD). Using their CI/CD system and an in-house tool called Amino, they are then able to automatically spawn environments composed of many independent projects, where each project is pegged to a specific version (e.g. branch or tag).
The company is able to iterate much faster which has sped up application delivery at DSC.
About the Speaker
Prior to joining Dollar Shave Club, David’s worked at Splice, NationBuilder, and Yelp. David has a degree in Computer Science from Harvey Mudd College.
Follow David on Twitter: https://twitter.com/davidhuie
Slides from the presentation are below. We'll be posting video & transcripts shortly.
Join us at the next Santa Monica DevOps Mastermind Meetup!
Register here: https://www.meetup.com/DevOpsMastermind/
Learn how Cloud Posse recently architected and implemented WordPress for massive scale on Amazon EC2. We'll show you exactly the tools that we used and our recipe to both secure and power WordPress setups on AWS using Elastic Beanstalk, EFS, CodePipeline, Memcached, Aurora and Varnish.
Learn about some of the “best practices” for running a modern engineering organization that we practice at Cloud Posse. “Best Practices” are opinionated & proven strategies/tactics used to achieve some desired outcome.
These slides are a small part of what we think it takes to embrace a DevOps culture. It starts with the “Organization” embracing change and laying a foundation that will support the cross-disciplinary practice of DevOps.
Secrets are any sensitive piece of information (like a password, API token, TLS private key) that must be kept safe. This presentation is a practical guide covering what we've done at Cloud Posse to lock down secrets in production. It includes our answer to avoid the same pitfalls that Shape Shift encountered when they were hacked. The techniques presented are compatible with automated cloud environments and even legacy systems.
The other prime example is “Cloud Computing.” It was not a new concept, it’s just that no one had really cracked the nut to show us how to do it properly. That was until Amazon came along. With EC2 they made it accessible and showed us the possibilities; they let us write infrastructure as code. Our minds were blown with the possibilities! So everyone tried to copy what Amazon did, but unfortunately it was a little too late.
That's because now we have the container movement. The concept of “Containers” is also nothing new. In Linux, the core functionality has existed ever since 2008 when Google contributed their work on LXC – the technology behind containers – to the Linux Kernel. However, it wasn’t until Docker came along circa 2013 (5 years later!) and made it braindead easy for developers to run them that we started seeing an uptick in their adoption. Now Docker is taking a page out of VMware's playbook by following up with Enterprise tools for production with release of Universal Container Platform (“UCP”) & and the Docker Datacenter (“DDC”).
- Make it easy.
- Target developers.
- Let percolate through out the enterprise until resistance is futile.
- A dozen or more hypervisor technologies like VMware, Zen, KVM will lose massive market share.
- Elaborate Configuration Management tools like Puppet and Chef that were created to address the broken ways we used to configure software (basically emulated what humans would do by hand) will no longer be needed because we don’t write software as broken anymore.
- EC2 private-cloud knock offs like OpenStack, vCloud, Eucalyptus, CloudStack, etc that were designed to run your own private cloud on-prem like Amazon, are now overkill or at the very least passé (R.I.P.)
Hint: It’s not strictly technological.
First of all, we can agree that anytime we iterate, the second iteration is easier, better and faster than the first. Simply put, everything is less scary the second time around. Moving from the classic “bare metal” paradigm to a “virtualized” one was a massive endeavor. It was the “first” major paradigm shift of it’s kind. It took convincing of both C-Level execs and wrangling of Operations teams. Since it was a foreign concept, there was severe skepticism and push back at all stages. Flash forward 15 years later and there’s now fresh blood at the top. There’s a new guard who has moved up through the ranks that’s more accepting of new technology. Enterprises have gotten better at accepting change. More over, the tools of the trade have improved. We’re better at writing software — software that is more cloud friendly (aka “cloud native”).
- Containers will become first-class citizens, replacing VMs as the defacto unit of cloud.
- If you still need a VM, that’s cool; you’ll have a couple options:
- Leverage a VM running inside a container. There's a project by Rancher called “VM Containers” which does exactly this. Sound absurd? Not to Google. They run their entire Public Cloud – VMs & all – on top of Borg.
- Use Clear Containers by Intel which have minimal overhead, full machine-level isolation and can leverage the VT technology of modern CPU chipsets. Not to mention, it's fully Open Source!
- The brave will attempt using some sort of Unikernel, but it’s still too early to know for sure if that will be the way to go.
- Interest behind OpenStack (et al) will wane and innovation cease – they were ahead of their time. We learned A LOT from the experience – both what worked well and didn't. As a result, we'll see a significant brain-drain, with key contributors moving over to the Kubernetes camp.
- Kubernetes will replace OpenStack du jour and as a result we'll see a resurgence of bare-metal in the Enterprise
- Amazon’s ECS will be EOL’d and replaced with offerings of Kubernetes & Swarm.
- Kubernetes and Swarm will be battling it out for #1 because competition is good.
- The best features of Mesos will be cherry picked by both Kubernetes & Swarm, but Mesos fail to gain bigger foothold in the market.
Kelsey Hightower, a Google Developer Advocate and Google Cloud Platform evangelist, recently gave a very helpful screencast demonstrating some of the tips & tricks he uses when developing Go microservices for Kubernetes & docker. Among other things, he recommends being very verbose during the initialization of your app by outputting environment variables and port bindings. He also raises the important distinction between readiness probes and liveness probes and when to use them. Lastly, in the Q&A he explains why it's advantageous to use resource files instead of editing live configurations because the former fits better in to a pull-request workflow that many companies already use as part of the CI/CD pipeline.