GitOps with Terraform on Codefresh (Webinar)

Erik OstermanSlidesLeave a Comment

🚀 Infrastructure as code, pipelines as code, and now we even have code as code! =P In this talk, we show you how we build and deploy applications with Terraform using GitOps with Codefresh. Cloud Posse is a power user of Terraform and have written over 140 Terraform modules. We’ll share how we handle automation with security while making the process easy for engineers.

Erik Osterman is the founder of Cloud Posse, a DevOps professional services company that specializes in cloud migrations and release engineering. Previously he was the Director of Cloud Architecture, for CBS Interactive where he led cloud strategy across the organization.

 


Geodesic Cloud Automation Shell (Slides)

Erik OstermanSlidesLeave a Comment

🚀 Geodesic is a cloud automation shell. It’s the superset of all other tools including (terraform, terragrunt, chamber, aws-vault, aws-okta, kops, gomplate, helm, helmfile, aws cli, variant, etc) that we use to automate workflows. You can think of it like a swiss army knife for creating and building consistent platforms to be shared across team environments. It easily versions staging/production/dev environments in a repeatable manner that can be followed by any team member with only a single dependency: docker. Because of this, it works with Mac OSX, Linux, and Windows 10. Learn how you can use the geodesic shell to improve your DevOps workflows! These are the slides from the live demo at the Los Angeles Kubernetes Meetup.

Erik Osterman is the founder of Cloud Posse, a DevOps professional services company that specializes in cloud migrations and release engineering. Previously he was the Director of Cloud Architecture, for CBS Interactive where he led cloud strategy across the organization.

 


Scale 17x: Intro to GitOps

Erik OstermanSlidesLeave a Comment

We presented this year at Scale 17x the value of GitOps for an organization. The goal for operations should be absolute automation. With “Infrastructure as Code” we're able to employ all the best practices of traditional software development to infrastructure and operations- including code reviews and continuous delivery of changes. GitOps is the concept of using Git as the system of record for the desired state of configurations. The ultimate goal is to achieve repeatable processes to apply changes, in the same way, every time. Predictable rollouts let us know ahead of time what’s going to happen (e.g. before you merge). Auditable histories let us see what was done and by whom. Lastly, the process should be accessible to anyone, not just a few technically elite members of the team.

Erik Osterman is the founder of Cloud Posse, a DevOps professional services company that specializes in cloud migrations and release engineering. Previously he was the Director of Cloud Architecture, for CBS Interactive where he led cloud strategy across the organization.

 


November 2018 Edition of the SweetOps Newsletter

adminNewslettersLeave a Comment

 

 

Welcome to the November 2018 edition of the SweetOps newsletter.

We've got a lot to share with you this month as we've been exceptionally busy! Overall, our emphasis this month has been on growing the community around our open source adoption and investing heavily in Atlantis for terraform GitOps-style automation.

 

 

New Terraform Modules

  1. terraform-aws-iam-account-settings – Terraform module to provision general IAM account settings. It will create the IAM account alias for pretty login URLs and set the account password policy.
  2. terraform-aws-iam-user – Terraform Module to provision a basic IAM user suitable for humans. Supports automatic password encryption using Keybasepublic keys.
  3. terraform-null-smtp-mail – Terraform module to send transactional emails via an SMTP relay directly from within terraform. This is perfect for sending teams email notifications when infrastructure changes.

 

You can find all of the cloudposse terraform modules in the official terraform module registry.

Important Terraform Module Updates

  1. terraform-aws-rds-cluster – we've added support for Aurora Serverless as well as fixed some idempotency issues raised by the community.
  2. terraform-aws-cloudfront-s3-cdn – we've fixed support for regional S3 endpoints with the CloudFront CDN
  3. terraform-aws-elastic-beanstalk-environment – we've received a handful of PRs to extend the module capabilities with better support for autoscaling, nodejs, ELB security groups, and cloudwatch logs.
  4. terraform-aws-dynamodb – we've added support for local secondary indexes

Many thanks to @johncblandii, @br0nhy, @bober2000, @pabardina, @rverma-nikiai, @aknysh, @jamisonhyatt and the many others who took the time to open issues, submit pull requests and review code.

 

GitOps with Atlantis and Terraform

 

We recently held a meetup during #connectweek in Pasadena (CA) where we gave a live demo using Atlantis with Terraform to provision AWS user accounts using only Pull Requests. Atlantis is the secret for enabling teams to collaborate on Terraform. We have our own fork of atlantis that we're maintaining until the essential security features we introduced get upstreamed.

 

Slides from the presentation are available on our blog.

 

Interesting Links & Projects

Here are some interesting links that circulated this month in our slack team.

  1. Very cool Open Source SIEM security and compliance dashboard for surfacing events.
  2. Interesting workaround to achieve Tillerless Helm today by running tiller on localhost.
  3. HashiCorp, the company behind Terraform, has raised an additional $100M
  4. 10 Habits DevOps practitioners must break
  5. Monitoring dashboard for Kubernetes Jobs that makes it easy to see which are running and if their latest status was “succeeded” or “failed”.
  6. Two Objects not Namespaced by the Linux Kernel and therefore shared by linux containers.
  7. Malicious Life Podcast – The True stories behind the world of cybercrime

For more links, subscribe to our @cloudposse twitter feed.

Follow us on Twitter @cloudposse

New Alpine Package Repository

 

As part of Geodesic, our framework for provisioning and automating infrastructure, we needed a better way to distribute our toolchain (aws-vault,   chamber, kops, gomplate, etc) that was fast and reliable that made it easy to version pin tools for individual customers. Previously, we shipped all tools with the geodesic docker base image pinned to a specific release, but this was bad because it forced a customer to upgrade their entire toolchain when they just wanted one upgraded package.

 

The fix was to distribute all of our tools as versioned alpine packages. Our alpine repository is available at apk.cloudposse.com. More instructions for configuring this repository can be found in the github repository where we define all packages.

 

https://github.com/cloudposse/packages

 

What does this mean if you use the make based package installer? Not much. We'll continue to support Makefile based package installations because we need a way to install these packages natively on OSX/Linux/Windows/etc that is not alpine specific. In the future, we might distribute the packages for Debian and RPM-based distros as well.

 

Terragrunt with Geodesic

 

Terragrunt is a popular tool for automating terraform deployments. It makes it easy to reuse modules, initialize terraform state, and orchestrate complex multi-stage terraform build-outs.

 

For a demo we recently did on using Atlantis with Terraform, we put together an example of using Terragrunt on Geodesic. Long and short of it is that it's easy, and there's not much to it. For an example, refer to our reference architectures. Here's an example of adding a user using terraform and terragrunt in combination with our terraform-root-modules.

 

Slack Community

Over the past month, we've seen a sharp uptick in the number of daily conversations especially in the #terraform, #release-engineering, and #kubernetes channels. We're grateful for everyone helping out to answer questions and sharing interesting articles. If you haven't already, sign up for the SweetOps slack team today.

Join Slack Community

Reviews & Testimonials

We've set up a place for our community members to leave us testimonials. If our modules or slack community have helped you, we would love to know. A few words from you would totally make our day and help us reach others.

Please leave us a testimonial, if you've found some of our open source repos helpful.

 

All the best,

 

Erik Osterman

CEO / Founder, Cloud Posse, LLC

 

 

How to use Terraform with Teams using Atlantis (#GitOps)

adminMeetupLeave a Comment

GitOps is where everything, including infrastructure, is maintained in Git and controlled via a combination of Pull Requests and CI/CD pipelines. Reduce the learning curve for new devs by providing a familiar, repeatable process. Use Code Reviews to catch bugs and increase operational competency. Provide transparency to the rest of the team with Pull Requests. This presentation had a live demo of using Atlantis with Terraform that showed how to easily add and remove users from AWS IAM safely & securely using simply GitHub Pull Requests.

Erik Osterman is the founder of Cloud Posse, a DevOps professional services company that specializes in cloud migrations and release engineering. Previously he was the Director of Cloud Architecture, for CBS Interactive where he led cloud strategy across the organization.

 


Effortless Blue/Green Deployments on Kubernetes with Helm

adminMeetupLeave a Comment

Last night was our first ever Pasadena “DevOps Mastermind” meetup.

First speaker up was Dan Garfield. He talked about how to achieve Blue/Green deployments. Blue/Green has been around for a long time but what are the “best practices” when using Kubernetes? How does it change when using Helm? Last night we learned from Dan the differences as he demonstrated how to pull it off effectively with repeatability using Codefresh. When using Helm, the picture changes slightly, keeping a history so rollbacks work properly is critical and requires structuring your Helm Chart accordingly. Check out the slides!

Dan Garfield is a Google Developer Expert, Chief Evangelist of Codefresh, and Kubernetes, Helm, Istio, and Docker meetup organizer. His talks have been featured at Kubecon, Swampup, DeveloperWeek, and many other places. He focuses on DevOps, and Deployment Strategies in a micro-service world.


Effortless Helm Chart Deployments

adminCI/CD, DevOps, MeetupLeave a Comment

Learn how to deploy complex service-oriented architectures easily using Helmfiles. Forget umbrella charts and manual helm deployments. Helmfile is the missing piece of the puzzle. Helmfiles are the declarative way to deploy Helm charts in a 12-factor compatible way. They're great for deploying all your kubernetes services and even for Codefresh continuous delivery to Kubernetes. We'll show you exactly how we do it with a live demo, including public repos for all our helmfiles.

SweetOps Town Hall Meeting

adminMeetupLeave a Comment

Today we had our first “Town Hall” meeting where members of our SweetOps community (slack.cloudposse.com) got together on a Zoom conference call to talk shop.

Remember to vote when we should have our next call.

Discussion Points

  • GitOps – CI/CD Automation of Terraform
  • Git ChatOps
  • OAuth2 Proxy
  • Govcloud

Pain Points

There were a few shared pain points by members.

  • Keeping modules up to date is tedious, especially in busy environments where modules are released frequently. We wish there was a way to identify out of date modules and easily update those module references. Erik mentioned that he submitted a “Feature Request” for Dependabot to support HCL. Please ? if you think this would be awesome!
  • Loren mentioned that using Open Source in Govcloud is difficult because Govcloud lags way behind the commercial AWS cloud in features. For example, EC2 instance tags cannot be set on launch. Other times, API endpoint URLs just don't work because they've been hardcoded to the ones used by the commercial cloud offering.
  • Erik mentioned that OAuth2 Proxy appears to be an abandoned project. With over 90+ open pull requests, broken OpenID Connect support and no new releases in over a year, maybe it's time for a new maintainer. Apparently, others have suggested this too.

Technologies

A number of technologies came up in our discussions

Services

Other Links

Attendees

Resolutions

  • Hold these “Town Hall” meetings more often (perhaps weekly)  and at different times so that we can include members in all geographies and time zones.
  • Ensure we record and publish the next call

Geodesic Logo!

adminUncategorizedLeave a Comment

We've finally settled on a logo for Geodesic, our solution for implementing rock-solid cloud architectures.

In mathematics, a geodesic line is the shortest distance between two points on a sphere. It's also a solid structure composed of geometric shapes such as hexagons.

 

We like to think of geodesic as the shortest path to a rock-solid cloud infrastructure. The geodesic logo is a hexagon with a cube suspended at its center. The cube represents this geodesic container, which is central to everything and at the same time is what ties everything together.

 

But look a little closer and you’ll notice there’s much more to it. It's also an isometric shape of a cube with a missing piece. This represents its pluggable design, which lets anyone extend it to suit their vision.

Top 5 DevOps Bad Habbits

adminDevOps, OpEdLeave a Comment

Recently, we were asked to name our top (5) DevOps “Worst Practices” (or anti-patterns). Here's what we came up with…

#1. Not looking outside the organization to see how others are solving the problem. Always building new things rather than looking for readymade solutions (open source, SaaS, or enterprise offerings) which leads to piles of technical debt & inevitable snowflake infrastructures.

#2. Not building easy tools that the rest of the company can use. We must never forget who we are serving. Developers are our customers too.

#3. Not treating DevOps as a shared responsibility. It needs to be embedded into the engineering organization, not relegated to a select few individuals. “DevOps” is more of a philosophy than a job title.

#4. Not treating Infrastructure as Code. We call this new paradigm GitOps, where Git system of record for all infrastructure and CI/CD is our delivery mechanism.

#5. Never ever commit to `master`. We don't do it in regular software projects, we shouldn't do it for ops. Everyone should be following the standard Git Workflow on their Infrastructure Code (Feature Branching, Pull Requests, Code Reviews, CI/CD). This increases transparency and helps the rest of the team stay up to date with everything going on.