OSCON - Managing Distributed Systems Using BOSH
Austin, Texas
2016-05-16 to 2016-05-19


Description

Managing your platform is essential for operability. BOSH is an open source tool for predictably managing the long term lifecycle of distributed systems. We’ll learn to create a BOSH release: a versioned collection of configuration properties, configuration templates, start-up scripts, source code, binary artifacts, and anything else required to build and deploy software in a reproducible way.



Abstract



Everyone who operates software has a platform with a story around managing applications. But a platform is also software, so we need to manage the lifecycle of the platform itself. BOSH is an open source tool for predictably managing the real-world lifecycle of services in distributed systems. While with traditional configuration management tools we can maintain consistent state for individual machines, BOSH enables us to converge services as distributed systems, to keep consistent the state of a collection of compute, storage, and networking resources.

While BOSH is known as the infrastructure automation layer for the open source Cloud Foundry platform, it can also be used to produce versioned, deployable artifacts for arbitrary infrastructure components. A BOSH release is a versioned collection of configuration properties, configuration templates, start-up scripts, source code, binary artifacts, and anything else required to build and deploy software in a reproducible way.

We’ll take a high-level look at the functions BOSH performs - packaging, distribution, orchestration, and management - and then dive into the details. We’ll peel back the surface layers and look at the BOSH director, and then a stemcell (Ubuntu image with an agent), a release (source packaging), and a manifest (configuration details).

Managing the underlying distributed system and persistence layers continues to be more challenging than deploying stateless apps. Databases, cluster schedulers, and other persistence layers are among the components deployable by BOSH, and we’ll walk through multiple deployments, ensuring that participants have a chance to understand how BOSH addresses four aspects of release engineering:

* Completeness - capturing all code, binary artifacts, and configuration as a release that is packaged and can be 100% transferred “behind-the-firewall”

* Reproducibility - versioning the application, OS, persistence layer, and configuration together

* Consistency - used for development and production operations

* Agility - integrates with continuous delivery, allowing for upgrades to the platform and persistence layers

BOSH is in essence a narrow AI, optimized for automation of repetitive tasks in management of distributed systems. Learning how to operate it will give attendees insight into optimizations they can make in their own platforms and pipelines, while giving them insights into operability for all those long term needs.



Git repo for tutorial (with slides and links)

Snagged a front row seat to @bridgetkromhout's tutorial on distributed systems with BOSH #OSCON pic.twitter.com/sxyqL7Xosv

— Kenny Bastani (@kennybastani) May 17, 2016