Why cloud governance needs to be an open source affair

Stacklet arguably shouldn’t exist. The company just launched Stacklet Platform around the open source project Cloud Custodian, but one of the cloud providers probably should have built something similar first. Stacklet makes it straightforward to embrace a policy/governance as code model to provide real-time policy enforcement across all clouds via detection, notification and remediation, using a simple, declarative language.

SEE: Cheat sheet: The most important cloud advances of the decade (free PDF) (TechRepublic)

Every cloud has this need–a way to do policy as code at scale–yet it’s Stacklet developers (along with a growing community) that built Cloud Custodian. Perhaps the reason why, said Stacklet co-founder and Cloud Custodian creator Kapil Thangavelu, is that individual vendors are focused on a comparatively narrow view of the world. Open source, by contrast, “has allowed us to source many different ideas and use cases from lots of different organizations and in many different contexts.” 

Thinking broadly

Here’s that Thangavelu quote in context: 

A lot of the providers take a very narrow view of a given problem domain because that’s the focus for a given team. Open source has allowed us to source many different ideas and use cases from lots of different organizations and in many different contexts….The challenge of just looking at a single piece of the problem is that you’re fragmenting the end user with 20 different tools to do 20 different things. And that ends up being problematic when you actually drive to a holistic transformation of being well-managed.

Keep in mind that Cloud Custodian emerged from work Thangavelu was doing at Capital One, which is a big company with over 50,000 employees and tens of billions in revenue. It was a laboratory primed to help Thangavelu “service the different needs from different groups within the enterprise: audit, risk, security, application teams, lines of business,” he said. That helped make Cloud Custodian incredibly useful within his enterprise. But just one enterprise.

Open source increased the scope and utility of Cloud Custodian beyond one company’s needs.  

“As we’ve gotten to open source, that pool of use cases simply expanded,” he noted. No matter how creative your product managers, they’re always necessarily constrained by the needs of the business they’re running. By contrast, Thangavelu continued, “Open source is the strongest way to achieve [expanded scope] because your usage and your users address a wider swath of needs than any given company has. They represent the needs of a large diverse set of interests. And they’re all pulling in different directions.”

This push-and-pull from a growing Cloud Custodian community has made it a useful tool for organizations that may have thousands or even tens of thousands of diverse policies to manage. These different organizations can thus iterate on their own policies while collaborating on the core, underlying code to make it better for everyone.

Not that this open source approach is easy.

A question of balance

For the Stacklet team, one of the hardest challenges, Thangavelu noted, is balancing project needs against product needs. “The difficulty of maintaining a community while building and going at the pace I want to go in open source, while building a product at the same time, has been an interesting conundrum,” he stressed. Why? It turns out that company and community sometimes need to build at different paces.

Take, for example, backwards compatibility and operational simplicity, two hallmarks of Cloud Custodian development. These are sometimes at odds with community desire to build features in a number of new directions. Now add the complexity (and power) of enabling these Cloud Custodian users to become contributors of those features they want, while keeping the project cohesive. “Because if you have a growing community that feels empowered to contribute [it’s very different from] one where you’re simply managing a product backlog in a more traditional closed source fashion,” he said, which is what some open source companies do. 

Sound complicated? It is, and it’s made more so by also trying to juggle a revenue model that allows Stacklet to make enough money to sustain ongoing, core Cloud Custodian development without becoming a proprietary software company dependent on itself for all innovation. Fortunately, he concluded, there’s a growing sense in the industry that it’s important “to understand the open source software that enterprises adopt, and to also have some responsibility for what they’re building on top of.” As I’ve written before, Stacklet has done a fantastic job of encouraging this exact kind of customer-driven innovation. Long may it continue. 

Disclosure: I work for AWS, but the views expressed herein are mine.

Cloud and Everything as a Service Newsletter

This is your go-to resource for XaaS, AWS, Microsoft Azure, Google Cloud Platform, cloud engineering jobs, and cloud security news and tips.
Delivered Mondays

Sign up today

Also see