By challenging assumptions and embracing experimentation, individuals and teams can unlock fresh ideas. To this end, collaboration fueled by diverse perspectives further strengthens this innovation cycle. In this episode, Kelsey Hightower shares his experiences, from challenging the status quo in large organizations to embracing the collaborative spirit of open-source communities. Discover how Kelsey’s contributions to Puppet and his role in the development of Kubernetes shaped the landscape of modern infrastructure.
Edited transcription
Eschewing a traditional computer science degree heavy on theory, Kelsey prioritized practical skills. Certifications, contracting work, and even running his own computer store became his training ground. By the time he landed his first full-time job, Kelsey possessed a strong foundation and a desire to make a real impact.
Large organizations, however, proved a challenge. Accustomed to questioning processes and constant learning, Kelsey encountered a different reality. He found that “there’re so many people that walk around those kinds of environments that have 20 years of one year’s experience; They only know how to do the process, not change the process.”
But it was also around that time that Kelsey found the open-source community, which consolidated as the pivot of his tech career. Unlike his experience in his full-time job, the community provided him opportunities to interact with industry leaders, contribute to projects, and embrace the concept of “learning in public”: “It felt like a lot of the people who were designing these technologies, you could just jump online, IRC, mailing list, ask them questions, and they will take the time to respond,” Kelsey recalls.
As Kelsey identified areas in his job where automation could be applied, he actively sought and implemented open-source tools like Puppet. However, as he found, large enterprises are often cautious about adopting new technologies, especially ones that haven’t yet reached a certain level of maturity.
Undeterred, Kelsey turned to open-source contribution as a learning and experience engine. Nights and weekends were dedicated to honing his Python skills and actively participating in the community. “[I was] learning how to join, contribute at home,” he explains. “That’s when I opened the door to the community.” His expertise and contributions didn’t go unnoticed. Puppet Labs, impressed by his work, offered him a position, moving from a practitioner using open-source tools to a leader within the open-source space, working directly for an open-source company.
Moving to an open-source development company
While working at a large enterprise, Kelsey focused on maintaining existing systems. In turn, joining Puppet Labs meant “moving into the people who create things” and innovating. Despite the change, the transition felt natural. “I already knew most of the engineers,” Kelsey explains, “because I’d been contributing to the same codebase.” The same was true about decision-making in what was now a community-driven environment in which consensus was reached through open communication. “That’s when I opened my first Twitter account just to say, ‘Hey guys, we’re working on this. Please upvote the issue,’” Kelsey recalls.
Learning to collaborate and being open to acquiring new skills will get you further than labels, Kelsey affirms. “The industry gets hung up on labels,” he observes, and points out that “you can collaborate effectively regardless of your team’s name; Just calling yourselves ‘DevOps’ doesn’t guarantee results.”
Even the most brilliant idea needs buy-in. “You can’t just be right; You really have to do way more work to get everybody else on board to be successful,” says Kelsey, who learned that strong communication and alignment within the organization are what drive change.
On the other hand, working with a highly skilled team meant encountering plenty of good ideas. In this scenario, selecting the best solution inevitably led to situations where some good ideas wouldn’t be implemented, which could be demotivating for some team members: “That is a really hard challenge because smart people get told ‘no.’”
Public speaking and social media
In open-source projects, influencing change requires effective communication to build consensus across a global community. “You have no choice but to be a communicator,” says Kelsey, highlighting the stark contrast to limited communication in enterprises. Building alone isn’t enough. Open-source engineers need to be storytellers, explaining their work through documentation, presentations, and more. “Any engineer that’s good, when you build something, your next step is to tell the world about it… There is no world where you build it and they come and figure out what it is,” Kelsey says.
Given the tight relationship between being a communicator and developer in the open-source sphere, public speaking at meetups and conferences felt like a natural extension for Kelsey. Although initially hesitant, he embraced social media like Twitter to share his work, answer questions, and connect with a global audience. Soon, his online presence blossomed into a respected reputation within the tech community.
From Puppet to containers
Kelsey’s friction with Puppet began with his push to migrate their codebase from Ruby (limited in scalability) to Go. Internal resistance arose due to concerns about enterprise compatibility with legacy systems. “So we’re that company now… trapped now in our own success…?” he thought, frustrated.
The final straw came with Docker’s emergence. This containerization paradigm offered a fresh approach to system administration, but Puppet hesitated to adopt it. Kelsey, eager to explore this new world, transitioned to a smaller company.
While Docker boasted a user-friendly experience, complexities arose in large-scale deployments. As such, CoreOS, Kelsey’s new company, developed Fleet, their own orchestration tool built on etcd and systemd. However, it still faced limitations imposed by Docker’s control. To solve this issue, CoreOS developed Rocket as its own container runtime, offering greater freedom.
However, around that time, Kubernetes came to light. A container orchestration tool built to address the shortcomings of existing solutions, Kubernetes introduced the concept of pods, groups of tightly coupled containers that could be managed as a single unit. Pod’s flexibility in deployments was paired with Kubernetes’ superior networking capabilities.
Consequently, despite its early dominance, Docker couldn’t hold onto the orchestration crown. Docker prioritized the developer experience and building a strong image ecosystem, a strategy that, while successful, wasn’t as fit for complex deployments. As Kelsey puts it, “You haven’t mastered a tool until you understand when it should not be used.” Docker eventually introduced orchestration features like Docker Swarm, but it couldn’t match the momentum Kubernetes had already built.
Several factors fueled Kubernetes’ success. Among them, its well-defined API stands out as what allowed the community to expand Kubernetes’ functionality and integrate it with diverse tools and platforms. Just like Linux distributions like Red Hat and Ubuntu empower adoption, Kubernetes benefited from open-source distribution ecosystems and the collective knowledge of a vast developer community.
Building an Open-Source Colossus: The Village that Raised Kubernetes
Kubernetes, the ubiquitous container orchestration platform, isn’t the brainchild of a single genius. Instead, its success story is a testament to the power of collaboration in open-source communities. “I don’t know if anyone had the solo understanding,” Kelsey says, pointing out that a collaborative spirit is a cornerstone of successful open-source projects.
Interestingly, Kelsey sheds light on an unexpected way in which collaboration manifests: through competition. Companies like VMware, Heptio, and Microsoft Azure, despite being competitors, ultimately contributed to Kubernetes’ success. “Everyone was spreading out and saying, this is the way, this is the way,” says Kelsey. This lack of “arrogance to ignore” — even from competitors — embodies the open-source spirit. Diverse ideas, embraced from all corners, ultimately improve the project as a whole.
Drawing a parallel with Linux, another open-source giant project, Kelsey finds Kubernetes to be of a similar modular nature. Just as companies like Samsung can contribute specific modules to the Linux kernel, “CoreOS or Kubernetes gets the same treatment,” says Kelsey, highlighting contributions from various sources.
The bottom line
While Kelsey has recently retired from his full-time job, he remains actively involved in the tech community. Want to delve deeper into Kelsey Hightower’s work? Here are some starting points:
- You can find Kelsey’s talks on Youtube.
- Follow him on X at @kelseyhightower
Originally published at https://semaphoreci.com on April 9, 2024.
