The Phoenix Project changed how I lead Security Engineering

In the realm of Information Technology, few books have made as profound an impact as Gene Kim’s “The Phoenix Project.” (Book Link) This novel, draped in the garb of a corporate thriller, introduces us to an IT manager, Bill, who is thrust into the chaotic world of a project – code-named Phoenix – on the brink of disaster. As the book unfolds, Bill battles not only the looming deadline and a myriad of technical challenges but also navigates the intricate web of company politics and cross-departmental tensions.

In this post you will discover how I apply ‘The Phoenix Project’s’ lessons to manage my security engineering teams and achieve more with improved work management.

In the early stages of my management career, our CIO took a unique approach: he made “The Phoenix Project” required reading for everyone in our division. This wasn’t just a book recommendation; it was a strategic move designed to spark new ideas and drive change in our team.

Following our individual reading, we gathered offsite for a team meeting that took on the feel of a lively book club. We discussed “The Phoenix Project,” shared our takeaways, and broke into smaller groups to brainstorm strategies to apply the lessons from the book to our work.

This experience significantly shaped our team’s approach to work, but it also had a profound impact on me personally. As a new manager, this exercise provided valuable insight into effective team management. The leadership demonstrated by our CIO during this time was not only educational but also inspirational, offering a guiding light as I navigated the early days of my management career.

The real brilliance of “The Phoenix Project,” however, lies not in its engaging plot but in its ability to distill complex IT concepts into understandable and relatable terms. Through Bill’s journey, we’re introduced to the ‘Three Ways’ of DevOps, a methodology that can be applied beyond software development. As a security engineering leader, I’ve found invaluable insights in this book, lessons that I’ve applied to enhance my team’s efficiency and effectiveness.

At the heart of the book are four types of work: Business Projects, Internal Projects, Changes, and Unplanned Work. These are easily relatable to any security engineering team. Business Projects are proactive tasks related to strategy and growth, while Internal Projects focus on improving efficiency and effectiveness. Changes encompass work that mitigates risk or improves services, and Unplanned Work, the most disruptive, arises from unexpected issues or incidents.

Security engineering often includes all four. For example, building a new detection algorithm might be a Business Project, while optimizing code or enhancing documentation might be an Internal Project. Changes might include patching vulnerabilities or implementing a new access control system. And Unplanned Work could arise when a breach occurs or a system fails unexpectedly.

To manage these types of work effectively, I advocate adopting the ‘Three Ways’ philosophy from the book.

The First Way: Systems Thinking emphasizes the performance of the entire system, not just individual components. In a security context, we must ensure all elements of our security infrastructure work harmoniously together to provide robust defense. It means building systems that scale, remain resilient under pressure, and can recover quickly from issues. It encourages proactive planning for disaster recovery and incident response.

The book talks about the dangers of multitasking, the need for work in progress limits, and the power of focusing on throughput over individual task completion. These concepts, if internalized, can transform a software engineering team’s productivity and effectiveness.

For instance, as with any type of software development, the ‘Planned’ category of work is closely tied to feature development. These are tasks that are part of the product roadmap and have timelines attached. Understanding this, it becomes crucial to manage these tasks effectively, ensuring that work in progress is kept to a minimum to prevent context switching.

Meanwhile, ‘Unplanned’ work can crop up in the form of bugs, vulnerabilities, customer requests, or other technical issues that need to be addressed immediately. It is vital to have strategies in place to manage these unexpected tasks without disrupting the planned work.

Finally, the ‘Improvements’ category can be tied to refactoring or improving the codebase or infrastructure. Though it may not seem urgent, neglecting this kind of work could increase the technical debt and slow down future development.

In all these situations, the principles outlined in ‘The Phoenix Project’ can guide teams to manage their workload better, leading to more predictable deliverables and a smoother software development process.

The Second Way: Amplify Feedback Loops promotes the importance of swift and effective communication between teams. Security engineering is a shared responsibility, and it’s vital to have open channels of communication for identifying vulnerabilities, discussing potential threats, and mitigating risk. Regular reviews and retrospectives can be beneficial in learning from past incidents and continuously improving security measures.

Identifying and understanding the consumers of your team’s work is a fundamental step in effectively managing ‘Changes’. In the context of a security team, your customers could be internal teams such as the development, operations, or even executive teams. Equally, they could be external stakeholders such as auditors, clients, or regulatory bodies. Building empathy for these stakeholders is a key factor in creating security tools and procedures that are both effective and user-friendly.

Applying the ‘Changes’ concept from ‘The Phoenix Project’ to these relationships, it’s crucial to establish open lines of communication with these ‘customers’. This fosters an environment where your team can understand their needs, expectations, and pain points, thus ensuring the security measures you implement align with their requirements and work processes.

Moreover, building quick, non-disruptive feedback loops at the “point of performance” becomes essential, especially in the security domain. Security incidents are often time-sensitive, and capturing real-time feedback can significantly enhance your response effectiveness. However, it’s vital that these feedback mechanisms do not burden the operator or distract them from their primary task. Tools and procedures should be designed such that they facilitate swift, on-the-go feedback that can be analyzed and acted upon to improve processes continually. This approach aligns with the teachings of ‘The Phoenix Project’, emphasizing the importance of constant improvement and adjustment based on real-world feedback.

The Third Way: Culture of Continual Experimentation and Learning reinforces the importance of an innovative, risk-tolerant culture. In the ever-evolving landscape of cybersecurity, the ability to adapt and learn quickly is crucial. By encouraging a culture of experimentation and learning, you empower your team to stay at the forefront of new security strategies and technologies.

The insights from “The Phoenix Project” can be transformative for a security engineering team. By understanding and managing the different types of work and embodying the ‘Three Ways,’ we can create high-performing teams that are resilient, communicative, and continually evolving.

Remember, as in the book, the journey may not be smooth, but the lessons learned and the growth experienced along the way can make it worth the challenge. The Phoenix, after all, rises from the ashes stronger than before.