IT Infrastructure Deployments: Learning To “Work the Problem”

Valuable Lessons in IT Infrastructure and Technology Deployment Problem Solving Learned From the Apollo 13 Mission

On April 13th 1970, the Apollo 13 spacecraft suffered a devastating explosion.  The event triggered one of the most extraordinary problem-solving efforts in the history of mankind; as the NASA team fought to bring home the stricken craft and its crew from over two hundred thousand miles away.

In the movie “Apollo 13”, just after Mission Commander Jim Lovell says “Houston, we’ve had a problem,” the Flight Director, Gene Kranz, turns to his Operations team and says “Let’s work the problem people.  Let’s not make things worse by guessing.”  Powerful yet simple, Kranz’s statement defines both straightforward leadership, and a practical approach to problem resolution that dates back to NASA’s earliest days.  It also refers to a important quality that is too often ignored in problem-solving: hard earned, rock-solid competence.

Rock-solid competence is also a key success factor for a multi-site technology rollout. Let’s discuss how a rollout company can build a team where competence is – as it is at NASA – the norm, and not the exception.

The Origin of “Work the Problem”

After the process failures that culminated in the launch pad fire that killed the Apollo 1 crew, Kranz gave a powerful speech, since referred to as the “Kranz Dictum”.  The dictum defines a way forward by focusing on two qualities: “Tough” and “Competent”.  In Kranz’s words: “Tough means we are forever accountable for what we do, or what we fail to do. We will never again compromise our responsibilities.  Competent means we will never take anything for granted.  We will never be found short in our knowledge and in our skills.”

Although a technology rollout may not carry life or death consequences for poor performance, the underlying challenge is the same.  When attempting to solve a difficult problem, it can be tempting to just follow intuition.  As Kranz notes though, this approach often makes things worse rather than better.  True competence demands that we do things thoughtfully and mindfully, rather than by hope, intuition, or guesswork.

Gene Kranz is a genius of operational procedure, e.g. he devised the “go/no-go” launch status check system that is still used today.  during the Apollo 13 crisis, Kranz did what the best leaders do: he led his team to successfully overcome seemingly insurmountable challenges, using mindful, deliberative information processing and effective problem-solving processes.  The opportunity is there for technology rollout companies to do the same, but it takes commitment and effort.

Becoming “Tough” and “Competent”

The key word that Kranz uses to describe “Tough” in his speech is “Accountable”.  To create a “Tough” organization, there must be a culture of accountability that starts at the top.  Leadership should then continually reinforce this culture, by rewarding employees that demonstrate that accountability.  Along with accountability, team members must also be empowered to directly question the approach to be taken. This ensures that the risk of dangerous Groupthink is avoided, and the proposed solution is the best that can be achieved.

In technology rollout companies, building organizational Competence requires a long-term commitment to continuous improvement.   Leadership again plays a big part, by providing a structured approach to the building of skills and knowledge that regularly raises the bar for competence across the organization as a whole.

Technology rollout teams that embody these qualities are are well-equipped to craft and deliver innovative solutions to deployment problems, that would otherwise be considered insolvable.

A Checklist to Help You “Work the Problem”

To achieve a successful multi-site technology rollout, the qualities of Tough and Competent must be thoroughly applied to deployment problems. Taking it step by step, leaders must:

  • 1. Accurately define the problem to be overcome
  • 2. Determine the goals/objectives of the ideal solution
  • 3. Create an array of alternative, viable solution options
  • 4. Analyze the potential consequences for each solution option
  • 5. Use this analysis to select the best course of action
  • 6. Properly plan the implementation
  • 7. Implement the plan with full commitment
  • 8. Use incoming data to continually adapt the plan, as necessary

The Bottom-Line: It Takes Commitment and Determination to Fix Hard Problems

Over 40 years later, Apollo 13 is still one of the best examples of how rigorous competence and a structured approach to problem-solving can deliver incredible results – even in the most difficult situation.  As Apollo Flight Controller Jerry Bostick said, when describing how the mission control team coped with the stream of unexpected challenges that arose during the Apollo 13 mission:

When bad things happened, we just calmly laid out all the options, and failure was not one of them. We never panicked, and we never gave up on finding a solution.”

This was paraphrased in the movie as “Failure is not an option.”  This statement perfectly encapsulates the problem-solving approach that NASA used to successfully bring the crew home.  This approach – proven in the most difficult of circumstances – offers valuable lessons for multi-site technology rollouts.

When your deployment hits a bump in the road, make sure that your rollout company has the skills and processes that it takes to build a lasting solution.  It’s worth asking before you start how they approach problem resolution.  If you don’t like their answer, talk to us.  We’ll be happy to walk you through our methodologies, processes and training programs, and discuss practical examples of how we’ve successfully applied them.