If you have worked in large scale multi-site technology rollout account management for a while, it’s a certainty that you have either worked in or heard about, rollouts that were “doomed from the start”. With a project as complex as a multi-site technology rollout, to the unprepared – the problems can come thick and fast.
We use a practical example of just such a project, as the basis to discuss what you must do, when you find yourself cleaning up someone else’s messy rollout.
The Situation
A major enterprise decided to refresh its wide area network infrastructure, which spans over 40,000 sites. The contract was placed with a top-tier communications provider, who engaged a $1B+ deployment company to deploy and commission the infrastructure on-site. The initial engagement was for the first phase of the project, a total of 18,000 locations. The scope of work included:
- Extending the network to hundreds of existing customer locations across the country
- Installing and configuring thousands of new routers, as well as existing equipment
- Installing and connecting switches
- Testing the entire network.
As well as maintaining a consistently high-quality of deployment, the pace of the rollout was also defined as a key metric. In order to achieve the customer’s deadline for the first phase of the project, a steady pace of 30-50 sites per day must be delivered.
Very quickly, the project fell significantly behind schedule. Deployments were taking longer than expected, and were also suffering severe quality issues. This in turn triggered revisits, which further jammed the already packed schedule. As a result, even in their best week, the deployment partner could only deliver a handful of sites per day.
The prime contractor made multiple attempts to work with the deployment partner to get the project back on track – without luck. Eventually, under heavy pressure from the customer, they were left no alternative but to fire the partner and seek a new one.
After exhaustively researching options to save the failing project, the prime contractor engaged Concert Technologies to deliver the remainder of the first phase, with an option to continue with the project, if successful.
Step I: Understanding What Went Wrong
Before prescribing a solution to the rollout issues, it was first necessary to diagnose the problems. As soon as Concert engaged with the rollout, we began work to understand why and how things had gone so badly wrong. Interviews were held with key people in the customer and prime contractor teams. Very quickly, a set of root causes for the rollout failure were identified:
- Lack of effective, skilled project management
- Lack of an accurate, repeatable Work Order Packet (derived from SOW)
- Lack of communication between on-site POC, field techs, and other stakeholders
- Lack of an established QA process
Armed with this understanding, Concert had the information they needed to ensure that decisions taken in building the rollout plan were supported by the data rather than conjecture…
Step II: Decide How to Proceed
With a full understanding of the challenges that caused the previous plan to fail, Concert built a new rollout plan. Because of the large scale of the rollout and the issues that were encountered by the previous rollout company, it was essential that the plan focus on ensuring consistent, high-quality delivery – all the while maintaining maximum rollout speed.
Over and above addressing the shortcomings of the first rollout, there were two key customer requirements:
- Concert we must make contact with every site prior to arrival,
- Concert must provide regular status reports to key customer contacts.
Step III: Execute, Measure, Adjust
Now equipped with a solid plan, from a standing start Concert was able to deliver an initial phase of 750 customer sites in less than 30 days. While this achievement was much closer to the required run-rate than the previous provider, it was still below the target of 30-50 sites per day.
Because of this, upon completion of this phase, all aspects of the project were reviewed and areas of potential improvement were identified. Changes were made to the work order packet and supporting processes, to increase rollout pace in the next and subsequent phases.
As the project entered the next phase, Concert continued to focus on increasing rollout pace without impacting quality. Throughout the life of the project, Concert refined the processes and methodologies used to manage deployments. This enabled the Concert team to continuously increase rollout pace, without compromising rollout quality.
The Outcome: Rollout Success and a Happy Customer
Concert delivered the rollout at a pace that exceeded the customers’ requirement, without compromising quality – ultimately achieving a sustained rollout pace of 50 sites a day.
The success of the rollout was such that Concert remains a trusted provider over two decades later. This success is best described using a quote from the customer’s primary contact for the project:
“We have experienced enormous WAN growth, and our success relies heavily on Concert Technologies to provide new and upgraded WAN services in a timely manner. There are many variables to manage when performing these responsibilities. I want to take a moment and state that the team and I are very grateful for the excellent service that Concert provides. The team is exceptional and their efforts positively contribute to our ability to manage the network projects. I want to express appreciation for the hard work these folks put forth. It really makes a difference!”
Having delivered so successfully on a project on the brink of falling apart, Concert forged a strong partnership with the prime contractor that is still in place many years later. Additionally, the Prime contractor’s owner for the project is a customer of Concert services today – you can watch his video testimonial by clicking here.
The Bottom-Line: A Good Solution Requires An Understanding of the Problem
When your next multi-site technology rollout needs to deliver a volume of consistently good deployments, make sure that you have the necessary skills and processes. It’s worth talking to your technology rollout company before the rollout starts to see how they approach problem resolution.
