April 13, 2019
If you’ve been in the industry for a while, one of the best things in your work life is change management. But one of the worst is probably also change management. These two simple words elicit a range of emotional responses throughout the organization’s hierarchy, from the lowest engineer to the highest C-level. The engineers want to do their work. The C-level needs the job done without any impact to the business. A Change Advisory Board (CAB) or other such acronym exists to decide when significant changes can or cannot happen. Many times a change needs to be implemented but some part of the org is performing another maintenance, so infrastructure changes routinely get pushed back. However, the issue with this practice is that you end up running on continually outdated infrastructure and trying to do more on top of it.
They expected another three weeks to isolate, fix, and reattempt the maintenance. We executed it within 24 hours.
One of the things I’ve found personally beneficial with NetBrain is the ability to mock up changes in the infrastructure and see the effects before the change window and without a lab! Let’s be honest: not everyone has a lab environment and the budget to invest in one. NetBrain has allowed me as both a customer and a consultant to represent the changes and the impact therein visually.
Check out this 90-second video of NetBrain’s map-based visual change management capability.
How we reduced a 3-week fix to 24 hours
For example, a large customer was making a change from a traditional point-to-point hub-and-spoke segment to a full mesh. No big deal, right? Well, the customer made adjustments and was getting things going, and it went sideways faster than Clark Griswold in National Lampoon’s Vacation! I got the call on a Thursday night, and they expressed concern and wanted my help figuring out what was wrong, why, and the path forward. They rescheduled the maintenance for three weeks out.
Leveraging NetBrain, we built their environment out via an infrastructure discovery in less than 30 minutes. Once completed, you can go in and change variables (configurations) and see how the routing is affected. I showed them that the change they made was oversaturating a spoke due to a bad advertisement in OSPF. I showed them the new changes to make in the model and simulated the routing paths from A to B that they needed, and then we validated all 50 remote sites in the model for intranet, mesh, and Internet access. The entire process took less than 4 hours, with the majority of it being the extensive testing of each site’s future-state routing table.
Collect data before the change is made, execute the change, and gather data after to analyze the impact.
Now, this customer — like most who need changes — wanted the change to enhance their infrastructure. They were rolling out a new VoIP solution, and a mesh DMVPN would allow them to communicate directly to the sites dynamically without hairpinning through the hub, which was becoming costly to sustain such a large circuit. They wanted to utilize the Internet in some locations as well. (Think SD-WAN without going the full conversion.) The longer it took to make this change, the longer it would take to finish their rollout. They expected another three weeks to isolate, fix, and reattempt the maintenance. We executed it within 24 hours.
The next evening, Friday, we took our modeled changes and implemented them directly from NetBrain’s map! Talk about visually appealing and useful. We took benchmarks (think snapshots) before and after the change for our records and troubleshooting if needed. Next, we used some custom Qapps to validate our major changes — routing validity, connectivity confirmation, and OSPF/BGP neighborships. If anything didn’t hit the expected values, we got notified when the Qapp finished. The change took less than 20 minutes.
Verify the flow of traffic across key network paths both before and after major changes are deployed.
Why is all of this important, and what does it mean to you? Change management is a harsh reality of your individual capability to execute and your business process to prevent outages with checks and balances. NetBrain enables both to empower you to perform at a high level with confidence and assurance.
Pitch from NetBrain:
Learn more in the Overview of NetBrain’s Change Management Approach solution brief.