I recently had an opportunity to sit down with John Criddle, Senior Director IT Enterprise Infrastructure Operations at CVS Health. John is a very seasoned and respected leader having managed enterprise IT Infrastructure and Operations for over 20 years. CVS Health is a retail pharmacy and health care company with assets that includes CVS Pharmacy, CVS Caremark, CVS Specialty, and the retail clinic (MinuteClinic).
CVS has a very rich and complex network because of mergers and acquisitions which makes it hard to diagnose and troubleshoot. Also, its environment is very rich in firewalls because of the nature of the business. Because of its complexity, there are certain behaviors in the network that need to be addressed closely.
Three things I walked away with after speaking to John and having a good understanding of the complexity and importance of CVS’s network for running its day-to-day business operations.
Lesson #1: Limited visibility is bad for your network health.
Network maps must be created “on-demand” with details on the map to suit each specific workflow because a one-size-fits-all (i.e., one ‘static’ map) does not work for every unique use case (design, compliance, troubleshooting, etc.) Per John, “NetBrain greatly helps CVS Health in understanding network design, documentation, troubleshooting, and collaboration.
Maps are created on an adhoc basis to troubleshoot network issues, such as, sporadic voice issues caused by asynchronous communications, end-to-end IP connectivity, bottlenecks in the network, and so on.” Speaking broadly, IT teams within the organizations face tremendous pressure to sustain business requirements and Service Level Agreements (SLAs). Network downtime can pose significant impact on internal operations and external SLAs. There is no shortage of network monitoring tools to help engineers detect network outages, but the steps to diagnose the alarm are almost always manual.
One of the biggest challenges when trying to reduce mean-time-to-repair (MTTR) is having the visibility into the network to identify all the network elements that are part of the problem. Effective and faster troubleshooting requires a combination of experience, an intimate knowledge of the network’s design, and automation.
Lesson #2: Up-to-date network documentation is a must
Engineers spend a lot time creating network documentation which is not rich in detail and becomes obsolete as soon as changes take place in the network. Additionally, automatic benchmark capability is critical for network compliance and security so your documentation and network diagrams are always up-to-date. John added, “Network documentation is big since network engineers spend a lot of time creating the Microsoft® Visio diagrams. NetBrain checks each node in the network to determine firewall rules configured on the network elements”.
Lesson #3: Collaboration between teams enhances productivity
A wide majority of tasks today get resolved in a very collaborative manner between teams that could possibly be in the same building or spread across geographies. One good example of collaboration will be that of network and security teams working together in the event of a cyber-security attacks. Both teams surge to identify the nature of the attack, localize and block the source, identify and fix the compromised systems, and perform the threat analysis. Also, the CERT team will maintain a list of steps taken to resolve the vulnerability.
Effective collaboration between individuals and teams gets amplified with the right tools fostering transparency of data collection and analysis. John continued to add, “Our network security and core switching & routing teams use NetBrain for running diagnostics and collaboration. These quick diagnostics helps us to understand how devices are performing.”
It was gratifying to chat with one of our key customers and learn about what’s important to them, and where they see the value of NetBrain. John further commented, “There is so much more we can do with NetBrain that I’d like my team to explore.” I’m very appreciative of the time with John and look forward to stronger partnership between CVS Health and NetBrain.