Automation Success Overview – Part 4
In Part 1 of this overview, we examined the NetBrain Automation Success Framework as a whole. In Parts 2 and 3, we examined the elements of NetBrain that comprise Level-1…
March 6, 2017
Network documentation is great, but documentation is only as good as how accurate it is, if it exists at all. For years network engineers have limped along with an incomplete picture of their networks and the burden of creating and maintaining tedious spreadsheets. Documentation is priceless, but documentation is hard.
Several years ago I was part of a very large Cisco Identity Services Engine (ISE) project that required switch upgrades in order to utilize certain features on the wired network. Though this was an extremely large organization, there wasn’t good documentation to refer to in order to create a bill of materials for each location. In our department shared folder I found some diagrams for a few sites, but most of those diagrams were at least a few years old and deemed untrustworthy by my manager. The software we used to automatically discover the network was unreliable, vendor-specific, and limited to only a few platforms, so I had only one option: discover the network manually and create new documentation.
I spent hours and hours manually crawling the network using traceroute and commands like show cdp neighbors, show version, show ip interface brief and show interface description. At the end of a typical work day, I had a spreadsheet of all the switches at a particular site including hostnames, IP addresses, models, code versions, and interconnections. Some sites were so large it took me a few days.
A simple site inventory should have been available, but because it wasn’t I had the daunting task of creating it by hand. It took me weeks to go from site to site discovering each network, and by the time meetings began to discuss a particular location, its network had already changed. Even my brand-new documentation wasn’t completely accurate, and we needed reliable documentation for this project to be a success.
A funny saying in the networking industry is that a network diagram becomes out of date just about the time it’s done being printed. So how can we, as network engineers, systems administrators, and IT managers, improve this process and always have accurate, up to date network documentation at our fingertips?
Using a good script to automate the gathering of information would have saved me countless hours of tedious work. In fact, it would likely have driven the overall cost of the project down due to increased efficiency. The concept of automation itself isn’t new, but by and large, its application to networking is.
Today, networking professionals are adopting automation methods long used by systems teams who use various programming languages to efficiently manage their server environments. In the same way, network engineers in larger numbers are using the same tools and methods to gather network information, make configuration changes, and be better engineers.
But not every engineer knows Python or has the time to learn advanced Linux scripting, so dynamic network discovery software is also growing in popularity. Software that abstracts automated network discovery is incredibly valuable to engineers looking for an easy graphical user interface and programmatic reporting, but progress still needs to be made to accommodate multiple network vendors, hardware platforms, code versions, and network topologies. This is where things are going in the industry – vendor agnostic visibility, automation, and programmability.
After spending an entire workday crawling from switch to switch via command line, I knew that site’s network pretty well. In fact, during meetings with other engineers and project managers, I became the site’s subject matter expert with regard to topology though I had never been there before in my life. All it took was decent documentation.
If that documentation had been available as part of normal IT operations, the ISE project would have taken much less time and cost much less money. Think about that for your own network. How many times have you troubleshot an issue for much longer than you should have because you spent an inordinate amount of time searching for an IP address, a dependency, a physical location, or even a password?
If knowledge is power, then network engineers without documentation are powerless to be efficient and effective. Even having a very simple spreadsheet of inventory, IP addresses, and port mappings would give network professionals an incredible advantage in managing their infrastructures. The problem is that documentation has to be maintained in order for it to be valuable more than a few days. Spreadsheets don’t lend themselves to this, and using scripts requires significant maintenance as well.
Software that automates network discovery simplifies the entire processing by abstracting both network discovery and programmatic reporting. And since software companies are already abstracting configuration of routers, switches and other network functions with various SDN products, it’s natural that the discovery process and maintenance of real-time network state is also being automated.
Using a programmatic interface, a network engineer can easily schedule regular network discoveries and have at his or her fingertips an accurate, up to date resource at all times. And as the network industry conforms to open standards such as NETCONF, RESTCONF, REST APIs and native BASH support, network discovery software is better able to accommodate any vendor, any model, and any version of code.
How incredible would good network documentation have been as we kicked off my Cisco ISE project? And how incredible would real-time, trustworthy documentation be for your daily troubleshooting and management?
I’m tired of hunting through shared folders for old diagrams created five years ago in Word.
I’m tired of troubleshooting a technical problem only to find that the issue was due to the presence of a switch no one even knew about.
I’m tired of troubleshooting an issue only to discover that we were all wrong with how we thought traffic egressed the network during a failover scenario.
Manual discovery is from the stone age, and maintaining scripts is often not an option. We need good documentation. Thankfully, the barriers for creating and maintaining good network documentation are slowly being eliminated. The software is getting better, and vendors are slowly responding to the cries of the folks in the trenches. But the moral of the story is that even if it’s simple and plain, spend some time putting together accurate network documentation. It’s not glamorous work, and without good software it can be tedious. But though it’s hard, good documentation is priceless.