In the world of network programmability, I talk a lot about NETCONF and YANG and what it means to be programmable. The one thing I don’t talk about often enough and which is very important is the origins of programmability coming from the world of computer science. To truly realize the benefits of network programmability and deliver automation, it is worthwhile spending time understanding computer science and software development methodologies and leveraging their proven techniques in order to get full value out of network programmability.
Traditionally, software development was done using a waterfall model where programmers go off and build and test their separate pieces. Then, when ready, the pieces are integrated together to finish and test the final product. This approach has the problem that integration and system-level test happens late in the product development cycle. Problems found at this stage have to be isolated and fixed. If major problems are found at this late stage, the product release can end up being delayed.
The study of computer science and software development methodologies has shown that it is better to identify problems as soon as possible. The sooner they are found, the sooner they can be fixed, and the sooner the product can be shipped. This led to the development of the agile development method. In order to find problems more quickly as the software is being developed, a time-saving approach called Continuous Integration (CI) has become the key to most software development practices, especially the agile methodology. CI calls for software developers to integrate the full system and perform system-level tests regularly throughout the process, even multiple times a day. Ideally, each time the software updates are merged, it should trigger an automated code build and test sequence. As a result, developers are always building and testing the full product. This allows issues to identified and fixed as early as possible in the development cycle.
So, we’ve seen that Continuous Integration provides major benefits to software development. How can this methodology be applied in the area of network management and programmability? As a device maker, obviously, CI can be applied to your firmware image. However, to get the full benefit of CI, you also needed to automate the testing of your device. This is where a programmable interface driven by NETCONF and YANG can be leveraged. By making use of a NETCONF client you can develop automated device tests which are controlled and run over your programmable interface.
Naturally, you will want to leverage this capability and deploy a programmable interface. How can you drive it to do the testing? There are different levels at which you can approach this. The lowest level would be to make use of the netconf-console program which is included in the ConfD distribution. To use netconf-console, you set up input scripts that consist of raw NETCONF protocol packets in XML. Netconf-console feeds these into the device and then you capture the returned protocol packets and analyze the results. Effective, but it can be quite a bit of work at the protocol level.
The next higher level to approach implementing tests is to make use of a NETCONF client software library such as ncclient or the YANG Development Kit (YDK). To use these tools, you write software in Python or C++ which drives the NETCONF client calls to your device under test, and then use the return codes and values to analyze the test results in your test software.
The highest level which requires the least amount of software development is to make use of the NETCONF & YANG Automation Testing (NYAT) program. NYAT makes use of a free version of Cisco Network Services Orchestrator (NSO) along with an add-on package called DrNED Examiner to drive NETCONF & YANG testing as well as automation best practices testing. This approach can also drive tests that cycle through various configuration states using transactions.
To ultimately drive and deliver automation, you will benefit from a Continuous Integration approach to network equipment development in order to test and quickly find and fix bugs faster.
To learn more about NETCONF and YANG testing check out our “NETCONF and YANG Automation Testing Guide” and watch our “NETCONF & YANG Automation Testing Demo/Tutorial.” All of this will help you implement testing into your development process more efficient and help you to ship the best programmable product possible.