Extracting YANG Annotations from Your Data Models

CommunityWhen working with ConfD, developers make quite a bit of use of Tail-f YANG extension statements in their YANG data models to provide information about things such as backend instrumentation, e.g. “tailf:callpoint”, and northbound management interface auto-rendering customization, e.g. “tailf:cli-*”. It is common to use these inline in your YANG data models. However, when it comes time to provide your network element’s YANG data models to your customers, it is not desirable to include the Tail-f YANG extension statements as they relate to the internals of your system and are not needed by NETCONF clients. Also, oftentimes, their presence can cause problems for the NETCONF client.

ConfD provides an alternative to using the Tail-f YANG extensions inline in your YANG data models. That alternative is annotation modules using the tailf:tailf-annotate-module and tailf:tailf-annotate-statement Tail-f YANG extension statements. This feature allows the developer to remove the Tail-f YANG extension statements from the main YANG data model and place them into a separate annotation module which the confdc compiler will merge in during data model compilation.

You may have used “tailf:” extension statements in great numbers to generate, for example, a CLI of your liking and then discovered that those extensions are shipped in the YANG data models your customers get. Maybe your customers started to use the NETCONF interface to program your device and now use the NETCONF <get-schema> RPC to download your YANG model only to find a bunch of, to the NETCONF client, useless “tailf:cli-*” YANG extension statements all littering the YANG data models you provide.

So, extracting the Tail-f YANG extension statements to an annotation file and sanitizing the original YANG module of “tailf:” extensions is desirable in order to have “clean” YANG data models to supply to your users. However, doing that work manually would take a lot of time. You could write a script from scratch to do it, but that is no longer required.

I recently found a really simple way to extract those “tailf:” extension statements into a tailf-annotate-module/statement annotation module. The solution consists of a combination of the open-source pyang tool and the Python Beautiful Soup + lxml parser to do the work in just a few lines of Python. As you may know, items placed in an annotation file will no longer be included in the YANG data models shipped using, for example, NETCONF <get-schema>. You just provide the annotation file(s) with the original YANG data model to the ConfD confdc compiler to include them.

This solution is called ann-stmt and is available on ConfD-Developer on GitHub under ConfD-Demos. There are many more details in the project’s README file.

The key idea leveraged in this solution is that you can loosely convert YANG to YIN, an XML representation of YANG, and back again.  The solution works by using pyang to convert your YANG data model to YIN and then using Python Beautiful Soup + lxml to extract the annotations. When taking a peek at the Beautiful Soup BS4 documentation, you will notice that you can use the combination of Python BS4 and lxml for all sorts of XML parsing of your YANG data models when in YIN XML format and, of course, your XML configuration/data with very little code/effort.

As usual with GitHub projects, if you come up with improvements or corrections to the ann-stmt demo, please, open up a pull request to get it reviewed and added to the project.

Check out the code from ConfD-Developer on GitHub today!