Improving List Filtering Performance in ConfD

Community Performance optimization is a key component to our success here at Tail-f/Cisco and is always a goal we aim to achieve. In the world of networks, one fundamental way to increase performance is to reduce the amount of data that needs to cross the network. In order to achieve this, network management protocols include support for sending queries to a network device. These queries are used by the network device to filter down larger data sets and only send the data which matches the query criteria. This filtering will result in network performance optimization as long as the filter processing time is minimized.

Filtering has always been a key part of the ConfD core engine. With the release of ConfD 7.1, we introduced an important performance improvement in the Data Provider API (DPAPI) which allows list data to be filtered closer to the source. This feature, as you can probably guess, is called “List Filters”.

An excellent example of a northbound interface that supports data filtering in queries is NETCONF. There are two types of filtering, subtree, and XPath, which a NETCONF client can include when issuing a or a get-config> RPC call. Previously, this filter information was kept and processed in the ConfD core engine and not passed to the Data Providers. The ConfD core engine would first get the data from the Data Providers and then apply the filter before sending it back to the user. This can be costly for very large lists; both in terms of gathering and moving the data as well as the number of callback invocations.

The List Filters feature, which was introduced in ConfD 7.1, enables the ConfD core engine to pass filter information to the Data Provider and receive the data from the Data Provider already filtered. By having the filter processing in the Data Provider backend code, less data has to be moved to the ConfD core engine and fewer callbacks need to be called. This is another “tool in the box” for ConfD application developers to consider using when dealing with large list data retrieval.

Our new application note “List Filters in ConfD” introduces this important new feature: what it is, how it works, and how to implement it in your Data Provider code. Download your copy today!

Leave a Reply

Notify of