Most recently and happening more often than it used to, I have come across different projects that want to add NETCONF to their existing or exciting new products. The goal, to leverage current or planned state data stored in a database that they want to make accessible to NETCONF (or RESTCONF). Such state data can be stored in ConfD’s CDB operational datastore, but sometimes there are requirements to keep some or all of that state data external to ConfD in a totally separate external datastore. In addition, there are often misunderstandings on what ConfD’s CDB operational datastore can do and how to read the data using ConfD’s NETCONF implementation. i.e. how to filter data in queries.
Consequently, I created an application note “ConfD Operational Data and External Databases” demonstrating how to provide some or all state data through CDB and the ConfD Data Provider API, how different types of queries can be performed over NETCONF to retrieve the data, and how a high-performance integration can be done with databases where I used the well-established PostgreSQL and Cassandra databases as examples.
One goal of the application note is to provide guidance on how to utilize the features ConfD provides for various use cases where operational state data is provided over management interfaces such as NETCONF, RESTCONF, etc. Another equally important goal is to provide input and reasoning behind why organizations may want to use an external database for operational state data in the first place. The reasons are commonly one of the following three:
- You might, for example, be passing large amounts and or very frequently changing application state data between application instances. The manager will never see this data, it is just used for passing state in-between your applications, and, by the way, you need to replicate the data to standby nodes to maintain high availability. This replication needs to be throttled or will require ring mechanisms, eventual consistency schemes, etc.
- You have a need for multiple indexes and joins on the fly, etc, to present your data in real time directly to a web interface; not over NETCONF. ConfD’s CDB provides the basic key indexing functionality and a schema derived from the YANG model, but you may need the tools and features provided by databases that specialize in searching and organizing massive amounts of state data in an efficient manner.
- Legacy database bias, including project managers who convince you that changes and new things are bad and very costly. You should stay safe and not change any applications to adapt to the ConfD APIs that are currently integrated with a legacy database. Not even the possibility of using a wrapper is safe enough for your project manager. Then doing an external operational state database integration may be your only remaining option.
I look forward to continuing the conversation on how this type of integration is best made with ConfD and NETCONF. It is essential to rethink the reasons behind why we want to use external databases as we move into use-cases such as virtualized containerized systems where state data and telemetry plays an even more important role.