The Importance of Prototyping

This exchange recently took place in a meeting where we had been called in to help a customer with lab-to-production problems. They had implemented a homegrown CLI for their lab trials and had an engineer on site throughout the process so the customer really hadn’t seen the interface:

Me: “So, tell me about what your customer finds unacceptable with your current solution.”
Customer: “Well, they were happy with the command line interface look and feel and the fact that we had good coverage of the features in the early version.”
Me: “OK.”
Customer: “…but they were negatively surprised that there was no way to look at the entire configuration through the CLI…”
Me:?! (you know; looking startled, yet interested)
Customer: “Well, we kind of did not implement an equivalent of “show configuration” in the CLI as that was never part of our requirements documentation.”

We see engineering teams making increasing use of prototyping to develop more exciting products and push the operational productivity and efficiency envelope even further. Prototyping obviously plays a role in the pre-release phases as a means of qualifying requirements between product management and the development team. It is also a very powerful tool during alpha- or beta-programs with customers, reducing the risk of any misalignment in expectations by exposing a tangible implementation to the people that will eventually make productive use of it.

We increasingly see network equipment providers send extremely early software loads to customers for feedback – often prototypes running on (or installable on) Linux and commodity server hardware. These prototypes may not implement the full feature set and will likely not perform up to specification, but give customers a hands-on environment to base their feedback on. This approach increases the value of feedback and avoids knee-jerk reactions to feature descriptions in specification documents. It may also allow customers to do early conformance testing to ensure that their existing management frameworks (e.g. fault, performance and configuration management) work well with the incoming new product.

We’ve found that new software versions that allow for early exposure to customer feedback tend to perform better. Customer requirements commonly focus (and rightly so) on features specific to their business challenges at hand. And while manageability may not be one of those in the early stages, they eventually will become so as field deployments ramp up as equipment is being shifted into production use under the watchful eye of operations teams. It is not uncommon for new products to be sent back to the lab based on the inability to meet basic operations requirements.

The team at Tail-f is an interesting mix of people with backgrounds in computer science, network design and operations. As a team, we have accumulated several hundred man-years of experience helping network equipment providers add world-class management to their products. We enjoy helping customers with challenging feature requirements. And more and more find our customers both realize the competitive value of great manageability and automation features and are willing to act on it.

Getting back to my opening anecdote. Perhaps extreme, but not such an in frequent occurrence as you might think and is definitely something that would have been caught during the first minute or so had they brought a prototype to the operations teams. We were able to help in this situation by shedding their in-house CLI and integrating ConfD in their software stack. This added, among other things, a way to show the full configuration.