Metadata is the data of the data. Whether in the form of key-value pair labels or as tags, Metadata is the context that the data requires. With context, the data is meaningful and valuable to those analyzing it. AIOps is the business of using data and algorithms to facilitate network and IT infrastructure operations. The connection is clear: metadata is fundamental for any AIOps initiative’s success.
However, metadata has been managed historically as second-class data by monitoring and observability systems. Engineers will look for the best way to enrich the data that is being collected. However, the response they often receive from vendors is: “yes, you can upload a CSV file”. But this answer doesn’t satisfy the where, who and what-if scenarios. For example, where does that CSV file come from? Who generates it? What if it needs to be updated?
Contextual information (i.e., metadata) may exist in different form factors. Sometimes it’s embedded in the data itself or it may exist in CMDB or inventory systems, CRMs, or other types of repositories where the connections exist between the business and the infrastructure. To be clear, there is no one-size-fits-all when it comes to metadata and the different manifestations it can present.
At the end of the day, Metadata is simply data. It is data that needs to be collected to provide context to other types of data in a system. This is the principal Selector follows with its platform. There are no assumptions about where and how metadata can or cannot be collected. Selector’s platform can fetch metadata from any source using any format or collection mechanism. Metadata will be kept in our Metastore and updated as needed based on its nature and the collection mechanisms defined.
Also, a high-performance and dynamic data join mechanism will be executed by our Data Hypervisor (DHV). It will enrich all the incoming network telemetry, logs, events, etc., with the Metadata corresponding to them. Any data that the Selector platform collects can become metadata for other data streams. Think of this as the glue that connects the data silos together.
While this is something users may not need to deal with daily, this is a fundamental element that enables Selector Analytics to surface the relevant insights and correlations in the network and cloud infrastructure.
Data is more complex than ever. The ability to simply upload a CSV file with Metadata is no longer a viable option. Multiple data domains exist to collect and join, and the glue required may come from various sources and formats. True observability requires these connections. Therefore, it requires that Metadata be managed as first-class data throughout the entire system.
Let’s explore some of the benefits of Selector’s Analytics platform:
First, operations teams are no longer required to create, upload and maintain CSV files to get contextual information in their network data.
Second, by supporting all collection mechanisms for Metadata, Selector’s platform can extract it from any system, in any format and enable complex data joins to provide the contextual information required for accurate insights and correlations.
Interested in learning more about this feature? Contact us today.