Description
What do Netflix and Spotify have to do with Selector? In this Networking Field Day 30 presentation clip, Nitin Kumar, Selector Co-founder and CTO, explains how the techniques used by these services inspired our event correlation.
Transcript
[Video text]
Networking Field Day 30 Selector AI
A Deep Dive into Selector AI
Nitin Kumar
January 20, 2023
Me and my chief data scientist, we were just discussing Netflix recommendation systems. This is before we had done this. … We shared a lot of common taste in movies, and he would say, “Hey watch this movie or this one.” Also on Spotify, sometimes I’d be listening to music and suddenly [it’d] recommend: “you should listen to this.” I’m like: how did this software know that I like this?
Then we did some research on it. And we found that every movie, every song, can be decomposed into call features.
So, think of the movie Pulp Fiction. If the movie is Pulp Fiction, you and I understand Pulp Fiction as just a name, maybe [that it’s also] directed by Quentin Tarantino. That’s about it. But internally, the Netflix system breaks down Pulp Fiction into thousands of attributes … thousands of attributes … like this movie has a screenplay. This movie has this and that. It breaks down a movie into a thousand attributes.
Now you take another movie that will have another thousand attributes. Then, the system does an intersection of those attributes. So, if I’ve liked Pulp Fiction, most likely I’m going to like the other movie because, out of those thousand attributes—60 attributes in that movie, max 60 attributes in this movie—most likely Nitin is going to like [this other] movie. That’s how recommendation systems work. Then we realized that means recommendation systems are correlating these two movies. Pulp Fiction and Jackie Brown were correlated because of certain common attributes.
But wait. We can do this with network events as well. If two events are heavily correlated, there must be a root cause between them. In the case of the movie system, the commonality was my choice in movies or my choice in music. [inaudible] Something.
We felt that the same applies to networking events or any kind of events. If they have similar attributes, there must be a root cause between them: Joe must have issued that configuration … [laughter] because things went down. [interjection] Of course, when you start the implementation, you don’t believe it’s going to work; you want it to work; and then we’ve deployed this again and again and again, and it just works.
The magic of this algorithm is it’s domain independent. I do not have to model the underlying domain. Some folks might argue that you’re losing some fidelity. That’s okay, because as soon as you rely on understanding the domain, you become embedded in it, and every network looks very different. You then have to start taking care of every nuance in the rule modeling.
This thing … just works. Of course! The premise here is tags have to be well documented … you have to have good tagging. Let’s solve that problem. It’s a data problem. The rest of the world has solved that data problem. Even if the original tags are not clean, let’s put in some work to solve for that … because you can see the end result there. If I solve the problem of cleaning up tags, creating more tags, it’s a bounded problem. Maybe it’s hard. It’s a bounded problem.
That’s been one of our key differentiators as we go into a system in a deployment and we’re able to show correlations. We sometimes don’t understand their exact network topology. That’s not important for this part of the problem. And, it was inspired by Netflix and Spotify.
Audience member 1: I have to tell you, that was the best explanation I’ve ever heard.
Audience member 2: Seriously.
Nitin Kumar: Thank you.
Audience member 3: Yeah. It was great.Nitin Kumar: And it’s true! It’s true.