Optimising graph lookups in MongoDB

Graph databases such as Neo4J specialise in traversing graphs of relationships — such as those you might find in a social network. Many non-graph databases have been incorporating Graph Compute Engines to perform similar tasks. Since MongoDB 3.4 release, we now have the ability to perform simple graph traversal using the $graphLookup aggregation framework function.

Now we have a collection with records like this:

What we are doing here is starting with person 1476767, then following the elements of the friends array out to two levels — i.e.: to “friends of friends”.

Increasing the maxdepth exponentially increases the amount of data we have to cope with. This is the notorious “seven degrees of separation” effect — most people in a social network are linked by 6–7 hops, so once we get past that we are effectively traversing the entire set. Unfortunately, this meant that traversing more than 3 deep causes us to run out of memory:

For shallow networks, having an index on the connectToField makes an enormous difference. But as the depth increases, the index performance advantage decreases and eventually performance matches that of the unindexed case. In this example data that just happens to be at the “7 degrees of separation” but it will clearly depend on the nature of the data.

The $graphLookup operator is a very powerful addition to the MongoDB aggregation framework and continues the trend of providing richer query capabilities within the server. Mastering the aggregation framework is clearly a high priority for anyone wanting to exploit the full power of MongoDB.

Related posts:

Stock brokerage firms can charge up to $10 per trade, which isn’t much for those who make trades in high volume. However, new investors tend to have lighter pockets, so the fees make a difference…

Even when you are bored, make the best of your time by engaging in the following activities:. “Things to do when you are bored” is published by Srijita Chakraborty.