Friday, July 13, 2012

Visualizing communities of reddit.com

What does reddit look like? No doubt it has many faces. Alien faces mostly:

Many faces of reddit

Some time ago I created visualization of related subreddits, where you can type subreddit name and get network of related subreddits:

Network of books related communities

This network was only three levels deep. I didn't show anything further due to rendering limits of SVG.

Recently I added support for WebGL rendering, and now you can see networks with tens of thousands links and nodes (of course if you have modern browser and computer). I couldn't resist the temptation to create visualization of all subreddits to see the whole picture. And here it is:



This network has almost 3 800 communities and shows more than 14 000 connections between these communities. Feel free to check it out and come back to find out more: http://www.yasiv.com/allreddit.

How exactly does it work?

I get information about all subreddits (communities) from reddit.com/reddits and store it in Google App Engine. When I last time indexed all subreddits there was a limit of 25 000 items. Reddit doesn't give any more, but it is enough to get started. Oh, and there is no 'adult' communities in this list, sorry.

Some of subreddits in their description have links to other communities. I parse this information and consider every link as a connection between two communities. E.g. if you go to /r/pics there are links to other communities in description:

Example of links from /r/pics subreddit


Those become connections in visualization (but only if I can find referenced community among indexed ones).

I also excluded from visualization all isolated communities: If no other community has a link to the current I'm not showing it. Otherwise the whole visualization becomes overwhelmed with outer circle of not connected dots:

Outer circle is made of isolated subreddits

It's also very expensive to render them all at once (still have to improve performance of the drawing library).

So what?

For me it was an exercise. I wanted to see this face of reddit and I saw it. Satisfied :). 

But I also hope this visualization can help you discover new subreddits. It's fairly easy to search subreddits on specific subject. Just type something that matters to you in the search box on the top and visualization highlights related communities. E.g. here is a search for "book" related communities:

Books related subreddits within larger picture

For those of you who run ads on reddit this tool could also be useful when considering which communities should be your target audience.

Hope you enjoy it. Let me know what you think :).

5 comments:

  1. I'm coming from using Sigma.js, which is really a good library, but definitely VivaGraph is a must. Congrats!

    ReplyDelete
  2. Hi Andrie, Very nice tool. Where can I get the source code for Amazon Yasiv tool?

    ReplyDelete
  3. great work. impressive loading time.

    ReplyDelete