16.2 A short post on mapping the historical pool hashrate distribution

Other posts in this series

Recommended reading:
Naomi Robbins' posts
"Creating More Effective Graphs"

0. Introduction

For a while I've been trying to find better ways to present pool hashrate distributions. Pie charts are OK - most people understand them (although as you should all be aware, pie charts have severe problems and can mislead you), but I wanted something that a) was not misleading and b) could show proportions of the network attributable to specific pools. I also wanted to show cumulative data, rather than the plots I usually use to show the historical hashrate distribution.

What I really want to do is create a "chart" like this, but sadly I have no idea how it's done since it relies on a javascript toolkit. Instead, I've just focused on creating 2D images that will do the trick.

1. The charts

The first chart is a stacked histogram of number of blocks solved per week, based on blockchain coinbase signatures and on my own data. The second chart is the percentage of blocks solved by particular pools. Both are read from the bottom up (most recent data is at the top).

You'll notice a large amount of "Unknown". This is because either pools weren't signing the coinbase, or I hadn't started to keep track of the number of blocks solved by a pool. From September 2012 onward, the "Unknown" is reduced significantly and only represents solominers; several pools for which I couldn't screen scrape; and the urban-legendary "dark pools". The chart only starts from July 2011 since that's the first time pools identified themselves by signing the coinbase (first Eligius, followed by BitMinter and BTC Guild).

I like the fact that these charts don't exaggerate the proportion attributable to any particular pool the way a pie chart does, and trends are shown quite clearly. However these charts still have some significant problems that need to be solved:

    i) More data is required.  Much data is missing pre-September 2012. Also some data is missing after this date - if the pool wasn't signing the coinbase and the pool's website was not accessible, I have no record for the pool for that week. For example, look at GHash.IO (if you can find it) - at one point near the end it crashes from a significant portion of the network, to zero, and back up to an even larger proportion of the network. I'm fairly sure this is wrong. I'm going to try to get old (pre-coinbase signing) data from pools to fill in some of the earlier unknowns and more recent data to fix some of the errors.

    ii) Difficulty in identifying smaller pools. I need to find a way to make the charts interactive, so that "mousing over" a section will reveal the name of the pool that section represents.

    iii) Smoothing the data. Since block solving is a stochastic process, many of the "wiggles" in the chart represent variance. I want to use kernel smoothing to make the charts more legible, but at the same time not cause any significant cumulative errors.

This will mean quite a bit of work, but I think it will be worth it.

What do you think, readers? Are these useful charts enough for me to spend time on completing?

  

organofcorti.blogspot.com is a reader supported blog:

12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r


Find a typo or spelling error? Email me with the details at organofcorti@organofcorti.org and if you're the first to email me I'll pay you per error:
 

I'm terrible at proofreading, so some of these posts may be worth quite a bit to the keen reader.

<sixteenpoint two>