January 26th 2014 Weekly Hashrate Contributor and Network Statistics

Other weekly pool and network statistics posts


Welcome, miners.

Changelog:

• Everything - see below

Usual pools missing from results:

• I haven't rebuilt this part yet - TBA.

Errors:

• I haven't found any - please tell me if you find something.

0. Update almost complete
The reasons behind working on a new method of gathering and analysing network hashrate contribution statistics for the last few months:

• Orphaned blocks were often incorrectly reported by pools.
• There is a wide variety of block history reporting methods, some of which don't report block heights, and some of which don't report block creation times, some report block hashes, and some report only the transaction hash of the generation address transaction.
• I had no simple way to integrate new hashrate sources.
• I had no way to attribute blocks to generation addresses.

I decided to gather the pool statistics into one dataframe, and then grep the block chain for the reported block hashes or transaction hash. Once found, each block is marked as confirmed.

This means that I have an accurate record of orphaned blocks, but it also means that I can use the blockchain height and timestamp recorded on the blockchain to fill in the data that some pools do not report.

Only orphaned blocks can't be assigned a timestamp in this way, and for pools which do not report the creation time of an orphaned block I use the timestamp of the block that won the orphan race. For pools reporting neither block height nor time of orphaned blocks, I use the next time the pools solves a block. In these latter two cases some inaccuracy may occur.

Several important notes:

• There are several pools with missing historical data or are missing from historical data completely. This only affects the extent of the historical data charts (for example Figure 2) but not the accuracy - except for some early values of top three pool's combined hashrates.
• I have requested this data from several pools but have not yet heard back, except from BTC Guild which doesn't have the data available anymore, and BTC Mine which will deliver their complete history soon.
• I won't be including pool-hopping charts unless pool-hopping becomes common once more.

1. Weekly statistics using blockchain data
Using only blockchain data, the following statistics can be reported:

• An estimate of the hashrate (with the upper and lower 95% confidence interval bounds).
• Valid blocks solved.
• The percentage of network blocks solved.

The first table is a compilation of those statistics, including non pool sources such as CloudHashing and pools that don't have public reporting of statistics, such as Discus Fish. Note that actual pool hashrates (where available) will be more accurate.

2. Weekly statistics using pool reported data
The weekly statistics that can be calculated using all solved blocks - both valid and orphaned - and difficulty 1 shares per round are:

• A much more accurate estimate of the hashrate, confidence intervals are unnecessary.
• Orphan races lost, and percentage of solved blocks that were not added to the blockchain.
• "Luck" is the usual difficulty 1 equivalent shares per round / mining difficulty, or (equivalently) accepted shares / expected shares.
• CDF: The cumulative density function (CDF) measures the percentage of the time this number accepted shares / expected shares would be less than the calculated value, given the number of valid + invalid blocks.
• Bitcoin per Gigashare. This figure is not an indicator of how much a miner should have expected per one million Difficulty 1 shares (or one thousand difficulty 1000 shares, etc), since it doesn't take into account the reward method or fees charged. Rather, it should be considered as a "luck" index that also incorporates the number of orphaned blocks and the current reward per block.

Since BTC Guild doesn't report shares per block but does report transaction hashes for all blocks, luck calculations cannot be calculated but orphaned blocks can. Pools such as "Discus Fish" that don't have a public pool interface cannot be included.

3. Reused but unknown generation addresses
Unknown generation addresses that are not reused are probably solominers or private mining concerns that don't have share-holders wanting to follow transactions. However, reused addresses are probably from hash contributors that do not wish to remain anonymous. These need to be identified so they can be removed from the "Unknown" group. I'm not interested in identifying those who wish to remain completely anonymous, so I'm not trying to trace originating IP addresses (as Blockchain.info does).

Table 3: Blocks solved by unknown but re-used generation addresses Jan 27 2013 to Jan 26 2014

Unknown recurring generation address	Blocks solved this week	Percentage of network	Percentage of unknown	Estimate of hashrate	Blocks solved ever
1GuMujABuc8kvzDTyVJFpcf4vszPUgsjiU	9	0.72 %	10.98 %	114 Thps	17
18mhP1YWE2cheiGezjpbRqY29wFLdGWtN4	7	0.56 %	8.54 %	89 Thps	7
136eiekKeXJUTbbwj6NU4vP7AnVX5RjUx2	5	0.40 %	6.10 %	64 Thps	5
17SptDvV9jovDMXQPR7PVvNDcjnxuaf4ga	3	0.24 %	3.66 %	38 Thps	6
16d11Mgzwvge21tZxaq6pwXnbrPDzbtVNH	2	0.16 %	2.44 %	25 Thps	2
1G9QNTLvP3PbhDf6r9AhtwYQBxbt8rnnu8	2	0.16 %	2.44 %	25 Thps	2
1NcYzANW5z1WF1ssnEQG3FpxBhAwettRxw	2	0.16 %	2.44 %	31 Thps	2

4. Percentage of network blocks
The only change here is that I'm not bothering with percentage of the network hashrate, which was inaccurate due to an increase in the number of hashrate sources with estimated hashrates.

5. 51% attack chart
There are several changes here compared to the previous version: hashrates are all estimated from blocks solved, and the history goes back to the earliest date my data contains three known pools. As previously mentioned, some pool data may be missing from the earliest data points.

6. Network percentage history of the current top ten contributors.
Data is calculated from the number of blocks each contributor added to the blockchain during the week, rather than using hashrates and estimated hashrates, and because of this should be more accurate with a longer history. The other changes are cosmetic.

7. Average hashrate per solved block (valid + invalid)
The only changes here are cosmetic:

• Changes to facet labels and background.
• The last two weeks are included rather than just the last week.
• BTC Guild not included. This is because I've simplified data that I scrape and collect to improve robustness of the data collection. However, BTC Guild has a very good hashrate chart which has matched my past estimates quite well and which I regard as accurate.
• The network hashrate per 144 blocks is also not included, since I think it is of limited use to readers more interested in pools.

8. Pool "luck"
I have wanted to remove either the shares per round / network Difficulty chart or the CDF chart, since they measure the same thing. However I couldn't make up my mind as to which would be the better chart to keep. In the end I came up with something I think is better than both.

This chart is new and might look a little confusing at first, but is actually simple and intuitive to follow once you get the hang of it. The orange dots are the usual accepted shares / expected shares (equivalently, shares per round / network Difficulty). The background colours are accepted shares / expected shares confidence intervals for the number of blocks solved for the week. The greater the number of blocks solved (the higher the percentage of the network) the narrower the bounds.

The "luck" data points should be outside the upper or lower boundaries only rarely. Many data points outside this range indicate unusual and unlikely "luck".

Data only goes back for the last twelve months at most - any more data points than this becomes hard to read, and recent data is most important.

9. User hashrate and combined user hashrate densities
Cosmetic changes only.

10. Comments?
I hope the new and improved charts and tables are useful for you. Let me know what you think about the changes - are they helpful? Any requests? Also, please let me know if there are any obvious formatting or spelling errors in the new charts.

Cheers all!


organofcorti.blogspot.com is a reader supported blog:

1QC2KE4GZ4SZ8AnpwVT483D2E97SLHTGCG

Find a typo or spelling error? Email me with the details at organofcorti@organofcorti.org and if you're the first to email me I'll pay you per ten errors:

Please refer to the most recent blog post for current rates or rule changes.

I'm terrible at proofreading, so some of these posts may be worth quite a bit to the keen reader.
Exceptions:

• Errors in text repeated across multiple posts: I will only pay for the most recent errors rather every single occurrence.
• Errors in chart texts: Since I can't fix the chart texts (since I don't keep the data that generated them) I can't pay for them. Still, they would be nice to know about!

I write in British English.

<weeklypoolstatistics>