Friday, 23 November 2012

10.3 Canaries, coal mines and black swans

0. Introduction
In the last post in this series, I described a linear model that did a reasonable job at modelling the response of the network hashrate to the MTGOX US$BTC exchange rate and the previous week's network hashrate. In effect, it allowed us to predict the following week's network hashrate to 95% confidence interval of  +/- 15.1%. This post I want to do something just as interesting even though it does not provide any forecast of the network hashrate. What it does is detect bad air and black swans.

Canaries were used in underground coal mines to detect noxious gases. They'd respond to the gas well before the miners themselves were affected and provided a good early warning system that the air was bad. Black swans, on the other hand, are changes to a system due to factors that may seem clear in retrospect but are impossible to know beforehand.

I wanted an early warning system that won't simply detect abnormal changes we expect (for example, ASICs being added to the network, as discussed last post) but also unknown, unpredictable changes to the network. What I needed was a canary that detected both.

The simplest approach, which I've used here, is to create a model of relationship between the MTGOX US$BTC exchange rate and the network hashrate for a period of time that lacked significant changes to either. If the residuals are low and normally distributed, then the model is valid for that time period. Applying the model to the complete history of the MTGOX US$BTC exchange rate we can then determine if the model residuals are greatest at periods of greatest change in either network hashrate or price.

If you would like to compare the last post models to this post, now would be a good time to open the last post in another tab.

1. The model
I used the median MTGOX US$BTC exchange rate as the price, and chose the period from the last weekly median minimum that occurred - for the week ending 21st November 2011 - until now. I first looked for a lagged relationship between price and hashrate, and did not find one (see the CCF chart below). This is very interesting, since it means that the lagged relationship is also dependant on some other variable, or perhaps is non linear.

I did however find a weak autocorrelation for hashrate at lag = 1. I next looked for a linear relationship between hashrate and price without lag, and found one. I tested a few similar models and found the one with the best correlation and lowest most normal distributed residuals, and the lowest p value for each term (all p < 0.001):

Model C: log(H) ~ 3.96 + 0.52442 log(H, lag = 1) + 0.33676 log(H, lag = 3)  + 0.14132 log(P) + error 

H = weekly average network hashrate
P = weekly median MTGOX US$ / BTC exchange rate

Model C models the network hashrate well. The residuals can be assumed normally distributed, and result in 95% confidence interval error term of +/-9.3% .

Prior to November 2011 there were several times when the hashrate and price changed at a very different rate to the changes from November 2011 onward. How does our model handle those changes?

The model now has a much larger error. Any sudden change in hashrate or price away from the model quickly exceeds the 95% confidence interval. In fact about 44% of rounds exceed the original CI, and mostly when there's been a sudden change in hashrate without a change in price, or (more often) the converse.

2. Conclusions and discussion
Is model C detecting sudden changes, or is the error due to the lack of a price lag variable? I think the model does both - sudden changes away from the normal linear relationship exceed the 95% confidence interval because of the lack of the price lag variable. 

Model C is the proverbial canary. Calculate it each week and find a week that log(H) - model C (log(H, lag1), log(H, lag3),log(P)) > 9.3%, then there's a good chance ASICs are being added. If the converse is true,  some other variable is slowing the response of the network hashrate to the exchange rate. Happy hunting!

  • Model C: log(H) ~ 3.96 + 0.52442 log(H, lag = 1) + 0.33676 log(H, lag = 3)  + 0.14132 log(P) +/- 9.3% 
  • If  log(H) - model C (log(H, lag1), log(H, lag3),log(P)) > 9.3%, the network hashrate is increasing more quickly than indicated by the price (for example ASICs have been switched on).
  • If  log(H) - model C (log(H, lag1), log(H, lag3),log(P)) < 9.3%, the network hashrate is increasing more slowly than indicated by the price (for example an increase in D that makes mining unprofitable for many miners.

Donations help give me the time to analyse bitcoin mining related issues and write these posts. If you enjoy or find them helpful, please consider a small bitcoin donation:

<tenpoint three>

No comments:

Post a Comment

Comments are switched off until the current spam storm ends.