Selfish mining log 1

bingledack [3:57 PM]
joined selfish-mining

zbingledack [3:57 PM]
cleared the channel purpose

vlad2vlad [3:57 PM]
joined selfish-mining by invitation from @zbingledack, along with @cryptorebel, @freetrader, @kyuupichan, @elliotolds, @tomtomtom7, @iang, @deadalnix, @gregnie and some others.

zbingledack [4:02 PM]
Would like to understand @csw's point here in more detail, as well as the connection to the article about block consistency and how it relates to selfish mining.

zbingledack [4:04 PM]
uploaded this image: Screenshot_2017-05-12-04-59-52.png
Add Comment

zbingledack [4:04 PM]
https://web.archive.org/web/20160502203749/http://www.drcraigwright.net/consistency-distribution-transactions/
Dr. Craig Wright Blog
Bitcoin mining: consistency and the distribution of transactions - Dr. Craig Wright Blog
Misunderstandings in the bitcoin community have caused confusion about how bitcoin works. The bitcoin mining process is fundamentally competitive.
April 28th, 2016 at 6:30 AM



cryptorebel [4:06 PM]
was looking for that

zbingledack [4:10 PM]
If the selfish mining attack paper made an error in probability measurement (failing to use conditional probabilities when needed), it seems like it should be straightforward to dismantle the paper or at least show that the viability of the attack is different than assumed. I'm not yet able to contextualize CSW's comments in the screenshot above.

Also the article seems to say many people mistakenly think blocks are fixed before they are solved, but as far as I can tell that isn't a widespread belief. Perhaps he means that people don't take this fact seriously in terms of thinking through all the implications, such as on selfish mining. Here, too, I don't see the connection.
1 reply 5 days ago View thread

zbingledack [4:15 PM]
Here's the Selfish Mining paper: http://arxiv.org/pdf/1311.0243v2.pdf

zbingledack [4:21 PM]
I'm guessing the probability calculations in this section of the Selfish Mining are where the error is claimed to occur:

zbingledack [4:22 PM]
uploaded this image: Screenshot_2017-05-12-05-18-38.png
Add Comment

zbingledack [4:22 PM]
uploaded this image: Screenshot_2017-05-12-05-18-56.png
Add Comment

zbingledack [4:22 PM]
uploaded this image: Screenshot_2017-05-12-05-19-26.png
Add Comment

zbingledack [4:26 PM]
And just in case, here's the Appendix A with the details of the probability calculation:

zbingledack [4:27 PM]
uploaded this image: Screenshot_2017-05-12-05-24-50.png
Add Comment

zbingledack [4:33 PM]
@all
Feel free to invite anyone who may be interested and won't be disruptive

cypherblock [4:45 PM]
someone in the past wrote a python program to simulate this https://www.reddit.com/r/Bitcoin/comments/1vxszh/selfish_mine_simulation_in_python/
reddit
Selfish Mine simulation in python • r/Bitcoin
Hi All, I made a little code to simulate a selfish mine according to the paper of Eyal and Gün Sirer : "Majority is not Enough : Bitcoin Mining...


elliotolds [4:49 PM]
CSW's initial message about conditional probabilities makes no sense to me. The probability of finding the next block is independent of who found the last one. CSW's blog post about selfish mining is extremely bizarre, in that it doesn't address any of the important issues in selfish mining, and instead focuses on weird things that no one believes, like the content of blocks being fixed before they're mined

[4:50]
I wrote my own python program to simulate selfish mining after the paper came out. Selfish mining is legit and Sirer & Eyal's calculations are right, CSW is wrong.

elliotolds [4:58 PM]
An outside-view argument: Note that Sirer and Greg / other Core devs dislike each other almost as much as Core and BU folks hate each other, and are always criticizing each other's work. How likely is it that the Core folks would let a basic probability error in that paper go unnoticed?

zbingledack [5:00 PM]
I'm guessing CSW is saying it is a Monty Hall  style counterintuitive situation that many people get wrong. Need a bit more details/context on P(SS|H)


cypherblock [5:01 PM]
I also do not see the connection of the content of blocks being fixed thing. @elliotolds what have people concluded about this paper? What refutations or other conclusions exist?

zbingledack [5:01 PM]
Even Paul Erdős apparently didn't believe switching in the Monty Haul problem was profitable until shown a simulation.

[5:02]
And it's a very simple conditional probability problem.

elliotolds [5:05 PM]
@zbingledack that may be the claim, and even brilliant individual people can have weird mental blind spots, but the selfish mining paper has gotten lots of review from lots of smart people. I don't know anyone other than CSW who thinks he found this type of flaw.

@cypherblock the paper is widely accepted as legit. Core devs all accept it but they just claim that the paper isn't novel because everyone already knew about the attack.  People mostly disagree about the seriousness of the practical consequences / whether social factors will make the attack less likely.

[5:06]
the attack is detectable, for instance, so if a big mining pool were to use selfish mining, everyone else would notice when they started releasing long strings of blocks at the same time, and presumably they'd get bad PR

High. Core folk are far from infallible, and take criticism as an attack.

Try saying to Peter Todd that you can remove RBF from the node with breaking a single test and see how it goes.
Try saying to blockstream folks that their implementation of schnorr in element alpha is broken (related key attacks).

Shall I continue ? Nobody think fixing a proba in a freaking paper is worth being the target of a smear campaign.
elliotolds
An outside-view argument: Note that Sirer and Greg / other Core devs dislike each other almost as much as Core and BU folks hate each other, and are always criticizing each other's work. How likely is it that the Core folks would let a basic probability error in that paper go unnoticed?
Posted in #selfish-miningMay 11th at 4:58 PM

elliotolds [5:52 PM]
you're saying Core devs would be reluctant to point in an error in someone else's paper, because Core devs are afraid of being smeared? Maybe you're thinking of different Core devs? The ones I know take the typical geek delight in smacking down anyone else any chance they get to show how smart they are, and don't shy away from intellectual fights. The fact that Peter Todd will attack you if you criticize RBF strengthens my point. Core devs took the selfish mining paper as an attack on them, because Sirer was essentially claiming to have discovered something that they missed, which was a threat to their ego. We should expect Core to have been especially motivated to find any flaws in that paper.

Anyway, this is just the outside-view argument. If you know math and want to verify CSW is wrong for yourself, it's easy to read the paper.

[5:53]
CSW hasn't actually clarified what exactly his claim is. If someone here actually think's he's right, can you post a clear description of what the claimed probability error is,?

peter_r [6:04 PM]
I was speaking with CSW about this too, and I still can't make heads-or-tails about his conditional probability argument.  As far as I can tell, the work in Eyal and Sirer's paper is correct given their assumptions.  I'm open-minded to being proven wrong, but I really need to see a clear presentation of the arguments and a simulation.


deadalnix [7:20 PM]
@elliotolds I meant that core devs aren't as good as one might think they are. They are aggressively defneding a high priesthood status so they won't risk it going out of their confort zone.

[7:21]
@peter_r I think csw argument is that the assumption are not correct, which may well be true. Or not. Not sure.

elliotolds [7:23 PM]
I am amazed at the amount of deference some people give to CSW's argument, without anyone understanding what argument CSW is actually making.


cryptorebel [7:33 PM]
yeah I don't think he is saying there is a probability error in the math of the paper, but more he is saying there is a problem with the initial assumptions

[7:35]
He seemed to be saying that as a result of the random ordering of txs in blocks, and latency in the network, it causes extra processing time for the selfish miner making it less economically feasible. He was analyzing the game theory of it I think, and trying to show the incentives dont align.  But would be nice if he could clarify more what he is getting at

elliotolds [9:36 PM]
@cryptorebel, the screenshotted message makes it look like he think's the probability math is wrong.

Why would the ordering of transactions in a block make selfish mining less feasible? I don't see any mechanism there that makes sense.

cryptorebel [9:38 PM]
I was getting it from this part: "What seems to be misunderstood here is that separate miners can mine transactional data in any order. The addition of a nonce to seek a solution provides miners with the ability to add verified transactions in any order while they equally and fairly compete using their levels of computational power. As a consequence, miners do not benefit by pruning transactions in blocks or by seeking a common ordering of transactions. If a miner were to seek to align a transactional order with other miners, the likely result would be a scenario in which any miner seeking to align transactional positions would be economically disadvantaged due to the extra cost of this pre-processing."

[9:40]
I think the screen shotted thing is not about the existing probability math in the selfish mining paper, but additional math that Craig is introducing to help explain the true incentives and game theory, something to do with competition between miners, I am not smart enough to understand it though

elliotolds [9:42 PM]
what would a miner gain exactly by aligning transaction positions with other miners? I don't see why he's even talking about this

 22 replies Last reply 4 days ago View thread

bitsko [9:42 PM]
https://btcchat.slack.com/archives/G583BUJ7J/p1494494570144806  does this link work here ?

elliotolds [9:42 PM]
@bitsko yep


travin
[10:06 PM]
Keeping a pastebin here - `https://pastebin.com/U6X62eXN` Please note that image uploads are not included. Links are though.


----- May 12th -----
zbingledack [12:31 AM]
@elliotolds It's not deference, just interest in establishing priors for how seriously to investigate his other claims. This seems like it could be a more straightforward case than the Segwit critique, so I thought it worth delving into.

csw [2:22 AM]
All wrong sorry. The SM needs to wait until after the H miner has released.

There are issues with the gamma and the lack of a model based on k-shell network graphs and the issue of thinking nodes are the same as vertices.

The paper will come out in a matter of weeks.

[2:23]
I will discuss in a couple weeks. Sorry, but I have zero time today.

[2:24]
Please have a read of:
"Standard Deviations Flawed Assumptions Tortured Data and other ways to lie with Statistics"

Gary Smith

[2:25]
And no, it is not independent.

[2:26]
The argument that a SM waits and then releases following the HM means that they are always conditional.

Conditional probability seems to really by counter intuitive to most people.

zbingledack [3:26 AM]
Could you (or someone) just explain in words what probability is being referred to by P(SS|H)? Or what condition "|H" refers to?

Is it, "The probability that the selfish miner mines two blocks in a row, starting at a time when the honest mining contingent has just mined a block"?


cypherblock [6:31 AM]
uploaded this image: Screen Shot 2017-05-12 at 6.30.43 AM.png
  Add Comment

zbingledack [11:26 AM]
Yeah I looked up the notation beforehand. Just need the context of what specific events are being referred to in what scenario. (edited)

[11:29]
That is, what _precisely_ do we mean by SS and H in "P(SS|H)" and when exactly is this claimed to apply within the selfish mining scenarios?

freetrader [4:10 PM]
@zbingledack : why not invite Emin to discuss? (edited)
2 replies Last reply 4 days ago View thread

freetrader [4:11 PM]
or Ittay ?

elliotolds [5:20 PM]
I suggest figuring out what the argument actually is before inviting anyone else to discuss


freetrader [5:37 PM]
that is probably a good idea.

davids [8:02 PM]
joined selfish-mining by invitation from @bitsko


----- May 13th -----
csw [4:33 AM]
There are many flawed assumptions in the SM paper.There are around 4 papers in peer review at the moment. The nature of how the network works, the assumptions and more are incorrect.

Theory is fine, as long as it is tested post the theory with data.

deadalnix [4:37 AM]
@csw That'd be simpler if you would point out what assumption is flawed and how specifically.

csw [4:40 AM]
Most of what people think about Bitcoin, nodes etc

[4:41]
https://books.google.co.uk/books/about/Complex_Social_Networks.html?id=CEVSnW-cPnsC&source=kp_cover&redir_esc=y
Google Books
Complex Social Networks
This 2007 book provides a systematic and self-contained account of the fast-developing theory of complex social networks. Social networks are central to the understanding of most socio-economic phenomena in the modern world. The classical approach to studying them relies on a methodology that abstracts from their size and complexity. In contrast, the approach taken in this book keeps complexity at the core, whilst integrating it with the incentive considerations that are preeminent in traditional Show more…


[4:41]
That book is a good start.

[4:41]
The Kappa of Bitcoin connectivity is not what people see.

zbingledack [12:45 PM]
@freetrader Certainly fine to invite them in but yeah it seems like it may be a waste until we have the argument nailed down

csw [4:07 PM]
Let me help you slowly on this.

I will pull it all apart axiom by axiom. It seems my expectations are never clear enough for people.

[4:08]
Let use start with 6.1 on the topic of Gamma
https://www.cs.cornell.edu/~ie53/publications/btcProcFC.pdf

[4:09]
"Because selfish mining is reactive, and it springs into action only after the
honest nodes have discovered a block X, it may seem to be at a disadvantage.
But a savvy pool operator can perform a sybil attack on honest miners by adding
a significant number of zero-power miners to the Bitcoin miner network. These
virtual miners act as advance sensors by participating in data dissemination,
but do not mine new blocks"

We call these nodes, they are wallets, but leave that for now.

[4:11]
Gamma is set and modeled using using a low kappa Power Distribution network. Bitcoin is an extremely high kappa Poisson distributed network.

[4:13]
To see this:

Model the node connectivity.
1. Use a Kamada-Kawai algorithm for the layout.
2. Model distance and the spreading conditions

[4:14]
If you make the wrong assumption about a network, you get horribly wrong results.

If you make the assumption that shares trade using a Gaussian distribution in place of a power law, then we see large hedge traders (Long Term Capital) get into trouble.

[4:15]
The maths can be "good" but if it is the wrong maths modeling the wrong system, what does it matter?

csw [4:21 PM]
Let me start you with a little foundational reading
https://arxiv.org/pdf/cond-mat/9903357.pdf

[4:23]
*Key assumption 1 - Selfish Miners*
_"The virtual miners are managed by the pool, and once they hear of block X, they ignore it and start propagating block P. The random
peer-to-peer structure of the Bitcoin overlay network will eventually propagate X to all miners, but the propagation of X under these conditions will be strictly slower than that of block P."_

Here the addition of a small number of nodes we start to impact the network.

_"By adding enough virtual nodes, the pool operator can thus increase γ"_.

What is *enough*?

csw [4:31 PM]
In the SM Model, these are pool members that do not mine. They are a Babaioff et al (2012) Sybil.

Babaioff, M., Dobzinski, S., Oren, S., Zohar, A.: On Bitcoin and red balloons. In: ACM Conference
on Electronic Commerce. pp. 56–73 (2012)
http://delivery.acm.org/10.1145/2230000/2229022/p56-babaioff.pdf?ip=81.148.139.225&id=2229022&acc=ACTIVE%20SERVICE&key=4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E5020CFA9523E86D4%2E4D4702B0C3E38B35&CFID=622263&CFTOKEN=98972933&__acm__=1494707443_903c80bb8dc201ee247ff83f4331a5f9

Does everyone follow thus far?

The SM authors have used depth and gamma that is incorrect, but at least (on p 61) Babaioff et al (2012) _"There is no Sybil-proof reward scheme in which information propagation and no duplication are dominant strategy for all nodes at depth 3 or less."_ Babaioff  et al (2012) have considered the effect of distinct depths in the system.

For some depths, the network was demonstrated to be Sybil proof. The statement is that _"Notice that this scheme exhibits in equilibrium low overhead, Sybil proofness, and provides the nodes with an incentive to propagate information."_ (edited)

[4:32]
So, can be see that it is extremely important first of all to have a correct model of the network?

[4:33]
Modeling the wrong network means little - this is a paper on the Bitcoin network, not a separate and distinct free scale network graph.

[4:34]
@freetrader @zbingledack @peter_r @elliotolds et...

This is a start

[4:37]
So, to make this blindingly clear and to help you see the first of MANY flawed assumptions and models, can we agree that the mathematics of the wrong model does not matter, correct or not, I am unconcerned. If you chose the wrong model, you will be wrong.


csw [4:45 PM]
*Questions to see if any of you know:*
1. What is the condition for "unbounded spread" in Bitcoin?
2. How many Giant components does Bitcoin exhibit?
3. How many edges / vertices are there in the Bitcoin network? Is this directed or undirected?
4. What is the Eigenvector of the network and nodes?

These questions are critical, if you cannot answer these, you cannot start to understand the system in Bitcoin.

PS - I know all these answers and have empirically validated the theory AFTER the theory and not mined to find data that suits my point.

Centrality is good in network propagation, it is not the same as centralisation. I suggest that people read up on this.

Now, what is the Normailised Closeness Centrality of Bitcoin?
Between-ness centrality?
Eigen Vector Centrality?

[4:45]
@vlad2vlad @satoshi @mwilcox

satoshi [4:45 PM]
joined selfish-mining by invitation from @csw, along with @mwilcox

csw [4:47 PM]
So.... you are supporting a paper that has NOT even noted the nature of the network. That has not defined why the increase of node count increases Gamma and I have not even gotten to conditional probability and the various real and deep issues in this paper.


vlad2vlad [4:51 PM]
This is really in-depth!!!  Wonder if we have any miners in here.

csw [4:51 PM]
To work, the addition of these gamma altering Sibyls needs to alter Gamma.

We are adding nodes that in their introduction are acting in a manner analogous to the deletion of a random node. This is a consequence of vertices reconnecting to this new Sibyl node and redefining the network.  Even in a low connectivity Barabási form network would we see a dramatic increase in mean-shortest path length (or a dramatic decrease in the clustering coefficient). It is rare for this and Bitcoin is far more robust than a Small world model

[4:52]
This is one of the minor points that in itself invalidates the entirety of this (Selfish Miner) paper.

[4:52]
As you get each of these points, I will have you go deeper into the rabbit hole that is Bitcoin and you can learn why this for of attack is not feasible.


[4:53]
That is all for now....

Your turn to look at and answer those questions I posited and see why they are crucial for this discussion and why this paper needed to have done this.

xhiggy [4:53 PM]
Thanks

csw [4:54 PM]
Also, please let me know if this is suitably referenced... I can add far more, but it is enough for me.

vlad2vlad [4:57 PM]
Oh I think it's enough!  :).   Thanks!!!

cryptorebel [5:29 PM]
this is amazingly fascinating

btcalbin [9:26 PM]
joined selfish-mining by invitation from @bitsko

zbingledack [11:51 PM]
Thanks, Craig. That is a lot to consider and - for me at least - a lot of new topics to study before I can make any judgments.

[11:53]
Perhaps some who are more familiar with the base concepts can help bring the group up to speed, piece by piece, perhaps starting with the basics of network morphology, explaining kappa and gamma, etc. ("Gamma is set and modeled using using a low kappa Power Distribution network. Bitcoin is an extremely high kappa Poisson distributed network.") (edited)


----- May 14th -----
csw [3:14 AM]
This is only one of the issues. People are making assumptions and models with no relation to the system.

They could make a hypothesis of the system and test that, but where is the sensation in it?

elliotolds [5:35 AM]
csw's description does not make sense to me. csw says "Gamma is set and modeled using using a low kappa Power Distribution network. Bitcoin is an extremely high kappa Poisson distributed network."   The selfish mining paper doesn't model the network topology. Gamma is just the fraction of honest miners who mine on the selfish miner's block. csw seems to think that in practice gamma will be low, but the selfish mining paper still shows selfish mining is profitable when gamma is 0.

csw [5:59 AM]
And that is also wrong, but I am doing one point at a time.

[5:59]
Seeing as you keep jumping, it just means nothing is sorted.

[6:00]
Start looking at each area.

[6:00]
Lets work on the first assumption. Then we will work on what the inconsistencies with the rest of the model.

[6:01]
At no point have I said, this is it - now it is all wrong. I am TRYING to step people through an extremely complex area and have them learn step by step.


[6:03]
And again, the model is wrong, but until you start to actially understand the nature of the network, then it is no good starting to debate the maths.

If you have the wrong model, it does not matter how good the maths is... And it is also flawed, but one thing at a time.

[6:03]
IFF Bitcoin was a low kappa power law system, then they would have some argument. It is not.

csw [6:17 AM]
@elliotolds
What is the effect of changing from a Poisson to a Power law network?
What is the distance and how does this change centrality measurements?
In Babaioff (2012) what was the study of Gamma referring to?
How were these findings used in the Selfish Miner paper?
If Bitcoin was degraded to a Small World system, what would change?

Babaioff, M., Dobzinski, S., Oren, S., Zohar, A.: On Bitcoin and red balloons. In: ACM Conference
on Electronic Commerce. pp. 56–73 (2012)

[6:18]
If you cannot answer those questions, you cannot even start to answer the assumptions used in SM.

[6:23]
Let's think on Percolation theory...

It the propagation of blocks and Transactions *Subcritical* or *supercritical* or something in between?

[6:23]
What is the Sharp threshold for Bitcoin's Percolation model?

[6:24]
A little more reading:

Smirnov, Stanislav (2001). "Critical percolation in the plane: conformal invariance, Cardy's formula, scaling limits". Comptes Rendus de l'Académie des Sciences - Series I - Mathematics. 333 (3): 239–244.

[6:28]
Next, to go back to why this matters...

The first transaction released is on an open network with large inter-connectivity.  It is an SEIR-C model and SIR as a simplification. The nodes that have received a block are not.

Once you have recieved a TX, that node is now immunised from a competing block.

Node response:
 1 Receive a valid TX or Block
 2 Reject an equal but competing block that comes even a second later

As the nodes reject the new block, they do not forward it

This means that the distance and centrality values for the first node to release are NOT the same as the later ones.

You cannot model the transmission using the same formula and data... Bad assumptions lead to bad models. It is simpler, but wrong.