API tools faq
paste
Advertisement
Guest User

ver2

a guest
Apr 22nd, 2016
76
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 7.37 KB | None | 0 0
raw download clone embed print report
  1. **You are a victim of [branch prediction][1] fail.**
  2.  
  3. ----------
  4.  
  5. ##What is Branch Prediction?
  6.  
  7. Consider a railroad junction:
  8.  
  9. [![][2]][5]
  10. <sub>[Image][5] by Mecanismo, via Wikimedia Commons. Used under the [CC-By-SA 3.0](//creativecommons.org/licenses/by-sa/3.0/deed.en) license.</sub>
  11.  
  12. Now for the sake of argument, suppose this is back in the 1800s - before long distance or radio communication.
  13.  
  14. You are the operator of a junction and you hear a train coming. You have no idea which way it is supposed to go. You stop the train to ask the driver which direction they want. And then you set the switch appropriately.
  15.  
  16. *Trains are heavy and have a lot of inertia. So they take forever to start up and slow down.*
  17.  
  18. Is there a better way? You guess which direction the train will go!
  19.  
  20. - If you guessed right, it continues on.
  21. - If you guessed wrong, the captain will stop, back up, and yell at you to flip the switch. Then it can restart down the other path.
  22.  
  23. **If you guess right every time**, the train will never have to stop.<br>
  24. **If you guess wrong too often**, the train will spend a lot of time stopping, backing up, and restarting.
  25.  
  26. ----------
  27.  
  28. **Consider an if-statement:** At the processor level, it is a branch instruction:
  29.  
  30. ![enter image description here][3]
  31.  
  32. You are a processor and you see a branch. You have no idea which way it will go. What do you do? You halt execution and wait until the previous instructions are complete. Then you continue down the correct path.
  33.  
  34. *Modern processors are complicated and have long pipelines. So they take forever to "warm up" and "slow down".*
  35.  
  36. Is there a better way? You guess which direction the branch will go!
  37.  
  38. - If you guessed right, you continue executing.
  39. - If you guessed wrong, you need to flush the pipeline and roll back to the branch. Then you can restart down the other path.
  40.  
  41. **If you guess right every time**, the execution will never have to stop.<br>
  42. **If you guess wrong too often**, you spend a lot of time stalling, rolling back, and restarting.
  43.  
  44. ----------
  45.  
  46. This is branch prediction. I admit it's not the best analogy since the train could just signal the direction with a flag. But in computers, the processor doesn't know which direction a branch will go until the last moment.
  47.  
  48. So how would you strategically guess to minimize the number of times that the train must back up and go down the other path? You look at the past history! If the train goes left 99% of the time, then you guess left. If it alternates, then you alternate your guesses. If it goes one way every 3 times, you guess the same...
  49.  
  50. ***In other words, you try to identify a pattern and follow it.*** This is more or less how branch predictors work.
  51.  
  52. Most applications have well-behaved branches. So modern branch predictors will typically achieve >90% hit rates. But when faced with unpredictable branches with no recognizable patterns, branch predictors are virtually useless.
  53.  
  54. Further reading: ["Branch predictor" article on Wikipedia][1].
  55.  
  56. ----------
  57.  
  58. ##As hinted from above, the culprit is this if-statement:
  59.  
  60. if (data[c] >= 128)
  61. sum += data[c];
  62.  
  63. Notice that the data is evenly distributed between 0 and 255.
  64. When the data is sorted, roughly the first half of the iterations will not enter the if-statement. After that, they will all enter the if-statement.
  65.  
  66. This is very friendly to the branch predictor since the branch consecutively goes the same direction many times.
  67. Even a simple saturating counter will correctly predict the branch except for the few iterations after it switches direction.
  68.  
  69. **Quick visualization:**
  70.  
  71. T = branch taken
  72. N = branch not taken
  73.  
  74. data[] = 0, 1, 2, 3, 4, ... 126, 127, 128, 129, 130, ... 250, 251, 252, ...
  75. branch = N N N N N ... N N T T T ... T T T ...
  76.  
  77. = NNNNNNNNNNNN ... NNNNNNNTTTTTTTTT ... TTTTTTTTTT (easy to predict)
  78.  
  79. However, when the data is completely random, the branch predictor is rendered useless because it can't predict random data.
  80. Thus there will probably be around 50% misprediction. (no better than random guessing)
  81.  
  82. data[] = 226, 185, 125, 158, 198, 144, 217, 79, 202, 118, 14, 150, 177, 182, 133, ...
  83. branch = T, T, N, T, T, T, T, N, T, N, N, T, T, T, N ...
  84.  
  85. = TTNTTTTNTNNTTTN ... (completely random - hard to predict)
  86.  
  87.  
  88. ----------
  89.  
  90. **So what can be done?**
  91.  
  92. If the compiler isn't able to optimize the branch into a conditional move, you can try some hacks if you are willing to sacrifice readability for performance.
  93.  
  94. Replace:
  95.  
  96. if (data[c] >= 128)
  97. sum += data[c];
  98.  
  99. with:
  100.  
  101. int t = (data[c] - 128) >> 31;
  102. sum += ~t & data[c];
  103.  
  104. This eliminates the branch and replaces it with some bitwise operations.
  105.  
  106. <sub>(Note that this hack is not strictly equivalent to the original if-statement. But in this case, it's valid for all the input values of `data[]`.)</sub>
  107.  
  108. **Benchmarks: Core i7 920 @ 3.5 GHz**
  109.  
  110. C++ - Visual Studio 2010 - x64 Release
  111.  
  112. // Branch - Random
  113. seconds = 11.777
  114.  
  115. // Branch - Sorted
  116. seconds = 2.352
  117.  
  118. // Branchless - Random
  119. seconds = 2.564
  120.  
  121. // Branchless - Sorted
  122. seconds = 2.587
  123.  
  124. Java - Netbeans 7.1.1 JDK 7 - x64
  125.  
  126. // Branch - Random
  127. seconds = 10.93293813
  128.  
  129. // Branch - Sorted
  130. seconds = 5.643797077
  131.  
  132. // Branchless - Random
  133. seconds = 3.113581453
  134.  
  135. // Branchless - Sorted
  136. seconds = 3.186068823
  137.  
  138. Observations:
  139.  
  140. - **With the Branch:** There is a huge difference between the sorted and unsorted data.
  141. - **With the Hack:** There is no difference between sorted and unsorted data.
  142. - In the C++ case, the hack is actually a tad slower than with the branch when the data is sorted.
  143.  
  144. A general rule of thumb is to avoid data-dependent branching in critical loops. (such as in this example)
  145.  
  146. ----------
  147.  
  148.  
  149. **Update :**
  150.  
  151. - GCC 4.6.1 with `-O3` or `-ftree-vectorize` on x64 is able to generate a conditional move. So there is no difference between the sorted and unsorted data - both are fast.
  152.  
  153. - VC++ 2010 is unable to generate conditional moves for this branch even under `/Ox`.
  154.  
  155. - Intel Compiler 11 does something miraculous. It [interchanges the two loops][4], thereby hoisting the unpredictable branch to the outer loop. So not only is it immune the mispredictions, it is also twice as fast as whatever VC++ and GCC can generate! In other words, ICC took advantage of the test-loop to defeat the benchmark...
  156.  
  157. - If you give the Intel Compiler the branchless code, it just out-right vectorizes it... and is just as fast as with the branch (with the loop interchange).
  158.  
  159. - 32bit ARM code contains conditional bits in most instructions to make it possible to eliminate branching in short cycles at all. The innermost loop of C++ code will result in just two instructions for ARMv4 and later: one CMP instruction which updates the ALU result flags and one SUM instruction with according bit being set. [More about this](http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0473e/BABCHFCB.html)
  160.  
  161. This goes to show that even mature modern compilers can vary wildly in their ability to optimize code...
  162.  
  163.  
  164. [1]: //en.wikipedia.org/wiki/Branch_predictor
  165. [2]: //i.stack.imgur.com/muxnt.jpg
  166. [3]: //i.stack.imgur.com/pyfwC.png
  167. [4]: //en.wikipedia.org/wiki/Loop_interchange
  168. [5]: //commons.wikimedia.org/wiki/File:Entroncamento_do_Transpraia.JPG
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!