API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Jan 31st, 2020
305
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 54.58 KB | None | 0 0
raw download clone embed print report
  1. bazar@ao756:~/apt-p2p$ apt-p2p.py
  2. Traceback (most recent call last):
  3. File "/usr/local/bin/apt-p2p.py", line 15, in <module>
  4. from twisted.application import service, internet, app, strports
  5. ImportError: No module named twisted.application
  6. bazar@ao756:~/apt-p2p$ pip install twisted
  7. Collecting twisted
  8. Downloading https://files.pythonhosted.org/packages/06/28/2a433e147de68c8416aa0179c45e67b67161f5c0f24aaaf1723f6229f574/Twisted-19.10.0-cp37-cp37m-manylinux1_x86_64.whl (3.1MB)
  9. 100% |████████████████████████████████| 3.1MB 53kB/s
  10. Collecting hyperlink>=17.1.1 (from twisted)
  11. Downloading https://files.pythonhosted.org/packages/7f/91/e916ca10a2de1cb7101a9b24da546fb90ee14629e23160086cf3361c4fb8/hyperlink-19.0.0-py2.py3-none-any.whl
  12. Collecting Automat>=0.3.0 (from twisted)
  13. Downloading https://files.pythonhosted.org/packages/e5/11/756922e977bb296a79ccf38e8d45cafee446733157d59bcd751d3aee57f5/Automat-0.8.0-py2.py3-none-any.whl
  14. Collecting incremental>=16.10.1 (from twisted)
  15. Downloading https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl
  16. Collecting attrs>=17.4.0 (from twisted)
  17. Downloading https://files.pythonhosted.org/packages/a2/db/4313ab3be961f7a763066401fb77f7748373b6094076ae2bda2806988af6/attrs-19.3.0-py2.py3-none-any.whl
  18. Collecting constantly>=15.1 (from twisted)
  19. Downloading https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl
  20. Collecting PyHamcrest>=1.9.0 (from twisted)
  21. Downloading https://files.pythonhosted.org/packages/ac/6c/a641af18e416e6501c10b03742387176626a1d48196100160df796f36632/PyHamcrest-2.0.0-py3-none-any.whl (51kB)
  22. 100% |████████████████████████████████| 61kB 822kB/s
  23. Collecting zope.interface>=4.4.2 (from twisted)
  24. Downloading https://files.pythonhosted.org/packages/fc/1f/eeda511f9d857b63e248bbb2084f04e6c6a1863901954ba0872965c0895c/zope.interface-4.7.1-cp37-cp37m-manylinux1_x86_64.whl (169kB)
  25. 100% |████████████████████████████████| 174kB 25kB/s
  26. Requirement already satisfied: idna>=2.5 in /usr/lib/python3/dist-packages (from hyperlink>=17.1.1->twisted) (2.6)
  27. Requirement already satisfied: six in /usr/lib/python3/dist-packages (from Automat>=0.3.0->twisted) (1.13.0)
  28. Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from zope.interface>=4.4.2->twisted) (44.0.0)
  29. Installing collected packages: hyperlink, attrs, Automat, incremental, constantly, PyHamcrest, zope.interface, twisted
  30. Successfully installed Automat-0.8.0 PyHamcrest-2.0.0 attrs-19.3.0 constantly-15.1.0 hyperlink-19.0.0 incremental-17.5.0 twisted-19.10.0 zope.interface-4.7.1
  31. bazar@ao756:~/apt-p2p$ apt-p2p.py
  32. Traceback (most recent call last):
  33. File "/usr/local/bin/apt-p2p.py", line 15, in <module>
  34. from twisted.application import service, internet, app, strports
  35. ImportError: No module named twisted.application
  36. bazar@ao756:~/apt-p2p$ pip install twisted.application
  37. Collecting twisted.application
  38. Could not install packages due to an EnvironmentError: 404 Client Error: Not Found for url: https://pypi.org/simple/twisted-application/
  39.  
  40. bazar@ao756:~/apt-p2p$ sudo apt install python-twisted
  41. [sudo] пароль для bazar:
  42. Чтение списков пакетов… Готово
  43. Построение дерева зависимостей
  44. Чтение информации о состоянии… Готово
  45. Будут установлены следующие дополнительные пакеты:
  46. python-asn1crypto python-attr python-automat python-cffi-backend python-constantly
  47. python-cryptography python-enum34 python-hamcrest python-hyperlink python-idna
  48. python-incremental python-ipaddress python-openssl python-pyasn1 python-pyasn1-modules
  49. python-service-identity python-twisted-bin python-twisted-core python-zope.interface
  50. Предлагаемые пакеты:
  51. python-attr-doc python-cryptography-doc python-cryptography-vectors python-enum34-doc
  52. python-openssl-doc python-openssl-dbg python-twisted-bin-dbg python-pampy python-qt3
  53. python-serial python-wxgtk3.0
  54. Следующие НОВЫЕ пакеты будут установлены:
  55. python-asn1crypto python-attr python-automat python-cffi-backend python-constantly
  56. python-cryptography python-enum34 python-hamcrest python-hyperlink python-idna
  57. python-incremental python-ipaddress python-openssl python-pyasn1 python-pyasn1-modules
  58. python-service-identity python-twisted python-twisted-bin python-twisted-core
  59. python-zope.interface
  60. Обновлено 0 пакетов, установлено 20 новых пакетов, для удаления отмечено 0 пакетов, и 3 пакетов не обновлено.
  61. Необходимо скачать 2 487 kB/2 841 kB архивов.
  62. После данной операции объём занятого дискового пространства возрастёт на 18,6 MB.
  63. Хотите продолжить? [Д/н]
  64. Пол:1 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-asn1crypto all 0.24.0-1build1 [72,8 kB]
  65. Пол:2 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-attr all 18.2.0-1build1 [29,7 kB]
  66. Пол:3 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-automat all 0.8.0-0ubuntu2 [27,1 kB]
  67. Пол:4 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-constantly all 15.1.0-1build1 [8 168 B]
  68. Пол:5 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-ipaddress all 1.0.17-1build1 [18,4 kB]
  69. Пол:6 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-hamcrest all 1.9.0-2 [24,7 kB]
  70. Пол:7 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-idna all 2.6-2build1 [32,7 kB]
  71. Пол:8 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-hyperlink all 19.0.0-1 [33,5 kB]
  72. Пол:9 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-incremental all 16.10.1-3.1 [14,8 kB]
  73. Пол:10 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-openssl all 19.0.0-1build1 [43,1 kB]
  74. Пол:11 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-pyasn1 all 0.4.2-3build1 [46,6 kB]
  75. Пол:12 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-pyasn1-modules all 0.2.1-0.2build1 [32,8 kB]
  76. Пол:13 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-service-identity all 18.1.0-5build1 [10,6 kB]
  77. Пол:14 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-twisted-bin amd64 18.9.0-6 [15,2 kB]
  78. Пол:15 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-zope.interface amd64 4.6.0-4 [84,8 kB]
  79. Пол:16 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-twisted-core all 18.9.0-6 [1 989 kB]
  80. Пол:17 http://ru.archive.ubuntu.com/ubuntu focal/universe amd64 python-twisted all 18.9.0-6 [3 728 B]
  81. Получено 2 487 kB за 16с (159 kB/s)
  82. Выбор ранее не выбранного пакета python-asn1crypto.
  83. (Чтение базы данных … на данный момент установлено 472853 файла и каталога.)
  84. Подготовка к распаковке …/00-python-asn1crypto_0.24.0-1build1_all.deb …
  85. Распаковывается python-asn1crypto (0.24.0-1build1) …
  86. Выбор ранее не выбранного пакета python-attr.
  87. Подготовка к распаковке …/01-python-attr_18.2.0-1build1_all.deb …
  88. Распаковывается python-attr (18.2.0-1build1) …
  89. Выбор ранее не выбранного пакета python-automat.
  90. Подготовка к распаковке …/02-python-automat_0.8.0-0ubuntu2_all.deb …
  91. Распаковывается python-automat (0.8.0-0ubuntu2) …
  92. Выбор ранее не выбранного пакета python-cffi-backend.
  93. Подготовка к распаковке …/03-python-cffi-backend_1.13.2-1_amd64.deb …
  94. Распаковывается python-cffi-backend (1.13.2-1) …
  95. Выбор ранее не выбранного пакета python-constantly.
  96. Подготовка к распаковке …/04-python-constantly_15.1.0-1build1_all.deb …
  97. Распаковывается python-constantly (15.1.0-1build1) …
  98. Выбор ранее не выбранного пакета python-enum34.
  99. Подготовка к распаковке …/05-python-enum34_1.1.6-2ubuntu1_all.deb …
  100. Распаковывается python-enum34 (1.1.6-2ubuntu1) …
  101. Выбор ранее не выбранного пакета python-ipaddress.
  102. Подготовка к распаковке …/06-python-ipaddress_1.0.17-1build1_all.deb …
  103. Распаковывается python-ipaddress (1.0.17-1build1) …
  104. Выбор ранее не выбранного пакета python-cryptography.
  105. Подготовка к распаковке …/07-python-cryptography_2.6.1-4ubuntu1_amd64.deb …
  106. Распаковывается python-cryptography (2.6.1-4ubuntu1) …
  107. Выбор ранее не выбранного пакета python-hamcrest.
  108. Подготовка к распаковке …/08-python-hamcrest_1.9.0-2_all.deb …
  109. Распаковывается python-hamcrest (1.9.0-2) …
  110. Выбор ранее не выбранного пакета python-idna.
  111. Подготовка к распаковке …/09-python-idna_2.6-2build1_all.deb …
  112. Распаковывается python-idna (2.6-2build1) …
  113. Выбор ранее не выбранного пакета python-hyperlink.
  114. Подготовка к распаковке …/10-python-hyperlink_19.0.0-1_all.deb …
  115. Распаковывается python-hyperlink (19.0.0-1) …
  116. Выбор ранее не выбранного пакета python-incremental.
  117. Подготовка к распаковке …/11-python-incremental_16.10.1-3.1_all.deb …
  118. Распаковывается python-incremental (16.10.1-3.1) …
  119. Выбор ранее не выбранного пакета python-openssl.
  120. Подготовка к распаковке …/12-python-openssl_19.0.0-1build1_all.deb …
  121. Распаковывается python-openssl (19.0.0-1build1) …
  122. Выбор ранее не выбранного пакета python-pyasn1.
  123. Подготовка к распаковке …/13-python-pyasn1_0.4.2-3build1_all.deb …
  124. Распаковывается python-pyasn1 (0.4.2-3build1) …
  125. Выбор ранее не выбранного пакета python-pyasn1-modules.
  126. Подготовка к распаковке …/14-python-pyasn1-modules_0.2.1-0.2build1_all.deb …
  127. Распаковывается python-pyasn1-modules (0.2.1-0.2build1) …
  128. Выбор ранее не выбранного пакета python-service-identity.
  129. Подготовка к распаковке …/15-python-service-identity_18.1.0-5build1_all.deb …
  130. Распаковывается python-service-identity (18.1.0-5build1) …
  131. Выбор ранее не выбранного пакета python-twisted-bin:amd64.
  132. Подготовка к распаковке …/16-python-twisted-bin_18.9.0-6_amd64.deb …
  133. Распаковывается python-twisted-bin:amd64 (18.9.0-6) …
  134. Выбор ранее не выбранного пакета python-zope.interface.
  135. Подготовка к распаковке …/17-python-zope.interface_4.6.0-4_amd64.deb …
  136. Распаковывается python-zope.interface (4.6.0-4) …
  137. Выбор ранее не выбранного пакета python-twisted-core.
  138. Подготовка к распаковке …/18-python-twisted-core_18.9.0-6_all.deb …
  139. Распаковывается python-twisted-core (18.9.0-6) …
  140. Выбор ранее не выбранного пакета python-twisted.
  141. Подготовка к распаковке …/19-python-twisted_18.9.0-6_all.deb …
  142. Распаковывается python-twisted (18.9.0-6) …
  143. Настраивается пакет python-enum34 (1.1.6-2ubuntu1) …
  144. Настраивается пакет python-asn1crypto (0.24.0-1build1) …
  145. Настраивается пакет python-attr (18.2.0-1build1) …
  146. Настраивается пакет python-zope.interface (4.6.0-4) …
  147. Настраивается пакет python-pyasn1 (0.4.2-3build1) …
  148. Настраивается пакет python-idna (2.6-2build1) …
  149. Настраивается пакет python-pyasn1-modules (0.2.1-0.2build1) …
  150. Настраивается пакет python-twisted-bin:amd64 (18.9.0-6) …
  151. Настраивается пакет python-hamcrest (1.9.0-2) …
  152. Настраивается пакет python-incremental (16.10.1-3.1) …
  153. Настраивается пакет python-constantly (15.1.0-1build1) …
  154. Настраивается пакет python-hyperlink (19.0.0-1) …
  155. Настраивается пакет python-ipaddress (1.0.17-1build1) …
  156. Настраивается пакет python-cffi-backend (1.13.2-1) …
  157. Настраивается пакет python-automat (0.8.0-0ubuntu2) …
  158. Настраивается пакет python-cryptography (2.6.1-4ubuntu1) …
  159. Настраивается пакет python-openssl (19.0.0-1build1) …
  160. Настраивается пакет python-service-identity (18.1.0-5build1) …
  161. Настраивается пакет python-twisted-core (18.9.0-6) …
  162. Настраивается пакет python-twisted (18.9.0-6) …
  163. Обрабатываются триггеры для man-db (2.9.0-2) …
  164. bazar@ao756:~/apt-p2p$ pip install twisted.application
  165. Collecting twisted.application
  166. Could not install packages due to an EnvironmentError: 404 Client Error: Not Found for url: https://pypi.org/simple/twisted-application/
  167.  
  168. bazar@ao756:~/apt-p2p$ apt-p2p.py
  169. 2020-01-31 15:04:08+0300 [-] Log opened.
  170. 2020-01-31 15:04:08+0300 [-] Loading config files: '/etc/apt-p2p/apt-p2p.conf', '/home/bazar/.apt-p2p/apt-p2p.conf', ''
  171. 2020-01-31 15:04:08+0300 [-] Successfully loaded config files: ''
  172. 2020-01-31 15:04:08+0300 [-] Starting application with uid/gid None/None
  173. 2020-01-31 15:04:08+0300 [-] Traceback (most recent call last):
  174. 2020-01-31 15:04:08+0300 [-] File "/usr/local/bin/apt-p2p.py", line 68, in <module>
  175. 2020-01-31 15:04:08+0300 [-] DHT = __import__(config.get('DEFAULT', 'DHT')+'.DHT', globals(), locals(), ['DHT'])
  176. 2020-01-31 15:04:08+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/DHT.py", line 17, in <module>
  177. 2020-01-31 15:04:08+0300 [-] from khashmir import Khashmir
  178. 2020-01-31 15:04:08+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/khashmir.py", line 23, in <module>
  179. 2020-01-31 15:04:08+0300 [-] from db import DB
  180. 2020-01-31 15:04:08+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/db.py", line 5, in <module>
  181. 2020-01-31 15:04:08+0300 [-] from pysqlite2 import dbapi2 as sqlite
  182. 2020-01-31 15:04:08+0300 [-] ImportError: No module named pysqlite2
  183. bazar@ao756:~/apt-p2p$ pip install pysqlite2
  184. Collecting pysqlite2
  185. Could not install packages due to an EnvironmentError: 404 Client Error: Not Found for url: https://pypi.org/simple/pysqlite2/
  186.  
  187. bazar@ao756:~/apt-p2p$ sudo apt install libsqlite3-dev
  188. Чтение списков пакетов… Готово
  189. Построение дерева зависимостей
  190. Чтение информации о состоянии… Готово
  191. Предлагаемые пакеты:
  192. sqlite3-doc
  193. Следующие НОВЫЕ пакеты будут установлены:
  194. libsqlite3-dev
  195. Обновлено 0 пакетов, установлено 1 новых пакетов, для удаления отмечено 0 пакетов, и 3 пакетов не обновлено.
  196. Необходимо скачать 684 kB архивов.
  197. После данной операции объём занятого дискового пространства возрастёт на 2 332 kB.
  198. Пол:1 http://ru.archive.ubuntu.com/ubuntu focal/main amd64 libsqlite3-dev amd64 3.30.1-1ubuntu1 [684 kB]
  199. Получено 684 kB за 3с (222 kB/s)
  200. Выбор ранее не выбранного пакета libsqlite3-dev:amd64.
  201. (Чтение базы данных … на данный момент установлено 474374 файла и каталога.)
  202. Подготовка к распаковке …/libsqlite3-dev_3.30.1-1ubuntu1_amd64.deb …
  203. Распаковывается libsqlite3-dev:amd64 (3.30.1-1ubuntu1) …
  204. Настраивается пакет libsqlite3-dev:amd64 (3.30.1-1ubuntu1) …
  205. bazar@ao756:~/apt-p2p$ apt-p2p.py
  206. 2020-01-31 15:07:01+0300 [-] Log opened.
  207. 2020-01-31 15:07:01+0300 [-] Loading config files: '/etc/apt-p2p/apt-p2p.conf', '/home/bazar/.apt-p2p/apt-p2p.conf', ''
  208. 2020-01-31 15:07:01+0300 [-] Successfully loaded config files: ''
  209. 2020-01-31 15:07:01+0300 [-] Starting application with uid/gid None/None
  210. 2020-01-31 15:07:01+0300 [-] Traceback (most recent call last):
  211. 2020-01-31 15:07:01+0300 [-] File "/usr/local/bin/apt-p2p.py", line 68, in <module>
  212. 2020-01-31 15:07:01+0300 [-] DHT = __import__(config.get('DEFAULT', 'DHT')+'.DHT', globals(), locals(), ['DHT'])
  213. 2020-01-31 15:07:01+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/DHT.py", line 17, in <module>
  214. 2020-01-31 15:07:01+0300 [-] from khashmir import Khashmir
  215. 2020-01-31 15:07:01+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/khashmir.py", line 23, in <module>
  216. 2020-01-31 15:07:01+0300 [-] from db import DB
  217. 2020-01-31 15:07:01+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/db.py", line 5, in <module>
  218. 2020-01-31 15:07:01+0300 [-] from pysqlite2 import dbapi2 as sqlite
  219. 2020-01-31 15:07:01+0300 [-] ImportError: No module named pysqlite2
  220. bazar@ao756:~/apt-p2p$ python -c 'import _sqlite3'
  221. bazar@ao756:~/apt-p2p$ apt-p2p.py
  222. 2020-01-31 15:07:14+0300 [-] Log opened.
  223. 2020-01-31 15:07:14+0300 [-] Loading config files: '/etc/apt-p2p/apt-p2p.conf', '/home/bazar/.apt-p2p/apt-p2p.conf', ''
  224. 2020-01-31 15:07:14+0300 [-] Successfully loaded config files: ''
  225. 2020-01-31 15:07:14+0300 [-] Starting application with uid/gid None/None
  226. 2020-01-31 15:07:14+0300 [-] Traceback (most recent call last):
  227. 2020-01-31 15:07:14+0300 [-] File "/usr/local/bin/apt-p2p.py", line 68, in <module>
  228. 2020-01-31 15:07:14+0300 [-] DHT = __import__(config.get('DEFAULT', 'DHT')+'.DHT', globals(), locals(), ['DHT'])
  229. 2020-01-31 15:07:14+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/DHT.py", line 17, in <module>
  230. 2020-01-31 15:07:14+0300 [-] from khashmir import Khashmir
  231. 2020-01-31 15:07:14+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/khashmir.py", line 23, in <module>
  232. 2020-01-31 15:07:14+0300 [-] from db import DB
  233. 2020-01-31 15:07:14+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/db.py", line 5, in <module>
  234. 2020-01-31 15:07:14+0300 [-] from pysqlite2 import dbapi2 as sqlite
  235. 2020-01-31 15:07:14+0300 [-] ImportError: No module named pysqlite2
  236. bazar@ao756:~/apt-p2p$ ls
  237. apt_p2p apt_p2p_Khashmir bootstrap1 debian downloader1 LICENSE.md setup.py TODO.md
  238. apt-p2p.conf apt-p2p.py build docs downloader2 README.md test.py
  239. bazar@ao756:~/apt-p2p$ cd apt_p2p_Khashmir/
  240. bazar@ao756:~/apt-p2p/apt_p2p_Khashmir$ ls
  241. actions.py db.py __init__.py khash.py krpc.py node.py util.py
  242. bencode.py DHT.py khashmir.py knode.py ktable.py stats.py
  243. bazar@ao756:~/apt-p2p/apt_p2p_Khashmir$ cat k
  244. khashmir.py khash.py knode.py krpc.py ktable.py
  245. bazar@ao756:~/apt-p2p/apt_p2p_Khashmir$ cat k
  246. khashmir.py khash.py knode.py krpc.py ktable.py
  247. bazar@ao756:~/apt-p2p/apt_p2p_Khashmir$ cat khashmir.py
  248.  
  249. """The main Khashmir program.
  250.  
  251. @var isLocal: a compiled regular expression suitable for testing if an
  252. IP address is from a known local or private range
  253. """
  254.  
  255. import warnings
  256. warnings.simplefilter("ignore", DeprecationWarning)
  257.  
  258. from datetime import datetime, timedelta
  259. from random import randrange, shuffle
  260. from sha import sha
  261. from copy import copy
  262. import os, re
  263.  
  264. from twisted.internet.defer import Deferred
  265. from twisted.internet.base import DelayedCall
  266. from twisted.internet import protocol, reactor
  267. from twisted.python import log
  268. from twisted.trial import unittest
  269.  
  270. from db import DB
  271. from ktable import KTable
  272. from knode import KNodeBase, KNodeRead, KNodeWrite, NULL_ID
  273. from khash import newID, newIDInRange
  274. from actions import FindNode, FindValue, GetValue, StoreValue
  275. from stats import StatsLogger
  276. import krpc
  277.  
  278. isLocal = re.compile('^(192\.168\.[0-9]{1,3}\.[0-9]{1,3})|'+
  279. '(10\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})|'+
  280. '(172\.0?1[6-9]\.[0-9]{1,3}\.[0-9]{1,3})|'+
  281. '(172\.0?2[0-9]\.[0-9]{1,3}\.[0-9]{1,3})|'+
  282. '(172\.0?3[0-1]\.[0-9]{1,3}\.[0-9]{1,3})|'+
  283. '(127\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})$')
  284.  
  285. class KhashmirBase(protocol.Factory):
  286. """The base Khashmir class, with base functionality and find node, no key-value mappings.
  287.  
  288. @type _Node: L{node.Node}
  289. @ivar _Node: the knode implementation to use for this class of DHT
  290. @type config: C{dictionary}
  291. @ivar config: the configuration parameters for the DHT
  292. @type pinging: C{dictionary}
  293. @ivar pinging: the node's that are currently being pinged, keys are the
  294. node id's, values are the Deferred or DelayedCall objects
  295. @type port: C{int}
  296. @ivar port: the port to listen on
  297. @type store: L{db.DB}
  298. @ivar store: the database to store nodes and key/value pairs in
  299. @type node: L{node.Node}
  300. @ivar node: this node
  301. @type table: L{ktable.KTable}
  302. @ivar table: the routing table
  303. @type token_secrets: C{list} of C{string}
  304. @ivar token_secrets: the current secrets to use to create tokens
  305. @type stats: L{stats.StatsLogger}
  306. @ivar stats: the statistics gatherer
  307. @type udp: L{krpc.hostbroker}
  308. @ivar udp: the factory for the KRPC protocol
  309. @type listenport: L{twisted.internet.interfaces.IListeningPort}
  310. @ivar listenport: the UDP listening port
  311. @type next_checkpoint: L{twisted.internet.interfaces.IDelayedCall}
  312. @ivar next_checkpoint: the delayed call for the next checkpoint
  313. """
  314.  
  315. _Node = KNodeBase
  316.  
  317. def __init__(self, config, cache_dir='/tmp'):
  318. """Initialize the Khashmir class and call the L{setup} method.
  319.  
  320. @type config: C{dictionary}
  321. @param config: the configuration parameters for the DHT
  322. @type cache_dir: C{string}
  323. @param cache_dir: the directory to store all files in
  324. (optional, defaults to the /tmp directory)
  325. """
  326. self.config = None
  327. self.pinging = {}
  328. self.setup(config, cache_dir)
  329.  
  330. def setup(self, config, cache_dir):
  331. """Setup all the Khashmir sub-modules.
  332.  
  333. @type config: C{dictionary}
  334. @param config: the configuration parameters for the DHT
  335. @type cache_dir: C{string}
  336. @param cache_dir: the directory to store all files in
  337. """
  338. self.config = config
  339. self.port = config['PORT']
  340. self.store = DB(os.path.join(cache_dir, 'khashmir.' + str(self.port) + '.db'))
  341. self.node = self._loadSelfNode('', self.port)
  342. self.table = KTable(self.node, config)
  343. self.token_secrets = [newID()]
  344. self.stats = StatsLogger(self.table, self.store)
  345.  
  346. # Start listening
  347. self.udp = krpc.hostbroker(self, self.stats, config)
  348. self.udp.protocol = krpc.KRPC
  349. self.listenport = reactor.listenUDP(self.port, self.udp)
  350.  
  351. # Load the routing table and begin checkpointing
  352. self._loadRoutingTable()
  353. self.refreshTable(force = True)
  354. self.next_checkpoint = reactor.callLater(60, self.checkpoint)
  355.  
  356. def Node(self, id, host = None, port = None):
  357. """Create a new node.
  358.  
  359. @see: L{node.Node.__init__}
  360. """
  361. n = self._Node(id, host, port)
  362. n.table = self.table
  363. n.conn = self.udp.connectionForAddr((n.host, n.port))
  364. return n
  365.  
  366. def __del__(self):
  367. """Stop listening for packets."""
  368. self.listenport.stopListening()
  369.  
  370. def _loadSelfNode(self, host, port):
  371. """Create this node, loading any previously saved one."""
  372. id = self.store.getSelfNode()
  373. if not id or not id.endswith(self.config['VERSION']):
  374. id = newID(self.config['VERSION'])
  375. return self._Node(id, host, port)
  376.  
  377. def checkpoint(self):
  378. """Perform some periodic maintenance operations."""
  379. # Create a new token secret
  380. self.token_secrets.insert(0, newID())
  381. if len(self.token_secrets) > 3:
  382. self.token_secrets.pop()
  383.  
  384. # Save some parameters for reloading
  385. self.store.saveSelfNode(self.node.id)
  386. self.store.dumpRoutingTable(self.table.buckets)
  387.  
  388. # DHT maintenance
  389. self.store.expireValues(self.config['KEY_EXPIRE'])
  390. self.refreshTable()
  391.  
  392. self.next_checkpoint = reactor.callLater(randrange(int(self.config['CHECKPOINT_INTERVAL'] * .9),
  393. int(self.config['CHECKPOINT_INTERVAL'] * 1.1)),
  394. self.checkpoint)
  395.  
  396. def _loadRoutingTable(self):
  397. """Load the previous routing table nodes from the database.
  398.  
  399. It's usually a good idea to call refreshTable(force = True) after
  400. loading the table.
  401. """
  402. nodes = self.store.getRoutingTable()
  403. for rec in nodes:
  404. n = self.Node(rec[0], rec[1], int(rec[2]))
  405. self.table.insertNode(n, contacted = False)
  406.  
  407. #{ Local interface
  408. def addContact(self, host, port, callback=None, errback=None):
  409. """Ping this node and add the contact info to the table on pong.
  410.  
  411. @type host: C{string}
  412. @param host: the IP address of the node to contact
  413. @type port: C{int}
  414. @param port:the port of the node to contact
  415. @type callback: C{method}
  416. @param callback: the method to call with the results, it must take 1
  417. parameter, the contact info returned by the node
  418. (optional, defaults to doing nothing with the results)
  419. @type errback: C{method}
  420. @param errback: the method to call if an error occurs
  421. (optional, defaults to calling the callback with the error)
  422. """
  423. n = self.Node(NULL_ID, host, port)
  424. self.sendJoin(n, callback=callback, errback=errback)
  425.  
  426. def findNode(self, id, callback):
  427. """Find the contact info for the K closest nodes in the global table.
  428.  
  429. @type id: C{string}
  430. @param id: the target ID to find the K closest nodes of
  431. @type callback: C{method}
  432. @param callback: the method to call with the results, it must take 1
  433. parameter, the list of K closest nodes
  434. """
  435. # Mark the bucket as having been accessed
  436. self.table.touch(id)
  437.  
  438. # Start with our node
  439. nodes = [copy(self.node)]
  440.  
  441. # Start the finding nodes action
  442. state = FindNode(self, id, callback, self.config, self.stats)
  443. reactor.callLater(0, state.goWithNodes, nodes)
  444.  
  445. def insertNode(self, node, contacted = True):
  446. """Try to insert a node in our local table, pinging oldest contact if necessary.
  447.  
  448. If all you have is a host/port, then use L{addContact}, which calls this
  449. method after receiving the PONG from the remote node. The reason for
  450. the separation is we can't insert a node into the table without its
  451. node ID. That means of course the node passed into this method needs
  452. to be a properly formed Node object with a valid ID.
  453.  
  454. @type node: L{node.Node}
  455. @param node: the new node to try and insert
  456. @type contacted: C{boolean}
  457. @param contacted: whether the new node is known to be good, i.e.
  458. responded to a request (optional, defaults to True)
  459. """
  460. # Don't add any local nodes to the routing table
  461. if not self.config['LOCAL_OK'] and isLocal.match(node.host):
  462. log.msg('Not adding local node to table: %s/%s' % (node.host, node.port))
  463. return
  464.  
  465. old = self.table.insertNode(node, contacted=contacted)
  466.  
  467. if (isinstance(old, self._Node) and old.id != self.node.id and
  468. (datetime.now() - old.lastSeen) >
  469. timedelta(seconds=self.config['MIN_PING_INTERVAL'])):
  470.  
  471. # Bucket is full, check to see if old node is still available
  472. df = self.sendPing(old)
  473. df.addErrback(self._staleNodeHandler, old, node, contacted)
  474. elif not old and not contacted:
  475. # There's room, we just need to contact the node first
  476. df = self.sendPing(node)
  477. # Also schedule a future ping to make sure the node works
  478. def rePing(newnode, self = self):
  479. if newnode.id not in self.pinging:
  480. self.pinging[newnode.id] = reactor.callLater(self.config['MIN_PING_INTERVAL'],
  481. self.sendPing, newnode)
  482. return newnode
  483. df.addCallback(rePing)
  484.  
  485. def _staleNodeHandler(self, err, old, node, contacted):
  486. """The pinged node never responded, so replace it."""
  487. self.table.invalidateNode(old)
  488. self.insertNode(node, contacted)
  489. return err
  490.  
  491. def nodeFailed(self, node):
  492. """Mark a node as having failed a request and schedule a future check.
  493.  
  494. @type node: L{node.Node}
  495. @param node: the new node to try and insert
  496. """
  497. exists = self.table.nodeFailed(node)
  498.  
  499. # If in the table, schedule a ping, if one isn't already sent/scheduled
  500. if exists and node.id not in self.pinging:
  501. self.pinging[node.id] = reactor.callLater(self.config['MIN_PING_INTERVAL'],
  502. self.sendPing, node)
  503.  
  504. def sendPing(self, node):
  505. """Ping the node to see if it's still alive.
  506.  
  507. @type node: L{node.Node}
  508. @param node: the node to send the join to
  509. """
  510. # Check for a ping already underway
  511. if (isinstance(self.pinging.get(node.id, None), DelayedCall) and
  512. self.pinging[node.id].active()):
  513. self.pinging[node.id].cancel()
  514. elif isinstance(self.pinging.get(node.id, None), Deferred):
  515. return self.pinging[node.id]
  516.  
  517. self.stats.startedAction('ping')
  518. df = node.ping(self.node.id)
  519. self.pinging[node.id] = df
  520. df.addCallbacks(self._pingHandler, self._pingError,
  521. callbackArgs = (node, datetime.now()),
  522. errbackArgs = (node, datetime.now()))
  523. return df
  524.  
  525. def _pingHandler(self, dict, node, start):
  526. """Node responded properly, update it and return the node object."""
  527. self.stats.completedAction('ping', start)
  528. del self.pinging[node.id]
  529. # Create the node using the returned contact info
  530. n = self.Node(dict['id'], dict['_krpc_sender'][0], dict['_krpc_sender'][1])
  531. reactor.callLater(0, self.insertNode, n)
  532. return n
  533.  
  534. def _pingError(self, err, node, start):
  535. """Error occurred, fail node."""
  536. log.msg("action ping failed on %s/%s: %s" % (node.host, node.port, err.getErrorMessage()))
  537. self.stats.completedAction('ping', start)
  538.  
  539. # Consume unhandled errors
  540. self.pinging[node.id].addErrback(lambda ping_err: None)
  541. del self.pinging[node.id]
  542.  
  543. self.nodeFailed(node)
  544. return err
  545.  
  546. def sendJoin(self, node, callback=None, errback=None):
  547. """Join the DHT by pinging a bootstrap node.
  548.  
  549. @type node: L{node.Node}
  550. @param node: the node to send the join to
  551. @type callback: C{method}
  552. @param callback: the method to call with the results, it must take 1
  553. parameter, the contact info returned by the node
  554. (optional, defaults to doing nothing with the results)
  555. @type errback: C{method}
  556. @param errback: the method to call if an error occurs
  557. (optional, defaults to calling the callback with the error)
  558. """
  559. if errback is None:
  560. errback = callback
  561. self.stats.startedAction('join')
  562. df = node.join(self.node.id)
  563. df.addCallbacks(self._joinHandler, self._joinError,
  564. callbackArgs = (node, datetime.now()),
  565. errbackArgs = (node, datetime.now()))
  566. if callback:
  567. df.addCallbacks(callback, errback)
  568.  
  569. def _joinHandler(self, dict, node, start):
  570. """Node responded properly, extract the response."""
  571. self.stats.completedAction('join', start)
  572. # Create the node using the returned contact info
  573. n = self.Node(dict['id'], dict['_krpc_sender'][0], dict['_krpc_sender'][1])
  574. reactor.callLater(0, self.insertNode, n)
  575. return (dict['ip_addr'], dict['port'])
  576.  
  577. def _joinError(self, err, node, start):
  578. """Error occurred, fail node."""
  579. log.msg("action join failed on %s/%s: %s" % (node.host, node.port, err.getErrorMessage()))
  580. self.stats.completedAction('join', start)
  581. self.nodeFailed(node)
  582. return err
  583.  
  584. def findCloseNodes(self, callback=lambda a: None):
  585. """Perform a findNode on the ID one away from our own.
  586.  
  587. This will allow us to populate our table with nodes on our network
  588. closest to our own. This is called as soon as we start up with an
  589. empty table.
  590.  
  591. @type callback: C{method}
  592. @param callback: the method to call with the results, it must take 1
  593. parameter, the list of K closest nodes
  594. (optional, defaults to doing nothing with the results)
  595. """
  596. id = self.node.id[:-1] + chr((ord(self.node.id[-1]) + 1) % 256)
  597. self.findNode(id, callback)
  598.  
  599. def refreshTable(self, force = False):
  600. """Check all the buckets for those that need refreshing.
  601.  
  602. @param force: refresh all buckets regardless of last bucket access time
  603. (optional, defaults to False)
  604. """
  605. def callback(nodes):
  606. pass
  607.  
  608. for bucket in self.table.buckets:
  609. if force or (datetime.now() - bucket.lastAccessed >
  610. timedelta(seconds=self.config['BUCKET_STALENESS'])):
  611. # Choose a random ID in the bucket and try and find it
  612. id = newIDInRange(bucket.min, bucket.max)
  613. self.findNode(id, callback)
  614.  
  615. def shutdown(self):
  616. """Closes the port and cancels pending later calls."""
  617. self.listenport.stopListening()
  618. try:
  619. self.next_checkpoint.cancel()
  620. except:
  621. pass
  622. for nodeid in self.pinging.keys():
  623. if isinstance(self.pinging[nodeid], DelayedCall) and self.pinging[nodeid].active():
  624. self.pinging[nodeid].cancel()
  625. del self.pinging[nodeid]
  626. self.store.close()
  627.  
  628. def getStats(self):
  629. """Gather the statistics for the DHT."""
  630. return self.stats.formatHTML()
  631.  
  632. #{ Remote interface
  633. def krpc_ping(self, id, _krpc_sender = None):
  634. """Pong with our ID.
  635.  
  636. @type id: C{string}
  637. @param id: the node ID of the sender node
  638. @type _krpc_sender: (C{string}, C{int})
  639. @param _krpc_sender: the sender node's IP address and port
  640. """
  641. if _krpc_sender is not None:
  642. n = self.Node(id, _krpc_sender[0], _krpc_sender[1])
  643. reactor.callLater(0, self.insertNode, n, False)
  644.  
  645. return {"id" : self.node.id}
  646.  
  647. def krpc_join(self, id, _krpc_sender = None):
  648. """Add the node by responding with its address and port.
  649.  
  650. @type id: C{string}
  651. @param id: the node ID of the sender node
  652. @type _krpc_sender: (C{string}, C{int})
  653. @param _krpc_sender: the sender node's IP address and port
  654. """
  655. if _krpc_sender is not None:
  656. n = self.Node(id, _krpc_sender[0], _krpc_sender[1])
  657. reactor.callLater(0, self.insertNode, n, False)
  658. else:
  659. _krpc_sender = ('127.0.0.1', self.port)
  660.  
  661. return {"ip_addr" : _krpc_sender[0], "port" : _krpc_sender[1], "id" : self.node.id}
  662.  
  663. def krpc_find_node(self, id, target, _krpc_sender = None):
  664. """Find the K closest nodes to the target in the local routing table.
  665.  
  666. @type target: C{string}
  667. @param target: the target ID to find nodes for
  668. @type id: C{string}
  669. @param id: the node ID of the sender node
  670. @type _krpc_sender: (C{string}, C{int})
  671. @param _krpc_sender: the sender node's IP address and port
  672. """
  673. if _krpc_sender is not None:
  674. n = self.Node(id, _krpc_sender[0], _krpc_sender[1])
  675. reactor.callLater(0, self.insertNode, n, False)
  676. else:
  677. _krpc_sender = ('127.0.0.1', self.port)
  678.  
  679. nodes = self.table.findNodes(target)
  680. nodes = map(lambda node: node.contactInfo(), nodes)
  681. token = sha(self.token_secrets[0] + _krpc_sender[0]).digest()
  682. return {"nodes" : nodes, "token" : token, "id" : self.node.id}
  683.  
  684.  
  685. class KhashmirRead(KhashmirBase):
  686. """The read-only Khashmir class, which can only retrieve (not store) key/value mappings."""
  687.  
  688. _Node = KNodeRead
  689.  
  690. #{ Local interface
  691. def findValue(self, key, callback):
  692. """Get the nodes that have values for the key from the global table.
  693.  
  694. @type key: C{string}
  695. @param key: the target key to find the values for
  696. @type callback: C{method}
  697. @param callback: the method to call with the results, it must take 1
  698. parameter, the list of nodes with values
  699. """
  700. # Mark the bucket as having been accessed
  701. self.table.touch(key)
  702.  
  703. # Start with ourself
  704. nodes = [copy(self.node)]
  705.  
  706. # Search for others starting with the locally found ones
  707. state = FindValue(self, key, callback, self.config, self.stats)
  708. reactor.callLater(0, state.goWithNodes, nodes)
  709.  
  710. def valueForKey(self, key, callback, searchlocal = True):
  711. """Get the values found for key in global table.
  712.  
  713. Callback will be called with a list of values for each peer that
  714. returns unique values. The final callback will be an empty list.
  715.  
  716. @type key: C{string}
  717. @param key: the target key to get the values for
  718. @type callback: C{method}
  719. @param callback: the method to call with the results, it must take 2
  720. parameters: the key, and the values found
  721. @type searchlocal: C{boolean}
  722. @param searchlocal: whether to also look for any local values
  723. """
  724.  
  725. def _getValueForKey(nodes, key=key, response=callback, self=self, searchlocal=searchlocal):
  726. """Use the found nodes to send requests for values to."""
  727. # Get any local values
  728. if searchlocal:
  729. l = self.store.retrieveValues(key)
  730. if len(l) > 0:
  731. node = copy(self.node)
  732. node.updateNumValues(len(l))
  733. nodes = nodes + [node]
  734.  
  735. state = GetValue(self, key, self.config['RETRIEVE_VALUES'], response, self.config, self.stats)
  736. reactor.callLater(0, state.goWithNodes, nodes)
  737.  
  738. # First lookup nodes that have values for the key
  739. self.findValue(key, _getValueForKey)
  740.  
  741. #{ Remote interface
  742. def krpc_find_value(self, id, key, _krpc_sender = None):
  743. """Find the number of values stored locally for the key, and the K closest nodes.
  744.  
  745. @type key: C{string}
  746. @param key: the target key to find the values and nodes for
  747. @type id: C{string}
  748. @param id: the node ID of the sender node
  749. @type _krpc_sender: (C{string}, C{int})
  750. @param _krpc_sender: the sender node's IP address and port
  751. """
  752. if _krpc_sender is not None:
  753. n = self.Node(id, _krpc_sender[0], _krpc_sender[1])
  754. reactor.callLater(0, self.insertNode, n, False)
  755.  
  756. nodes = self.table.findNodes(key)
  757. nodes = map(lambda node: node.contactInfo(), nodes)
  758. num_values = self.store.countValues(key)
  759. return {'nodes' : nodes, 'num' : num_values, "id": self.node.id}
  760.  
  761. def krpc_get_value(self, id, key, num, _krpc_sender = None):
  762. """Retrieve the values stored locally for the key.
  763.  
  764. @type key: C{string}
  765. @param key: the target key to retrieve the values for
  766. @type num: C{int}
  767. @param num: the maximum number of values to retrieve, or 0 to
  768. retrieve all of them
  769. @type id: C{string}
  770. @param id: the node ID of the sender node
  771. @type _krpc_sender: (C{string}, C{int})
  772. @param _krpc_sender: the sender node's IP address and port
  773. """
  774. if _krpc_sender is not None:
  775. n = self.Node(id, _krpc_sender[0], _krpc_sender[1])
  776. reactor.callLater(0, self.insertNode, n, False)
  777.  
  778. l = self.store.retrieveValues(key)
  779. if num == 0 or num >= len(l):
  780. return {'values' : l, "id": self.node.id}
  781. else:
  782. shuffle(l)
  783. return {'values' : l[:num], "id": self.node.id}
  784.  
  785.  
  786. class KhashmirWrite(KhashmirRead):
  787. """The read-write Khashmir class, which can store and retrieve key/value mappings."""
  788.  
  789. _Node = KNodeWrite
  790.  
  791. #{ Local interface
  792. def storeValueForKey(self, key, value, callback=None):
  793. """Stores the value for the key in the global table.
  794.  
  795. No status in this implementation, peers respond but don't indicate
  796. status of storing values.
  797.  
  798. @type key: C{string}
  799. @param key: the target key to store the value for
  800. @type value: C{string}
  801. @param value: the value to store with the key
  802. @type callback: C{method}
  803. @param callback: the method to call with the results, it must take 3
  804. parameters: the key, the value stored, and the result of the store
  805. (optional, defaults to doing nothing with the results)
  806. """
  807. def _storeValueForKey(nodes, key=key, value=value, response=callback, self=self):
  808. """Use the returned K closest nodes to store the key at."""
  809. if not response:
  810. def _storedValueHandler(key, value, sender):
  811. """Default callback that does nothing."""
  812. pass
  813. response = _storedValueHandler
  814. action = StoreValue(self, key, value, self.config['STORE_REDUNDANCY'], response, self.config, self.stats)
  815. reactor.callLater(0, action.goWithNodes, nodes)
  816.  
  817. # First find the K closest nodes to operate on.
  818. self.findNode(key, _storeValueForKey)
  819.  
  820. #{ Remote interface
  821. def krpc_store_value(self, id, key, value, token, _krpc_sender = None):
  822. """Store the value locally with the key.
  823.  
  824. @type key: C{string}
  825. @param key: the target key to store the value for
  826. @type value: C{string}
  827. @param value: the value to store with the key
  828. @param token: the token to confirm that this peer contacted us previously
  829. @type id: C{string}
  830. @param id: the node ID of the sender node
  831. @type _krpc_sender: (C{string}, C{int})
  832. @param _krpc_sender: the sender node's IP address and port
  833. """
  834. if _krpc_sender is not None:
  835. n = self.Node(id, _krpc_sender[0], _krpc_sender[1])
  836. reactor.callLater(0, self.insertNode, n, False)
  837. else:
  838. _krpc_sender = ('127.0.0.1', self.port)
  839.  
  840. for secret in self.token_secrets:
  841. this_token = sha(secret + _krpc_sender[0]).digest()
  842. if token == this_token:
  843. self.store.storeValue(key, value)
  844. return {"id" : self.node.id}
  845. raise krpc.KrpcError, (krpc.KRPC_ERROR_INVALID_TOKEN, 'token is invalid, do a find_nodes to get a fresh one')
  846.  
  847.  
  848. class Khashmir(KhashmirWrite):
  849. """The default Khashmir class (currently the read-write L{KhashmirWrite})."""
  850. _Node = KNodeWrite
  851.  
  852.  
  853. class SimpleTests(unittest.TestCase):
  854.  
  855. timeout = 10
  856. DHT_DEFAULTS = {'VERSION': 'A000', 'PORT': 9977,
  857. 'CHECKPOINT_INTERVAL': 300, 'CONCURRENT_REQS': 8,
  858. 'STORE_REDUNDANCY': 6, 'RETRIEVE_VALUES': -10000,
  859. 'MAX_FAILURES': 3, 'LOCAL_OK': True,
  860. 'MIN_PING_INTERVAL': 900,'BUCKET_STALENESS': 3600,
  861. 'KRPC_TIMEOUT': 9, 'KRPC_INITIAL_DELAY': 2,
  862. 'KEY_EXPIRE': 3600, 'SPEW': True, }
  863.  
  864. def setUp(self):
  865. d = self.DHT_DEFAULTS.copy()
  866. d['PORT'] = 4044
  867. self.a = Khashmir(d)
  868. d = self.DHT_DEFAULTS.copy()
  869. d['PORT'] = 4045
  870. self.b = Khashmir(d)
  871.  
  872. def tearDown(self):
  873. self.a.shutdown()
  874. self.b.shutdown()
  875. os.unlink(self.a.store.db)
  876. os.unlink(self.b.store.db)
  877.  
  878. def testAddContact(self):
  879. self.failUnlessEqual(len(self.a.table.buckets), 1)
  880. self.failUnlessEqual(len(self.a.table.buckets[0].nodes), 0)
  881.  
  882. self.failUnlessEqual(len(self.b.table.buckets), 1)
  883. self.failUnlessEqual(len(self.b.table.buckets[0].nodes), 0)
  884.  
  885. self.a.addContact('127.0.0.1', 4045)
  886. reactor.iterate()
  887. reactor.iterate()
  888. reactor.iterate()
  889. reactor.iterate()
  890. reactor.iterate()
  891. reactor.iterate()
  892. reactor.iterate()
  893. reactor.iterate()
  894.  
  895. self.failUnlessEqual(len(self.a.table.buckets), 1)
  896. self.failUnlessEqual(len(self.a.table.buckets[0].nodes), 1)
  897. self.failUnlessEqual(len(self.b.table.buckets), 1)
  898. self.failUnlessEqual(len(self.b.table.buckets[0].nodes), 1)
  899.  
  900. def testStoreRetrieve(self):
  901. self.a.addContact('127.0.0.1', 4045)
  902. reactor.iterate()
  903. reactor.iterate()
  904. reactor.iterate()
  905. reactor.iterate()
  906. self.got = 0
  907. self.a.storeValueForKey(sha('foo').digest(), 'foobar')
  908. reactor.iterate()
  909. reactor.iterate()
  910. reactor.iterate()
  911. reactor.iterate()
  912. reactor.iterate()
  913. reactor.iterate()
  914. self.a.valueForKey(sha('foo').digest(), self._cb)
  915. reactor.iterate()
  916. reactor.iterate()
  917. reactor.iterate()
  918. reactor.iterate()
  919. reactor.iterate()
  920. reactor.iterate()
  921. reactor.iterate()
  922. reactor.iterate()
  923. reactor.iterate()
  924. reactor.iterate()
  925. reactor.iterate()
  926. reactor.iterate()
  927. reactor.iterate()
  928. reactor.iterate()
  929. reactor.iterate()
  930. reactor.iterate()
  931. reactor.iterate()
  932. reactor.iterate()
  933. reactor.iterate()
  934. reactor.iterate()
  935. reactor.iterate()
  936.  
  937. def _cb(self, key, val):
  938. if not val:
  939. self.failUnlessEqual(self.got, 1)
  940. elif 'foobar' in val:
  941. self.got = 1
  942.  
  943.  
  944. class MultiTest(unittest.TestCase):
  945.  
  946. timeout = 30
  947. num = 20
  948. DHT_DEFAULTS = {'VERSION': 'A000', 'PORT': 9977,
  949. 'CHECKPOINT_INTERVAL': 300, 'CONCURRENT_REQS': 8,
  950. 'STORE_REDUNDANCY': 6, 'RETRIEVE_VALUES': -10000,
  951. 'MAX_FAILURES': 3, 'LOCAL_OK': True,
  952. 'MIN_PING_INTERVAL': 900,'BUCKET_STALENESS': 3600,
  953. 'KRPC_TIMEOUT': 9, 'KRPC_INITIAL_DELAY': 2,
  954. 'KEY_EXPIRE': 3600, 'SPEW': True, }
  955.  
  956. def _done(self, val):
  957. self.done = 1
  958.  
  959. def setUp(self):
  960. self.l = []
  961. self.startport = 4088
  962. for i in range(self.num):
  963. d = self.DHT_DEFAULTS.copy()
  964. d['PORT'] = self.startport + i
  965. self.l.append(Khashmir(d))
  966. reactor.iterate()
  967. reactor.iterate()
  968.  
  969. for i in self.l:
  970. i.addContact('127.0.0.1', self.l[randrange(0,self.num)].port)
  971. i.addContact('127.0.0.1', self.l[randrange(0,self.num)].port)
  972. i.addContact('127.0.0.1', self.l[randrange(0,self.num)].port)
  973. reactor.iterate()
  974. reactor.iterate()
  975. reactor.iterate()
  976. reactor.iterate()
  977. reactor.iterate()
  978. reactor.iterate()
  979.  
  980. for i in self.l:
  981. self.done = 0
  982. i.findCloseNodes(self._done)
  983. while not self.done:
  984. reactor.iterate()
  985. for i in self.l:
  986. self.done = 0
  987. i.findCloseNodes(self._done)
  988. while not self.done:
  989. reactor.iterate()
  990.  
  991. def tearDown(self):
  992. for i in self.l:
  993. i.shutdown()
  994. os.unlink(i.store.db)
  995.  
  996. reactor.iterate()
  997.  
  998. def testStoreRetrieve(self):
  999. for i in range(10):
  1000. K = newID()
  1001. V = newID()
  1002.  
  1003. for a in range(3):
  1004. self.done = 0
  1005. def _scb(key, value, result):
  1006. self.done = 1
  1007. self.l[randrange(0, self.num)].storeValueForKey(K, V, _scb)
  1008. while not self.done:
  1009. reactor.iterate()
  1010.  
  1011.  
  1012. def _rcb(key, val):
  1013. if not val:
  1014. self.done = 1
  1015. self.failUnlessEqual(self.got, 1)
  1016. elif V in val:
  1017. self.got = 1
  1018. for x in range(3):
  1019. self.got = 0
  1020. self.done = 0
  1021. self.l[randrange(0, self.num)].valueForKey(K, _rcb)
  1022. while not self.done:
  1023. reactor.iterate()
  1024. bazar@ao756:~/apt-p2p/apt_p2p_Khashmir$ ls
  1025. actions.py db.py __init__.py khash.py krpc.py node.py util.py
  1026. bencode.py DHT.py khashmir.py knode.py ktable.py stats.py
  1027. bazar@ao756:~/apt-p2p/apt_p2p_Khashmir$ apt-p2p.py
  1028. 2020-01-31 15:12:28+0300 [-] Log opened.
  1029. 2020-01-31 15:12:28+0300 [-] Loading config files: '/etc/apt-p2p/apt-p2p.conf', '/home/bazar/.apt-p2p/apt-p2p.conf', ''
  1030. 2020-01-31 15:12:28+0300 [-] Successfully loaded config files: ''
  1031. 2020-01-31 15:12:28+0300 [-] Starting application with uid/gid None/None
  1032. 2020-01-31 15:12:28+0300 [-] Traceback (most recent call last):
  1033. 2020-01-31 15:12:28+0300 [-] File "/usr/local/bin/apt-p2p.py", line 68, in <module>
  1034. 2020-01-31 15:12:28+0300 [-] DHT = __import__(config.get('DEFAULT', 'DHT')+'.DHT', globals(), locals(), ['DHT'])
  1035. 2020-01-31 15:12:28+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/DHT.py", line 17, in <module>
  1036. 2020-01-31 15:12:28+0300 [-] from khashmir import Khashmir
  1037. 2020-01-31 15:12:28+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/khashmir.py", line 23, in <module>
  1038. 2020-01-31 15:12:28+0300 [-] from db import DB
  1039. 2020-01-31 15:12:28+0300 [-] File "/usr/local/lib/python2.7/dist-packages/apt_p2p_Khashmir/db.py", line 5, in <module>
  1040. 2020-01-31 15:12:28+0300 [-] from pysqlite2 import dbapi2 as sqlite
  1041. 2020-01-31 15:12:28+0300 [-] ImportError: No module named pysqlite2
  1042. bazar@ao756:~/apt-p2p/apt_p2p_Khashmir$ nano db.py
  1043. bazar@ao756:~/apt-p2p/apt_p2p_Khashmir$ cd ..
  1044. bazar@ao756:~/apt-p2p$ ls
  1045. apt_p2p apt_p2p_Khashmir bootstrap1 debian downloader1 LICENSE.md setup.py TODO.md
  1046. apt-p2p.conf apt-p2p.py build docs downloader2 README.md test.py
  1047. bazar@ao756:~/apt-p2p$ ./apt_p2p.py
  1048. bash: ./apt_p2p.py: Нет такого файла или каталога
  1049. bazar@ao756:~/apt-p2p$ python apt_p2p.py
  1050. python: can't open file 'apt_p2p.py': [Errno 2] No such file or directory
  1051. bazar@ao756:~/apt-p2p$ python apt_p2p
  1052. apt_p2p/ apt_p2p_Khashmir/
  1053. bazar@ao756:~/apt-p2p$ python apt_p2p
  1054. apt_p2p/ apt_p2p_Khashmir/
  1055. bazar@ao756:~/apt-p2p$ chmod +x apt-p2p.py
  1056. apt_p2p/ bootstrap1/ downloader1/ .gitignore test.py
  1057. apt-p2p.conf build/ downloader2/ LICENSE.md TODO.md
  1058. apt_p2p_Khashmir/ debian/ .gbp.conf README.md
  1059. apt-p2p.py docs/ .git/ setup.py
  1060. bazar@ao756:~/apt-p2p$ chmod +x apt-p2p.py
  1061. bazar@ao756:~/apt-p2p$ python apt_p2p.py
  1062. python: can't open file 'apt_p2p.py': [Errno 2] No such file or directory
  1063. bazar@ao756:~/apt-p2p$ python apt
  1064. apt_p2p/ apt_p2p_Khashmir/ apt-p2p.py
  1065. bazar@ao756:~/apt-p2p$ python apt-p2p.py
  1066. 2020-01-31 15:14:28+0300 [-] Log opened.
  1067. 2020-01-31 15:14:28+0300 [-] Loading config files: '/etc/apt-p2p/apt-p2p.conf', '/home/bazar/.apt-p2p/apt-p2p.conf', ''
  1068. 2020-01-31 15:14:28+0300 [-] Successfully loaded config files: ''
  1069. 2020-01-31 15:14:28+0300 [-] Starting application with uid/gid None/None
  1070. 2020-01-31 15:14:29+0300 [-] Starting main application server
  1071. 2020-01-31 15:14:29+0300 [-] Traceback (most recent call last):
  1072. 2020-01-31 15:14:29+0300 [-] File "apt-p2p.py", line 73, in <module>
  1073. 2020-01-31 15:14:29+0300 [-] from apt_p2p.apt_p2p import AptP2P
  1074. 2020-01-31 15:14:29+0300 [-] File "/home/bazar/apt-p2p/apt_p2p/apt_p2p.py", line 11, in <module>
  1075. 2020-01-31 15:14:29+0300 [-] from twisted.web2 import static
  1076. 2020-01-31 15:14:29+0300 [-] ImportError: No module named web2
  1077. bazar@ao756:~/apt-p2p$ nano apt
  1078. apt_p2p/ apt-p2p.conf apt_p2p_Khashmir/ apt-p2p.py
  1079. bazar@ao756:~/apt-p2p$ nano apt-p2p.py
  1080. bazar@ao756:~/apt-p2p$ cat apt-p2p.py | grep twisted.web2
  1081. bazar@ao756:~/apt-p2p$ cd apt_p2p/
  1082. bazar@ao756:~/apt-p2p/apt_p2p$ nano apt_p2p.py
  1083. bazar@ao756:~/apt-p2p/apt_p2p$
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!