API tools faq
paste
Advertisement
Guest User

Help with security groups ("in use" error) in juju/openstack

a guest
May 10th, 2012
150
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 14.12 KB | None | 0 0
raw download clone embed print report
  1. Scenario:
  2.  
  3. juju bootstrap
  4. echo -e "\nAguardando ambiente ser inicializado..."
  5. sleep 60;
  6. juju deploy --repository ~/charms local:hadoop-master
  7. juju deploy --repository ~/charms local:hadoop-slave
  8. echo -e "\nAguardando instanciamento do Master e Slave do Hadoop..."
  9. sleep 200;
  10. juju add-relation hadoop-slave hadoop-master
  11. juju expose hadoop-master
  12. sleep 10;
  13. for i in {1..10} ; do
  14. juju add-unit hadoop-slave; sleep 10;
  15. done
  16.  
  17. There are 13 instances. The database is clean, starting from id 1.
  18.  
  19. At this point, many instances doesnt work!!
  20.  
  21. Considering the code where the error is happening we have:
  22.  
  23. /usr/lib/python2.7/dist-packages/nova/api/ec2/cloud.py", line 827
  24. @require_context
  25. def security_group_in_use(context, group_id):
  26. session = get_session()
  27. with session.begin():
  28. # Are there any instances that haven't been deleted
  29. # that include this group?
  30. inst_assoc = session.query(models.SecurityGroupInstanceAssociation).\
  31. filter_by(security_group_id=group_id).\
  32. filter_by(deleted=False).\
  33. all()
  34. for ia in inst_assoc:
  35. num_instances = session.query(models.Instance).\
  36. filter_by(deleted=False).\
  37. filter_by(id=ia.instance_id).\
  38. count()
  39. if num_instances:
  40. return True
  41.  
  42. return False
  43.  
  44. The database state at this moment is:
  45.  
  46. mysql> select * from security_groups;
  47. +---------------------+------------+------------+---------+----+----------------+---------
  48. | created_at | deleted | id | name | description |
  49. +---------------------+------------+------------+---------+----+----------------+---------
  50. | 2012-05-10 18:17:30 | 0 | 1 | default | default |
  51. | 2012-05-10 18:17:30 | 0 | 2 | juju-sample | juju group for sample |
  52. | 2012-05-10 18:17:31 | 0 | 3 | juju-sample-0 | juju group for sample machine 0 |
  53. | 2012-05-10 18:18:39 | 0 | 4 | juju-sample-1 | juju group for sample machine 1 |
  54. | 2012-05-10 18:18:42 | 0 | 5 | juju-sample-2 | juju group for sample machine 2 |
  55. | 2012-05-10 18:22:15 | 0 | 6 | juju-sample-3 | juju group for sample machine 3 |
  56. | 2012-05-10 18:22:26 | 0 | 7 | juju-sample-4 | juju group for sample machine 4 |
  57. | 2012-05-10 18:22:37 | 0 | 8 | juju-sample-5 | juju group for sample machine 5 |
  58. | 2012-05-10 18:22:49 | 0 | 9 | juju-sample-6 | juju group for sample machine 6 |
  59. | 2012-05-10 18:23:00 | 0 | 10 | juju-sample-7 | juju group for sample machine 7 |
  60. | 2012-05-10 18:23:12 | 0 | 11 | juju-sample-8 | juju group for sample machine 8 |
  61. | 2012-05-10 18:23:23 | 0 | 12 | juju-sample-9 | juju group for sample machine 9 |
  62. | 2012-05-10 18:23:34 | 0 | 13 | juju-sample-10 | juju group for sample machine 10 |
  63. | 2012-05-10 18:23:46 | 0 | 14 | juju-sample-11 | juju group for sample machine 11 |
  64. | 2012-05-10 18:23:58 | 0 | 15 | juju-sample-12 | juju group for sample machine 12 |
  65. +---------------------+------------+------------+---------+----+----------------+---------
  66.  
  67. mysql> select * from security_group_instance_association where deleted = 0;
  68. +---------------------+------------+------------+---------+----+--------
  69. | created_at | deleted | id | security_group_id | instance_id |
  70. +---------------------+------------+------------+---------+----+--------
  71. | 2012-05-10 18:17:32 | 0 | 1 | 2 | 1 |
  72. | 2012-05-10 18:17:32 | 0 | 2 | 3 | 1 |
  73. | 2012-05-10 18:18:40 | 0 | 3 | 2 | 2 |
  74. | 2012-05-10 18:18:40 | 0 | 4 | 4 | 2 |
  75. | 2012-05-10 18:18:43 | 0 | 5 | 2 | 3 |
  76. | 2012-05-10 18:18:43 | 0 | 6 | 5 | 3 |
  77. | 2012-05-10 18:22:16 | 0 | 7 | 2 | 4 |
  78. | 2012-05-10 18:22:16 | 0 | 8 | 6 | 4 |
  79. | 2012-05-10 18:22:27 | 0 | 9 | 2 | 5 |
  80. | 2012-05-10 18:22:27 | 0 | 10 | 7 | 5 |
  81. | 2012-05-10 18:22:38 | 0 | 11 | 2 | 6 |
  82. | 2012-05-10 18:22:38 | 0 | 12 | 8 | 6 |
  83. | 2012-05-10 18:22:50 | 0 | 13 | 2 | 7 |
  84. | 2012-05-10 18:22:50 | 0 | 14 | 9 | 7 |
  85. | 2012-05-10 18:23:01 | 0 | 15 | 2 | 8 |
  86. | 2012-05-10 18:23:01 | 0 | 16 | 10 | 8 |
  87. | 2012-05-10 18:23:13 | 0 | 17 | 2 | 9 |
  88. | 2012-05-10 18:23:13 | 0 | 18 | 11 | 9 |
  89. | 2012-05-10 18:23:24 | 0 | 19 | 2 | 10 |
  90. | 2012-05-10 18:23:24 | 0 | 20 | 12 | 10 |
  91. | 2012-05-10 18:23:35 | 0 | 21 | 2 | 11 |
  92. | 2012-05-10 18:23:36 | 0 | 22 | 13 | 11 |
  93. | 2012-05-10 18:23:47 | 0 | 23 | 2 | 12 |
  94. | 2012-05-10 18:23:47 | 0 | 24 | 14 | 12 |
  95. | 2012-05-10 18:23:59 | 0 | 25 | 2 | 13 |
  96. | 2012-05-10 18:23:59 | 0 | 26 | 15 | 13 |
  97. +---------------------+------------+------------+---------+----+--------
  98. 26 rows in set (0.00 sec)
  99.  
  100. mysql> select id from instances where deleted = 0;
  101. +----+
  102. | id |
  103. +----+
  104. | 1 |
  105. | 2 |
  106. | 3 |
  107. | 4 |
  108. | 5 |
  109. | 6 |
  110. | 7 |
  111. | 8 |
  112. | 9 |
  113. | 10 |
  114. | 11 |
  115. | 12 |
  116. | 13 |
  117. +----+
  118. 13 rows in set (0.00 sec)
  119.  
  120. Trying to create a new unit generate the same error. How juju works with 0 for the first instance, it will then create the juju-sample-13 (the 14th instance).
  121.  
  122. 2012-05-10 15:47:42 DEBUG nova.api.ec2 [req-c12d2887-b42d-42fc-8636-5c1ff55e35f6 de0bba964c7b4948bef8bb04d7111cae fed67a76052340e6b225879aed674846] action: DeleteSecurityGroup from (pid=23347) __call__ /usr/lib/python2.7/dist-packages/nova/api/ec2/__init__.py:435
  123. 2012-05-10 15:47:42 DEBUG nova.api.ec2 [req-c12d2887-b42d-42fc-8636-5c1ff55e35f6 de0bba964c7b4948bef8bb04d7111cae fed67a76052340e6b225879aed674846] arg: GroupName val: juju-sample-13 from (pid=23347) __call__ /usr/lib/python2.7/dist-packages/nova/api/ec2/__init__.py:437
  124. 2012-05-10 15:47:42 ERROR nova.api.ec2 [req-c12d2887-b42d-42fc-8636-5c1ff55e35f6 de0bba964c7b4948bef8bb04d7111cae fed67a76052340e6b225879aed674846] Unexpected error raised: Group not valid. Reason: In Use
  125. 2012-05-10 15:47:42 TRACE nova.api.ec2 Traceback (most recent call last):
  126. 2012-05-10 15:47:42 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/nova/api/ec2/__init__.py", line 582, in __call__
  127. 2012-05-10 15:47:42 TRACE nova.api.ec2 result = api_request.invoke(context)
  128. 2012-05-10 15:47:42 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/nova/api/ec2/apirequest.py", line 81, in invoke
  129. 2012-05-10 15:47:42 TRACE nova.api.ec2 result = method(context, **args)
  130. 2012-05-10 15:47:42 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/nova/api/ec2/cloud.py", line 827, in delete_security_group
  131. 2012-05-10 15:47:42 TRACE nova.api.ec2 raise exception.InvalidGroup(reason="In Use")
  132. 2012-05-10 15:47:42 TRACE nova.api.ec2 InvalidGroup: Group not valid. Reason: In Use
  133. 2012-05-10 15:47:42 TRACE nova.api.ec2
  134. 2012-05-10 15:47:42 ERROR nova.api.ec2 [req-c12d2887-b42d-42fc-8636-5c1ff55e35f6 de0bba964c7b4948bef8bb04d7111cae fed67a76052340e6b225879aed674846] Environment: {"CONTENT_TYPE": "text/plain", "SCRIPT_NAME": "/services/Cloud", "REQUEST_METHOD": "GET", "HTTP_HOST": "10.129.10.44:8773", "PATH_INFO": "", "SERVER_PROTOCOL": "HTTP/1.0", "QUERY_STRING": "AWSAccessKeyId=08d1790ca04646f3b116331a6565d2a7&Action=DeleteSecurityGroup&GroupName=juju-sample-13&Signature=mx6NgS2EAdiu%2BiIz4T%2B0LyPHV%2B9M2T7W8NCsEOSqvY8%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2012-05-10T18%3A47%3A29Z&Version=2008-12-01", "HTTP_USER_AGENT": "Twisted PageGetter", "SERVER_NAME": "10.129.10.44", "REMOTE_ADDR": "172.16.0.2", "wsgi.url_scheme": "http", "SERVER_PORT": "8773", "GATEWAY_INTERFACE": "CGI/1.1"}
  135. 2012-05-10 15:47:42 ERROR nova.api.ec2 [req-c12d2887-b42d-42fc-8636-5c1ff55e35f6 de0bba964c7b4948bef8bb04d7111cae fed67a76052340e6b225879aed674846] UnknownError: An unknown error has occurred. Please try your request again.
  136.  
  137. --
  138.  
  139. The database:
  140.  
  141. New line:
  142. mysql> select * from security_groups;
  143. +---------------------+------------+------------+---------+----+----------------+---------
  144. | created_at | deleted | id | name | description |
  145. +---------------------+------------+------------+---------+----+----------------+---------
  146. | 2012-05-10 18:17:30 | 0 | 1 | default | default |
  147. | 2012-05-10 18:17:30 | 0 | 2 | juju-sample | juju group for sample |
  148. | 2012-05-10 18:17:31 | 0 | 3 | juju-sample-0 | juju group for sample machine 0 |
  149. | 2012-05-10 18:18:39 | 0 | 4 | juju-sample-1 | juju group for sample machine 1 |
  150. | 2012-05-10 18:18:42 | 0 | 5 | juju-sample-2 | juju group for sample machine 2 |
  151. | 2012-05-10 18:22:15 | 0 | 6 | juju-sample-3 | juju group for sample machine 3 |
  152. | 2012-05-10 18:22:26 | 0 | 7 | juju-sample-4 | juju group for sample machine 4 |
  153. | 2012-05-10 18:22:37 | 0 | 8 | juju-sample-5 | juju group for sample machine 5 |
  154. | 2012-05-10 18:22:49 | 0 | 9 | juju-sample-6 | juju group for sample machine 6 |
  155. | 2012-05-10 18:23:00 | 0 | 10 | juju-sample-7 | juju group for sample machine 7 |
  156. | 2012-05-10 18:23:12 | 0 | 11 | juju-sample-8 | juju group for sample machine 8 |
  157. | 2012-05-10 18:23:23 | 0 | 12 | juju-sample-9 | juju group for sample machine 9 |
  158. | 2012-05-10 18:23:34 | 0 | 13 | juju-sample-10 | juju group for sample machine 10 |
  159. | 2012-05-10 18:23:46 | 0 | 14 | juju-sample-11 | juju group for sample machine 11 |
  160. | 2012-05-10 18:23:58 | 0 | 15 | juju-sample-12 | juju group for sample machine 12 |
  161. | 2012-05-10 18:42:56 | 0 | 16 | juju-sample-13 | juju group for sample machine 13 |
  162. +---------------------+------------+------------+---------+----+----------------+---------
  163.  
  164.  
  165. mysql> select * from security_group_instance_association where deleted = 0;
  166. +---------------------+------------+------------+---------+----+--------
  167. | created_at | deleted | id | security_group_id | instance_id |
  168. +---------------------+------------+------------+---------+----+--------
  169. | 2012-05-10 18:17:32 | 0 | 1 | 2 | 1 |
  170. | 2012-05-10 18:17:32 | 0 | 2 | 3 | 1 |
  171. | 2012-05-10 18:18:40 | 0 | 3 | 2 | 2 |
  172. | 2012-05-10 18:18:40 | 0 | 4 | 4 | 2 |
  173. | 2012-05-10 18:18:43 | 0 | 5 | 2 | 3 |
  174. | 2012-05-10 18:18:43 | 0 | 6 | 5 | 3 |
  175. | 2012-05-10 18:22:16 | 0 | 7 | 2 | 4 |
  176. | 2012-05-10 18:22:16 | 0 | 8 | 6 | 4 |
  177. | 2012-05-10 18:22:27 | 0 | 9 | 2 | 5 |
  178. | 2012-05-10 18:22:27 | 0 | 10 | 7 | 5 |
  179. | 2012-05-10 18:22:38 | 0 | 11 | 2 | 6 |
  180. | 2012-05-10 18:22:38 | 0 | 12 | 8 | 6 |
  181. | 2012-05-10 18:22:50 | 0 | 13 | 2 | 7 |
  182. | 2012-05-10 18:22:50 | 0 | 14 | 9 | 7 |
  183. | 2012-05-10 18:23:01 | 0 | 15 | 2 | 8 |
  184. | 2012-05-10 18:23:01 | 0 | 16 | 10 | 8 |
  185. | 2012-05-10 18:23:13 | 0 | 17 | 2 | 9 |
  186. | 2012-05-10 18:23:13 | 0 | 18 | 11 | 9 |
  187. | 2012-05-10 18:23:24 | 0 | 19 | 2 | 10 |
  188. | 2012-05-10 18:23:24 | 0 | 20 | 12 | 10 |
  189. | 2012-05-10 18:23:35 | 0 | 21 | 2 | 11 |
  190. | 2012-05-10 18:23:36 | 0 | 22 | 13 | 11 |
  191. | 2012-05-10 18:23:47 | 0 | 23 | 2 | 12 |
  192. | 2012-05-10 18:23:47 | 0 | 24 | 14 | 12 |
  193. | 2012-05-10 18:23:59 | 0 | 25 | 2 | 13 |
  194. | 2012-05-10 18:23:59 | 0 | 26 | 15 | 13 |
  195. | 2012-05-10 18:42:57 | 0 | 27 | 2 | 14 |
  196. | 2012-05-10 18:42:57 | 0 | 28 | 16 | 14 |
  197. +---------------------+------------+------------+---------+----+--------
  198.  
  199.  
  200. mysql> select id from instances where deleted = 0;
  201. +----+
  202. | id |
  203. +----+
  204. | 1 |
  205. | 2 |
  206. | 3 |
  207. | 4 |
  208. | 5 |
  209. | 6 |
  210. | 7 |
  211. | 8 |
  212. | 9 |
  213. | 10 |
  214. | 11 |
  215. | 12 |
  216. | 13 |
  217. | 14 | <--
  218. +----+
  219.  
  220. inst_assoc = session.query(models.SecurityGroupInstanceAssociation).\
  221. filter_by(security_group_id=group_id).\
  222. filter_by(deleted=False).\
  223. all()
  224. +---------------------+------------+------------+---------+----+--------------
  225. | created_at | deleted | id | security_group_id | instance_id |
  226. | 2012-05-10 18:42:57 | 0 | 28 | 16 | 14 |
  227.  
  228.  
  229. for ia in inst_assoc:
  230. num_instances = session.query(models.Instance).\
  231. filter_by(deleted=False).\
  232. filter_by(id=ia.instance_id).\
  233. count()
  234.  
  235. Count will return 1, so, True.
  236.  
  237.  
  238. Once the dabase was empty, all those security rules was created by juju and nova. I'm not good with python, but seems that these tests will always be true, returning true in the function and generating the error.
  239.  
  240. Maybe the order of insertion on the database can be important. I don't know when the '14' instance was inserted in 'instances' table. If the tests occurs before this, ok. But, if 14 is inserted before the test, the error will happen.
  241.  
  242. I don't know if the problem is exactly that. Maybe. I don't know the nova and juju codes. I'm justing testing something.
  243.  
  244.  
  245. Cheers.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!