API tools faq
paste
Advertisement
jimklimov

JNLP torn down - stacktrace

jimklimov
Aug 13th, 2019
233
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 9.27 KB | None | 0 0
raw download clone embed print report
  1. (From IRC)
  2.  
  3. Is anyone around that can tell me about how Jenkins jobs can persist around the restart of a master server?
  4.  
  5. I've read about this mostly in the context of last year or so in pipelines' evolution... but somehow got an understanding that standalone agents (swarm, jnlp... those that are not child processes of master JVM, not launched over SSH from it, etc.) whose JVMs keep running regardless of master restart - jobs running in them would also survive and resume reporting their logs when the master comes back alive and the agent is reconnected.
  6.  
  7. So I tried with a swarm client, hoping to put our longest-running jobs like stress/survivability/etc. tests there, jobs which run for hours and block opportunities for master (or plugin) updates
  8.  
  9. But when the master was restarted (deliberately not very gracefully first, as "systemctl restart jenkins"), then upon restart the job log contained a few new lines - a stacktrace about lost remoting channel (possibly closed still as part of master teardown) :
  10.  
  11. FATAL: command execution failed
  12. java.io.IOException
  13. at hudson.remoting.Channel.close(Channel.java:1428)
  14. at hudson.remoting.Channel.close(Channel.java:1405)
  15. at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:844)
  16. at hudson.slaves.SlaveComputer.kill(SlaveComputer.java:811)
  17. at hudson.model.AbstractCIBase.killComputer(AbstractCIBase.java:89)
  18. at jenkins.model.Jenkins.access$2100(Jenkins.java:312)
  19. at jenkins.model.Jenkins$19.run(Jenkins.java:3464)
  20. at hudson.model.Queue._withLock(Queue.java:1379)
  21. at hudson.model.Queue.withLock(Queue.java:1256)
  22. at jenkins.model.Jenkins._cleanUpDisconnectComputers(Jenkins.java:3458)
  23. at jenkins.model.Jenkins.cleanUp(Jenkins.java:3336)
  24. at hudson.WebAppMain.contextDestroyed(WebAppMain.java:379)
  25. at org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:965)
  26. at org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:566)
  27. at org.eclipse.jetty.server.handler.ContextHandler.stopContext(ContextHandler.java:942)
  28. at org.eclipse.jetty.servlet.ServletContextHandler.stopContext(ServletContextHandler.java:376)
  29. at org.eclipse.jetty.webapp.WebAppContext.stopWebapp(WebAppContext.java:1503)
  30. at org.eclipse.jetty.webapp.WebAppContext.stopContext(WebAppContext.java:1467)
  31. at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:1009)
  32. at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:288)
  33. at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:569)
  34. at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
  35. at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:178)
  36. at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:199)
  37. at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:124)
  38. at org.eclipse.jetty.server.Server.doStop(Server.java:464)
  39. at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
  40. at winstone.Launcher.shutdown(Launcher.java:313)
  41. at winstone.ShutdownHook.run(ShutdownHook.java:25)
  42. Caused: hudson.remoting.ChannelClosedException: Channel "unknown": Remote call on JNLP4-connect connection from webproxy.localdomain/10.0.0.2:48049 failed. The channel is closing down or has closed down
  43. at hudson.remoting.Channel.call(Channel.java:950)
  44. at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
  45. at com.sun.proxy.$Proxy105.isAlive(Unknown Source)
  46. at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1150)
  47. at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1142)
  48. at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:155)
  49. at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
  50. at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
  51. at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
  52. at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
  53. at hudson.model.Build$BuildExecution.build(Build.java:206)
  54. at hudson.model.Build$BuildExecution.doRun(Build.java:163)
  55. at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
  56. at hudson.model.Run.execute(Run.java:1815)
  57. at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
  58. at hudson.model.ResourceController.execute(ResourceController.java:97)
  59. at hudson.model.Executor.run(Executor.java:429)
  60. FATAL: Unable to delete script file /tmp/jenkins3171735084038584934.sh
  61. java.nio.channels.ClosedChannelException
  62. at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209)
  63. at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
  64. at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
  65. at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
  66. at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
  67. at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
  68. at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
  69. at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
  70. at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
  71. at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
  72. at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:784)
  73. at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
  74. at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:314)
  75. at hudson.remoting.Channel.close(Channel.java:1452)
  76. at hudson.remoting.Channel.close(Channel.java:1405)
  77. at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:844)
  78. at hudson.slaves.SlaveComputer.kill(SlaveComputer.java:811)
  79. at hudson.model.AbstractCIBase.killComputer(AbstractCIBase.java:89)
  80. at jenkins.model.Jenkins.access$2100(Jenkins.java:312)
  81. at jenkins.model.Jenkins$19.run(Jenkins.java:3464)
  82. at hudson.model.Queue._withLock(Queue.java:1379)
  83. at hudson.model.Queue.withLock(Queue.java:1256)
  84. at jenkins.model.Jenkins._cleanUpDisconnectComputers(Jenkins.java:3458)
  85. at jenkins.model.Jenkins.cleanUp(Jenkins.java:3336)
  86. at hudson.WebAppMain.contextDestroyed(WebAppMain.java:379)
  87. at org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:965)
  88. at org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:566)
  89. at org.eclipse.jetty.server.handler.ContextHandler.stopContext(ContextHandler.java:942)
  90. at org.eclipse.jetty.servlet.ServletContextHandler.stopContext(ServletContextHandler.java:376)
  91. at org.eclipse.jetty.webapp.WebAppContext.stopWebapp(WebAppContext.java:1503)
  92. at org.eclipse.jetty.webapp.WebAppContext.stopContext(WebAppContext.java:1467)
  93. at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:1009)
  94. at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:288)
  95. at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:569)
  96. at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
  97. at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:178)
  98. at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:199)
  99. at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:124)
  100. at org.eclipse.jetty.server.Server.doStop(Server.java:464)
  101. at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
  102. at winstone.Launcher.shutdown(Launcher.java:313)
  103. at winstone.ShutdownHook.run(ShutdownHook.java:25)
  104. Caused: hudson.remoting.ChannelClosedException: Channel "unknown": Remote call on JNLP4-connect connection from webproxy.localdomain/10.0.0.2:48049 failed. The channel is closing down or has closed down
  105. at hudson.remoting.Channel.call(Channel.java:950)
  106. at hudson.FilePath.act(FilePath.java:1072)
  107. at hudson.FilePath.act(FilePath.java:1061)
  108. at hudson.FilePath.delete(FilePath.java:1542)
  109. at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:123)
  110. at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
  111. at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
  112. at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
  113. at hudson.model.Build$BuildExecution.build(Build.java:206)
  114. at hudson.model.Build$BuildExecution.doRun(Build.java:163)
  115. at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
  116. at hudson.model.Run.execute(Run.java:1815)
  117. at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
  118. at hudson.model.ResourceController.execute(ResourceController.java:97)
  119. at hudson.model.Executor.run(Executor.java:429)
  120. Build step 'Execute shell' marked build as failure
  121. [BFA] Scanning build for known causes...
  122. [BFA] No failure causes found
  123. [BFA] Done. 0s
  124. Collecting metadata...
  125. Metadata collection done.
  126. Finished: FAILURE
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!