Advertisement
Guest User

Untitled

a guest
Nov 12th, 2019
72
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 5.47 KB | None | 0 0
  1. This looks like a crio bug which requires all containers to be privileged. Cherry-picking https://github.com/openshift/machine-config-operator/pull/1222 will help solve this.
  2.  
  3. grep -rn "Permission denied" ./journal.log -C 5 | less
  4.  
  5. ./journal.log-2067069-Oct 29 04:16:44 master0.beeper.os.fyre.ibm.com hyperkube[18023]: I1029 04:16:44.499764 18023 kubelet_pods.go:1346] Generating status for "etcd-member-master0.beeper.os.fyre.ibm.com_openshift-etcd(2b8c55350762ca4c7b3958480bbd7481)"
  6. ./journal.log-2067070-Oct 29 04:16:44 master0.beeper.os.fyre.ibm.com hyperkube[18023]: I1029 04:16:44.499929 18023 kubelet_node_status.go:283] Setting node annotation to enable volume controller attach/detach
  7. ./journal.log-2067071-Oct 29 04:16:44 master0.beeper.os.fyre.ibm.com hyperkube[18023]: I1029 04:16:44.502073 18023 kubelet_node_status.go:468] Recording NodeHasSufficientMemory event message for node master0.beeper.os.fyre.ibm.com
  8. ./journal.log-2067072-Oct 29 04:16:44 master0.beeper.os.fyre.ibm.com hyperkube[18023]: I1029 04:16:44.502335 18023 kubelet_node_status.go:468] Recording NodeHasNoDiskPressure event message for node master0.beeper.os.fyre.ibm.com
  9. ./journal.log-2067073-Oct 29 04:16:44 master0.beeper.os.fyre.ibm.com hyperkube[18023]: I1029 04:16:44.502435 18023 kubelet_node_status.go:468] Recording NodeHasSufficientPID event message for node master0.beeper.os.fyre.ibm.com
  10. ./journal.log:2067074:Oct 29 04:16:44 master0.beeper.os.fyre.ibm.com hyperkube[18023]: I1029 04:16:44.502715 18023 status_manager.go:382] Ignoring same status for pod "etcd-member-master0.beeper.os.fyre.ibm.com_openshift-etcd(2b8c55350762ca4c7b3958480bbd7481)", status: {Phase:Running Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2019-10-25 19:24:12 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2019-10-29 04:15:28 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [etcd-member]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2019-10-29 04:15:28 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [etcd-member]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2019-10-25 19:24:12 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.16.5.196 PodIP:10.16.5.196 StartTime:2019-10-25 19:24:12 +0000 UTC InitContainerStatuses:[{Name:discovery State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2019-10-25 18:13:26 +0000 UTC,FinishedAt:2019-10-25 18:13:26 +0000 UTC,ContainerID:cri-o://39de2ff401150d54c4768596f601ab54a36873c11cc0b066455a736d1f8199b8,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:true RestartCount:2 Image:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3d073e4de33f20162ddd17a996787db3307371e06e2c1b3aa46757799f03d194 ImageID:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3d073e4de33f20162ddd17a996787db3307371e06e2c1b3aa46757799f03d194 ContainerID:cri-o://39de2ff401150d54c4768596f601ab54a36873c11cc0b066455a736d1f8199b8} {Name:certs State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2019-10-25 18:13:27 +0000 UTC,FinishedAt:2019-10-25 18:13:27 +0000 UTC,ContainerID:cri-o://d8124134e28e408dc76e4e993a228c20d1244956500c06623e0276476a33625e,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:true RestartCount:0 Image:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6feefefa56c94e345442bd439d928cfa41ba985a256b41443325876e156c669c ImageID:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6feefefa56c94e345442bd439d928cfa41ba985a256b41443325876e156c669c ContainerID:cri-o://d8124134e28e408dc76e4e993a228c20d1244956500c06623e0276476a33625e}] ContainerStatuses:[{Name:etcd-member State:{Waiting:&ContainerStateWaiting{Reason:CrashLoopBackOff,Message:Back-off 40s restarting failed container=etcd-member pod=etcd-member-master0.beeper.os.fyre.ibm.com_openshift-etcd(2b8c55350762ca4c7b3958480bbd7481),} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:/bin/sh: line 3: /run/etcd/environment: Permission denied
  11. ./journal.log-2067075-Oct 29 04:16:44 master0.beeper.os.fyre.ibm.com hyperkube[18023]: ,StartedAt:2019-10-29 04:16:16 +0000 UTC,FinishedAt:2019-10-29 04:16:16 +0000 UTC,ContainerID:cri-o://a5167212c9a508327681abca9e8f167460306e3ee53f8d6bf2079c0e0efb1304,}} Ready:false RestartCount:5 Image:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd863c15756acf03fa8bfb77e6ec28921fe5edaf82684574b75246eaf39377d4 ImageID:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd863c15756acf03fa8bfb77e6ec28921fe5edaf82684574b75246eaf39377d4 ContainerID:cri-o://a5167212c9a508327681abca9e8f167460306e3ee53f8d6bf2079c0e0efb1304} {Name:etcd-metrics State:{Waiting:nil Running:&ContainerStateRunning{StartedAt:2019-10-25 18:13:50 +0000 UTC,} Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:2019-10-25 18:13:29.229445 I | etcdmain: ServerTLS: cert = /etc/ssl/etcd/system:etcd-metric:etcd-0.beeper.os.fyre.ibm.com.crt, key = /etc/ssl/etcd/system:etcd-metric:etcd-0.beeper.os.fyre.ibm.com.key, ca = /etc/ssl/etcd/metric-ca.crt, trusted-ca = , client-cert-auth = false, crl-file =
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement