-
Choerodon平台版本: 0.13.0
-
遇到问题的执行步骤:
平台运行使用了一段时间没出现问题,然后按照开发文档开发了一个demo项目,在持续集成maven打包的时候出错了,然后gitlab runner里面重试了几次,最后gitlab就挂掉了
重启过gitlab,启动失败 -
文档地址:
-
环境信息(如:节点信息):
-
报错日志:
-
原因分析:
-
疑问:
Choerodon平台版本: 0.13.0
遇到问题的执行步骤:
平台运行使用了一段时间没出现问题,然后按照开发文档开发了一个demo项目,在持续集成maven打包的时候出错了,然后gitlab runner里面重试了几次,最后gitlab就挂掉了
文档地址:
环境信息(如:节点信息):
报错日志:
原因分析:
疑问:
Hi,参考下这个
kubectl describe pod gitlab-85d587c458-nmm28 -n c7n-system
kubectl logs gitlab-85d587c458-nmm28 -n c7n-system
请问一下还有没有别的方法可以尝试一下?
重启下nfs服务试试
请问下,这个可能是什么原因导致gitlab服务down掉了呢?
可能是nfs服务内部出现了问题
gitlab再次down掉,还是一样的报错,然后我看了一下nfs服务的日志,如下,还请帮忙看看是否有问题
I0218 02:20:07.235199 1 main.go:63] Provisioner choerodon.io/nfs-provisioner specified
I0218 02:20:07.235374 1 main.go:87] Setting up NFS server!
I0218 02:20:07.830997 1 server.go:144] starting RLIMIT_NOFILE rlimit.Cur 1048576, rlimit.Max 1048576
I0218 02:20:07.831045 1 server.go:155] ending RLIMIT_NOFILE rlimit.Cur 1048576, rlimit.Max 1048576
I0218 02:20:07.849741 1 server.go:129] Running NFS server!
I0218 02:20:12.915751 1 leaderelection.go:185] attempting to acquire leader lease kube-system/choerodon.io-nfs-provisioner…
I0218 02:20:30.328591 1 leaderelection.go:194] successfully acquired lease kube-system/choerodon.io-nfs-provisioner
I0218 02:20:30.328891 1 controller.go:631] Starting provisioner controller choerodon.io/nfs-provisioner_nfs-provisioner-65bdd5f955-mxwzz_b8d1df5d-3323-11e9-891c-0a580ae940a9!
I0218 02:20:30.336526 1 event.go:221] Event(v1.ObjectReference{Kind:“Endpoints”, Namespace:“kube-system”, Name:“choerodon.io-nfs-provisioner”, UID:“295d24bf-1507-11e9-9a59-4ccc6aaa26b9”, APIVersion:“v1”, ResourceVersion:“4601498”, FieldPath:""}): type: ‘Normal’ reason: ‘LeaderElection’ nfs-provisioner-65bdd5f955-mxwzz_b8d1df5d-3323-11e9-891c-0a580ae940a9 became leader
I0218 02:20:30.429065 1 controller.go:680] Started provisioner controller choerodon.io/nfs-provisioner_nfs-provisioner-65bdd5f955-mxwzz_b8d1df5d-3323-11e9-891c-0a580ae940a9!
_E0218 02:22:16.449022 1 main.go:97] NFS server Exited Unexpectedly with err: ganesha.nfsd failed with error: signal: segmentation fault, output: _
I0218 02:22:17.449294 1 server.go:129] Running NFS server!
_E0219 03:41:22.892929 1 main.go:97] NFS server Exited Unexpectedly with err: ganesha.nfsd failed with error: signal: segmentation fault, output: _
I0219 03:41:23.893255 1 server.go:129] Running NFS server!
重启nfs后,再重启gitlab,gitlab恢复正常, 重启nfs服务的日志如下,
I0219 05:37:31.504708 1 main.go:63] Provisioner choerodon.io/nfs-provisioner specified
I0219 05:37:31.504786 1 main.go:87] Setting up NFS server!
I0219 05:37:31.644002 1 server.go:144] starting RLIMIT_NOFILE rlimit.Cur 1048576, rlimit.Max 1048576
I0219 05:37:31.644017 1 server.go:155] ending RLIMIT_NOFILE rlimit.Cur 1048576, rlimit.Max 1048576
I0219 05:37:31.644503 1 server.go:129] Running NFS server!
I0219 05:37:36.680246 1 leaderelection.go:185] attempting to acquire leader lease kube-system/choerodon.io-nfs-provisioner…
I0219 05:37:54.081985 1 leaderelection.go:194] successfully acquired lease kube-system/choerodon.io-nfs-provisioner
I0219 05:37:54.082232 1 controller.go:631] Starting provisioner controller choerodon.io/nfs-provisioner_nfs-provisioner-65bdd5f955-qbqnt_76aac593-3408-11e9-9604-0a580ae940ac!
I0219 05:37:54.082208 1 event.go:221] Event(v1.ObjectReference{Kind:“Endpoints”, Namespace:“kube-system”, Name:“choerodon.io-nfs-provisioner”, UID:“295d24bf-1507-11e9-9a59-4ccc6aaa26b9”, APIVersion:“v1”, ResourceVersion:“4803561”, FieldPath:""}): type: ‘Normal’ reason: ‘LeaderElection’ nfs-provisioner-65bdd5f955-qbqnt_76aac593-3408-11e9-9604-0a580ae940ac became leader
I0219 05:37:54.182572 1 controller.go:680] Started provisioner controller choerodon.io/nfs-provisioner_nfs-provisioner-65bdd5f955-qbqnt_76aac593-3408-11e9-9604-0a580ae940ac!
_E0219 05:39:23.493498 1 main.go:97] NFS server Exited Unexpectedly with err: ganesha.nfsd failed with error: signal: segmentation fault, output: _
I0219 05:39:24.493682 1 server.go:129] Running NFS server!