0.17升级到0.18，gitlab-ci认证失败

duhongx · 2020 年3 月 25 日 03:40

Choerodon平台版本: 0.18.0
遇到问题的执行步骤:

今天上午将猪齿鱼从0.17升级到0.18，升级过程中有一个步骤对不上，

这个步骤本地的猪齿鱼登录后，在管理界面的平台服务->平台接口下面找不到devops-service，如下

现在遇到的问题是gitlab的ci中访问harbor失败，如下

这个要怎么修改才能解决？

文档地址:
https://choerodon.io/zh/docs/installation-configuration/update/more/0.17-to-0.18/
环境信息(如:节点信息):
报错日志:
原因分析:

提出您分析问题的过程,以便我们能更准确的找到问题所在
疑问:

提出您对于遇到和解决该问题时的疑问

ChangpingShi · 2020 年3 月 25 日 03:48

执行这个操作了么

duhongx · 2020 年3 月 25 日 04:13

对的，就是这个没办法执行，现在是gitlab的ci里提示harbor 鉴权失败

duhongx · 2020 年3 月 25 日 05:48

gitlab ci里的harbor的相关信息是配置在devops-service对应的环境变量里么？

[root@dev ldap]# kubectl get cm -n c7n-system devops-service -o yaml
apiVersion: v1
data:
  application.yml: |
    spring:
      datasource:
        url: jdbc:mysql://localhost/devops_service?useUnicode=true&characterEncoding=utf-8&useSSL=false
        username: choerodon
        password: choerodon
      redis:
        host: localhost
      http:
        encoding:
         charset: UTF-8
         force: true
         enabled: true
    services:
      harbor:
        baseUrl: "harbor.example.com"
        username: "123456"
        password: "123456"
        insecureSkipTlsVerify: false
      gitlab:
        url: "gitlab.example.com"
        sshUrl: "gitlab.example.com"
        password: 123456
        projectLimit: 100
      helm:
        url: "helm.example.com"
      sonarqube:
        url: "http://sonarqube.example.com"
      gateway:
        url: "http://api.example.com"
    template:
      url: https://github.com/choerodon/choerodon-devops-templates.git
      version: 0.13.0
    choerodon:
      saga:
        consumer:
          thread-num: 5  # 消费线程数
          poll-interval: 3 # 拉取消息的间隔(秒)，默认1秒
          enabled: true # 是否启用消费端
      schedule:
        consumer:
          enabled: true # 启用任务调度消费端
          thread-num: 1 # 任务调度消费线程数
          poll-interval-ms: 1000 # 拉取间隔，默认1000毫秒
      websocket:
        max-redis-msg-listener-concurrency: 500
        security: false
    agent:
      version: "0.5.0"
      serviceUrl: "agent.example.com"
      certManagerUrl: "agent.example.com"
      repoUrl: "helm.example.com"
    eureka:
      instance:
        preferIpAddress: true
        leaseRenewalIntervalInSeconds: 1
        leaseExpirationDurationInSeconds: 3
      client:
        serviceUrl:
          defaultZone: http://localhost:8000/eureka/
        registryFetchIntervalSeconds: 1
    mybatis:
      mapperLocations: classpath*:/mapper/*.xml
      configuration:
        mapUnderscoreToCamelCase: true
    feign:
      hystrix:
        shareSecurityContext: true
        command:
          default:
            execution:
              isolation:
                thread:
                  timeoutInMilliseconds: 30000
    ribbon:
      ConnectTimeout: 50000
      ReadTimeout: 50000

    asgard-servie:
      ribbon:
        ConnectTimeout: 50000
        ReadTimeout: 50000

    cert:
      testCert: false # 证书是否为测试证书
    logging:
      level:
        io.choerodon.event.consumer: debug
      config: classpath:logback-spring.xml
kind: ConfigMap
metadata:
  annotations:
    choerodon.io/feature: spring-cloud-config
    choerodon.io/service: devops-service
    choerodon.io/version: 0.16.4
  creationTimestamp: "2019-05-18T04:51:23Z"
  name: devops-service
  namespace: c7n-system
  resourceVersion: "1906793"
  selfLink: /api/v1/namespaces/c7n-system/configmaps/devops-service
  uid: 95d19509-7928-11e9-8298-20040ff6e100

修改如下内容

        baseUrl: "harbor.example.com"
        username: "123456"
        password: "123456"

能解决问题么？

ChangpingShi · 2020 年3 月 25 日 05:53

1.0.18.13以上版本，限制了使用harbor admin用户去推送镜像，你执行下同步harbor权限就可以了。

2.为什么不能执行来着，请截图告知
3.改成harbor.example.com不能解决问题

duhongx · 2020 年3 月 25 日 06:05

执行下同步harbor权限，这个怎么执行？

文档中是在管理界面的平台服务–>平台接口–>devops-service，我本地的猪齿鱼管理界面的平台服务–>平台接口中没有devops-service呀，

duhongx · 2020 年3 月 25 日 06:07

本地的猪齿鱼管理界面的平台服务–>平台接口中没有devops-service，这个要怎么解决？按照0.17升级到0.18的文档来操作的，每个服务升级完成后通过curl检查都是UP的。但就是平台服务–>平台接口中没有devops-service。

ChangpingShi · 2020 年3 月 25 日 06:17

那你先手动调用下devops的/v1/upgrade?version=0.18.13这个接口

duhongx · 2020 年3 月 25 日 06:21

[root@dev ldap]# kubectl exec -it -n c7n-system devops-service-75684f6778-mrkhp /bin/bash
root@devops-service-75684f6778-mrkhp:/# ss -tnlp
bash: ss: command not found
root@devops-service-75684f6778-mrkhp:/# ps -ef
UID         PID   PPID  C STIME TTY          TIME CMD
root          1      0  0 00:59 ?        00:00:00 tini -- /bin/sh -c /enterpoint.sh java $JAVA_OPTS $SKYWALKING_OPTS -jar /devops-service.jar
root          6      1  0 00:59 ?        00:00:00 /bin/sh -c /enterpoint.sh java $JAVA_OPTS $SKYWALKING_OPTS -jar /devops-service.jar
root          7      6  2 00:59 ?        00:09:08 java -jar /devops-service.jar
root      16597      0  0 06:19 pts/0    00:00:00 /bin/bash
root      16608  16597  0 06:19 pts/0    00:00:00 ps -ef
root@devops-service-75684f6778-mrkhp:/# curl devops-service:8060/v1/upgrade?version=0.18.13
1585117207555root@devops-service-75684f6778-mrkhp:/# 
root@devops-service-75684f6778-mrkhp:/#

是这样执行么？

duhongx · 2020 年3 月 25 日 06:25

[root@dongfang-his ldap]# kubectl get pods -n c7n-system devops-service-75684f6778-mrkhp -o yaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    choerodon.io/metrics-group: spring-boot
    choerodon.io/metrics-path: /actuator/prometheus
  creationTimestamp: "2020-03-25T00:59:23Z"
  generateName: devops-service-75684f6778-
  labels:
    choerodon.io/metrics-port: "8061"
    choerodon.io/release: devops-service
    choerodon.io/service: devops-service
    choerodon.io/version: 0.18.18
    pod-template-hash: 75684f6778
  name: devops-service-75684f6778-mrkhp
  namespace: c7n-system
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: devops-service-75684f6778
    uid: 6df5b34b-c9b4-45cb-a90f-7a0f13e5ecbd
  resourceVersion: "67947884"
  selfLink: /api/v1/namespaces/c7n-system/pods/devops-service-75684f6778-mrkhp
  uid: bae4e85b-d437-4777-bba7-008249fb9f54
spec:
  containers:
  - env:
    - name: AGENT_CERTMANAGERURL
      value: https://openchart.choerodon.com.cn/choerodon/infra/
    - name: AGENT_REPOURL
      value: https://openchart.choerodon.com.cn/choerodon/c7n/
    - name: AGENT_SERVICEURL
      value: ws://devops.his.cn/agent/
    - name: AGENT_VERSION
      value: 0.18.1
    - name: EUREKA_CLIENT_SERVICEURL_DEFAULTZONE
      value: http://register-server.c7n-system:8000/eureka/
    - name: PRO_API_HOST
      value: gateway.choerodon.com.cn
    - name: PRO_CLIENT_ID
      value: devops
    - name: PRO_COOKIE_SERVER
      value: choerodon.com.cn
    - name: PRO_DEVOPS_HOST
      value: localhost:8060
    - name: PRO_FILE_SERVER
      value: http://minio.choerodon.com.cn
    - name: PRO_HEADER_TITLE_NAME
      value: Choerodon
    - name: PRO_HTTP
      value: http
    - name: PRO_LOCAL
      value: "true"
    - name: PRO_TITLE_NAME
      value: Choerodon
    - name: SECURITY_IGNORED
      value: /ci,/webhook,/v2/api-docs,/agent/**,/ws/**,/webhook/**
    - name: SERVICES_GATEWAY_URL
      value: http://api.his.cn
    - name: SERVICES_GITLAB_PASSWORD
      value: password
    - name: SERVICES_GITLAB_PROJECTLIMIT
      value: "100"
    - name: SERVICES_GITLAB_SSHURL
      value: gitlab.his.cn:2289
    - name: SERVICES_GITLAB_URL
      value: http://gitlab.his.cn
    - name: SERVICES_HARBOR_BASEURL
      value: https://registry.his.cn
    - name: SERVICES_HARBOR_INSECURESKIPTLSVERIFY
      value: "true"
    - name: SERVICES_HARBOR_PASSWORD
      value: Harbor12345
    - name: SERVICES_HARBOR_USERNAME
      value: admin
    - name: SERVICES_HELM_URL
      value: http://chart.his.cn
    - name: SERVICES_SONARQUBE_PASSWORD
      value: admin
    - name: SERVICES_SONARQUBE_URL
      value: http://sonar.his.cn
    - name: SERVICES_SONARQUBE_USERNAME
      value: admin
    - name: SPRING_CLOUD_CONFIG_ENABLED
      value: "true"
    - name: SPRING_CLOUD_CONFIG_URI
      value: http://register-server.c7n-system:8000/
    - name: SPRING_DATASOURCE_PASSWORD
      value: password
    - name: SPRING_DATASOURCE_URL
      value: jdbc:mysql://c7n-mysql.c7n-system.svc:3306/devops_service?useUnicode=true&characterEncoding=utf-8&useSSL=false
    - name: SPRING_DATASOURCE_USERNAME
      value: choerodon
    - name: SPRING_REDIS_DATABASE
      value: "11"
    - name: SPRING_REDIS_HOST
      value: c7n-redis.c7n-system.svc
    - name: TEMPLATE_URL
      value: https://github.com/choerodon/choerodon-devops-templates.git
    - name: TEMPLATE_VERSION
      value: 0.17.0
    image: registry.cn-shanghai.aliyuncs.com/choerodon/devops-service:0.18.18
    imagePullPolicy: IfNotPresent
    name: devops-service
    ports:
    - containerPort: 8060
      name: http
      protocol: TCP
    readinessProbe:
      exec:
        command:
        - /bin/sh
        - -c
        - curl -s localhost:8061/actuator/health --fail && nc -z localhost 8060
      failureThreshold: 3
      initialDelaySeconds: 60
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 60
    resources:
      limits:
        memory: 4Gi
      requests:
        memory: 2Gi
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-mjt6w
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: 192.168.1.213
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  volumes:
  - name: default-token-mjt6w
    secret:
      defaultMode: 420
      secretName: default-token-mjt6w
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2020-03-25T00:59:23Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2020-03-25T01:00:31Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2020-03-25T01:00:31Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2020-03-25T00:59:23Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: docker://8134f0a6eeaeef2c371668dcec0e395fa91761aee3dd74f4020153c3c7f8424e
    image: registry.cn-shanghai.aliyuncs.com/choerodon/devops-service:0.18.18
    imageID: docker-pullable://registry.cn-shanghai.aliyuncs.com/choerodon/devops-service@sha256:1b3c1fc4d9ec62a1c809016cd627c3f7e82a2390a66e2bcc31845847f4946386
    lastState: {}
    name: devops-service
    ready: true
    restartCount: 0
    state:
      running:
        startedAt: "2020-03-25T00:59:28Z"
  hostIP: 192.168.1.213
  phase: Running
  podIP: 172.22.119.182
  qosClass: Burstable
  startTime: "2020-03-25T00:59:23Z"

devops-service的信息是这样的

ChangpingShi · 2020 年3 月 25 日 06:25

能通过网关路由，带token，用postMan执行么

duhongx · 2020 年3 月 25 日 06:26

token是怎么获取的？

ChangpingShi · 2020 年3 月 25 日 06:28

访问界面任何接口都是有token的，你随意取一个就行

duhongx · 2020 年3 月 25 日 06:28

好的，我试试

duhongx · 2020 年3 月 25 日 06:38

还是这样呀？

ChangpingShi · 2020 年3 月 25 日 06:42

你看下devops-servic的日志，有没有这样的log同步创建harbor用户账号成功

ChangpingShi · 2020 年3 月 25 日 06:45

或者你测试现在ci能不能执行成功

duhongx · 2020 年3 月 25 日 06:46

devops-service

2020-03-25 14:46:02.661  INFO 7 --- [nio-8060-exec-9] i.c.d.a.s.i.DevopsCheckLogServiceImpl    : start upgrade task
2020-03-25 14:46:02.662  INFO 7 --- [ devops-upgrade] i.c.d.a.s.i.DevopsCheckLogServiceImpl    : version not matched

version 不匹配呀

duhongx · 2020 年3 月 25 日 06:49

duhongx · 2020 年3 月 25 日 06:49

现在可以了，这个是什么问题呀？就是从0.17.x升级到0.18的高版本，默认不能使用harbor的admin来登录harbor?