Problem with Task Fix Issue with VolumeMounts in Kubernetes

Hello Community,

i am stucking on the Fix Issue with VolumeMounts in Kubernetes Task. This is the Task Description:

"We deployed a Nginx and PHPFPM based application on Kubernetes cluster last week and it had been working fine. This morning one of the team members made a change somewhere which caused some issues, and the application stopped working. Please look into the issue and fix it:

The pod name is nginx-phpfpm and configmap name is nginx-config, Nginx is running on port 8099. Figure out the issues and fix them.

Once you think issue is fixed copy /tmp/index.php from jump host to nginx-container under nginx document root and try to curl the nginx URL from nginx-container itself, you should be able to run the php page you copied. This step isn’t mandatory but it will be helpful to test the setup.

Note: The kubectl utility on jump_host has been configured to work with the kubernetes cluster."

This is the pod config:

thor@jump_host ~$ cat pod.yml
apiVersion: v1
kind: Pod
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{“apiVersion”:“v1”,“kind”:“Pod”,“metadata”:{“annotations”:{},“name”:“nginx-phpfpm”,“namespace”:“default”},“spec”:{“containers”:[{“image”:“php:7.2-fpm”,“name”:“php-fpm-container”,“volumeMounts”:[{“mountPath”:“/usr/share/nginx/html”,“name”:“shared-files”}]},{“image”:“nginx:latest”,“name”:“nginx-container”,“volumeMounts”:[{“mountPath”:“/var/www/html”,“name”:“shared-files”},{“mountPath”:“/etc/nginx/nginx.conf”,“name”:“nginx-config-volume”,“subPath”:“nginx.conf”}]}],“volumes”:[{“emptyDir”:{},“name”:“shared-files”},{“configMap”:{“name”:“nginx-config”},“name”:“nginx-config-volume”}]}}
creationTimestamp: “2020-12-29T07:45:51Z”
managedFields:

  • apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
    f:status:
    f:conditions:
    .: {}
    k:{“type”:“PodScheduled”}:
    .: {}
    f:lastProbeTime: {}
    f:lastTransitionTime: {}
    f:message: {}
    f:reason: {}
    f:status: {}
    f:type: {}
    manager: kube-scheduler
    operation: Update
    time: “2020-12-29T07:45:51Z”
  • apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
    f:metadata:
    f:annotations:
    .: {}
    f:kubectl.kubernetes.io/last-applied-configuration: {}
    f:spec:
    f:containers:
    k:{“name”:“nginx-container”}:
    .: {}
    f:image: {}
    f:imagePullPolicy: {}
    f:name: {}
    f:resources: {}
    f:terminationMessagePath: {}
    f:terminationMessagePolicy: {}
    f:volumeMounts:
    .: {}
    k:{“mountPath”:“/etc/nginx/nginx.conf”}:
    .: {}
    f:mountPath: {}
    f:name: {}
    f:subPath: {}
    k:{“mountPath”:“/var/www/html”}:
    .: {}
    f:mountPath: {}
    f:name: {}
    k:{“name”:“php-fpm-container”}:
    .: {}
    f:image: {}
    f:imagePullPolicy: {}
    f:name: {}
    f:resources: {}
    f:terminationMessagePath: {}
    f:terminationMessagePolicy: {}
    f:volumeMounts:
    .: {}
    k:{“mountPath”:“/usr/share/nginx/html”}:
    .: {}
    f:mountPath: {}
    f:name: {}
    f:dnsPolicy: {}
    f:enableServiceLinks: {}
    f:restartPolicy: {}
    f:schedulerName: {}
    f:securityContext: {}
    f:terminationGracePeriodSeconds: {}
    f:volumes:
    .: {}
    k:{“name”:“nginx-config-volume”}:
    .: {}
    f:configMap:
    .: {}
    f:defaultMode: {}
    f:name: {}
    f:name: {}
    k:{“name”:“shared-files”}:
    .: {}
    f:emptyDir: {}
    f:name: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: “2020-12-29T07:45:51Z”
  • apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
    f:status:
    f:conditions:
    k:{“type”:“ContainersReady”}:
    .: {}
    f:lastProbeTime: {}
    f:lastTransitionTime: {}
    f:message: {}
    f:reason: {}
    f:status: {}
    f:type: {}
    k:{“type”:“Initialized”}:
    .: {}
    f:lastProbeTime: {}
    f:lastTransitionTime: {}
    f:status: {}
    f:type: {}
    k:{“type”:“Ready”}:
    .: {}
    f:lastProbeTime: {}
    f:lastTransitionTime: {}
    f:message: {}
    f:reason: {}
    f:status: {}
    f:type: {}
    f:containerStatuses: {}
    f:hostIP: {}
    f:startTime: {}
    manager: kubelet
    operation: Update
    time: “2020-12-29T07:45:58Z”
    name: nginx-phpfpm
    namespace: default
    resourceVersion: “615”
    selfLink: /api/v1/namespaces/default/pods/nginx-phpfpm
    uid: 0173c130-4ca1-443f-8d78-f7361d2fe9d9
    spec:
    containers:
  • image: php:7.2-fpm
    imagePullPolicy: IfNotPresent
    name: php-fpm-container
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    • mountPath: /usr/share/nginx/html
      name: shared-files
    • mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-62w57
      readOnly: true
  • image: nginx:latest
    imagePullPolicy: Always
    name: nginx-container
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    • mountPath: /var/www/html
      name: shared-files
    • mountPath: /etc/nginx/nginx.conf
      name: nginx-config-volume
      subPath: nginx.conf
    • mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-62w57
      readOnly: true
      dnsPolicy: ClusterFirst
      enableServiceLinks: true
      nodeName: node01
      preemptionPolicy: PreemptLowerPriority
      priority: 0
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: default
      serviceAccountName: default
      terminationGracePeriodSeconds: 30
      tolerations:
  • effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  • effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
    volumes:
  • emptyDir: {}
    name: shared-files
  • configMap:
    defaultMode: 420
    name: nginx-config
    name: nginx-config-volume
  • name: default-token-62w57
    secret:
    defaultMode: 420
    secretName: default-token-62w57
    status:
    conditions:
  • lastProbeTime: null
    lastTransitionTime: “2020-12-29T07:45:58Z”
    status: “True”
    type: Initialized
  • lastProbeTime: null
    lastTransitionTime: “2020-12-29T07:45:58Z”
    message: ‘containers with unready status: [php-fpm-container nginx-container]’
    reason: ContainersNotReady
    status: “False”
    type: Ready
  • lastProbeTime: null
    lastTransitionTime: “2020-12-29T07:45:58Z”
    message: ‘containers with unready status: [php-fpm-container nginx-container]’
    reason: ContainersNotReady
    status: “False”
    type: ContainersReady
  • lastProbeTime: null
    lastTransitionTime: “2020-12-29T07:45:58Z”
    status: “True”
    type: PodScheduled
    containerStatuses:
  • image: nginx:latest
    imageID: “”
    lastState: {}
    name: nginx-container
    ready: false
    restartCount: 0
    started: false
    state:
    waiting:
    reason: ContainerCreating
  • image: php:7.2-fpm
    imageID: “”
    lastState: {}
    name: php-fpm-container
    ready: false
    restartCount: 0
    started: false
    state:
    waiting:
    reason: ContainerCreating
    hostIP: 172.17.0.39
    phase: Pending
    qosClass: BestEffort
    startTime: “2020-12-29T07:45:58Z”

and thats the corresponding CM:

thor@jump_host ~$ kubectl get cm nginx-config -o yaml
apiVersion: v1
data:
nginx.conf: |
events {
}
http {
server {
listen 8099 default_server;
listen [::]:8099 default_server;

    # Set nginx to serve files from the shared volume!
    root /var/www/html;
    index  index.html index.htm index.php;
    server_name _;
    location / {
      try_files $uri $uri/ =404;
    }
    location ~ \.php$ {
      include fastcgi_params;
      fastcgi_param REQUEST_METHOD $request_method;
      fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
      fastcgi_pass 127.0.0.1:9000;
    }
  }
}

kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{“apiVersion”:“v1”,“data”:{“nginx.conf”:“events {\n}\nhttp {\n server {\n listen 8099 default_server;\n listen [::]:8099 default_server;\n\n # Set nginx to serve files from the shared volume!\n root /var/www/html;\n index index.html index.htm index.php;\n server_name _;\n location / {\n try_files $uri $uri/ =404;\n }\n location ~ \.php$ {\n include fastcgi_params;\n fastcgi_param REQUEST_METHOD $request_method;\n fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;\n fastcgi_pass 127.0.0.1:9000;\n }\n }\n}\n”},“kind”:“ConfigMap”,“metadata”:{“annotations”:{},“name”:“nginx-config”,“namespace”:“default”}}
creationTimestamp: “2020-12-29T07:45:51Z”
managedFields:

  • apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
    f:data:
    .: {}
    f:nginx.conf: {}
    f:metadata:
    f:annotations:
    .: {}
    f:kubectl.kubernetes.io/last-applied-configuration: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: “2020-12-29T07:45:51Z”
    name: nginx-config
    namespace: default
    resourceVersion: “574”
    selfLink: /api/v1/namespaces/default/configmaps/nginx-config
    uid: bc399247-fdd8-46ba-b49a-07b164587a27

the VolumeMount points to /var/www/html, same as in CM.
Tried to copy the File to Container with:

kubectl cp /tmp/index.php nginx-phpfpm:/var/www/html -c nginx-container

but still get failures:

thor@jump_host ~$ kubectl logs nginx-phpfpm -c nginx-container
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
127.0.0.1 - - [29/Dec/2020:07:53:20 +0000] “GET / HTTP/1.1” 404 27 “-” “curl/7.64.0”
2020/12/29 07:53:20 [error] 28#28: *1 FastCGI sent in stderr: “Primary script unknown” while reading response header from upstream, client: 127.0.0.1, server: _, request: “GET / HTTP/1.1”, upstream: “fastcgi://127.0.0.1:9000”, host: “localhost:8099”

I am really unsure whats the solution here so asking Community for help, may i really just overlooked a small piece but all help to understand this one is much appreciated.

Hi,

The nginx config sets the document root to /var/www/html however the volume is mounted at /usr/share/www/html. Delete the pod and set the nginx-container volume mount to /var/www/html. Then copy the index.php file and curl localhost:8099 from the nginx-container. You should be able to view the php you copied. Give it a go again. Good luck.

3 Likes

Thanks adeelak, that helped me a lot and got the task finished within minutes, much appreciate your help!

Martin,

I am new to Kubernetes. I saved what I got from
kubectl get pod nginx-phpfpm -o yaml
command to a file and changed ‘/usr/share/nginx/’ to ‘/var/www/’ in 3 places. Now which command I need to give to apply this file?

Hello, @supriya.pannamboor
You can use kubectl replace command.

kubectl replace -f <file-name>.yaml --force

It will delete the existing one and will recreate new with latest changes accordingly YAML file.

1 Like

player001,

Thank you so much for your help. It worked for me.

Hi @Tej-Singh-Rana , I made the changes accordingly in the yaml file , however im not able to figure out what all unnecessary fields to remove from the yaml file . I’m getting this error.

error: error validating “/tmp/po.yaml”: error validating data: invalid object to validate; if you choose to ignore these errors, turn validation off with --validate=false


@mihsan @Tej-Singh-Rana @rahul456 @mmumshad Task failed but doesnt show any reason for that. Could you please take a look? Thank you.

@oqtay.tasinov Have you refer the task to review. If so can you share the link here to have a look?