5.k8s 日常运维事项

作者: 哆啦A梦_ca52 | 来源:发表于2019-11-30 16:12 被阅读0次

5.k8s 日常运维事项
Hadoop相关文章索引（2）——Hadoop运维主题
运维内功——日常运维篇
企业选择多云管理平台六大注意事项
运维自救攻略：如何避免“锅”从天而降？
日常运维命令
运维日常命令
redis日常运维
zk日常运维
DBA日常运维

k8s 运维示例：
和运维相关的日常运维事宜

手动调整pod数量：
kubectl scale 对运行在k8s 环境中的pod 数量进行扩容(增加)或缩容(减小
当前pod数量：
root@k8s-master1:~# kubectl get deployment -n linux36
NAME READY UP-TO-DATE AVAILABLE AGE
linux36-nginx-deployment 1/1 1 1 3h22m
linux36-tomcat-app1-deployment 1/1 1 1 4h
#查看命令使用帮助
# kubectl --help | grep scale
scale Set a new size for a Deployment, ReplicaSet, Replication Controller, or Job
# kubectl scale --help
#执行扩容/缩容
# kubectl scale deployment/linux36-tomcat-app1-deployment --replicas=2 -n linux36
deployment.extensions/linux36-tomcat-app1-deployment scaled
#验证手动扩容结果
root@k8s-master1:~# kubectl get deployment -n linux36
NAME READY UP-TO-DATE AVAILABLE AGE
linux36-nginx-deployment 1/1 1 1 3h23m
linux36-tomcat-app1-deployment 2/2 2 2 4h1m

查看一个yaml几个pod
root@master:/opt/k8s-data/dockerfile/web/linux37/tomcat-app2# kubectl get  deployment  -n linux37 
NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
linux37-nginx-deployment         2/2     2            2           5h29m
linux37-tomcat-app1-deployment   1/1     1            1           10h
linux37-tomcat-app2-deployment   1/1     1            1           9h

扩建4个pod

root@master:/opt/k8s-data/dockerfile/web/linux37/tomcat-app1# kubectl scale deployment linux37-tomcat-app1-deployment --replicas=4 -n linux37 deployment.extensions linux37-tomcat-app1-deployment scaled
查看4个
root@master:/opt/k8s-data/dockerfile/web/linux37/tomcat-app1# kubectl get deployment -n linux37 | grep 4
linux37-tomcat-app1-deployment   4/4     4            4           11h
查看pod信息
root@master:/opt/k8s-data/dockerfile/web/linux37/tomcat-app1# kubectl get pod -n linux37
NAME                                              READY   STATUS    RESTARTS   AGE
linux37-nginx-deployment-76c4687b54-l2n26         1/1     Running   0          121m
linux37-nginx-deployment-76c4687b54-r6mk9         1/1     Running   0          121m
linux37-tomcat-app1-deployment-798998586b-ck7g7   1/1     Running   0          2m29s
linux37-tomcat-app1-deployment-798998586b-kb8dm   1/1     Running   0          2m29s
linux37-tomcat-app1-deployment-798998586b-lz58s   1/1     Running   0          86m
linux37-tomcat-app1-deployment-798998586b-vwdfd   1/1     Running   0          2m29s
缩减pod
root@master:/opt/k8s-data/dockerfile/web/linux37/tomcat-app1# kubectl scale deployment linux37-tomcat-app1-deployment --replicas=1 -n linux37 deployment.extensions linux37-nginx-deployment scaled

动态修改资源内容kubectl edit：
用于临时修改某些配置后需要立即生效的场景

查看资源的名称
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl get deployment -n linux37
NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
linux37-nginx-deployment         1/1     1            1           20h
linux37-tomcat-app1-deployment   1/1     1            1           25h
linux37-tomcat-app2-deployment   1/1     1            1           24h


查看pod的信息
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl describe pod linux37-nginx-deployment-76c4687b54-87ctw -n linux37

root@master:~# kubectl edit deployment linux37-nginx-deployment -n linux37
 45         image: harbor.wyh.net/linux37/nginx:v2
修改镜像为v2
再次查看pod信息就变成v2了
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl describe pod linux37-nginx-deployment-86f758d6f-z9snf -n linux37 | grep v2
    Image:          harbor.wyh.net/linux37/nginx:v2
  Normal  Pulling    3m25s  kubelet, 192.168.200.206  Pulling image "harbor.wyh.net/linux37/nginx:v2"
  Normal  Pulled     3m25s  kubelet, 192.168.200.206  Successfully pulled image "harbor.wyh.net/linux37/nginx:v2"
这个修改并不会保存到yaml文件里
edit命令临时修改还可以不建议用在永久化的配置

定义node资源标签：
lable是一个键值对，创建pod的时候会查询那些node有这个标签，只会将pod创建在符合指定label值的node节点
上。

查看node信息
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl get node
NAME              STATUS                     ROLES    AGE     VERSION
192.168.200.197   Ready                      <none>   4d22h   v1.14.7
192.168.200.198   Ready,SchedulingDisabled   master   6d22h   v1.14.7
192.168.200.206   Ready                      node     4d21h   v1.14.7
192.168.200.207   Ready                      node     4d21h   v1.14.7

打标签
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl label node 192.168.200.197 project=linux37
node/192.168.200.197 labeled
查看node信息发现多了一个标签
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl describe node 192.168.200.197 | grep  project=linux37
                    project=linux37
查看pod都运行在那些节点上
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl get pod -n linux37 -o wide
NAME                                              READY   STATUS    RESTARTS   AGE   IP               NODE              NOMINATED NODE   READINESS GATES
linux37-nginx-deployment-76c4687b54-77b9k         1/1     Running   0          19m   172.31.167.107   192.168.200.206   <none>           <none>
linux37-nginx-deployment-76c4687b54-cff5p         1/1     Running   0          19m   172.31.104.180   192.168.200.207   <none>           <none>
linux37-tomcat-app1-deployment-798998586b-lz58s   1/1     Running   1          16h   172.31.104.172   192.168.200.207   <none>           <none>
linux37-tomcat-app2-deployment-f5749f98c-lcwqm    1/1     Running   1          15h   172.31.104.171   192.168.200.207   <none>           <none>

root@master:/opt/k8s-data/yaml/web/linux37/nginx# vim nginx.yaml 
 54       nodeSelector:
 55         project: linux37
添加pod节点指定放在哪个节点的标签
执行更新pod
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl apply -f nginx.yaml 
deployment.extensions/linux37-nginx-deployment configured
service/linux37-nginx-service unchanged
查看pod的节点在哪里node信息里
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl get pod -o wide
NAME                               READY   STATUS    RESTARTS   AGE   IP               NODE              NOMINATED NODE   READINESS GATES
net-test-cd766cb69-2rfpz           1/1     Running   0          72m   172.31.180.118   192.168.200.197   <none>           <none>
net-test-cd766cb69-8lk5h           1/1     Running   0          97m   172.31.104.177   192.168.200.207   <none>           <none>
net-test-cd766cb69-94zhm           1/1     Running   0          72m   172.31.180.114   192.168.200.197   <none>           <none>
net-test-cd766cb69-vgsn7           1/1     Running   0          97m   172.31.104.178   192.168.200.207   <none>           <none>
nginx-deployment-995964986-7v2rr   1/1     Running   0          97m   172.31.104.179   192.168.200.207   <none>           <none>
查看他的project的
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl describe node 192.168.200.206  | grep 37
                    project=linux37
删除linux37的标签
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl label node 192.168.200.206 project-
node/192.168.200.206 labeled
再看一下就没有标签了
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl describe node 192.168.200.206  | grep 37

业务镜像版本升级及回滚：
在指定的deployment中通过kubectl set image指定新版本的镜像:tag 来实现更新代码的目的。
构建三个不同版本的nginx镜像，第一次使用v1版本，后组逐渐升级到v2与v3，测试镜像版本升级与回滚操作

升级到镜像到指定版本：
root@master:/opt/k8s-data/yaml/web/linux37/nginx# vim nginx.yaml 
 20         image: harbor.wyh.net/linux37/nginx:v1
修改为v1的版本
执行版本
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl apply -f nginx.yaml 

root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl get deployment -n linux37
NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
linux37-nginx-deployment         5/5     5            5           23h
linux37-tomcat-app1-deployment   1/1     1            1           37m
linux37-tomcat-app2-deployment   1/1     1            1           36m
查看现在的pod的版本
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl describe pod linux37-nginx-deployment-774844fc84-5v52d -n linux37 | grep v1
    Image:          harbor.wyh.net/linux37/nginx:v1


镜像更新命令格式为
root@master:/opt/k8s-data/yaml/web/linux37/nginx# kubectl set image deployment/linux37-nginx-deployment linux37-nginx-container=harbor.wyh.net/linux37/nginx:v2 -n linux37
查看更新后就变 成v2版本了
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl describe  pod linux37-nginx-deployment-d7d97866c-2lv5w  -n linux37 | grep v2
    Image:          harbor.wyh.net/linux37/nginx:v2
  Normal  Pulling    110s  kubelet, 192.168.200.197  Pulling image "harbor.wyh.net/linux37/nginx:v2"
回滚到上一个版本：
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl rollout undo deployment/linux37-nginx-deployment -n linux37
deployment.extensions/linux37-nginx-deployment rolled back
查看版本已经回滚到v1了
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl describe pod  linux37-nginx-deployment-774844fc84-bdc9l -n linux37 | grep v
Start Time:     Sat, 30 Nov 2019 07:08:54 +0000
    Image:          harbor.wyh.net/linux37/nginx:v1

查看历史版本
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl rollout history deployment/linux37-nginx-deployment -n linux37
deployment.extensions/linux37-nginx-deployment 
REVISION  CHANGE-CAUSE
1         <none>
2         <none>
3         <none>
回滚到指定版本
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl rollout undo deployment/linux37-nginx-deployment --to-revision=1 -n linux37
deployment.extensions/linux37-nginx-deployment rolled back

配置主机为封锁状态且不参与调度：
kubectl --help | grep cordon #警戒线
cordon Mark node as unschedulable #标记为警戒，即不参加pod调度
uncordon Mark node as schedulable #去掉警戒，即参加pod调度

查看node的信息
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl get node
NAME              STATUS                     ROLES    AGE    VERSION
192.168.200.197   Ready                      <none>   5d3h   v1.14.7
192.168.200.198   Ready,SchedulingDisabled   master   7d3h   v1.14.7
192.168.200.206   Ready                      node     5d1h   v1.14.7
192.168.200.207   Ready                      node     5d1h   v1.14.7

标记为可以在这个节点创建pod
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl uncordon 192.168.200.198
node/192.168.200.198 uncordoned

再次查看node的信息
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl get node
NAME              STATUS   ROLES    AGE    VERSION
192.168.200.197   Ready    <none>   5d3h   v1.14.7
192.168.200.198   Ready    master   7d3h   v1.14.7
192.168.200.206   Ready    node     5d1h   v1.14.7
192.168.200.207   Ready    node     5d1h   v1.14.7

设置192.168.200.198不参加调度
root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl cordon 192.168.200.198
node/192.168.200.198 cordoned

root@master:/opt/k8s-data/dockerfile/web/linux37/nginx# kubectl get node
NAME              STATUS                     ROLES    AGE    VERSION
192.168.200.197   Ready                      <none>   5d3h   v1.14.7
192.168.200.198   Ready,SchedulingDisabled   master   7d3h   v1.14.7
192.168.200.206   Ready                      node     5d1h   v1.14.7
192.168.200.207   Ready                      node     5d1h   v1.14.7

5.k8s 日常运维事项
k8s 运维示例：和运维相关的日常运维事宜扩建4个pod 动态修改资源内容kubectl edit：用于临时修改...
Hadoop相关文章索引（2）——Hadoop运维主题
hadoop运维笔记1 Hadoop集群日常运维 Hadoop运维经验杂谈 Hadoop运维笔记之调整hdfs...
运维内功——日常运维篇
以下为日常运维时每一个运维人员的内心准则！ 1.当发现故障的时候,应该先恢复再排查,实在无计可施可以试试重启; 2...
企业选择多云管理平台六大注意事项
企业选择多云管理平台六大注意事项 1、是否足够简单，学习曲线有多长 2、是否可实现自动化环境部署，日常运维作业...
运维自救攻略：如何避免“锅”从天而降？
开始这个话题之前，我们先来看看运维充（苦)实(bi)的工作日常。不被理解的工作日常 ——运维是做什么的？ ——装...
日常运维命令
计算空间du -h --max-depth=1 /path 连接数netstat -n | awk '/^tcp/...
运维日常命令
1.修改系统时间date -s "2017-11-18 10:18:00"clock -w2.查看系统的内核una...
redis日常运维
1.启动redis cd ~/redis/bin ./redis-server ../config/redis-2...
zk日常运维
日常维护操作启动ZK服务：bin/zkServer.sh start 查看ZK服务状态：bin/zkServer...
DBA日常运维
表空间： 1、查询表属于哪个表空间： 2、给表空间添加数据文件： 3、增加表空间数据文件的大小(之前初始化大小2G...