Welcome to tom’s docs!¶
Contents:
kubernetes¶
Contents:
k8s install¶
Contents:
before install¶
- 升级内核至 4.10 以上
可以参考 http://blogtt.readthedocs.io/en/latest/ubuntu/update-kernel/index.html
kubeadm-init-before-v1.8.3¶
在运行 kubeadm init 之前的动作
kubelet 服务检查¶
后来发现，在这里应该测试一下 kubelet.service。
原来，虽然我 apt install kubelet , 但是， 遗留了之前 kubeadm 的一些配置.(应该把它们清空的.) 如: /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
让我们来吧。
删除这个新安装的 kubelet¶
root@km:/etc/cni/net.d#
root@km:/etc/cni/net.d# apt remove kubelet
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
ebtables golang-1.8-go golang-1.8-race-detector-runtime golang-1.8-src kubernetes-cni socat
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
kubeadm kubelet
0 upgraded, 0 newly installed, 2 to remove and 11 not upgraded.
After this operation, 274 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 113087 files and directories currently installed.)
Removing kubeadm (1.8.3-00) ...
Removing kubelet (1.8.3-00) ...
root@km:/etc/cni/net.d#
root@km: ~# cd /lib/systemd/system
root@km:/lib/systemd/system# ls kube*
ls: cannot access 'kube*': No such file or directory
root@km:/lib/systemd/system#
删除原 conf 文件¶
删除从 /lib/systemd/system/kubelet.service 下看到的这个 conf 文件
root@km:/opt/cni/bin# cd /etc/systemd/system/kubelet.service.d/
root@km:/etc/systemd/system/kubelet.service.d# ls
10-kubeadm.conf
root@km:/etc/systemd/system/kubelet.service.d# rm 10-kubeadm.conf
root@km:/etc/systemd/system/kubelet.service.d# cd ..
root@km:/etc/systemd/system# rmdir kubelet.service.d/
root@km:/etc/systemd/system#
再安装 kubelet¶
root@km:/lib/systemd/system# apt install kubelet
Reading package lists... Done
Setting up kubelet (1.8.3-00) ...
root@km:/lib/systemd/system# systemctl start kubelet
root@km:/lib/systemd/system# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2017-11-17 17:34:35 CST; 1s ago
Docs: http://kubernetes.io/docs/
Process: 25074 ExecStart=/usr/bin/kubelet (code=exited, status=1/FAILURE)
Main PID: 25074 (code=exited, status=1/FAILURE)
Nov 17 17:34:35 km systemd[1]: kubelet.service: Unit entered failed state.
Nov 17 17:34:35 km systemd[1]: kubelet.service: Failed with result 'exit-code'.
root@km:/lib/systemd/system#
还出错了
- ::
root@km:/lib/systemd/system# cat kubelet.service [Unit] Description=kubelet: The Kubernetes Node Agent Documentation=http://kubernetes.io/docs/
[Service] ExecStart=/usr/bin/kubelet Restart=always StartLimitInterval=0 RestartSec=10
[Install] WantedBy=multi-user.target root@km:/lib/systemd/system# journalctl -xe ….. Nov 17 17:36:51 km kubelet[25463]: error: failed to run Kubelet: Running with swap on is not supported, please disable swap! or set –fail-swap-on flag to false. /proc/swaps contained: [Filename
从信息中看到，要把 swap 关闭，或者带 –fail-swap-on 参数。 那就带参数试一下先。
root@km:/lib/systemd/system# kubelet --fail-swap-on=false
成功逃过这个错误。 但是报了另一个错误。
root@km:/lib/systemd/system# kubelet --fail-swap-on=false
W1117 17:48:12.355020 27348 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
I1117 17:48:12.361647 27348 docker_service.go:207] Docker cri networking managed by kubernetes.io/no-op
error: failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "cgroupfs" is different from docker cgroup driver: "systemd"
这个显然与 docker 相关。
root@km:/lib/systemd/system# cat /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": ["https://0d6wdn2y.mirror.aliyuncs.com"]
}
那好了，先把这个后来加进来的 “exec-opts”: [“native.cgroupdriver=systemd”], 删除了吧。
root@km:/lib/systemd/system# cat /etc/docker/daemon.json {
“registry-mirrors”: [“https://0d6wdn2y.mirror.aliyuncs.com”]} root@km:/lib/systemd/system# sudo systemctl restart docker … root@km:/lib/systemd/system# kubelet –fail-swap-on=false I1117 17:52:52.954982 28490 feature_gate.go:156] feature gates: map[] I1117 17:52:52.955098 28490 controller.go:114] kubelet config controller: starting controller I1117 17:52:52.955117 28490 controller.go:118] kubelet config controller: validating combination of defaults and flags I1117 17:52:52.974917 28490 client.go:75] Connecting to docker on unix:///var/run/docker.sock I1117 17:52:52.974977 28490 client.go:95] Start docker client with request timeout=2m0s W1117 17:52:52.977139 28490 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d I1117 17:52:52.983192 28490 feature_gate.go:156] feature gates: map[] W1117 17:52:52.983420 28490 server.go:289] –cloud-provider=auto-detect is deprecated. The desired cloud provider should be set explicitly W1117 17:52:52.983465 28490 server.go:324] standalone mode, no API client I1117 17:52:52.983810 28490 manager.go:149] cAdvisor running in container: “/sys/fs/cgroup/cpu,cpuacct/user.slice” W1117 17:52:53.017865 28490 manager.go:157] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused W1117 17:52:53.018035 28490 manager.go:166] unable to connect to CRI-O api service: Get http://%2Fvar%2Frun%2Fcrio.sock/info: dial unix /var/run/crio.sock: connect: no such file or directory I1117 17:52:53.049774 28490 fs.go:139] Filesystem UUIDs: map[90ef4dc9-3b9e-4031-ad7f-209b328d2f3b:/dev/sda1 2017-02-15-20-36-22-00:/dev/sr0 3687da9b-d812-42a1-b42c-a3d3e7178372:/dev/dm-0 8679fa71-e60b-4686-a5fa-9a0cc5023568:/dev/dm-1] I1117 17:52:53.049836 28490 fs.go:140] Filesystem partitions: map[tmpfs:{mountpoint:/run major:0 minor:19 fsType:tmpfs blockSize:0} /dev/mapper/ubuntu–vg-root:{mountpoint:/var/lib/docker/aufs major:252 minor:0 fsType:ext4 blockSize:0} /dev/sda1:{mountpoint:/boot major:8 minor:1 fsType:ext2 blockSize:0}] I1117 17:52:53.052104 28490 manager.go:216] Machine: {NumCores:4 CpuFrequency:2097571 MemoryCapacity:8371290112 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:78cb13728eba6f6c819e6dea599a5db9 SystemUUID:564D7A67-BDF7-E109-61AC-DDC9929A346F BootID:8f8d7bb3-abb6-4d6a-b1ee-261cd1a2cc74 Filesystems:[{Device:tmpfs DeviceMajor:0 DeviceMinor:19 Capacity:837132288 Type:vfs Inodes:1021886 HasInodes:true} {Device:/dev/mapper/ubuntu–vg-root DeviceMajor:252 DeviceMinor:0 Capacity:75452612608 Type:vfs Inodes:4694016 HasInodes:true} {Device:/dev/sda1 DeviceMajor:8 DeviceMinor:1 Capacity:494512128 Type:vfs Inodes:124928 HasInodes:true}] DiskMap:map[252:0:{Name:dm-0 Major:252 Minor:0 Size:76793511936 Scheduler:none} 252:1:{Name:dm-1 Major:252 Minor:1 Size:8589934592 Scheduler:none} 2:0:{Name:fd0 Major:2 Minor:0 Size:4096 Scheduler:deadline} 8:0:{Name:sda Major:8 Minor:0 Size:85899345920 Scheduler:deadline}] NetworkDevices:[{Name:cni0 MacAddress:0a:58:0a:f4:00:01 Speed:0 Mtu:1500} {Name:ens160 MacAddress:00:0c:29:9a:34:6f Speed:10000 Mtu:1500} {Name:flannel.1 MacAddress:b2:6d:04:a9:38:8d Speed:0 Mtu:1450}] Topology:[{Id:0 Memory:8371290112 Cores:[{Id:0 Threads:[0] Caches:[]} {Id:1 Threads:[1] Caches:[]} {Id:2 Threads:[2] Caches:[]} {Id:3 Threads:[3] Caches:[]}] Caches:[{Size:20971520 Type:Unified Level:3}]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} I1117 17:52:53.053151 28490 manager.go:222] Version: {KernelVersion:4.4.0-62-generic ContainerOsVersion:Ubuntu 16.04.3 LTS DockerVersion:17.05.0-ce DockerAPIVersion:1.29 CadvisorVersion: CadvisorRevision:} W1117 17:52:53.053992 28490 server.go:232] No api server defined - no events will be sent to API server. I1117 17:52:53.054023 28490 server.go:422] –cgroups-per-qos enabled, but –cgroup-root was not specified. defaulting to / I1117 17:52:53.056245 28490 container_manager_linux.go:252] container manager verified user specified cgroup-root exists: / I1117 17:52:53.056284 28490 container_manager_linux.go:257] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>}]} ExperimentalQOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s} I1117 17:52:53.056462 28490 container_manager_linux.go:288] Creating device plugin handler: false W1117 17:52:53.061351 28490 kubelet_network.go:69] Hairpin mode set to “promiscuous-bridge” but kubenet is not enabled, falling back to “hairpin-veth” I1117 17:52:53.061391 28490 kubelet.go:517] Hairpin mode set to “hairpin-veth” W1117 17:52:53.065042 28490 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d I1117 17:52:53.071790 28490 docker_service.go:207] Docker cri networking managed by kubernetes.io/no-op I1117 17:52:53.102961 28490 docker_service.go:224] Setting cgroupDriver to cgroupfs I1117 17:52:53.140011 28490 remote_runtime.go:43] Connecting to runtime service unix:///var/run/dockershim.sock I1117 17:52:53.142110 28490 kuberuntime_manager.go:178] Container runtime docker initialized, version: 17.05.0-ce, apiVersion: 1.29.0 W1117 17:52:53.142331 28490 probe.go:215] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating. I1117 17:52:53.144299 28490 server.go:718] Started kubelet v1.8.3 E1117 17:52:53.144359 28490 kubelet.go:1234] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data for container / W1117 17:52:53.144478 28490 kubelet.go:1318] No api server defined - no node status update will be sent. I1117 17:52:53.144496 28490 server.go:128] Starting to listen on 0.0.0.0:10250 I1117 17:52:53.145570 28490 kubelet_node_status.go:280] Setting node annotation to enable volume controller attach/detach I1117 17:52:53.146415 28490 server.go:296] Adding debug handlers to kubelet server. I1117 17:52:53.159723 28490 fs_resource_analyzer.go:66] Starting FS ResourceAnalyzer I1117 17:52:53.159773 28490 status_manager.go:136] Kubernetes client is nil, not starting status manager. I1117 17:52:53.159791 28490 kubelet.go:1768] Starting kubelet main sync loop. I1117 17:52:53.159842 28490 kubelet.go:1779] skipping pod synchronization - [container runtime is down PLEG is not healthy: pleg was last seen active 2562047h47m16.854775807s ago; threshold is 3m0s] W1117 17:52:53.160705 28490 container_manager_linux.go:869] CPUAccounting not enabled for pid: 28490 W1117 17:52:53.160728 28490 container_manager_linux.go:872] MemoryAccounting not enabled for pid: 28490 E1117 17:52:53.160816 28490 container_manager_linux.go:603] [ContainerManager]: Fail to get rootfs information unable to find data for container / I1117 17:52:53.160887 28490 volume_manager.go:246] Starting Kubelet Volume Manager I1117 17:52:53.228692 28490 factory.go:355] Registering Docker factory W1117 17:52:53.228739 28490 manager.go:265] Registration of the rkt container factory failed: unable to communicate with Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused W1117 17:52:53.228910 28490 manager.go:276] Registration of the crio container factory failed: Get http://%2Fvar%2Frun%2Fcrio.sock/info: dial unix /var/run/crio.sock: connect: no such file or directory I1117 17:52:53.228931 28490 factory.go:54] Registering systemd factory I1117 17:52:53.229203 28490 factory.go:86] Registering Raw factory I1117 17:52:53.229466 28490 manager.go:1140] Started watching for new ooms in manager I1117 17:52:53.230193 28490 manager.go:311] Starting recovery of all containers I1117 17:52:53.342529 28490 manager.go:316] Recovery completed I1117 17:52:53.487640 28490 kubelet_node_status.go:280] Setting node annotation to enable volume controller attach/detach E1117 17:52:53.522074 28490 summary.go:92] Failed to get system container stats for “/user.slice/user-1000.slice/session-553.scope”: failed to get cgroup stats for “/user.slice/user-1000.slice/session-553.scope”: failed to get container info for “/user.slice/user-1000.slice/session-553.scope”: unknown container “/user.slice/user-1000.slice/session-553.scope” W1117 17:52:53.522142 28490 helpers.go:847] eviction manager: no observation found for eviction signal allocatableNodeFs.available I1117 17:53:03.522396 28490 kubelet_node_status.go:280] Setting node annotation to enable volume controller attach/detach E1117 17:53:03.554477 28490 summary.go:92] Failed to get system container stats for “/user.slice/user-1000.slice/session-553.scope”: failed to get cgroup stats for “/user.slice/user-1000.slice/session-553.scope”: failed to get container info for “/user.slice/user-1000.slice/session-553.scope”: unknown container “/user.slice/user-1000.slice/session-553.scope” W1117 17:53:03.554543 28490 helpers.go:847] eviction manager: no observation found for eviction signal allocatableNodeFs.available
这下是真的成功了。
打断它。把 –fail-swap-on=false 参数，加到 启动文件下吧。
cd /lib/systemd/system
root@km:/lib/systemd/system# vi kubelet.service
root@km:/lib/systemd/system# cat kubelet.service
root@km:/lib/systemd/system# systemctl daemon-reload
root@km:/lib/systemd/system# systemctl start kubelet
root@km:/lib/systemd/system# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2017-11-17 17:54:21 CST; 15s ago
Docs: http://kubernetes.io/docs/
Main PID: 28824 (kubelet)
Tasks: 14
Memory: 22.5M
CPU: 1.005s
CGroup: /system.slice/kubelet.service
└─28824 /usr/bin/kubelet --fail-swap-on=false
Nov 17 17:54:21 km kubelet[28824]: I1117 17:54:21.815998 28824 factory.go:54] Registering systemd factory
Nov 17 17:54:21 km kubelet[28824]: I1117 17:54:21.816270 28824 factory.go:86] Registering Raw factory
Nov 17 17:54:21 km kubelet[28824]: I1117 17:54:21.816520 28824 manager.go:1140] Started watching for new ooms in manager
Nov 17 17:54:21 km kubelet[28824]: I1117 17:54:21.817206 28824 manager.go:311] Starting recovery of all containers
Nov 17 17:54:21 km kubelet[28824]: I1117 17:54:21.915504 28824 manager.go:316] Recovery completed
Nov 17 17:54:22 km kubelet[28824]: I1117 17:54:22.050494 28824 kubelet_node_status.go:280] Setting node annotation to enable volume controller attach/detach
Nov 17 17:54:22 km kubelet[28824]: W1117 17:54:22.082192 28824 helpers.go:847] eviction manager: no observation found for eviction signal allocatableNodeFs.available
Nov 17 17:54:31 km systemd[1]: Started kubelet: The Kubernetes Node Agent.
Nov 17 17:54:32 km kubelet[28824]: I1117 17:54:32.082476 28824 kubelet_node_status.go:280] Setting node annotation to enable volume controller attach/detach
Nov 17 17:54:32 km kubelet[28824]: W1117 17:54:32.114163 28824 helpers.go:847] eviction manager: no observation found for eviction signal allocatableNodeFs.available
root@km:/lib/systemd/system#
如果说, 想保证重开机也生效,那就直接 https://askubuntu.com/questions/214805/how-do-i-disable-swap 来吧.
sudo swapoff -a
成功了。
kubelet stop¶
因为 kubeadm v1.8.3 在 kubeadm init 的过程中会自动启动 kubelet ,所以这里要把 kubelet stop了。
root@km:/lib/systemd/system# systemctl stop kubelet
好了。kubelet 这一块是OK了。
kubeadm-install-ubuntu-v1.8.4¶
Environment¶
所有节点都要安装 kubeadm, kubelet, kubectl
安装时，全使用 root 用户。直到 kubeadm join 成功后，全使用 非root用户
192.168.31.120 km master
192.168.31.119 kn1 node
192.168.31.118 kn2 node
加代理¶
准备FQ网络
- 命令行
加代理原因：kubeadm init 会去检查最新版本，及最新版本镜像是什么，镜像是否要更新。 如果本地有了相同的docker image id，就不会下载，不会更新。 这意味着，我们前几天的手工build kubeadm，达成 在 /etc/kubernetes/mainfest/ 下的 *.yaml 文件 加上 “imagePullPolicy: IfNotPresent” , 没有意义了。
root@km:~# export http_proxy="http://192.168.31.239:8118/"
root@km:~# export https_proxy="http://192.168.31.239:8118/"
root@km:~# export no_proxy="localhost,127.0.0.1,192.168.31.120,10.96.0.10,github.com,ubuntu.com"
- apt
加代理原因： apt update 要去 google.com 下载
root@km:~# cat /etc/apt/apt.conf
Acquire::http::proxy "http://192.168.31.239:8118/";
Acquire::https::proxy "https://192.168.31.239:8118/";
Acquire::no::proxy "ubuntu.com";
root@km:~#
- docker
加代理原因： 当 kubeadm init 中下载完了最新的images后，要通过 docker 专有的环境去下载镜像，并启动。这个过程，需要代理。
重点是加了 Environment=”HTTP_PROXY=http://192.168.31.239:8118/” “HTTPS_PROXY=http://192.168.31.239:8118/” “NO_PROXY=localhost,127.0.0.1,docker.io”
root@km:~# cat /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket firewalld.service
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
Environment="HTTP_PROXY=http://192.168.31.239:8118/" "HTTPS_PROXY=http://192.168.31.239:8118/" "NO_PROXY=localhost,127.0.0.1,docker.io"
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
root@km:~#
卸载之前已安装的kube*¶
root@km:~# sudo apt remove kubelet kubeadm kubectl -y
卸载完之后检查
root@km:~# which kubeadm
root@km:~# which kubelet
root@km:~# which kubectl
具体如下
卸载¶
jlch@kn1:~$ sudo apt remove kubelet kubeadm kubectl
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package 'kubeadm' is not installed, so not removed
Package 'kubelet' is not installed, so not removed
The following packages were automatically installed and are no longer required:
ebtables kubernetes-cni socat
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
kubectl
0 upgraded, 0 newly installed, 1 to remove and 86 not upgraded.
After this operation, 72.4 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 67947 files and directories currently installed.)
Removing kubectl (1.7.3-01) ...
jlch@kn1:~$
检查¶
jlch@kn1:~$ which kubelet
jlch@kn1:~$ which kubeadm
jlch@kn1:~$
root@kn1:~# which kubectl
/usr/local/bin/kubectl
root@kn1:~# kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.3", GitCommit:"2c2fe6e8278a5db2d15a013987b53968c743f2a1", GitTreeState:"clean", BuildDate:"2017-08-03T07:00:21Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
root@kn1:~# mv /usr/local/bin/kubectl /usr/local/bin/kubectl.v1.7.3
root@kn1:~# which kubectl
/usr/bin/kubectl
root@kn1:~#
root@kn1:~# su - jlch
jlch@kn1:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.4", GitCommit:"9befc2b8928a9426501d3bf62f72849d5cbcd5a3", GitTreeState:"clean", BuildDate:"2017-11-20T05:28:34Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
jlch@kn1:~$
安装docer¶
看官网
根据官网安装其它¶
root@kn1:~# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
OK
root@kn1:~# cat /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
安装kube*¶
安装kubelet kubeadm kubectl¶
看官网
root@kn1:~# apt-get install -y kubelet kubeadm kubectl
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
kubeadm kubectl kubelet
0 upgraded, 3 newly installed, 0 to remove and 87 not upgraded.
Need to get 46.0 MB of archives.
After this operation, 326 MB of additional disk space will be used.
Get:1 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubelet amd64 1.8.4-00 [19.2 MB]
Get:2 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubectl amd64 1.8.4-00 [8,612 kB]
Get:3 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubeadm amd64 1.8.4-00 [18.1 MB]
Fetched 46.0 MB in 1min 22s (557 kB/s)
Selecting previously unselected package kubelet.
(Reading database ... 67946 files and directories currently installed.)
Preparing to unpack .../kubelet_1.8.4-00_amd64.deb ...
Unpacking kubelet (1.8.4-00) ...
Selecting previously unselected package kubectl.
Preparing to unpack .../kubectl_1.8.4-00_amd64.deb ...
Unpacking kubectl (1.8.4-00) ...
Selecting previously unselected package kubeadm.
Preparing to unpack .../kubeadm_1.8.4-00_amd64.deb ...
Unpacking kubeadm (1.8.4-00) ...
Setting up kubelet (1.8.4-00) ...
Setting up kubectl (1.8.4-00) ...
Setting up kubeadm (1.8.4-00) ...
Installing new version of config file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf ...
root@kn1:~#
kubeadm init¶
kubeadm init¶
kubeadm init --pod-network-cidr=10.244.0.0/16 --skip-preflight-checks
安装 flannel¶
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
flannel 各节点开端口
ufw allow 8472
测试¶
ping 各pod 的 IP
kubeadm join¶
看 kubeadm-join.rst
kubeadm join¶
Environment¶
已安装 kubeadm, kubelet, kubectl
安装时，全使用 root 用户。直到 kubeadm join 成功后，全使用 非root用户
192.168.31.120 km master
192.168.31.119 kn1 node
192.168.31.118 kn2 node
kubeadm join¶
/etc/kubernetes/pki/ca.crt already exists¶
sudo kubeadm join –token ce4253.8322cc2590378260 192.168.31.120:6443 –discovery-token-ca-cert-hash sha256:bb0b9ef27e5ffef06776ca10a87ed548cefedc703ddaf904316c87d4a7f3655d
这个来自于 master节点， kubeadm init 后的提示。
jlch@kn1:~$ sudo kubeadm join --token ce4253.8322cc2590378260 192.168.31.120:6443 --discovery-token-ca-cert-hash sha256:bb0b9ef27e5ffef06776ca10a87ed548cefedc703ddaf904316c87d4a7f3655d
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[preflight] WARNING: docker version is greater than the most recently validated version. Docker version: 17.05.0-ce. Max validated version: 17.03
[preflight] Some fatal errors occurred:
/etc/kubernetes/pki/ca.crt already exists
/etc/kubernetes/kubelet.conf already exists
running with swap on is not supported. Please disable swap
[preflight] If you know what you are doing, you can skip pre-flight checks with `--skip-preflight-checks`
jlch@kn1:~$ mkdir etc.kubernetes
jlch@kn1:~$ mkdir etc.kubernetes/pki/
jlch@kn1:~$ sudo mv /etc/kubernetes/pki/ca.crt etc.kubernetes/pki/
jlch@kn1:~$ sudo mv /etc/kubernetes/kubelet.conf etc.kubernetes/
Please disable swap¶
jlch@kn1:~$ sudo kubeadm join --token ce4253.8322cc2590378260 192.168.31.120:6443 --discovery-token-ca-cert-hash sha256:bb0b9ef27e5ffef06776ca10a87ed548cefedc703ddaf904316c87d4a7f3655d
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[preflight] WARNING: docker version is greater than the most recently validated version. Docker version: 17.05.0-ce. Max validated version: 17.03
[preflight] Some fatal errors occurred:
running with swap on is not supported. Please disable swap
[preflight] If you know what you are doing, you can skip pre-flight checks with `--skip-preflight-checks`
jlch@kn1:~$ sudo swapoff -a
再来¶
jlch@kn1:~$ sudo kubeadm join --token ce4253.8322cc2590378260 192.168.31.120:6443 --discovery-token-ca-cert-hash sha256:bb0b9ef27e5ffef06776ca10a87ed548cefedc703ddaf904316c87d4a7f3655d
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[preflight] WARNING: docker version is greater than the most recently validated version. Docker version: 17.05.0-ce. Max validated version: 17.03
[discovery] Trying to connect to API Server "192.168.31.120:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.31.120:6443"
[discovery] Requesting info from "https://192.168.31.120:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.31.120:6443"
[discovery] Successfully established connection with API Server "192.168.31.120:6443"
[bootstrap] Detected server version: v1.8.4
[bootstrap] The server supports the Certificates API (certificates.k8s.io/v1beta1)
Node join complete:
* Certificate signing request sent to master and response
received.
* Kubelet informed of new secure connection details.
Run 'kubectl get nodes' on the master to see this machine join.
jlch@kn1:~$
成功了。
回 master 检查一下。
upgrade-v1.8.3-failure-install-v1.8.3¶
Installing Docker¶
root@km:~# cat /etc/apt/sources.list.d/docker.list
deb https://apt.dockerproject.org/repo ubuntu-xenial main
root@km:~# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://0d6wdn2y.mirror.aliyuncs.com"]
}
root@km:~# vi /etc/docker/daemon.json
root@km:~# cat /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": ["https://0d6wdn2y.mirror.aliyuncs.com"]
}
root@km:~# apt-get install -y curl apt-transport-https
Reading package lists... Done
Building dependency tree
Reading state information... Done
apt-transport-https is already the newest version (1.2.24).
curl is already the newest version (7.47.0-1ubuntu2.4).
The following packages were automatically installed and are no longer required:
golang-1.8-go golang-1.8-race-detector-runtime golang-1.8-src
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up etcd (2.2.5+dfsg-1ubuntu1) ...
insserv: warning: current start runlevel(s) (empty) of script `etcd' overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (0 1 2 3 4 5 6) of script `etcd' overrides LSB defaults (0 1 6).
Installing kubeadm, kubelet and kubectl¶
root@km:~# apt-get install -y apt-transport-https
Reading package lists... Done
Building dependency tree
Reading state information... Done
apt-transport-https is already the newest version (1.2.24).
The following packages were automatically installed and are no longer required:
golang-1.8-go golang-1.8-race-detector-runtime golang-1.8-src
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
root@km:~# cat /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
root@km:~# apt-get install -y kubelet kubeadm kubectl
Reading package lists... Done
Building dependency tree
Reading state information... Done
kubeadm is already the newest version (1.8.3-00).
kubectl is already the newest version (1.8.3-00).
kubectl set to manually installed.
kubelet is already the newest version (1.8.3-00).
The following packages were automatically installed and are no longer required:
golang-1.8-go golang-1.8-race-detector-runtime golang-1.8-src
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
root@km:~#
去下一个页面吧 https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
kubeadm-install-v1.8.3¶
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
env¶
192.168.31.120 km master
192.168.31.119 kn1 node
192.168.31.118 kn2 node
Initializing your master¶
kubeadm init --pod-network-cidr=10.244.0.0/16
如果遇到类似下面错误
[preflight] Some fatal errors occurred: Port 10250 is in use /etc/kubernetes/manifests is not empty /var/lib/kubelet is not empty
则，参考 https://github.com/kubernetes/kubernetes/issues/37063 运行下面命令：
kubeadm reset
systemctl start kubelet.service
之后，再次运行
kubeadm init --pod-network-cidr=10.244.0.0/16
被墙了，出不去，我了个去，怎么办？
https://mritd.me/2016/10/29/set-up-kubernetes-cluster-by-kubeadm/#21安装包从哪来
好吧，那就去 hub.docker.com 中配置吧
找到所有要配置的 image¶
找 /etc/kubernetes/manifests/¶
root@km:~# cd /etc/kubernetes/manifests/
root@km:/etc/kubernetes/manifests# ls
etcd.yaml kube-apiserver.yaml kube-controller-manager.yaml kube-scheduler.yaml
root@km:/etc/kubernetes/manifests#
root@km:/etc/kubernetes/manifests# cat *.yaml | grep image
image: gcr.io/google_containers/etcd-amd64:3.0.17
image: gcr.io/google_containers/kube-apiserver-amd64:v1.8.3
image: gcr.io/google_containers/kube-controller-manager-amd64:v1.8.3
image: gcr.io/google_containers/kube-scheduler-amd64:v1.8.3
root@km:/etc/kubernetes/manifests#
找源码¶
https://github.com/kubernetes/kubernetes/tree/master/cmd/kubeadm
root@km:~/kubernetes.20171116/cmd/kubeadm# git clone https://github.com/kubernetes/kubernetes kubernetes.20171116
root@km:~# cd kubernetes.20171116
root@km:~/kubernetes.20171116# cd cmd/kubeadm/
root@km:~/kubernetes.20171116/cmd/kubeadm# grep gcr.io -r ./
./app/apis/kubeadm/v1alpha1/defaults.go: DefaultImageRepository = "gcr.io/google_containers"
./app/phases/selfhosting/selfhosting_test.go: image: gcr.io/google_containers/kube-apiserver-amd64:v1.7.4
./app/phases/selfhosting/selfhosting_test.go: image: gcr.io/google_containers/kube-apiserver-amd64:v1.7.4
./app/phases/selfhosting/selfhosting_test.go: image: gcr.io/google_containers/kube-controller-manager-amd64:v1.7.4
./app/phases/selfhosting/selfhosting_test.go: image: gcr.io/google_containers/kube-controller-manager-amd64:v1.7.4
./app/phases/selfhosting/selfhosting_test.go: image: gcr.io/google_containers/kube-scheduler-amd64:v1.7.4
./app/phases/selfhosting/selfhosting_test.go: image: gcr.io/google_containers/kube-scheduler-amd64:v1.7.4
./app/phases/selfhosting/selfhosting_test.go: - image: gcr.io/google_containers/busybox
./app/phases/selfhosting/selfhosting_test.go: "image": "gcr.io/google_containers/busybox"
./app/phases/selfhosting/selfhosting_test.go: - image: gcr.io/google_containers/busybox
./app/phases/upgrade/staticpods_test.go:imageRepository: gcr.io/google_containers
./app/util/template_test.go: validTmplOut = "image: gcr.io/google_containers/pause-amd64:3.0"
./app/util/template_test.go: doNothing = "image: gcr.io/google_containers/pause-amd64:3.0"
./app/util/template_test.go: ImageRepository: "gcr.io/google_containers",
./app/util/template_test.go: ImageRepository: "gcr.io/google_containers",
./app/images/images_test.go: gcrPrefix = "gcr.io/google_containers"
./app/constants/constants.go: DefaultCIImageRepository = "gcr.io/kubernetes-ci-images"
root@km:~/kubernetes.20171116/cmd/kubeadm#
最后，居然什么也没有找到。哎呀。好吧。
回到 https://gitee.com/tomt/tom_docker_registry_push_pull.git ，看了一下，也就是这4个需要更新一下。
gcr.io/google_containers/kube-apiserver-amd64:v1.8.3
gcr.io/google_containers/kube-proxy-amd64:v1.8.3
gcr.io/google_containers/kube-scheduler-amd64:v1.8.3
gcr.io/google_containers/kube-controller-manager-amd64:v1.8.3
好了。明确了image, 那就做吧。
github hub.docker.io 配置 image¶
注意，在 hub.docker.io 下，自动构建仓库，时，取 github的仓库，对这个仓库名称有限制
- 不能大于3个 “-” 号
- 长度不能太长
这样呢，gcr.io/google_containers/kube-controller-manager-amd64:v1.8.3， 在github上本来要创建名称为 dockerlibraryk8s-kube-controller-manager-amd64 ，那这个不符合要求的。 怎么办？改个名称咯。 我把它修改成 dockerlibraryk8s-kube-cm-amd64了。 记得最后，pull 下来的时候，docker tag 一下。
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# ls
dockerlibraryk8s.sh k8s-v1.8.3.txt
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# cat dockerlibraryk8s.sh
#images=(heapster-influxdb-amd64:v1.3.3 heapster-amd64:v1.4.0 heapster-grafana-amd64:v4.4.3)
#images=(heapster-grafana-amd64:v4.4.3)
for imageName in `cat k8s-v1.8.3.txt`; do
echo $imageName
echo ${imageName%:*}
echo ${imageName#*:}
imageNamelast=dockerlibraryk8s-${imageName%:*}
echo docker pull tomtsang/$imageNamelast
docker pull tomtsang/$imageNamelast
echo docker tag tomtsang/$imageNamelast gcr.io/google_containers/$imageName
docker tag tomtsang/$imageNamelast gcr.io/google_containers/$imageName
echo docker rmi tomtsang/$imageNamelast
docker rmi tomtsang/$imageNamelast
done
echo "game over"
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# docker pull tomtsang/dockerlibraryk8s-kube-cm-amd64
Using default tag: latest
latest: Pulling from tomtsang/dockerlibraryk8s-kube-cm-amd64
0ffadd58f2a6: Already exists
18c5c31a1ebe: Pull complete
Digest: sha256:4e738f80a607772c205ca597c1d5874ee50ac40f0a5e88ab85084fd45b684ac0
Status: Downloaded newer image for tomtsang/dockerlibraryk8s-kube-cm-amd64:latest
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# docker tag tomtsang/dockerlibraryk8s-kube-cm-amd64 gcr.io/google_containers/dockerlibraryk8s-kube-cm-amd64:v1.8.3
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# docker tag gcr.io/google_containers/dockerlibraryk8s-kube-cm-amd64:v1.8.3 gcr.io/google_containers/kube-controller-manager-amd64:v1.8.3
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# docker rmi tomtsang/dockerlibraryk8s-kube-cm-amd64
Untagged: tomtsang/dockerlibraryk8s-kube-cm-amd64:latest
Untagged: tomtsang/dockerlibraryk8s-kube-cm-amd64@sha256:4e738f80a607772c205ca597c1d5874ee50ac40f0a5e88ab85084fd45b684ac0
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3#
使用 http://git.oschina.net/tomt/tom_k8s_kubernetes_install 这个仓库吧。
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# git remote -v
origin http://git.oschina.net/tomt/tom_k8s_kubernetes_install (fetch)
origin http://git.oschina.net/tomt/tom_k8s_kubernetes_install (push)
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# ls
dockerlibraryk8s.sh k8s-v1.8.3.txt
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# pwd
/root/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# ls
dockerlibraryk8s.sh k8s-v1.8.3.txt
root@km:~/tom_k8s_kubernetes_install/k8/docker-library-k8s-v1.8.3# ./dockerlibraryk8s.sh
到现在为止，应该 4 个 image都成功pull 到了 master节点了。
docker registry push¶
因为我们后面要在 node 节点上使用，所以干脆就直接 push 到 docker registry 去吧。
kubeadm init¶
root@km:~# export
...
declare -x http_proxy="http://192.168.31.239:8118/"
declare -x https_proxy="http://192.168.31.239:8118/"
declare -x no_proxy="localhost,127.0.0.1,192.168.31.120,10.96.0.10,github.com,ubuntu.com"
root@km:~# kubeadm init --pod-network-cidr=10.244.0.0/16 --skip-preflight-checks
....
....
....
To start using your cluster, you need to run (as a regular user):
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token ce4253.8322cc2590378260 192.168.31.120:6443 --discovery-token-ca-cert-hash sha256:bb0b9ef27e5ffef06776ca10a87ed548cefedc703ddaf904316c87d4a7f3655d
有image, 而不能启动docker container.
kubeadm-init-use-local-image¶
因为 kubeadm 安装 要提前下载好docker images, 并使用 这些个 docker images. 但是，我们公司的网络FQ的下载速度太慢，时不时会断了。所以，我们考虑使用本地下载好的这些 images。
要使用 local images, 那就要去修改 kubeadm 的代码，并重新build。好吧，我们build 吧。
env¶
192.168.31.120 km master
192.168.31.119 kn1 node
192.168.31.118 kn2 node
kubeadm-build¶
见 kubeadm-build.rst
加代理¶
root@km:/etc/kubernetes/manifests# export
declare -x HOME="/root"
declare -x LANG="en_US.UTF-8"
declare -x LANGUAGE="en_US:en"
declare -x LESSCLOSE="/usr/bin/lesspipe %s %s"
declare -x LESSOPEN="| /usr/bin/lesspipe %s"
declare -x LOGNAME="root"
declare -x LS_COLORS="rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:"
declare -x MAIL="/var/mail/root"
declare -x NO_PROXY="localhost,127.0.0.1/8,192.168.31.1/24"
declare -x OLDPWD="/etc/kubernetes"
declare -x PATH="/home/jlch/.ana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
declare -x PWD="/etc/kubernetes/manifests"
declare -x SHELL="/bin/bash"
declare -x SHLVL="1"
declare -x SUDO_COMMAND="/bin/su"
declare -x SUDO_GID="1000"
declare -x SUDO_UID="1000"
declare -x SUDO_USER="jlch"
declare -x TERM="xterm"
declare -x USER="root"
declare -x USERNAME="root"
declare -x http_proxy="http://192.168.31.10:1080"
declare -x https_proxy="http://192.168.31.10:1080"
declare -x no_proxy="localhost,127.0.0.1,192.168.31.120,192.168.0.0/16,10.96.0.0/16,github.com"
kubeadm init¶
把这个 kubeadm 放到 km 下
- ::
- root@km:/etc/kubernetes/manifests# chmod a+rx /usr/bin/kubeadm root@km:/etc/kubernetes/manifests# kubeadm version kubeadm version: &version.Info{Major:”1”, Minor:”8+”, GitVersion:”v1.8.3-dirty”, GitCommit:”f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd”, GitTreeState:”dirty”, BuildDate:”2017-11-20T07:05:42Z”, GoVersion:”go1.9.2”, Compiler:”gc”, Platform:”linux/amd64”} root@km:/etc/kubernetes/manifests# root@km:/etc/kubernetes/manifests# kubeadm init –pod-network-cidr=10.244.0.0/16 –skip-preflight-checks [kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters. unable to get URL “https://dl.k8s.io/release/stable-1.8.txt”: Get https://dl.k8s.io/release/stable-1.8.txt: net/http: TLS handshake timeout root@km:/etc/kubernetes/manifests#
不成功呀。连接上了代理，还是会 pull image
其它测试¶
root@km:/etc/kubernetes/manifests# kubeadm alpha phase etcd local
unable to get URL "https://dl.k8s.io/release/stable-1.8.txt": Get https://dl.k8s.io/release/stable-1.8.txt: net/http: TLS handshake timeout
root@km:/etc/kubernetes/manifests#
kubeadm-build¶
build kubeadm¶
修改 Kubeadm 使得 /etc/kubernetes/manifests/ 下的 *.yaml 文件带有 imagePullPolicy: IfNotPresent
env¶
192.168.31.114 jlch
gopath¶
cd ~/gopath/src/github.com/kubernetes/
git clone https://github.com/kubernetes/kubernetes.git
cd kubernetes
修改吧¶
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes$ git status
Not currently on any branch.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: cmd/kubeadm/app/phases/controlplane/manifests.go
modified: cmd/kubeadm/app/phases/etcd/local.go
no changes added to commit (use "git add" and/or "git commit -a")
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes$ git diff cmd/kubeadm/app/phases/controlplane/manifests.go
diff --git a/cmd/kubeadm/app/phases/controlplane/manifests.go b/cmd/kubeadm/app/phases/controlplane/manifests.go
index 7d2784d..5b2833f 100644
--- a/cmd/kubeadm/app/phases/controlplane/manifests.go
+++ b/cmd/kubeadm/app/phases/controlplane/manifests.go
@@ -75,6 +75,7 @@ func GetStaticPodSpecs(cfg *kubeadmapi.MasterConfiguration, k8sVersion *version.
kubeadmconstants.KubeAPIServer: staticpodutil.ComponentPod(v1.Container{
Name: kubeadmconstants.KubeAPIServer,
Image: images.GetCoreImage(kubeadmconstants.KubeAPIServer, cfg.GetControlPlaneImageRepository(), cfg.KubernetesVersion, cfg.UnifiedControlPlaneImage),
+ ImagePullPolicy: v1.PullIfNotPresent,
Command: getAPIServerCommand(cfg, k8sVersion),
VolumeMounts: mounts.GetVolumeMounts(kubeadmconstants.KubeAPIServer),
LivenessProbe: staticpodutil.ComponentProbe(int(cfg.API.BindPort), "/healthz", v1.URISchemeHTTPS),
@@ -84,6 +85,7 @@ func GetStaticPodSpecs(cfg *kubeadmapi.MasterConfiguration, k8sVersion *version.
kubeadmconstants.KubeControllerManager: staticpodutil.ComponentPod(v1.Container{
Name: kubeadmconstants.KubeControllerManager,
Image: images.GetCoreImage(kubeadmconstants.KubeControllerManager, cfg.GetControlPlaneImageRepository(), cfg.KubernetesVersion, cfg.UnifiedControlPlaneImage),
+ ImagePullPolicy: v1.PullIfNotPresent,
Command: getControllerManagerCommand(cfg, k8sVersion),
VolumeMounts: mounts.GetVolumeMounts(kubeadmconstants.KubeControllerManager),
LivenessProbe: staticpodutil.ComponentProbe(10252, "/healthz", v1.URISchemeHTTP),
@@ -93,6 +95,7 @@ func GetStaticPodSpecs(cfg *kubeadmapi.MasterConfiguration, k8sVersion *version.
kubeadmconstants.KubeScheduler: staticpodutil.ComponentPod(v1.Container{
Name: kubeadmconstants.KubeScheduler,
Image: images.GetCoreImage(kubeadmconstants.KubeScheduler, cfg.GetControlPlaneImageRepository(), cfg.KubernetesVersion, cfg.UnifiedControlPlaneImage),
+ ImagePullPolicy: v1.PullIfNotPresent,
Command: getSchedulerCommand(cfg),
VolumeMounts: mounts.GetVolumeMounts(kubeadmconstants.KubeScheduler),
LivenessProbe: staticpodutil.ComponentProbe(10251, "/healthz", v1.URISchemeHTTP),
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes$ git diff cmd/kubeadm/app/phases/etcd/local.go
diff --git a/cmd/kubeadm/app/phases/etcd/local.go b/cmd/kubeadm/app/phases/etcd/local.go
index e947794..8cc609a 100644
--- a/cmd/kubeadm/app/phases/etcd/local.go
+++ b/cmd/kubeadm/app/phases/etcd/local.go
@@ -54,6 +54,7 @@ func GetEtcdPodSpec(cfg *kubeadmapi.MasterConfiguration) v1.Pod {
Name: kubeadmconstants.Etcd,
Command: getEtcdCommand(cfg),
Image: images.GetCoreImage(kubeadmconstants.Etcd, cfg.ImageRepository, "", cfg.Etcd.Image),
+ ImagePullPolicy: v1.PullIfNotPresent,
// Mount the etcd datadir path read-write so etcd can store data in a more persistent manner
VolumeMounts: []v1.VolumeMount{staticpodutil.NewVolumeMount(etcdVolumeName, cfg.Etcd.DataDir, false)},
LivenessProbe: staticpodutil.ComponentProbe(2379, "/health", v1.URISchemeHTTP),
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes$
make¶
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes$ make
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes$ cd _output/
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes/_output$ ls
bin local
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes/_output$ cd bin/
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes/_output/bin$ ls
apiextensions-apiserver conversion-gen defaulter-gen e2e.test genfeddocs genman genyaml gke-certificates-controller hyperkube kube-aggregator kube-controller-manager kubefed kubemark kube-scheduler openapi-gen
cloud-controller-manager deepcopy-gen e2e_node.test gendocs genkubedocs genswaggertypedocs ginkgo go-bindata kubeadm kube-apiserver kubectl kubelet kube-proxy linkcheck teststale
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes/_output/bin$ ./kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.3-dirty", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"dirty", BuildDate:"2017-11-20T07:05:42Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
jlch@mon1:~/gopath/src/github.com/kubernetes/kubernetes/_output/bin$
好了，现在我们要的就是 kubeadm 文件。
delete node¶
clear
cat k8.export.sh
export KUBECONFIG=$HOME/admin.conf
ls
k get node
kubectl drain kn1 --delete-local-data --force --ignore-daemonsets
kubectl delete node kn1
kubectl drain kn2 --delete-local-data --force --ignore-daemonsets
kubectl delete node kn2
k get node
clear
kubectl drain km --delete-local-data --force --ignore-daemonsets
kubeadm reset
k get pod --all-namespaces
docker ps
kubernetes upgrade¶
Contents:
Upgrading kubeadm clusters from 1.7.3 to 1.8.3¶
Upgrading kubeadm clusters from 1.7 to 1.8
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/
env¶
192.168.31.120 km, master
192.168.31.119 kn1
192.168.31.118 kn2
下载 kubeadm¶
root@km:~# sudo chmod a+rx /usr/bin/kubeadm
root@km:~# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.2", GitCommit:"bdaeafa71f6c7c04636251031f93464384d54963", GitTreeState:"clean", BuildDate:"2017-10-24T19:38:10Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
root@km:~#
上传¶
root@km:~# kubeadm config upload from-file --config ./admin.conf
unable to decode config from "./admin.conf" [no kind "Config" is registered for version "v1"]
不是这样的 conf 文件呀。
root@km:~# kubeadm config upload from-flags []
unable to get URL "https://dl.k8s.io/release/stable-1.8.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.8.txt: read tcp 192.168.31.120:47118->172.217.24.16:443: read: connection reset by peer
root@km:~#
要FQ呀。
proxy FQ¶
root@km:~# cat proxy.sh
#!/bin/bash
NO_PROXY=localhost,127.0.0.1/8,192.168.31.1/24
export NO_PROXY
export http_proxy=http://192.168.31.10:1080/
export https_proxy=http://192.168.31.10:1080/
root@km:~#
kubeadm upgrade apply v1.8.3¶
FQ后，重新来过
root@km:~# cp kubeadm.v1.8.3 kubeadm
root@km:~# mv kubeadm /usr/bin/
root@km:~# chmod a+rx /usr/bin/kubeadm
root@km:~# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-08T18:27:48Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
root@km:~# kubeadm config upload from-flags []
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
root@km:~# kubeadm upgrade plan
[preflight] Running pre-flight checks
[upgrade] Making sure the cluster is healthy:
[upgrade/health] Checking API Server health: Healthy
[upgrade/health] Checking Node health: All Nodes are healthy
[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.7.3
[upgrade/versions] kubeadm version: v1.8.3
[upgrade/versions] Latest stable version: v1.8.3
[upgrade/versions] Latest version in the v1.7 series: v1.7.10
Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT AVAILABLE
Kubelet 3 x v1.7.3 v1.7.10
Upgrade to the latest version in the v1.7 series:
COMPONENT CURRENT AVAILABLE
API Server v1.7.3 v1.7.10
Controller Manager v1.7.3 v1.7.10
Scheduler v1.7.3 v1.7.10
Kube Proxy v1.7.3 v1.7.10
Kube DNS 1.14.5 1.14.5
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.7.10
_____________________________________________________________________
Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT AVAILABLE
Kubelet 3 x v1.7.3 v1.8.3
Upgrade to the latest stable version:
COMPONENT CURRENT AVAILABLE
API Server v1.7.3 v1.8.3
Controller Manager v1.7.3 v1.8.3
Scheduler v1.7.3 v1.8.3
Kube Proxy v1.7.3 v1.8.3
Kube DNS 1.14.5 1.14.5
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.8.3
_____________________________________________________________________
root@km:~# kubeadm upgrade apply v1.8.3
[preflight] Running pre-flight checks
[upgrade] Making sure the cluster is healthy:
[upgrade/health] Checking API Server health: Healthy
[upgrade/health] Checking Node health: All Nodes are healthy
[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade/version] You have chosen to upgrade to version "v1.8.3"
[upgrade/versions] Cluster version: v1.7.3
[upgrade/versions] kubeadm version: v1.8.3
[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y
[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler]
[upgrade/prepull] Prepulling image for component kube-scheduler.
[upgrade/prepull] Prepulling image for component kube-apiserver.
[upgrade/prepull] Prepulling image for component kube-controller-manager.
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
[upgrade/prepull] Prepulled image for component kube-apiserver.
[upgrade/prepull] Prepulled image for component kube-controller-manager.
[upgrade/prepull] Prepulled image for component kube-scheduler.
[upgrade/prepull] Successfully prepulled the images for all the control plane components
[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.8.3"...
[upgrade/staticpods] Writing upgraded Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests025451644"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests025451644/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests025451644/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests025451644/kube-scheduler.yaml"
[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests188726187/kube-apiserver.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[upgrade/apply] FATAL: couldn't upgrade control plane. kubeadm has tried to recover everything into the earlier state. Errors faced: [timed out waiting for the condition]
root@km:~#
报错了。
解决报错¶
加入 kubeadm init 参数¶
通过 kubeadm upgrade –help
发现，要先把之前 kubeadm init 的参数，作为configmap传进来。 回忆一下，之前是 kubeadm init –pod-network-cidr=10.244.0.0/16，所以这里把它加进去
root@km:~# kubeadm config upload from-flags [--pod-network-cidr=10.244.0.0/16]
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
root@km:~#
root@km:~# k get configmap kubeadm-config -n kube-system
NAME DATA AGE
kubeadm-config 1 1d
root@km:~# k describe configmap kubeadm-config -n kube-system
Name: kubeadm-config
Namespace: kube-system
Labels: <none>
Annotations: <none>
Data
====
MasterConfiguration:
----
api:
advertiseAddress: 192.168.31.120
bindPort: 6443
authorizationModes:
- Node
- RBAC
certificatesDir: /etc/kubernetes/pki
cloudProvider: ""
etcd:
caFile: ""
certFile: ""
dataDir: /var/lib/etcd
endpoints: null
image: ""
keyFile: ""
imageRepository: gcr.io/google_containers
kubernetesVersion: v1.8.3
networking:
dnsDomain: cluster.local
podSubnet: ""
serviceSubnet: 10.96.0.0/12
nodeName: km
token: ""
tokenTTL: 24h0m0s
unifiedControlPlaneImage: ""
Events: <none>
root@km:~#
呀！没把参数 –pod-network-cidr=10.244.0.0/16，加入呀（如果加入了 podSubnet: “” 会变成 podSubnet: 10.244.0.0/16 的）
看来刚刚不能加 [] 这个中括号。 再来。
root@km:~# kubeadm config upload from-flags --pod-network-cidr=10.244.0.0/16
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
root@km:~# k get cm kubeadm-config -n kube-system -o yaml
apiVersion: v1
data:
MasterConfiguration: |
api:
advertiseAddress: 192.168.31.120
bindPort: 6443
authorizationModes:
- Node
- RBAC
certificatesDir: /etc/kubernetes/pki
cloudProvider: ""
etcd:
caFile: ""
certFile: ""
dataDir: /var/lib/etcd
endpoints: null
image: ""
keyFile: ""
imageRepository: gcr.io/google_containers
kubernetesVersion: v1.8.3
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
nodeName: km
token: ""
tokenTTL: 24h0m0s
unifiedControlPlaneImage: ""
kind: ConfigMap
metadata:
creationTimestamp: 2017-11-14T10:04:45Z
name: kubeadm-config
namespace: kube-system
resourceVersion: "10542232"
selfLink: /api/v1/namespaces/kube-system/configmaps/kubeadm-config
uid: 3de0d764-c923-11e7-96bf-000c299a346f
root@km:~#
成功。
kubeadm upgrade plan¶
root@km:~# kubeadm upgrade plan
[preflight] Running pre-flight checks
[upgrade] Making sure the cluster is healthy:
[upgrade/health] Checking API Server health: Healthy
[upgrade/health] Checking Node health: All Nodes are healthy
[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.7.3
[upgrade/versions] kubeadm version: v1.8.3
[upgrade/versions] Latest stable version: v1.8.3
[upgrade/versions] Latest version in the v1.7 series: v1.7.10
Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT AVAILABLE
Kubelet 3 x v1.7.3 v1.7.10
Upgrade to the latest version in the v1.7 series:
COMPONENT CURRENT AVAILABLE
API Server v1.7.3 v1.7.10
Controller Manager v1.7.3 v1.7.10
Scheduler v1.7.3 v1.7.10
Kube Proxy v1.7.3 v1.7.10
Kube DNS 1.14.5 1.14.5
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.7.10
_____________________________________________________________________
Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT AVAILABLE
Kubelet 3 x v1.7.3 v1.8.3
Upgrade to the latest stable version:
COMPONENT CURRENT AVAILABLE
API Server v1.7.3 v1.8.3
Controller Manager v1.7.3 v1.8.3
Scheduler v1.7.3 v1.8.3
Kube Proxy v1.7.3 v1.8.3
Kube DNS 1.14.5 1.14.5
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.8.3
_____________________________________________________________________
root@km:~#
kubeadm upgrade apply¶
如果说kubernetes的配置文件不是默认/etc/kubernetes/admin.conf，那则在这里加 –kubeconfig /root/admin.conf 就行了。 如：
root@km:~# kubeadm upgrade apply v1.8.3 --kubeconfig /root/admin.conf --force
多打印一些信息吧
root@km:~# kubeadm upgrade apply v1.8.3 --print-config --skip-preflight-checks
[preflight] Skipping pre-flight checks
[upgrade] Making sure the cluster is healthy:
[upgrade/health] Checking API Server health: Healthy
[upgrade/health] Checking Node health: All Nodes are healthy
[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade/config] Configuration used:
api:
advertiseAddress: 192.168.31.120
bindPort: 6443
authorizationModes:
- Node
- RBAC
certificatesDir: /etc/kubernetes/pki
cloudProvider: ""
etcd:
caFile: ""
certFile: ""
dataDir: /var/lib/etcd
endpoints: null
image: ""
keyFile: ""
imageRepository: gcr.io/google_containers
kubernetesVersion: v1.8.3
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
nodeName: km
token: ef36c0.2aa43b08712e4d54
tokenTTL: 24h0m0s
unifiedControlPlaneImage: ""
[upgrade/version] You have chosen to upgrade to version "v1.8.3"
[upgrade/versions] Cluster version: v1.7.3
[upgrade/versions] kubeadm version: v1.8.3
[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y
[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler]
[upgrade/prepull] Prepulling image for component kube-scheduler.
[upgrade/prepull] Prepulling image for component kube-apiserver.
[upgrade/prepull] Prepulling image for component kube-controller-manager.
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
[upgrade/prepull] Prepulled image for component kube-apiserver.
[upgrade/prepull] Prepulled image for component kube-controller-manager.
[upgrade/prepull] Prepulled image for component kube-scheduler.
[upgrade/prepull] Successfully prepulled the images for all the control plane components
[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.8.3"...
[upgrade/staticpods] Writing upgraded Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests076595912"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests076595912/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests076595912/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests076595912/kube-scheduler.yaml"
[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests094178183/kube-apiserver.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[upgrade/apply] FATAL: couldn't upgrade control plane. kubeadm has tried to recover everything into the earlier state. Errors faced: [timed out waiting for the condition]
root@km:~#
还是失败了。 彻底失败了。
没有办法了。
全新安装 v1.8.3¶
cephfs¶
Contents:
使用 cephfs 完成 statefulset 的练习¶
环境¶
k8s-master 192.168.31.120 km master
k8s-node1 192.168.31.119 kn1 node1
k8s-node2 192.168.31.118 kn2 node2
cephfs-admin 192.168.31.115
cephfs-monitor 192.168.31.114
cephfs-client 192.168.31.172
各 k8s-node 安装完 ceph-common(sudo apt install ceph-common -y)
准备¶
在进行操作前，请完成下面的操作：
1）阅读
http://www.cnblogs.com/iiiiher/p/7159810.html
https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client
注意：
https://github.com/kubernetes-incubator/external-storage 这个仓库，是官方提供之外的 External storage plugins, provisioners, and helper libraries ，因为我们在[官方文档 中看到了 nfs 是不支持 provisioners ](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#provisioner)的，所以要来这里了哟。
2）git clone
git clone https://github.com/kubernetes-incubator/external-storage
cd external-storage/nfs-client/deploy/
3）按步骤来进行。
现在打开 https://github.com/kubernetes-incubator/external-storage/tree/master/ceph/cephfs
开始动手了
step¶
Compile the provisioner¶
见 cephfs-k8s-make-by-go-get.rst
这个地方，请参看 http://blogtt.readthedocs.io/en/latest/k8s/cephfs/cephfs-k8s-make-by-go-get.html
Make the container image and push to the registry¶
这个地方，看公司情况。主要是 把 docker image 放到仓库去，然后让 k8s各个节点机，docker pull 这个make 后的 image.
Start Kubernetes local cluster¶
略
配置 configmap¶
这个小步骤，https://github.com/kubernetes-incubator/external-storage/tree/master/ceph/cephfs 并没有写。但是确实是要走的。
jlch@km:~/cephfs$ ls
cephfs_provisioner cephfs-provisioner.go ceph-secret-admin.yaml CHANGELOG.md claim.yaml class.yaml configmap.yaml deployment.yaml Dockerfile local-start.sh Makefile OWNERS README.md test-pod.yaml
jlch@km:~/cephfs$ k apply -f configmap.yaml
jlch@km:~/cephfs$ k get cm
NAME DATA AGE
cephfs-provisioner 1 23h
jlch@km:~/cephfs$
Create a Ceph admin secret¶
在 cephfs-admin 下
ceph auth get client.admin 2>&1 |grep "key = " |awk '{print $3'} |xargs echo -n > /tmp/secret
把这个/tmp/secret 弄到 k8s-master 的 /tmp/secret
在 k8s-master 下
kubectl create secret generic ceph-secret-admin --from-file=/tmp/secret --namespace=kube-system
Start CephFS provisioner¶
docker run -ti -v /root/.kube:/kube -v /var/run/kubernetes:/var/run/kubernetes --privileged --net=host cephfs-provisioner /usr/local/bin/cephfs-provisioner -master=http://127.0.0.1:8080 -kubeconfig=/kube/config -id=cephfs-provisioner-1
这个官方的配置肯定与我们的实际不同，修改一下配置哟。
- kube配置文件
因为我们的机器是 /root/admin.conf 了哈, 所以直接修改名字成 config
root@km:~# cat k8.export.sh
sudo cp /etc/kubernetes/admin.conf $HOME/
sudo chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
root@km:~# cp admin.conf config
root@km:~#
- image
我们 make 后的 image 是 quay.io/external_storage/cephfs-provisioner:latest
- ceph参数 -master
看一下service
root@km:~/cephfs# k get svc --all-namespaces | grep default | grep kubernetes
default kubernetes 10.96.0.1 <none> 443/TCP 71d
root@km:~/cephfs#
所以我们这里是 -master=https://10.96.0.1/ 或者 -master=https://10.96.0.1:443/
综合一下，我们的操作应该是：
root@km:~# docker run -ti -v /root/:/kube -v /var/run/kubernetes:/var/run/kubernetes --privileged --net=host quay.io/external_storage/cephfs-provisioner /usr/local/bin/cephfs-provisioner -master=https://10.96.0.1/ -kubeconfig=/kube/config -id=cephfs-provisioner-1
好了，基于 docker 方式的 pv 应该是跑起来了。
方法2 docker
docker run -ti -v /home/jlch:/kube -v /var/run/kubernetes:/var/run/kubernetes --privileged --net=host quay.io/external_storage/cephfs-provisioner /usr/local/bin/cephfs-provisioner -master=https://10.96.0.1/ -kubeconfig=/kube/admin.conf -id=cephfs-provisioner-1
对应于 方法1 deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: cephfs-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: cephfs-provisioner
spec:
containers:
- name: cephfs-provisioner
image: "quay.io/external_storage/cephfs-provisioner:latest" # 对应 镜像
imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
valueFrom:
configMapKeyRef:
key: provisioner.name
name: cephfs-provisioner
command: # 这里对应 命令
- "/usr/local/bin/cephfs-provisioner"
args: # 这里对应三个参数
- "-id=cephfs-provisioner-1"
- "-master=https://10.96.0.1/"
- "-kubeconfig=/kube/admin.conf"
volumeMounts: # 对应 -v
- mountPath: /kube
name: kube-config
- mountPath: /var/run/kubernetes
name: kube-run-env
volumes:
- name: kube-config
hostPath:
# directory location on host
path: /home/jlch
# this field is optional
type: Directory
- name: kube-run-env
hostPath:
# directory location on host
path: /var/run/kubernetes
# this field is optional
type: Directory
Create a CephFS Storage Class¶
Replace Ceph monitor’s IP in class.yaml with your own and create storage class:
kubectl create -f class.yaml
Create a claim¶
kubectl create -f claim.yaml
Create a Pod using the claim¶
kubectl create -f test-pod.yaml
因为我之前安装过 flannel 的缘故，所以部署在 kn2 上的时候，报出 Failed create pod sandbox. 错误来了。
jlch@km:~/cephfs$ k describe pod test-pod
Name: test-pod
Namespace: default
Node: kn2/192.168.31.118
Start Time: Fri, 24 Nov 2017 11:29:18 +0800
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"test-pod","namespace":"default"},"spec":{"containers":[{"args":["-c","touch /mnt/S...
Status: Pending
IP:
Containers:
test-pod:
Container ID:
Image: gcr.io/google_containers/busybox:v1.1.1
Image ID:
Port: <none>
Command:
/bin/sh
Args:
-c
touch /mnt/SUCCESS && exit 0 || exit 1
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/mnt from pvc (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-np6qz (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
pvc:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: claim1
ReadOnly: false
default-token-np6qz:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-np6qz
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.alpha.kubernetes.io/notReady:NoExecute for 300s
node.alpha.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 39s default-scheduler Successfully assigned test-pod to kn2
Normal SuccessfulMountVolume 39s kubelet, kn2 MountVolume.SetUp succeeded for volume "default-token-np6qz"
Normal SuccessfulMountVolume 38s kubelet, kn2 MountVolume.SetUp succeeded for volume "pvc-a2bdd1d1-d0c7-11e7-85d4-000c299a346f"
Warning FailedCreatePodSandBox 31s (x8 over 38s) kubelet, kn2 Failed create pod sandbox.
Warning FailedSync 31s (x8 over 38s) kubelet, kn2 Error syncing pod
Normal SandboxChanged 31s (x7 over 38s) kubelet, kn2 Pod sandbox changed, it will be killed and re-created.
jlch@km:~/cephfs$
这里同样地，参考一下 cephfs-k8s-deployment-faq.rst 就可以了。
查看一下pod
jlch@km:~/cephfs$ k get pod
NAME READY STATUS RESTARTS AGE
cephfs-provisioner-cff8d95c-6tgcs 1/1 Running 1 22m
jlch@km:~/cephfs$
我的pod 去哪里了？？？ 噢。因为我的 pod 是一次性的任务，它直接Completed了。所以要 带 -a 参数。
jlch@km:~/cephfs$ k get pod -a
NAME READY STATUS RESTARTS AGE
cephfs-provisioner-cff8d95c-6tgcs 1/1 Running 1 23m
test-pod 0/1 Completed 0 2m
jlch@km:~/cephfs$
好了，确实是pod执行完了。
确认 cephfs 数据¶
那怎么看我的效果呢？
cephu@ceph-client:~/mycephfs$ sudo mount -t ceph 192.168.31.114:6789:/ /mnt/mycephfs -o name=admin,secretfile=admin.secret
unable to read secretfile: No such file or directory
error reading secret file
failed to parse ceph_options
cephu@ceph-client:~/mycephfs$ cd # 这个目录不对，没有 admin.secret 文件
cephu@ceph-client:~$ ls
admin.secret mycephfs release.asc
cephu@ceph-client:~$ cat admin.secret # 看到了吧，这个密码，就是 cephfs 的密码呀。重要
AQCtj+****************IQZJnaJSPEz4jdGw==
cephu@ceph-client:~$
cephu@ceph-client:~$ sudo mount -t ceph 192.168.31.114:6789:/ /mnt/mycephfs -o name=admin,secretfile=admin.secret # mount 成功
cephu@ceph-client:~$ ls /mnt/mycephfs/
a.txt b.txt c.txt h.txt volumes
cephu@ceph-client:~$ cd /mnt/mycephfs/
cephu@ceph-client:/mnt/mycephfs$ ls
a.txt b.txt c.txt h.txt volumes
cephu@ceph-client:/mnt/mycephfs$ cd volumes/
cephu@ceph-client:/mnt/mycephfs/volumes$ ls
_deleting k8s kubernetes _kubernetes:kubernetes-dynamic-pvc-5467e02a-c132-11e7-bda7-000c299a346f.meta _kubernetes:kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148.meta
cephu@ceph-client:/mnt/mycephfs/volumes$ cd kubernetes/
cephu@ceph-client:/mnt/mycephfs/volumes/kubernetes$ ls
haha hahaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa hahaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb kubernetes-dynamic-pvc-5467e02a-c132-11e7-bda7-000c299a346f kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148
cephu@ceph-client:/mnt/mycephfs/volumes/kubernetes$ ll
total 0
drwxr-xr-x 1 root root 0 Nov 24 11:29 ./
drwxr-xr-x 1 root root 0 Nov 24 11:29 ../
drwxr-xr-x 1 root root 0 Nov 1 14:35 haha/
drwxr-xr-x 1 root root 0 Nov 1 14:36 hahaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
drwxr-xr-x 1 root root 0 Nov 1 14:38 hahaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/
drwxr-xr-x 1 root root 0 Nov 4 15:37 kubernetes-dynamic-pvc-5467e02a-c132-11e7-bda7-000c299a346f/
drwxr-xr-x 1 root root 0 Nov 24 11:36 kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148/ # 哈哈，从时间上看，这个就是我们要的文件夹。
cephu@ceph-client:/mnt/mycephfs/volumes/kubernetes$ ll kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148/
ls: reading directory 'kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148/': Input/output error # what? 又是这个问题。升级内核去吧。我们这里不升级了哈。见 cephfs-k8s-faq.rst
total 0
drwxr-xr-x 1 root root 0 Nov 24 11:36 ./
drwxr-xr-x 1 root root 0 Nov 24 11:29 ../
cephu@ceph-client:/mnt/mycephfs/volumes/kubernetes$ cd
好了，这里没用，换一个内核版本 4.10.0 以上的机器来吧。先umount 吧。
cephu@ceph-client:/mnt/mycephfs/volumes/kubernetes$ cd
cephu@ceph-client:~$ umount /mnt/mycephfs/
cephu@ceph-client:~$ ls /mnt/mycephfs/
cephu@ceph-client:~$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=4067060k,nr_inodes=1016765,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=817444k,mode=755)
/dev/mapper/ubuntu--vg-root on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
/dev/sda1 on /boot type ext2 (rw,relatime,block_validity,barrier,user_xattr,acl)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
/dev/mapper/ubuntu--vg-root on /var/lib/docker/aufs type ext4 (rw,relatime,errors=remount-ro,data=ordered)
none on /var/lib/docker/aufs/mnt/3ebf0690df4cb6798be2be8c6bee8a77eacfa7e89c42acacc2a97ea2bc3af09a type aufs (rw,relatime,si=781e745ed7325e3c,dio,dirperm1)
nsfs on /run/docker/netns/default type nsfs (rw)
shm on /var/lib/docker/containers/9e4d9c65734a4e566b4303071ec85f0ea1a18efb2f077e51d09d8457039b725d/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
none on /var/lib/docker/aufs/mnt/1a10a53a1baa1de36489cc6f35562790a3659b0dfa08331ee3521d5b89f4f848 type aufs (rw,relatime,si=781e745c1d80ee3c,dio,dirperm1)
shm on /var/lib/docker/containers/9ceeadb3b8184a36d946bd1aeb98a50a0682b75de141c2195373faa3786bbe66/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
tmpfs on /run/user/113 type tmpfs (rw,nosuid,nodev,relatime,size=817444k,mode=700,uid=113,gid=120)
tmpfs on /run/user/1003 type tmpfs (rw,nosuid,nodev,relatime,size=817444k,mode=700,uid=1003,gid=1003)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=817444k,mode=700,uid=1000,gid=1000)
cephu@ceph-client:~$ ls /mnt/mycephfs/
cephu@ceph-client:~$
可以了。
查内核版本
jlch@kn1:~$ uname -a
Linux kn1 4.12.0-041200-generic #201707022031 SMP Mon Jul 3 00:32:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
jlch@kn1:~$
查一下mount下已有的ceph
jlch@kn1:~$ mount | grep ceph
jlch@kn1:~$
把密码文件admin.secret搞过来
jlch@kn1:~$ ls
10-kubeadm.conf apt.conf etc.kubernetes hpa-example.sh k3-dockerlibraryk8s.sh linux-image-4.12.0-041200-generic_4.12.0-041200.201707022031_amd64.deb nginx-dockerlibraryk8s.sh redis.sh
admin.conf busybox-dockerlibraryk8s.sh hello-frontend.sh image k8 liveness-dockerlibraryk8s.sh node-hello.sh registry.crt
admin.secret docker hello-go-gke.sh job-wq-2 k8.export.sh mydockersimages.tar proxy.sh xtrabackup-dockerlibraryk8s.sh
mount
jlch@kn1:~$ sudo mount -t ceph 192.168.31.114:6789:/ /mnt/mycephfs -o name=admin,secretfile=admin.secret
jlch@kn1:~$ mount | grep ceph
192.168.31.114:6789:/ on /mnt/mycephfs type ceph (rw,relatime,name=admin,secret=<hidden>,acl)
jlch@kn1:~$
成功了，查一下文件
jlch@kn1:~$ ls /mnt/mycephfs/
a.txt b.txt c.txt h.txt volumes/
jlch@kn1:~$ cd /mnt/mycephfs/volumes/
jlch@kn1:/mnt/mycephfs/volumes$ ls
_deleting k8s kubernetes _kubernetes:kubernetes-dynamic-pvc-5467e02a-c132-11e7-bda7-000c299a346f.meta _kubernetes:kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148.meta
jlch@kn1:/mnt/mycephfs/volumes$ cd kubernetes/
jlch@kn1:/mnt/mycephfs/volumes/kubernetes$ ll
total 0
drwxr-xr-x 1 root root 5 Nov 24 11:29 ./
drwxr-xr-x 1 root root 5 Nov 24 11:29 ../
drwxr-xr-x 1 root root 0 Nov 1 14:35 haha/
drwxr-xr-x 1 root root 0 Nov 1 14:36 hahaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
drwxr-xr-x 1 root root 0 Nov 1 14:38 hahaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/
drwxr-xr-x 1 root root 1 Nov 4 15:37 kubernetes-dynamic-pvc-5467e02a-c132-11e7-bda7-000c299a346f/
drwxr-xr-x 1 root root 1 Nov 24 11:36 kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148/ # 哈哈，从时间上看，这个就是我们要的文件夹。
jlch@kn1:/mnt/mycephfs/volumes/kubernetes$ cd kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148/
jlch@kn1:/mnt/mycephfs/volumes/kubernetes/kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148$ ls
SUCCESS
jlch@kn1:/mnt/mycephfs/volumes/kubernetes/kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148$
哈哈，找到了之前由 test-pod 创建的文件 SUCCESS。
到此，https://github.com/kubernetes-incubator/external-storage/tree/master/ceph/cephfs 的几个步骤已全走完了。
game over!
cephfs-k8s-make-by-go-get¶
ENV¶
k8s-master 192.168.31.120 km master
k8s-node1 192.168.31.119 kn1 node1
k8s-node2 192.168.31.118 kn2 node2
ceph-client 192.168.31.172
ceph-mon1 192.168.31.114
这次的make, 可以在任何地方完成，只要满足：golang 1.7 以上的版本
我在 km,ceph-client,ceph-mon1 上都完成过
安装golang¶
如果已有安装，请忽略这一步
安装 golang 1.7 以上的版本。 我们这里安装 1.9.1
cd /home/jlch/ tar -xvf go1.9.2.linux-amd64.tar ls go export PATH=$PATH:/home/jlch/go/bin
验证go¶
go version
配置 GOPATH¶
mkdir gopath
export GOPATH=/home/jlch/gopath/
go get¶
go get github.com/kubernetes-incubator/external-storage
配置 Dockerfile¶
后来发现 docker image 的文件不对。
这个地方的 ENV CEPH_VERSION “jewel” 应该修改成 ENV CEPH_VERSION “luminous”
然后再 make
cd /github.com/kubernetes-incubator/external-storage/ceph/cephfs/
vi Dockerfile
看一下。
root@km:~/cephfs# cat ~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs/Dockerfile
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM centos:7
ENV CEPH_VERSION "jewel"
RUN rpm -Uvh https://download.ceph.com/rpm-$CEPH_VERSION/el7/noarch/ceph-release-1-1.el7.noarch.rpm && \
yum install -y epel-release && \
yum install -y ceph-common python-cephfs
COPY cephfs-provisioner /usr/local/bin/cephfs-provisioner
COPY cephfs_provisioner/cephfs_provisioner.py /usr/local/bin/cephfs_provisioner
CMD ["chmod", "o+x", "/usr/local/bin/cephfs_provisioner"]
root@km:~/cephfs#
make¶
cd gopath/src/
cd ./github.com/kubernetes-incubator/external-storage/
这里如果 make ceph/cephfs/ 直接这么走，会报如下错误。
[tom@test_240 external-storage]$ make ceph/cephfs/
make: 对“ceph/cephfs/”无需做任何事。
[tom@test_240 external-storage]$
所以要 cd ceph/cephfs/ && make ，如下：
[tom@test_240 external-storage]$ cd ceph/cephfs/
[tom@test_240 cephfs]$ ls
cephfs_provisioner ceph-secret-admin.yaml claim.yaml configmap.yaml Dockerfile Makefile README.md
cephfs-provisioner.go CHANGELOG.md class.yaml deployment.yaml local-start.sh OWNERS test-pod.yaml
[tom@test_240 cephfs]$ make
CGO_ENABLED=0 GOOS=linux go build -a -ldflags '-extldflags "-static"' -o cephfs-provisioner cephfs-provisioner.go
[tom@test_240 cephfs]$
这个时候，在 ceph/cephfs/ 下会多出一个 cephfs-provisioner 文件
[tom@test_240 cephfs]$ ls
cephfs_provisioner cephfs-provisioner cephfs-provisioner.go ceph-secret-admin.yaml CHANGELOG.md claim.yaml class.yaml configmap.yaml deployment.yaml Dockerfile local-start.sh Makefile OWNERS README.md test-pod.yaml
[tom@test_240 cephfs]$
make push¶
生成 docker image , quay.io/external_storage/cephfs-provisioner
make push
如果说出现下面这个样子，说明是 make 成了 docker image了，但是 Push 没有成功（应该是指 push 到 docker.io 没有成功）
79c182856123: Preparing
cf516324493c: Preparing
unauthorized: access to the requested resource is not authorized
make: *** [push] 错误 1
[tom@test_240 cephfs]$
push 到 registry¶
因为有 reg.jlch.com:5000 这个 registry 了，先登录
docker login reg.jlch.com:5000
docker tag quay.io/external_storage/cephfs-provisioner:latest reg.jlch.com:5000/quay.io/external_storage/cephfs-provisioner:20171114
docker push reg.jlch.com:5000/quay.io/external_storage/cephfs-provisioner:20171114
删除
docker rmi reg.jlch.com:5000/quay.io/external_storage/cephfs-provisioner:20171114
game over¶
k8s cephfs 在 deployment.yaml 中的使用¶
Environment¶
k8s-master 192.168.31.120 km master
k8s-node1 192.168.31.119 kn1 node1
k8s-node2 192.168.31.118 kn2 node2
配置 deployment.yaml¶
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: cephfs-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: cephfs-provisioner
spec:
containers:
- name: cephfs-provisioner
image: "quay.io/external_storage/cephfs-provisioner:latest" # 对应 镜像
imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
valueFrom:
configMapKeyRef:
key: provisioner.name
name: cephfs-provisioner
command: # 这里对应 命令
- "/usr/local/bin/cephfs-provisioner"
args: # 这里对应三个参数
- "-id=cephfs-provisioner-1"
- "-master=https://10.96.0.1/"
- "-kubeconfig=/kube/admin.conf"
volumeMounts: # 对应 -v
- mountPath: /kube
name: kube-config
- mountPath: /var/run/kubernetes
name: kube-run-env
volumes:
- name: kube-config
hostPath:
# directory location on host
path: /home/jlch
# this field is optional
type: Directory
- name: kube-run-env
hostPath:
# directory location on host
path: /var/run/kubernetes
# this field is optional
type: Directory
对应于
方法1 docker
docker run -ti -v /home/jlch:/kube -v /var/run/kubernetes:/var/run/kubernetes --privileged --net=host quay.io/external_storage/cephfs-provisioner /usr/local/bin/cephfs-provisioner -master=https://10.96.0.1/ -kubeconfig=/kube/admin.conf -id=cephfs-provisioner-1
apply¶
jlch@km:~/cephfs$ k apply -f deployment.yaml
deployment "cephfs-provisioner" created
jlch@km:~/cephfs$ k get pods
NAME READY STATUS RESTARTS AGE
cephfs-provisioner-cff8d95c-8b498 0/1 ContainerCreating 0 3s
describe¶
报错了
jlch@km:~/cephfs$ k describe pod cephfs-provisioner-cff8d95c-8b498
Name: cephfs-provisioner-cff8d95c-8b498
Namespace: default
Node: kn1/192.168.31.119
Start Time: Fri, 24 Nov 2017 11:13:06 +0800
Labels: app=cephfs-provisioner
pod-template-hash=79948517
Annotations: kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicaSet","namespace":"default","name":"cephfs-provisioner-cff8d95c","uid":"645fa2f3-d0c5-11e7-85d4-000c...
Status: Pending
IP:
Created By: ReplicaSet/cephfs-provisioner-cff8d95c
Controlled By: ReplicaSet/cephfs-provisioner-cff8d95c
Containers:
cephfs-provisioner:
Container ID:
Image: quay.io/external_storage/cephfs-provisioner:latest
Image ID:
Port: <none>
Command:
/usr/local/bin/cephfs-provisioner
Args:
-id=cephfs-provisioner-1
-master=https://10.96.0.1/
-kubeconfig=/kube/admin.conf
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment:
PROVISIONER_NAME: <set to the key 'provisioner.name' of config map 'cephfs-provisioner'> Optional: false
Mounts:
/kube from kube-config (rw)
/var/run/kubernetes from kube-run-env (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-np6qz (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
kube-config:
Type: HostPath (bare host directory volume)
Path: /home/jlch
kube-run-env:
Type: HostPath (bare host directory volume)
Path: /var/run/kubernetes
default-token-np6qz:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-np6qz
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.alpha.kubernetes.io/notReady:NoExecute for 300s
node.alpha.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 20s default-scheduler Successfully assigned cephfs-provisioner-cff8d95c-8b498 to kn1
Normal SuccessfulMountVolume 19s kubelet, kn1 MountVolume.SetUp succeeded for volume "kube-run-env"
Normal SuccessfulMountVolume 19s kubelet, kn1 MountVolume.SetUp succeeded for volume "kube-config"
Normal SuccessfulMountVolume 19s kubelet, kn1 MountVolume.SetUp succeeded for volume "default-token-np6qz"
Warning FailedCreatePodSandBox 12s (x7 over 18s) kubelet, kn1 Failed create pod sandbox.
Warning FailedSync 11s (x8 over 18s) kubelet, kn1 Error syncing pod
Normal SandboxChanged 11s (x7 over 17s) kubelet, kn1 Pod sandbox changed, it will be killed and re-created.
jlch@km:~/cephfs$
看到了吧。是 部署在 kn1 上的时候，报出 Failed create pod sandbox. 错误来了。 那就要去 kn1 上查看日志了。
root@kn1:~# vi /var/log/syslog
...
# 查找 ceph
Nov 24 09:09:19 kn1 systemd-udevd[23014]: Could not generate persistent MAC address for vethf620eecc: No such file or directory
Nov 24 09:09:19 kn1 kubelet[6008]: E1124 09:09:19.678998 6008 cni.go:301] Error adding network: "cni0" already has an IP address different from 10.244.1.1/24
Nov 24 09:09:19 kn1 kubelet[6008]: E1124 09:09:19.679048 6008 cni.go:250] Error while adding to cni network: "cni0" already has an IP address different from 10.244.1.1/24
Nov 24 09:09:19 kn1 kernel: [1788519.065714] cni0: port 1(vethf620eecc) entered disabled state
...
查找关键字 ceph， 找到了上面的信息，明显是与 cni0 相关。这上面讲了，”cni0” already has an IP address different from 10.244.1.1/24， 也就是说，可能现有已有一个cni0, 但是与kubernetes自动给它分配的ip: 10.244.1.1/24, 不一致了。 查一下ip,
root@kn1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:0c:29:6f:74:6a brd ff:ff:ff:ff:ff:ff
inet 192.168.31.119/24 brd 192.168.31.255 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe6f:746a/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:96:2d:2c:25 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:96ff:fe2d:2c25/64 scope link
valid_lft forever preferred_lft forever
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
link/ether 9e:b7:c9:ed:7e:cb brd ff:ff:ff:ff:ff:ff
inet 10.244.2.0/32 scope global flannel.1
valid_lft forever preferred_lft forever
inet6 fe80::9cb7:c9ff:feed:7ecb/64 scope link
valid_lft forever preferred_lft forever
4639: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether 0a:58:0a:f4:02:01 brd ff:ff:ff:ff:ff:ff
inet 10.244.2.1/24 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::b401:beff:fedf:2203/64 scope link
valid_lft forever preferred_lft forever
4640: veth3fec5211@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether 72:3c:c8:35:61:6f brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::703c:c8ff:fe35:616f/64 scope link
valid_lft forever preferred_lft forever
root@kn1:~#
我去，果然是。这里面的是 10.244.2.1/24。我靠。 怎么办？怎么办？ 修改cni0呗。
修改 cni0¶
root@kn1:~# systemctl stop docker
root@kn1:~# ip a
root@kn1:~# brctl --help
root@kn1:~# brctl # 这里会提示怎么安装。
root@kn1:~# apt install bridge-utils
root@kn1:~# brctl --help
root@kn1:~# ifconfig cni0 down
root@kn1:~# brctl delbr cni0
root@kn1:~# ip a
root@kn1:~# systemctl start docker
root@kn1:~# ip a # 这个时候，docker 启动的 containers 会自动帮助把 cni0 启动起来的。
检查¶
最后的效果
root@kn1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:0c:29:6f:74:6a brd ff:ff:ff:ff:ff:ff
inet 192.168.31.119/24 brd 192.168.31.255 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe6f:746a/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:96:2d:2c:25 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:96ff:fe2d:2c25/64 scope link
valid_lft forever preferred_lft forever
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
link/ether ba:84:65:bb:4d:68 brd ff:ff:ff:ff:ff:ff
inet 10.244.1.0/32 scope global flannel.1
valid_lft forever preferred_lft forever
inet6 fe80::b884:65ff:febb:4d68/64 scope link
valid_lft forever preferred_lft forever
4639: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether 0a:58:0a:f4:01:01 brd ff:ff:ff:ff:ff:ff
inet 10.244.1.1/24 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::f48c:7eff:fecd:4e08/64 scope link
valid_lft forever preferred_lft forever
4640: veth3fec5211@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether 72:3c:c8:35:61:6f brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::703c:c8ff:fe35:616f/64 scope link
valid_lft forever preferred_lft forever
root@kn1:~#
好了，再回到 km ，检查 deployment,
jlch@km:~/cephfs$ k get pods
NAME READY STATUS RESTARTS AGE
cephfs-provisioner-cff8d95c-6tgcs 1/1 Running 1 3h
jlch@km:~/cephfs$
k8s 中 cephfs 成功的 yaml 文件¶
https://github.com/kubernetes-incubator/external-storage/tree/master/ceph/cephfs
对 我们好不容易成功安装的 k8s-cephfs 进行一个记录呀！
环境¶
k8s-master 192.168.31.120 km master
k8s-node1 192.168.31.119 kn1 node1
k8s-node2 192.168.31.118 kn2 node2
cephfs-admin 192.168.31.115
cephfs-monitor 192.168.31.114
cephfs-client 192.168.31.172
git remote¶
jlch@km:~/cephfs$ git remote -v
origin https://github.com/kubernetes-incubator/external-storage (fetch)
origin https://github.com/kubernetes-incubator/external-storage (push)
jlch@km:~/cephfs$ git log | head
commit f1eb2a4ddf944fdd35a16e686ae104c1db8753b2
Merge: 06aaf46 52a4da4
Author: Matthew Wong <mawong@redhat.com>
Date: Tue Nov 21 01:48:43 2017 -0500
Merge pull request #468 from sathieu/patch-1
flex: Fix file shbang
commit 06aaf46950c9f6f741b34afc1d9f7807bdbe078c
jlch@km:~/cephfs$
git status¶
记录一下主要的修改点
jlch@km:~/cephfs$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: ceph-secret-admin.yaml
modified: claim.yaml
modified: class.yaml
modified: deployment.yaml
modified: test-pod.yaml
no changes added to commit (use "git add" and/or "git commit -a")
jlch@km:~/cephfs$
git diff¶
ceph-secret-admin.yaml¶
jlch@km:~/cephfs$ git diff ceph-secret-admin.yaml
diff --git a/ceph/cephfs/ceph-secret-admin.yaml b/ceph/cephfs/ceph-secret-admin.yaml
index c3a741a..1ebaac3 100644
--- a/ceph/cephfs/ceph-secret-admin.yaml
+++ b/ceph/cephfs/ceph-secret-admin.yaml
@@ -5,4 +5,5 @@ metadata:
type: "kubernetes.io/cephfs"
data:
#Please note this value is base64 encoded.
- key: QVFDTXBIOVlNNFExQmhBQVhHTlF5eU9uZThac1hxV0dvbi9kSVE9PQ==
+ #key: QVFDTXBIOVlNNFExQmhBQVhHTlF5eU9uZThac1hxV0dvbi9kSVE9PQ==
+ key: AQCtj+5ZnNTvGRAA2RxAGcIQZJnaJSPEz4jdGw==
jlch@km:~/cephfs$
claim.yaml¶
jlch@km:~/cephfs$ git diff claim.yaml
diff --git a/ceph/cephfs/claim.yaml b/ceph/cephfs/claim.yaml
index 2dca6ae..280a7a9 100644
--- a/ceph/cephfs/claim.yaml
+++ b/ceph/cephfs/claim.yaml
@@ -9,4 +9,4 @@ spec:
- ReadWriteMany
resources:
requests:
- storage: 1Gi
+ storage: 10Gi
jlch@km:~/cephfs$
class.yaml¶
jlch@km:~/cephfs$ git diff class.yaml
diff --git a/ceph/cephfs/class.yaml b/ceph/cephfs/class.yaml
index 91c73c0..b2825a3 100644
--- a/ceph/cephfs/class.yaml
+++ b/ceph/cephfs/class.yaml
@@ -4,7 +4,7 @@ metadata:
name: cephfs
provisioner: ceph.com/cephfs
parameters:
- monitors: 172.24.0.6:6789
+ monitors: 192.168.31.114:6789
adminId: admin
adminSecretName: ceph-secret-admin
adminSecretNamespace: "kube-system"
jlch@km:~/cephfs$
test-pod.yaml¶
jlch@km:~/cephfs$ git diff test-pod.yaml
diff --git a/ceph/cephfs/test-pod.yaml b/ceph/cephfs/test-pod.yaml
index 4888676..ece87e8 100644
--- a/ceph/cephfs/test-pod.yaml
+++ b/ceph/cephfs/test-pod.yaml
@@ -5,7 +5,8 @@ metadata:
spec:
containers:
- name: test-pod
- image: gcr.io/google_containers/busybox:1.24
+ image: gcr.io/google_containers/busybox:latest
+ imagePullPolicy: IfNotPresent
command:
- "/bin/sh"
args:
jlch@km:~/cephfs$
deployment.yaml¶
jlch@km:~/cephfs$ git diff deployment.yaml
diff --git a/ceph/cephfs/deployment.yaml b/ceph/cephfs/deployment.yaml
index 37c5f87..dc0315c 100644
--- a/ceph/cephfs/deployment.yaml
+++ b/ceph/cephfs/deployment.yaml
@@ -14,6 +14,7 @@ spec:
containers:
- name: cephfs-provisioner
image: "quay.io/external_storage/cephfs-provisioner:latest"
+ imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
valueFrom:
@@ -24,3 +25,23 @@ spec:
- "/usr/local/bin/cephfs-provisioner"
args:
- "-id=cephfs-provisioner-1"
+ - "-master=https://10.96.0.1/"
+ - "-kubeconfig=/kube/admin.conf"
+ volumeMounts:
+ - mountPath: /kube
+ name: kube-config
+ - mountPath: /var/run/kubernetes
+ name: kube-run-env
+ volumes:
+ - name: kube-config
+ hostPath:
+ # directory location on host
+ path: /home/jlch
+ # this field is optional
+ type: Directory
+ - name: kube-run-env
+ hostPath:
+ # directory location on host
+ path: /var/run/kubernetes
+ # this field is optional
+ type: Directory
cat¶
jlch@km:~/cephfs$ ls
cephfs_provisioner cephfs-provisioner.go ceph-secret-admin.yaml CHANGELOG.md claim.yaml class.yaml configmap.yaml deployment.yaml Dockerfile local-start.sh Makefile OWNERS README.md test-pod.yaml
jlch@km:~/cephfs$ cat ceph-secret-admin.yaml
apiVersion: v1
kind: Secret
metadata:
name: ceph-secret-admin
type: "kubernetes.io/cephfs"
data:
#Please note this value is base64 encoded.
#key: QVFDTXBIOVlNNFExQmhBQVhHTlF5eU9uZThac1hxV0dvbi9kSVE9PQ==
key: AQCtj+5ZnNTvGRAA2RxAGcIQZJnaJSPEz4jdGw==
jlch@km:~/cephfs$ cat claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: claim1
annotations:
volume.beta.kubernetes.io/storage-class: "cephfs"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
jlch@km:~/cephfs$ cat class.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: cephfs
provisioner: ceph.com/cephfs
parameters:
monitors: 192.168.31.114:6789
adminId: admin
adminSecretName: ceph-secret-admin
adminSecretNamespace: "kube-system"
jlch@km:~/cephfs$ cat test-pod.yaml
kind: Pod
apiVersion: v1
metadata:
name: test-pod
spec:
containers:
- name: test-pod
image: gcr.io/google_containers/busybox:latest
imagePullPolicy: IfNotPresent
command:
- "/bin/sh"
args:
- "-c"
- "touch /mnt/SUCCESS && exit 0 || exit 1"
volumeMounts:
- name: pvc
mountPath: "/mnt"
restartPolicy: "Never"
volumes:
- name: pvc
persistentVolumeClaim:
claimName: claim1
jlch@km:~/cephfs$ cat deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: cephfs-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: cephfs-provisioner
spec:
containers:
- name: cephfs-provisioner
image: "quay.io/external_storage/cephfs-provisioner:latest"
imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
valueFrom:
configMapKeyRef:
key: provisioner.name
name: cephfs-provisioner
command:
- "/usr/local/bin/cephfs-provisioner"
args:
- "-id=cephfs-provisioner-1"
- "-master=https://10.96.0.1/"
- "-kubeconfig=/kube/admin.conf"
volumeMounts:
- mountPath: /kube
name: kube-config
- mountPath: /var/run/kubernetes
name: kube-run-env
volumes:
- name: kube-config
hostPath:
# directory location on host
path: /home/jlch
# this field is optional
type: Directory
- name: kube-run-env
hostPath:
# directory location on host
path: /var/run/kubernetes
# this field is optional
type: Directory
jlch@km:~/cephfs$
cephfs-k8s 中的 make¶
下载 make ，报错了。
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful# cd external-storage/ceph/cephfs/
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# ls
cephfs_provisioner cephfs-provisioner.go ceph-secret-admin.yaml CHANGELOG.md claim.yaml class.yaml configmap.yaml deployment.yaml Dockerfile local-start.sh Makefile OWNERS README.md test-pod.yaml
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# make
CGO_ENABLED=0 GOOS=linux go build -a -ldflags '-extldflags "-static"' -o cephfs-provisioner cephfs-provisioner.go
cephfs-provisioner.go:28:2: cannot find package "github.com/golang/glog" in any of:
/usr/lib/go-1.8/src/github.com/golang/glog (from $GOROOT)
/root/go/src/github.com/golang/glog (from $GOPATH)
cephfs-provisioner.go:29:2: cannot find package "github.com/kubernetes-incubator/external-storage/lib/controller" in any of:
/usr/lib/go-1.8/src/github.com/kubernetes-incubator/external-storage/lib/controller (from $GOROOT)
/root/go/src/github.com/kubernetes-incubator/external-storage/lib/controller (from $GOPATH)
cephfs-provisioner.go:30:2: cannot find package "k8s.io/api/core/v1" in any of:
/usr/lib/go-1.8/src/k8s.io/api/core/v1 (from $GOROOT)
/root/go/src/k8s.io/api/core/v1 (from $GOPATH)
cephfs-provisioner.go:31:2: cannot find package "k8s.io/apimachinery/pkg/apis/meta/v1" in any of:
/usr/lib/go-1.8/src/k8s.io/apimachinery/pkg/apis/meta/v1 (from $GOROOT)
/root/go/src/k8s.io/apimachinery/pkg/apis/meta/v1 (from $GOPATH)
cephfs-provisioner.go:32:2: cannot find package "k8s.io/apimachinery/pkg/util/uuid" in any of:
/usr/lib/go-1.8/src/k8s.io/apimachinery/pkg/util/uuid (from $GOROOT)
/root/go/src/k8s.io/apimachinery/pkg/util/uuid (from $GOPATH)
cephfs-provisioner.go:33:2: cannot find package "k8s.io/apimachinery/pkg/util/wait" in any of:
/usr/lib/go-1.8/src/k8s.io/apimachinery/pkg/util/wait (from $GOROOT)
/root/go/src/k8s.io/apimachinery/pkg/util/wait (from $GOPATH)
cephfs-provisioner.go:34:2: cannot find package "k8s.io/client-go/kubernetes" in any of:
/usr/lib/go-1.8/src/k8s.io/client-go/kubernetes (from $GOROOT)
/root/go/src/k8s.io/client-go/kubernetes (from $GOPATH)
cephfs-provisioner.go:35:2: cannot find package "k8s.io/client-go/rest" in any of:
/usr/lib/go-1.8/src/k8s.io/client-go/rest (from $GOROOT)
/root/go/src/k8s.io/client-go/rest (from $GOPATH)
cephfs-provisioner.go:36:2: cannot find package "k8s.io/client-go/tools/clientcmd" in any of:
/usr/lib/go-1.8/src/k8s.io/client-go/tools/clientcmd (from $GOROOT)
/root/go/src/k8s.io/client-go/tools/clientcmd (from $GOPATH)
cephfs-provisioner.go:37:2: cannot find package "k8s.io/kubernetes/pkg/api/v1/helper" in any of:
/usr/lib/go-1.8/src/k8s.io/kubernetes/pkg/api/v1/helper (from $GOROOT)
/root/go/src/k8s.io/kubernetes/pkg/api/v1/helper (from $GOPATH)
Makefile:27: recipe for target 'all' failed
make: *** [all] Error 1
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs#
少了包呀。
查一下，有没有go, GOPATH
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# which go
/usr/bin/go
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# echo $GOPATH
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs#
没有设置，那好了，设置成我们之前用过的地址。
先看一下，我们之前的go包：
root@km:~# cd kubernetes
root@km:~/kubernetes# ls
api BUILD.bazel cluster code-of-conduct.md docs federation hack LICENSE Makefile OWNERS pkg README.md SUPPORT.md third_party Vagrantfile WORKSPACE
build CHANGELOG.md cmd CONTRIBUTING.md examples Godeps labels.yaml logo Makefile.generated_files OWNERS_ALIASES plugin staging test translations vendor
root@km:~/kubernetes#
随便查一个，上面没有的包吧，如 client-go:
root@km:~/kubernetes# ls -lR > ls-lr
root@km:~/kubernetes# cat ls-lr | grep client-go
drwxr-xr-x 17 root root 4096 Aug 29 18:47 client-go
drwxr-xr-x 5 root root 4096 Aug 29 18:47 client-go
...
...
./staging/src/k8s.io/client-go:
./staging/src/k8s.io/client-go/discovery:
./staging/src/k8s.io/client-go/discovery/cached:
./staging/src/k8s.io/client-go/discovery/fake:
./staging/src/k8s.io/client-go/dynamic:
./staging/src/k8s.io/client-go/dynamic/fake:
./staging/src/k8s.io/client-go/examples:
...
...
lrwxrwxrwx 1 root root 34 Aug 29 18:47 client-go -> ../../staging/src/k8s.io/client-go
root@km:~/kubernetes#
有呀，这样就不用重复下载了。
root@km:~/kubernetes# cd staging/
root@km:~/kubernetes/staging# ls
BUILD godeps-json-updater.go OWNERS pkg prime-apimachinery.sh README.md src
root@km:~/kubernetes/staging# cd src
root@km:~/kubernetes/staging/src# ls
github.com golang.org gopkg.in k8s.io
root@km:~/kubernetes/staging/src# cd k8s.io/
root@km:~/kubernetes/staging/src/k8s.io# ls
api apiextensions-apiserver apimachinery apiserver client-go code-generator kube-aggregator kube-openapi kubernetes metrics sample-apiserver
root@km:~/kubernetes/staging/src/k8s.io# pwd
/root/kubernetes/staging/src/k8s.io
root@km:~/kubernetes/staging/src/k8s.io#
设置一下 GOPATH
root@km:~/kubernetes/staging# export GOPATH=/root/kubernetes/staging
root@km:~/kubernetes/staging# echo $GOPATH
/root/kubernetes/staging
root@km:~/kubernetes/staging#
重新来一下，并且 go get
root@km:~/kubernetes/staging# cd ~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# make
CGO_ENABLED=0 GOOS=linux go build -a -ldflags '-extldflags "-static"' -o cephfs-provisioner cephfs-provisioner.go
/root/kubernetes/staging/src/k8s.io/client-go/discovery/discovery_client.go:26:2: cannot find package "github.com/emicklei/go-restful-swagger12" in any of:
/usr/lib/go-1.8/src/github.com/emicklei/go-restful-swagger12 (from $GOROOT)
/root/kubernetes/staging/src/github.com/emicklei/go-restful-swagger12 (from $GOPATH)
/root/kubernetes/staging/src/k8s.io/client-go/discovery/discovery_client.go:27:2: cannot find package "github.com/golang/protobuf/proto" in any of:
/usr/lib/go-1.8/src/github.com/golang/protobuf/proto (from $GOROOT)
/root/kubernetes/staging/src/github.com/golang/protobuf/proto (from $GOPATH)
/root/kubernetes/staging/src/k8s.io/client-go/discovery/discovery_client.go:28:2: cannot find package "github.com/googleapis/gnostic/OpenAPIv2" in any of:
/usr/lib/go-1.8/src/github.com/googleapis/gnostic/OpenAPIv2 (from $GOROOT)
/root/kubernetes/staging/src/github.com/googleapis/gnostic/OpenAPIv2 (from $GOPATH)
/root/kubernetes/staging/src/k8s.io/client-go/tools/clientcmd/auth_loaders.go:26:2: cannot find package "github.com/howeyc/gopass" in any of:
/usr/lib/go-1.8/src/github.com/howeyc/gopass (from $GOROOT)
/root/kubernetes/staging/src/github.com/howeyc/gopass (from $GOPATH)
/root/kubernetes/staging/src/k8s.io/client-go/tools/clientcmd/client_config.go:28:2: cannot find package "github.com/imdario/mergo" in any of:
/usr/lib/go-1.8/src/github.com/imdario/mergo (from $GOROOT)
/root/kubernetes/staging/src/github.com/imdario/mergo (from $GOPATH)
/root/kubernetes/staging/src/k8s.io/client-go/util/flowcontrol/throttle.go:22:2: cannot find package "github.com/juju/ratelimit" in any of:
/usr/lib/go-1.8/src/github.com/juju/ratelimit (from $GOROOT)
/root/kubernetes/staging/src/github.com/juju/ratelimit (from $GOPATH)
cephfs-provisioner.go:29:2: cannot find package "github.com/kubernetes-incubator/external-storage/lib/controller" in any of:
/usr/lib/go-1.8/src/github.com/kubernetes-incubator/external-storage/lib/controller (from $GOROOT)
/root/kubernetes/staging/src/github.com/kubernetes-incubator/external-storage/lib/controller (from $GOPATH)
/root/kubernetes/staging/src/k8s.io/apimachinery/pkg/util/uuid/uuid.go:22:2: cannot find package "github.com/pborman/uuid" in any of:
/usr/lib/go-1.8/src/github.com/pborman/uuid (from $GOROOT)
/root/kubernetes/staging/src/github.com/pborman/uuid (from $GOPATH)
Makefile:27: recipe for target 'all' failed
make: *** [all] Error 1
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# go get github.com/emicklei/go-restful-swagger12
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# go get github.com/golang/protobuf/proto
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# go get github.com/googleapis/gnostic/OpenAPIv2
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# go get github.com/howeyc/gopass
package golang.org/x/crypto/ssh/terminal: unrecognized import path "golang.org/x/crypto/ssh/terminal" (https fetch: Get https://golang.org/x/crypto/ssh/terminal?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs#
呀，这里要去 https://golang.org/x/crypto/ssh/terminal 这要FQ的呀，怎么办？
农总来了。
打开浏览器，github.com, 搜索，crypto, 语言选择 go, 找到 golang/crypto, 打开，复制 url,
打开 另一个 terminal, 然后，从之前知道 GOPATH 是 /root/kubernetes/staging/ ,那进去。
root@km:~#
root@km:~# cd kubernetes/staging/
root@km:~/kubernetes/staging# ls
BUILD godeps-json-updater.go OWNERS pkg prime-apimachinery.sh README.md src
root@km:~/kubernetes/staging# cd src
root@km:~/kubernetes/staging/src# ls
github.com golang.org gopkg.in k8s.io
root@km:~/kubernetes/staging/src# cd golang.org/
root@km:~/kubernetes/staging/src/golang.org# ls
x
root@km:~/kubernetes/staging/src/golang.org# cd x
root@km:~/kubernetes/staging/src/golang.org/x# ls
net text
root@km:~/kubernetes/staging/src/golang.org/x#
好了，到这里看到了，确实是没有 crypto, 下载 golang 在 github.com 下的官方镜像吧。
root@km:~/kubernetes/staging/src/golang.org/x# git clone https://github.com/golang/crypto.git
Cloning into 'crypto'...
remote: Counting objects: 3889, done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 3889 (delta 6), reused 18 (delta 4), pack-reused 3863
Receiving objects: 100% (3889/3889), 2.82 MiB | 44.00 KiB/s, done.
Resolving deltas: 100% (2469/2469), done.
Checking connectivity... done.
root@km:~/kubernetes/staging/src/golang.org/x# ls
crypto net text
root@km:~/kubernetes/staging/src/golang.org/x#
好了，现在有了。
如果后续还有其它的 golang 的包，也这么处理吧。
回到之前的 terminal 吧。 接着 go get
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# go get github.com/kubernetes-incubator/external-storage/lib/controller
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs# make
CGO_ENABLED=0 GOOS=linux go build -a -ldflags '-extldflags "-static"' -o cephfs-provisioner cephfs-provisioner.go
# command-line-arguments
./cephfs-provisioner.go:65: cannot use cephFSProvisioner literal (type *cephFSProvisioner) as type controller.Provisioner in return argument:
*cephFSProvisioner does not implement controller.Provisioner (wrong type for Delete method)
have Delete(*"k8s.io/api/core/v1".PersistentVolume) error
want Delete(*"github.com/kubernetes-incubator/external-storage/vendor/k8s.io/api/core/v1".PersistentVolume) error
./cephfs-provisioner.go:69: cannot use cephFSProvisioner literal (type *cephFSProvisioner) as type controller.Provisioner in assignment:
*cephFSProvisioner does not implement controller.Provisioner (wrong type for Delete method)
have Delete(*"k8s.io/api/core/v1".PersistentVolume) error
want Delete(*"github.com/kubernetes-incubator/external-storage/vendor/k8s.io/api/core/v1".PersistentVolume) error
./cephfs-provisioner.go:134: cannot use options.PersistentVolumeReclaimPolicy (type "github.com/kubernetes-incubator/external-storage/vendor/k8s.io/api/core/v1".PersistentVolumeReclaimPolicy) as type "k8s.io /api/core/v1".PersistentVolumeReclaimPolicy in field value
./cephfs-provisioner.go:135: cannot use options.PVC.Spec.AccessModes (type []"github.com/kubernetes-incubator/external-storage/vendor/k8s.io/api/core/v1".PersistentVolumeAccessMode) as type []"k8s.io/api/cor e/v1".PersistentVolumeAccessMode in field value
./cephfs-provisioner.go:137: cannot use "k8s.io/api/core/v1".ResourceName("k8s.io/api/core/v1".ResourceStorage) (type "k8s.io/api/core/v1".ResourceName) as type "github.com/kubernetes-incubator/external-stor age/vendor/k8s.io/api/core/v1".ResourceName in map index
./cephfs-provisioner.go:137: cannot use options.PVC.Spec.Resources.Requests["k8s.io/api/core/v1".ResourceName("k8s.io/api/core/v1".ResourceStorage)] (type "github.com/kubernetes-incubator/external-storage/ve ndor/k8s.io/apimachinery/pkg/api/resource".Quantity) as type "k8s.io/apimachinery/pkg/api/resource".Quantity in map value
./cephfs-provisioner.go:147: cannot use "k8s.io/api/core/v1".CephFSVolumeSource literal (type *"k8s.io/api/core/v1".CephFSVolumeSource) as type *"k8s.io/api/core/v1".CephFSPersistentVolumeSource in field val ue
./cephfs-provisioner.go:318: cannot use clientset (type *"k8s.io/client-go/kubernetes".Clientset) as type "github.com/kubernetes-incubator/external-storage/vendor/k8s.io/client-go/kubernetes".Interface in ar gument to controller.NewProvisionController:
*"k8s.io/client-go/kubernetes".Clientset does not implement "github.com/kubernetes-incubator/external-storage/vendor/k8s.io/client-go/kubernetes".Interface (wrong type for Admissionregistration metho d)
have Admissionregistration() "k8s.io/client-go/kubernetes/typed/admissionregistration/v1alpha1".AdmissionregistrationV1alpha1Interface
want Admissionregistration() "github.com/kubernetes-incubator/external-storage/vendor/k8s.io/client-go/kubernetes/typed/admissionregistration/v1alpha1".AdmissionregistrationV1alpha1Interface
Makefile:27: recipe for target 'all' failed
make: *** [all] Error 2
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs#
又报错。
这个好像是说冲突了嘛。。。
农总来了。
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage# ln -s ~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/lib build/src/github.com/kubernetes-incubator/external-storage/
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage# make ceph/cephfs
cd ceph/cephfs; \
make container
make[1]: Entering directory '/root/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs'
CGO_ENABLED=0 GOOS=linux go build -a -ldflags '-extldflags "-static"' -o cephfs-provisioner cephfs-provisioner.go
docker build -t quay.io/external_storage/cephfs-provisioner:latest .
Sending build context to Docker daemon 36.23MB
Step 1/6 : FROM centos:7
7: Pulling from library/centos
d9aaf4d82f24: Pull complete
Digest: sha256:eba772bac22c86d7d6e72421b4700c3f894ab6e35475a34014ff8de74c10872e
Status: Downloaded newer image for centos:7
---> 196e0ce0c9fb
Step 2/6 : ENV CEPH_VERSION "jewel"
---> Running in c48e286ca165
---> e9689ae3f521
Removing intermediate container c48e286ca165
Step 3/6 : RUN rpm -Uvh https://download.ceph.com/rpm-$CEPH_VERSION/el7/noarch/ceph-release-1-1.el7.noarch.rpm && yum install -y epel-release && yum install -y ceph-common python-cephfs
---> Running in 69640c268019
warning: /var/tmp/rpm-tmp.6mAWB0: Header V4 RSA/SHA256 Signature, key ID 460f3994: NOKEY
Retrieving https://download.ceph.com/rpm-jewel/el7/noarch/ceph-release-1-1.el7.noarch.rpm
Preparing... ########################################
Updating / installing...
ceph-release-1-1.el7 ########################################
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
* base: mirrors.cn99.com
* extras: mirrors.cn99.com
* updates: mirrors.cn99.com
userspace-rcu.x86_64 0:0.7.16-1.el7
....
...
... # 好长的一断时间，大约半个小时后
Complete!
---> 21cae91539c0
Removing intermediate container b1c88201e72b
Step 4/6 : COPY cephfs-provisioner /usr/local/bin/cephfs-provisioner
---> 73664d0eca7a
Removing intermediate container c076e6de19f0
Step 5/6 : COPY cephfs_provisioner/cephfs_provisioner.py /usr/local/bin/cephfs_provisioner
---> 25bc4c6da1d1
Removing intermediate container 0eba40388f99
Step 6/6 : CMD chmod o+x /usr/local/bin/cephfs_provisioner
---> Running in 5bf6e5fbc6e1
---> 262cba7e52ed
Removing intermediate container 5bf6e5fbc6e1
Successfully built 262cba7e52ed
Successfully tagged quay.io/external_storage/cephfs-provisioner:latest
docker tag quay.io/external_storage/cephfs-provisioner:latest quay.io/external_storage/cephfs-provisioner:latest
make[1]: Leaving directory '/root/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs'
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage#
成功了，看一下 docker image 是不是多了一个 quay.io/external_storage/cephfs-provisioner:latest
root@km:~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
quay.io/external_storage/cephfs-provisioner latest 262cba7e52ed 5 minutes ago 485MB
gcr.io/google-samples/hello-frontend 1.0 d8ca9fb857d9 3 weeks ago 183MB
果然有了。
这样，make 算是完成了。
后来发现 docker image 的文件不对。
vi Dockerfile
root@km:~/cephfs# cat ~/kubernetes.io/TUTORIALS/Stateful-Applications/cephfs-stateful/external-storage/ceph/cephfs/Dockerfile
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM centos:7
ENV CEPH_VERSION "jewel"
RUN rpm -Uvh https://download.ceph.com/rpm-$CEPH_VERSION/el7/noarch/ceph-release-1-1.el7.noarch.rpm && \
yum install -y epel-release && \
yum install -y ceph-common python-cephfs
COPY cephfs-provisioner /usr/local/bin/cephfs-provisioner
COPY cephfs_provisioner/cephfs_provisioner.py /usr/local/bin/cephfs_provisioner
CMD ["chmod", "o+x", "/usr/local/bin/cephfs_provisioner"]
root@km:~/cephfs#
这个地方的 ENV CEPH_VERSION “jewel” 应该修改成 ENV CEPH_VERSION “luminous”
然后再 make
k8s 与 cephfs 相关的FAQ¶
Input/output error¶
cephu@ceph-client:/mnt/mycephfs/volumes/kubernetes$ ll kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148/
ls: reading directory 'kubernetes-dynamic-pvc-a2c667ad-d0c7-11e7-b656-0a580af40148/': Input/output error
total 0
drwxr-xr-x 1 root root 0 Nov 24 11:36 ./
drwxr-xr-x 1 root root 0 Nov 24 11:29 ../
cephu@ceph-client:/mnt/mycephfs/volumes/kubernetes$
这个问题，看了一下，https://github.com/kubernetes-incubator/external-storage/issues/345，
最后，有用户是这样回复的
I tried to update my ubuntu kernel from 4.4.0 to 4.10.0 (sudo apt install linux-image-4.10.0-28-generic) and after a reboot, the error is gone, everything works fine from now on :)
所以就是升级内核了。升级去吧。 升级一下，果然成功了。
# what? 又是这个问题。见 k8s
k8s source¶
kubernetes官网 https://kubernetes.io
kubernetes.io的中文翻译 https://k8smeetup.github.io/
Kubernetes相关资讯和情报链接分享 https://jimmysong.io/posts/kubernetes-material-share/
awesome-kubernetes https://github.com/ramitsurana/awesome-kubernetes
Kubernetes The Hard Way https://github.com/kelseyhightower/kubernetes-the-hard-way
kubernetes-handbook https://jimmysong.io/kubernetes-handbook/
node-performance-dashboard http://node-perf-dash.k8s.io/#/builds
awesome-cloud-native¶
fat¶
Contents:
fat tools¶
Contents:
keyboards¶
platform¶
首先,无论看不看下面, 先看这里 https://www.zhihu.com/question/52589498
open source¶
QUANTAXIS 量化金融策略框架¶
http://www.yutiansut.com/ https://github.com/yutiansut/quantaxis
Hikyuu Quant Framework¶
Hikyuu Quant Framework是一款基于C++/Python的开源量化交易研究框架，用于策略分析及回测（目前用于国内证券市场）
RQAlpha¶
nodequant¶
不知道开源不开源¶
http://36kr.com/p/5044349.html
创业公司有 Uquant（优矿） ^^^^^^^^^^^^^^
Ricequant（米筐）¶
JoinQuant（聚宽）¶
bigquant¶
windquant¶
devops¶
Contents:
跳转机¶
windows 防止电脑被远程控制¶
https://jingyan.baidu.com/article/a948d65159fc890a2dcd2ea8.html
linux¶
Contents:
systemd¶
Contents:
daemon-reload¶
每一次修改完了 *.service , 都要进行systemctl daemon-reload
[root@mariadb-node-0 zabbix]# vi /usr/lib/systemd/system/zabbix-agent.service
[root@mariadb-node-0 zabbix]# systemctl daemon-reload
[root@mariadb-node-0 zabbix]# systemctl stop zabbix-agent.service
[root@mariadb-node-0 zabbix]# systemctl start zabbix-agent.service
[root@mariadb-node-0 zabbix]# systemctl status zabbix-agent.service
● zabbix-agent.service - Zabbix Agent
Loaded: loaded (/usr/lib/systemd/system/zabbix-agent.service; disabled; vendor preset: disabled)
Active: active (running) since 二 2017-10-10 10:39:08 CST; 2s ago
Process: 149998 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 150001 (zabbix_agentd)
Memory: 2.5M
CGroup: /system.slice/zabbix-agent.service
├─150001 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
├─150002 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
├─150003 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
├─150004 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
├─150005 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
└─150006 /usr/sbin/zabbix_agentd: active checks #1 [idle 1 sec]
10月 10 10:39:08 mariadb-node-0 systemd[1]: Starting Zabbix Agent...
10月 10 10:39:08 mariadb-node-0 systemd[1]: PID file /run/zabbix/zabbix_agentd.pid not readable (yet?) after start.
10月 10 10:39:08 mariadb-node-0 systemd[1]: zabbix-agent.service: Supervising process 150001 which is not our child. We'll...exits.
10月 10 10:39:08 mariadb-node-0 systemd[1]: Started Zabbix Agent.
10月 10 10:39:09 mariadb-node-0 python[150008]: SELinux is preventing /usr/sbin/zabbix_agentd from using the setrlimit acc...ocess.
***** Plugin catchall (100. confidence) suggests **************************...
Hint: Some lines were ellipsized, use -l to show in full.
[root@mariadb-node-0 zabbix]#
systemctl list-units¶
systemctl list-units 可以查看所有已注册成 *.service 的服务。
root@km:/etc/systemd/system/kubelet.service.d# sudo systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Fri 2017-11-17 14:56:03 CST; 3s ago
Docs: http://kubernetes.io/docs/
Process: 30677 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE)
Main PID: 30677 (code=exited, status=1/FAILURE)
Nov 17 14:56:03 km systemd[1]: kubelet.service: Unit entered failed state.
Nov 17 14:56:03 km systemd[1]: kubelet.service: Failed with result 'exit-code'.
root@km:/etc/systemd/system/kubelet.service.d#
说明已有 kubelet 服务了哈。
但是，万一你不知道服务全名，这个时候，就可以用 list-units了。
root@km:/etc/systemd/system/kubelet.service.d# systemctl list-units | grep kube
kubelet.service loaded activating auto-restart kubelet: The Kubernetes Node Agent
root@km:/etc/systemd/system/kubelet.service.d#
10-kubeadm.conf¶
- ::
- root@km:/etc/systemd/system/kubelet.service.d# cat 10-kubeadm.conf [Service] Environment=”KUBELET_KUBECONFIG_ARGS=–bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf –kubeconfig=/etc/kubernetes/kubelet.conf” Environment=”KUBELET_SYSTEM_PODS_ARGS=–pod-manifest-path=/etc/kubernetes/manifests –allow-privileged=true” Environment=”KUBELET_NETWORK_ARGS=–network-plugin=cni –cni-conf-dir=/etc/cni/net.d –cni-bin-dir=/opt/cni/bin” Environment=”KUBELET_DNS_ARGS=–cluster-dns=10.96.0.10 –cluster-domain=cluster.local” Environment=”KUBELET_AUTHZ_ARGS=–authorization-mode=Webhook –client-ca-file=/etc/kubernetes/pki/ca.crt” Environment=”KUBELET_CADVISOR_ARGS=–cadvisor-port=0” Environment=”KUBELET_CERTIFICATE_ARGS=–rotate-certificates=true –cert-dir=/var/lib/kubelet/pki” #ExecStart=/usr/bin/kubelet -v 5 ExecStart=/usr/bin/kubelet -v 5 $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CERTIFICATE_ARGS
和下面这个
root@km:/etc/systemd/system/kubelet.service.d# cat 10-kubeadm.conf
[Service]
ExecStart=/usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true --cluster-dns=10.96.0.10 --cluster-domain=cluster.local --authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.crt --authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.crt --cadvisor-port=0 --rotate-certificates=true --cert-dir=/var/lib/kubelet/pki
这2个效果是一样的。
Centos7下的systemctl命令与service和chkconfig¶
grep¶
Contents:
substitute¶
替换
reference¶
http://linux.it.net.cn/e/shell/2015/0102/11241.html
http://www.cnblogs.com/qq78292959/archive/2012/03/08/2385080.html
http://blog.csdn.net/lzyzuixin/article/details/7638979
http://wiki.jikexueyuan.com/project/shell-learning/sed-search-and-replace.html
command¶
grep "https://api-cm.zhangtl.com/" -rl package.json | xargs sed -i 's/https:\/\/api-cm.zhangtl.com\//http:\/\/10.10.15.181:8000\//g'
grep "192.168.31.99" -rl ./ | xargs sed -i "s/192.168.31.99/192.168.31.105/g"
grep "192.168.31.105" -rl ./ | xargs sed -i "s/192.168.31.105/192.168.31.99/g"
linux¶
Contents:
telnet¶
http://www.linuxidc.com/Linux/2016-01/127983.htm
一.CentOS下查看系统是否已安装telnet
rpm -qa | grep telnet
telnet-0.17-48.el6.x86_64
telnet-server-0.17-48.el6.x86_64
telnet 是挂在 xinetd 底下的，所以同时查看是否安装了xinetd服务
rpm -qa | grep xinetd
xinetd-2.3.14-39.el6_4.x86_64
两者都没有的安装的话，yum安装以下服务
yum install xinetd telnet telnet-server -y
mount¶
Contents:
mount¶
没事,不要随便 mount .
我用 192.168.31.197, mount , 192.168.31.232 的 nfs , 然后,某一天, 突然发现, 192.168.31.197 , 出问题了.
ls /home/jlch/
死活,没有用.
最后, 农总, 发现是 某一个 device, 出错了, 导致的..
这个时候的表现是 高load average, 低cpu占用, 且 通过 top , 查不到进程占用资源. ps aux 也查不出(当然查不出, mount是直接操作到内核中, 并不是进程)
所以, 没事, 不要随便 mount, 且手工 Mount 后一定要及时 umount .
firewalld¶
Contents:
firewalld¶
常用命令
sudo firewall-cmd --permanent --add-port=7890/tcp; sudo firewall-cmd --reload; sudo firewall-cmd --permanent --list-port
snapd¶
Contents:
snapd¶
ubuntu 下
192.168.31.197 出现的问题, snapd 下的 gdm3 , 一直往 /var/log/syslog 下写数据, 妈妈呀, 谁受得了这个…. 后来, 重启后, 通过 查看 /var/log/syslog 可以得到一些信息, 很多信息来自于 它们3个, 所以, 我直接把它们关闭了.
systemctl status gdm3
systemctl stop gdm3
systemctl disable gdm3
systemctl status snapd
systemctl stop snapd
systemctl disable snapd
systemctl stop haproxy
systemctl disable haproxy
fsck¶
Contents:
linux shell¶
Contents:
ping ip¶
通过ping, 实现快速测试IP是否被占用¶
#!/bin/bash
# 通过ping, 实现快速测试IP是否被占用
for siteip in $(seq 1 254)
do
site="192.168.1.${siteip}"
ping -c1 -W1 ${site} &> /dev/null
if [ "$?" == "0" ]; then
echo "$site is UP"
else
echo "$site is DOWN"
fi
done
path¶
修改使用路径¶
root@km:~# which kubectl
/usr/local/bin/kubectl
root@km:~# kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.3", GitCommit:"2c2fe6e8278a5db2d15a013987b53968c743f2a1", GitTreeState:"clean", BuildDate:"2017-08-03T07:00:21Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server 192.168.31.120:6443 was refused - did you specify the right host or port?
root@km:~# which kubeadm
/usr/bin/kubeadm
root@km:~# echo $PATH
/home/jlch/.ana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
root@km:~# ls /usr/bin/kube*
/usr/bin/kubeadm /usr/bin/kubectl /usr/bin/kubelet
root@km:~# /usr/bin/kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-08T18:39:33Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server 192.168.31.120:6443 was refused - did you specify the right host or port?
从上面可知，/usr/local/bin:/usr/sbin 导致 kubectl 使用了 /usr/local/bin/kubectl ,而没有使用 /usr/bin/kubectl .如果说就是想使用，怎么办？
root@km:~# mv /usr/local/bin/kubectl /usr/local/bin/kubectl.delete
root@km:~# kubectl version
bash: /usr/local/bin/kubectl: No such file or directory
root@km:~#
哈哈，这就尴尬了。
root@km:~# ls /usr/bin/kube*
/usr/bin/kubeadm /usr/bin/kubectl /usr/bin/kubelet
root@km:~# echo $PATH
/home/jlch/.ana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
root@km:~# ls /usr/local/bin/kube*
/usr/local/bin/kubectl.delete
root@km:~#
看，这个时候，是不是很奇怪，明明应该向 /usr/bin/下找 kubectl 的。 问农总。 这个时候，切换一个 terminal , 比如shell 再登录，就可以了。
root@km:~# su - root
root@km:~# kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-08T18:39:33Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
root@km:~#
OK了。
linux network¶
Contents:
export¶
shell 下¶
- ::
- root@km:~# export http_proxy=”http://192.168.31.239:8118/” root@km:~# export https_proxy=”http://192.168.31.239:8118/” root@km:~# export no_proxy=”localhost,127.0.0.1,192.168.31.120,10.96.0.10,github.com,ubuntu.com” root@km:~# export … declare -x http_proxy=”http://192.168.31.239:8118/” declare -x https_proxy=”http://192.168.31.239:8118/” declare -x no_proxy=”localhost,127.0.0.1,192.168.31.120,10.96.0.10,github.com,ubuntu.com”
no_proxy
支持类型
export noproxy=.dev,localhost,127.0.0.1,.31.123,123
- 不支持类型
export no_proxy=192.,192.*,192.168.0.0/16
/etc/apt/apt.conf下¶
这下面的 no::proxy 不知道生效不生效。有知道的小伙伴，求赐教，谢。
root@km:~# cat /etc/apt/apt.conf
#Acquire::http::proxy "http://192.168.31.10:8031/";
Acquire::http::proxy "http://192.168.31.239:8118/";
#Acquire::https::proxy "https://192.168.31.10:8031/";
Acquire::https::proxy "https://192.168.31.239:8118/";
Acquire::no::proxy "ubuntu.com";
root@km:~#
linux cat¶
Contents:
EOF 输入¶
https://kubernetes.io/docs/setup/independent/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl
root@km:/opt/cni/bin# cat <<EOF >/etc/cni/net.d/10-flannel.conf
{
"name": "cbr0",
"type": "flannel",
"delegate": {
"isDefaultGateway": true
}
}
EOF
root@km:/opt/cni/bin#
linux chkconfig¶
Contents:
CentOS设置程序开机自启动的方法¶
转自：http://www.centos.bz/2011/09/centos-setup-process-startup-boot/
在CentOS系统下，主要有两种方法设置自己安装的程序开机启动。
/etc/rc.d/rc.local¶
1、把启动程序的命令添加到/etc/rc.d/rc.local文件中，比如下面的是设置开机启动httpd。
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
/usr/local/apache/bin/apachectl start
/etc/rc.d/init.d/¶
2、把写好的启动脚本添加到目录/etc/rc.d/init.d/，然后使用命令chkconfig设置开机启动。
chkconfig 功能说明：检查，设置系统的各种服务。
语法：chkconfig [–add][–del][–list][系统服务] 或 chkconfig [–level <等级代号>][系统服务][on/off/reset]
–add 添加服务 –del 删除服务 –list 查看各服务启动状态
比如我们设置自启动mysql：
#将mysql启动脚本放入所有脚本运行目录/etc/rc.d/init.d中
cp /lamp/mysql-5.0.41/support-files/mysql.server /etc/rc.d/init.d/mysqld
#改变权限
chown root.root /etc/rc.d/init.d/mysqld
#所有用户都可以执行，单只有root可以修改
chmod 755 /etc/rc.d/init.d/mysqld
#将mysqld 放入linux启动管理体系中
chkconfig --add mysqld
#查看全部服务在各运行级状态
chkconfig --list mysqld
#只要运行级别3启动，其他都关闭
chkconfig --levels 245 mysqld off
例如：我们把httpd的脚本写好后放进/etc/rc.d/init.d/目录，使用
chkconfig --add httpd
chkconfig httpd on
命令即设置好了开机启动。
示例1¶
我在 10.10.12.10 机器上，做了2个 redis, 让其开机自启动。
[jlch@kube-master-10 ~]$ cat /etc/rc.d/init.d/redis_6379
#!/bin/sh
#Configurations injected by install_server below....
EXEC=/usr/local/bin/redis-server
CLIEXEC=/usr/local/bin/redis-cli
PIDFILE=/var/run/redis_6379.pid
CONF="/etc/redis/6379.conf"
REDISPORT="6379"
###############
# SysV Init Information
# chkconfig: - 58 74
# description: redis_6379 is the redis daemon.
### BEGIN INIT INFO
# Provides: redis_6379
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Should-Start: $syslog $named
# Should-Stop: $syslog $named
# Short-Description: start and stop redis_6379
# Description: Redis daemon
### END INIT INFO
case "$1" in
start)
if [ -f $PIDFILE ]
then
echo "$PIDFILE exists, process is already running or crashed"
else
echo "Starting Redis server..."
$EXEC $CONF
fi
;;
stop)
if [ ! -f $PIDFILE ]
then
echo "$PIDFILE does not exist, process is not running"
else
PID=$(cat $PIDFILE)
echo "Stopping ..."
$CLIEXEC -p $REDISPORT shutdown
while [ -x /proc/${PID} ]
do
echo "Waiting for Redis to shutdown ..."
sleep 1
done
echo "Redis stopped"
fi
;;
status)
PID=$(cat $PIDFILE)
if [ ! -x /proc/${PID} ]
then
echo 'Redis is not running'
else
echo "Redis is running ($PID)"
fi
;;
restart)
$0 stop
$0 start
;;
*)
echo "Please use start, stop, restart or status as first argument"
;;
esac
示例2¶
[jlch@kube-master-10 ~]$ cat /etc/rc.d/init.d/redis_6379
#!/bin/sh
#Configurations injected by install_server below....
#EXEC=/usr/local/bin/redis-server
EXEC=/opt/redis-4.0.2-haitong/src/redis-server
CLIEXEC=/usr/local/bin/redis-cli
PIDFILE=/var/run/redis_6379.pid
#CONF="/etc/redis/6379.conf"
CONF="/opt/redis-4.0.2-haitong/redis.conf"
REDISPORT="6379"
###############
# SysV Init Information
# chkconfig: - 58 74
# description: redis_6379 is the redis daemon.
### BEGIN INIT INFO
# Provides: redis_6379
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Should-Start: $syslog $named
# Should-Stop: $syslog $named
# Short-Description: start and stop redis_6379
# Description: Redis daemon
### END INIT INFO
case "$1" in
start)
if [ -f $PIDFILE ]
then
echo "$PIDFILE exists, process is already running or crashed"
else
echo "Starting Redis server..."
$EXEC $CONF
fi
;;
stop)
if [ ! -f $PIDFILE ]
then
echo "$PIDFILE does not exist, process is not running"
else
PID=$(cat $PIDFILE)
echo "Stopping ..."
$CLIEXEC -p $REDISPORT shutdown
while [ -x /proc/${PID} ]
do
echo "Waiting for Redis to shutdown ..."
sleep 1
done
echo "Redis stopped"
fi
;;
status)
PID=$(cat $PIDFILE)
if [ ! -x /proc/${PID} ]
then
echo 'Redis is not running'
else
echo "Redis is running ($PID)"
fi
;;
restart)
$0 stop
$0 start
;;
*)
echo "Please use start, stop, restart or status as first argument"
;;
esac
[jlch@kube-master-10 ~]$
/etc/rc.d/rc.sysinit¶
3、把启动程序的命令添加到/etc/rc.d/rc.sysinit 文件中
脚本/etc/rc.d/rc.sysinit，完成系统服务程序启动，如系统环境变量设置、设置系统时钟、加载字体、检查加载文件系统、生成系统启动信息日志文件等
比如我们设置自启动apache：
echo"/usr/local/apache2/bin/apachectl start" >> /etc/rc.d/rc.sysinit
curl¶
Contents:
curl -v 参数作调试¶
加了这个 -v 参数后，可以看到更详细的内容。
从内网取
[root@test_240 conf.d]# curl -X POST "http://10.10.11.20:8000/transmitter-api-v2/session" -H "accept: application/json" -H "content-type: application/json" -d "{ \"login_name\": \"xiaozhao\", \"login_pwd\": \"wdxiaozhao\"}" -v
* About to connect() to 10.10.11.20 port 8000 (#0)
* Trying 10.10.11.20...
* Connected to 10.10.11.20 (10.10.11.20) port 8000 (#0)
> POST /transmitter-api-v2/session HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.10.11.20:8000
> accept: application/json
> content-type: application/json
> Content-Length: 54
>
* upload completely sent off: 54 out of 54 bytes
< HTTP/1.1 404 Not Found
< Server: nginx/1.8.0
< Date: Fri, 08 Dec 2017 06:47:01 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 63
< Connection: keep-alive
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, POST, PUT, UPDATE, DELETE, PATCH, OPTIONS
< Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, x-access-token
<
* Connection #0 to host 10.10.11.20 left intact
{"code":"ResourceNotFound","message":"/session does not exist"}
从阿里云过外网取
[tom@jlch_web_001 ~]$ curl -X POST "http://120.234.23.98:8000/transmitter-api-v2/session" -H "accept: application/json" -H "content-type: applicatin_name\": \"xiaozhao\", \"login_pwd\": \"wdxiaozhao\"}" -v
* About to connect() to 120.234.23.98 port 8000 (#0)
* Trying 120.234.23.98...
* Connected to 120.234.23.98 (120.234.23.98) port 8000 (#0)
> POST /transmitter-api-v2/session HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 120.234.23.98:8000
> accept: application/json
> content-type: application/json
> Content-Length: 54
>
* upload completely sent off: 54 out of 54 bytes
< HTTP/1.1 404 Not Found
< Content-Type: text/html
< Expires: 0
< Cache-control: private
< Content-Length: 334
<
* Connection #0 to host 120.234.23.98 left intact
Sorry, Page Not Found
防火墙，直接篡改了 404的错误页面，还是很666的…..
linux ssh¶
Contents:
ssh 连接 登录 很慢¶
env¶
10.10.15.181 => 10.10.13.12
问题¶
10.10.12.13, 生成信号（/home/jlch/OpenX/OpenQuote2/kline_signaller.js ）的时候，要连接 Mongo 10.10.13.12，但是，连接明显很慢，导致先连接 redis, 然后产生的信号没有连接上mongo并保存。
然后，发现，ssh 10.10.13.12, 明显太慢了，大约要 10s.
解决¶
登录 10.10.13.12 ，修改 10.10.13.12 的 /etc/ssh/sshd_config 文件
主要是 GSSAPIAuthentication 和 UseDNS 参数修改成 no
[jlch@mongodb1 ~]$ sudo cat /etc/ssh/sshd_config | grep GSSAPIAuthentication
#GSSAPIAuthentication yes
GSSAPIAuthentication no
[jlch@mongodb1 ~]$ sudo cat /etc/ssh/sshd_config | grep UseDNS
#UseDNS yes
UseDNS no
[jlch@mongodb1 ~]$ sudo systemctl restart sshd
然后再次登录，则很快了。
ssh ca digital_signature htpps¶
密码学笔记¶
加密方法可以分为两大类。一类是单钥加密（private key cryptography），还有一类叫做双钥加密（public key cryptography）
http://www.ruanyifeng.com/blog/2006/12/notes_on_cryptography.html
SSH原理与运用（一）：远程登录¶
http://www.ruanyifeng.com/blog/2011/12/ssh_remote_login.html
SSL/TLS协议运行机制的概述
http://www.ruanyifeng.com/blog/2014/02/ssl_tls.html
图解SSL/TLS协议
http://www.ruanyifeng.com/blog/2014/09/illustration-ssl.html
screen¶
Contents:
linux 技巧：使用 screen 管理你的远程会话¶
https://www.ibm.com/developerworks/cn/linux/l-cn-screen/
[jlch@udvbct-ekl-a ~]$ ps -xj
PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND
8217 8219 8217 8217 ? -1 S 1000 0:00 sshd: jlch@pts/5
8219 8220 8220 8220 pts/5 8278 Ss 1000 0:00 -bash
8220 8278 8278 8220 pts/5 8278 R+ 1000 0:00 ps -xj
[jlch@udvbct-ekl-a ~]$
ps -xj 可以看到 这些个 PPID, PID, PGID, SID, TTY, TPGID 这些概念.
linux rescue¶
Contents:
centos7营救模式¶
http://blog.csdn.net/u013753804/article/details/48392325
主要步骤：
插入U盘，重新开机，开机过程按下F11, 进入启动区引导选择，选择U盘启动,
选择troubleshooting
选择Rescue a CentOS system
然后就开始从U盘启动，加载内核，文件系统。。。。。经过漫长的等待
选择1
回车
终于来到rescue模式
救援模式挂载lvm出错-mount unknown filesystem type ‘lvm2_member’解决¶
http://blog.51cto.com/tonychiu/769934
rhel6救援模式挂载lvm出错：
mount unknown filesystem type 'lvm2_member'
解决办法：
1、从fdisk 信息不难看出，已经加密的 “*” ,肯定是我的引导分区，而我的真实/ 安装在/dev/sda2上的。
2、挂载出错
3、查看物理卷、逻辑卷：pvs
pvs
vgs
4、查看逻辑卷：lvdisplay
lvdisplay
5、如未激活，需要激活逻辑卷
vgchange -ay /dev/centos_miwifi-r1cm
备注：步骤3-5的命令有时报command not found，解决
bash-4.1#lvm
lvm >
这样再执行如上命令
6、挂载逻辑卷
mkdir /mnt/sdaroot
mount /dev/centos_miwifi-r1cm/root /mnt/sdaroot
ls /mnt/sdaroot/var/log/
原来系统 的 /var/log/ 转移
sh-4.1# mount /dev/sdc4 /mnt/upan/log/
sh-4.1# cp -a /mnt/sdaroot/var/log/ /mnt/upan/log/
这样就把 原来系统 的 /var/log/ 转移到了 /mnt/upan/log/ ，也就是U盘的 log 目录下。
如果说做一个 chroot, 则如下：
sh-4.1# chroot /mnt/sdaroot/
bash-4.1#
这里，在chroot之前 sh-4.1# 目录下的/ 是内存中的 / ，也叫伪根；chroot后 bash-4.1# 目录下的/ 才是系统真实的/ ，也叫真根
io-cpu-memory¶
关于 io-cpu-memory 的查询
[tom@check ~]$ iostat
Linux 3.10.0-327.4.4.el7.x86_64 (check) 2017年12月11日 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
3.82 0.00 0.32 0.94 0.00 94.92
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 49.91 1062.33 2048.76 9479441715 18281532382
dm-0 1.51 3.60 77.51 32160568 691671407
dm-1 1.60 3.17 3.24 28282128 28937428
dm-2 0.61 3.89 8.54 34709518 76238588
dm-3 48.04 1051.67 1959.46 9384260398 17484682850
[tom@check ~]$
[tom@check ~]$ vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 1 1796304 277276 12 8753928 0 0 133 256 0 1 4 0 95 1 0
1 2 1796300 276716 12 8755312 16 0 720 26215 2242 2412 15 3 81 2 0
1 1 1796296 275296 12 8756816 0 0 0 22747 1650 1171 12 1 86 1 0
1 0 1796296 273772 12 8760704 4 0 1716 19107 1643 1915 9 1 85 4 0
1 0 1796296 265688 12 8768892 0 0 6432 13806 1924 2063 9 1 86 5 0
1 0 1796296 264920 12 8769968 20 0 20 2041 1640 1026 14 0 85 0 0
[tom@check ~]$ top
top - 17:18:46 up 103 days, 6:38, 10 users, load average: 1.04, 0.93, 0.86
Tasks: 393 total, 1 running, 392 sleeping, 0 stopped, 0 zombie
%Cpu(s): 10.1 us, 0.8 sy, 0.0 ni, 85.3 id, 3.8 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 16216960 total, 261732 free, 7262872 used, 8692356 buff/cache
KiB Swap: 16773116 total, 14976812 free, 1796304 used. 8445804 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
15825 mysql 20 0 9864996 4.908g 7512 S 81.9 31.7 17954:43 mysqld
898 root 20 0 4372 204 64 S 1.1 0.0 529:06.74 rngd
37715 rabbitmq 20 0 6059776 49684 2952 S 1.1 0.3 596:35.47 beam.smp
65 root 20 0 0 0 0 S 0.4 0.0 24:16.56 rcuos/6
819 root 20 0 0 0 0 S 0.4 0.0 30:43.70 xfsaild/dm-3
1646 redis 20 0 150880 2908 800 S 0.4 0.0 111:13.10 redis-server
2862 tom 20 0 146412 2404 1428 R 0.4 0.0 0:00.23 top
4247 tom 20 0 934376 19612 4796 S 0.4 0.1 306:20.27 PM2 v1.1.3: God
4378 jlch 20 0 938180 23904 4876 S 0.4 0.1 437:56.05 PM2 v1.1.3: God
31674 tom 20 0 938164 22372 4748 S 0.4 0.1 42:37.57 node /home/tom/
45485 jenkins 20 0 9777.8m 1.186g 8236 S 0.4 7.7 242:55.06 java
46382 zabbix 20 0 84736 1252 1100 S 0.4 0.0 25:59.45 zabbix_agentd
1 root 20 0 189620 3388 1560 S 0.0 0.0 8:59.88 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.70 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 4:51.05 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
8 root rt 0 0 0 0 S 0.0 0.0 0:13.08 migration/0
9 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
redis¶
Contents:
redis-cluster¶
Contents:
redis-cluster-install-by-rpm¶
http://rpmfind.net/ http://rpmfind.net/linux/rpm2html/search.php?query=redis&submit=Search+… http://rpmfind.net/linux/RPM/remi/enterprise/7/x86_64/redis-4.0.2-1.el7.remi.x86_64.html
https://pkgs.org/download/redis https://centos.pkgs.org/7/remi-x86_64/redis-4.0.2-1.el7.remi.x86_64.rpm.html
— 改 fedora 版本
http://rpmfind.net/linux/rpm2html/search.php?query=redis&submit=Search+…&system=&arch=
step¶
安装 ruby, gem
gem install redis
启动 redis
systemctl status redis
systemctl start redis
systemctl status redis
redis-cli
systemctl stop redis
启动 redis-cluster
下面这一部分，可以参考一下，当时给 海通证券部署factor 的情况。
cd by-source/
ls
cd redis-deploy-conf/
ls
./redis-deploy-ip-port-passwd.sh 127.0.0.1 7000 wudang
./redis-deploy-ip-port-passwd.sh 127.0.0.1 7001 wudang
./redis-deploy-ip-port-passwd.sh 127.0.0.1 7002 wudang
./redis-deploy-ip-port-passwd.sh 127.0.0.1 7003 wudang
./redis-deploy-ip-port-passwd.sh 127.0.0.1 7004 wudang
./redis-deploy-ip-port-passwd.sh 127.0.0.1 7005 wudang
ls
ls /usr/lib/systemd/system/redis*
cat /usr/lib/systemd/system/redis.service
/usr/bin/redis-server node-7000/redis.conf &
/usr/bin/redis-server node-7001/redis.conf &
/usr/bin/redis-server node-7002/redis.conf &
/usr/bin/redis-server node-7003/redis.conf &
/usr/bin/redis-server node-7004/redis.conf &
/usr/bin/redis-server node-7005/redis.conf &
./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
redis-cluster-del-key¶
总体思路¶
先拿到 key 所在的 slot ，然后就能找到对应 的 IP:Port, 然后删除就行了。
查 cluster 信息¶
[jlch@check ~]$ redis-cli -h 192.168.31.166 -p 7000 -a wudang -c cluster nodes
查 key 所在的 slot¶
[jlch@check ~]$ redis-cli -h 192.168.31.166 -p 7000 -a wudang -c cluster keyslot 600000
删除 key¶
[jlch@check ~]$ redis-cli -h 192.168.31.168 -p 7004 -a wudang -c get 600000
[jlch@check ~]$ redis-cli -h 192.168.31.168 -p 7004 -a wudang -c del 600000
批量删除¶
http://blog.csdn.net/badyting/article/details/76176824
cat redis-cluster-del.sh
#!/bin/bash
echo "start"
redis_list=("127.0.0.1:6379" "127.0.0.1:6380")
for info in ${redis_list[@]}
do
echo "开始执行:$info"
ip=`echo $info | cut -d \: -f 1`
port=`echo $info | cut -d \: -f 2`
# 如果有密码，要加上 -a password
cat key.txt | xargs -t -n1 redis-cli -h $ip -p $port -c del
done
echo "game over"
redis-cluster conf 配置 requirepass¶
redis-cluster-requirepass-auth¶
redis-cluster-requirepass-auth 配置密码时，一定要另加一句 AUTH wudang
reference¶
http://www.yiibai.com/redis/connection_auth.html http://bbs.csdn.net/topics/391824759?page=1
[jlch@check ~]$ redis-cli -h 192.168.31.166 -p 7000 -a wudang -c
192.168.31.166:7000> AUTH wudang
(error) ERR Client sent AUTH, but no password is set
192.168.31.166:7000> CONFIG SET requirepass "wudang"
OK
192.168.31.166:7000> AUTH wudang
OK
192.168.31.166:7000>
不然，配置的密码，在连接时会报错的哟。
集群客户端命令（redis-cli -c -p port）¶
http://www.cnblogs.com/gossip/p/5993922.html
集群
cluster info ：打印集群的信息
cluster nodes ：列出集群当前已知的所有节点（ node），以及这些节点的相关信息。
节点
cluster meet <ip> <port> ：将 ip 和 port 所指定的节点添加到集群当中，让它成为集群的一份子。
cluster forget <node_id> ：从集群中移除 node_id 指定的节点。
cluster replicate <node_id> ：将当前节点设置为 node_id 指定的节点的从节点。
cluster saveconfig ：将节点的配置文件保存到硬盘里面。
槽(slot)
cluster addslots <slot> [slot ...] ：将一个或多个槽（ slot）指派（ assign）给当前节点。
cluster delslots <slot> [slot ...] ：移除一个或多个槽对当前节点的指派。
cluster flushslots ：移除指派给当前节点的所有槽，让当前节点变成一个没有指派任何槽的节点。
cluster setslot <slot> node <node_id> ：将槽 slot 指派给 node_id 指定的节点，如果槽已经指派给另一个节点，那么先让另一个节点删除该槽>，然后再进行指派。
cluster setslot <slot> migrating <node_id> ：将本节点的槽 slot 迁移到 node_id 指定的节点中。
cluster setslot <slot> importing <node_id> ：从 node_id 指定的节点中导入槽 slot 到本节点。
cluster setslot <slot> stable ：取消对槽 slot 的导入（ import）或者迁移（ migrate）。
键
cluster keyslot <key> ：计算键 key 应该被放置在哪个槽上。
cluster countkeysinslot <slot> ：返回槽 slot 目前包含的键值对数量。
cluster getkeysinslot <slot> <count> ：返回 count 个 slot 槽中的键
redis sentinel¶
https://redis.io/topics/sentinel
这种 cluster , 更关注的是 冗余，关注 数据的准确性，但是没有分片。
之前的那种 cluster, 可以实现分片。
redis conf 制作¶
docker 中 redis-cli 连接¶
用 docker 容器来执行一个 redis 客户端
docker run -t -i –rm redis /bin/bash
[jlch@udvbct-ekl-a ~]$ sudo docker pull redis
[jlch@udvbct-ekl-a ~]$ sudo docker run -t -i --rm redis /bin/bash
root@b0541c712009:/data# redis-cli -h 192.168.31.149
192.168.31.149:6379> ping
PONG
192.168.31.149:6379>
root@b0541c712009:/data# redis-cli -h 192.168.31.181
192.168.31.181:6379> ping
PONG
192.168.31.181:6379>
root@b0541c712009:/data#
Framework Benchmarks¶
Contents:
Web Framework Benchmarks¶
Contents:
Web Framewosrk Benchmarks¶
一个超级好的 Web Framewosrk Benchmarks
http://www.techempower.com/whatwedo.html
http://www.techempower.com/benchmarks/
http://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=update
这里面才是真正的压力测试结果。
- 里面可以看到 市面上 最全的
- web 架构方案， 数据库， 程序语言language，
的表现
docker¶
Contents:
nginx¶
Contents:
nginx-install¶
Contents:
nginx install ubuntu¶
ubuntu16.04 install ubuntu
cd
wget http://nginx.org/keys/nginx_signing.key
sudo apt-key add nginx_signing.key
jlch@ceph-client:~$ cat /etc/apt/sources.list.d/nginx.list
deb http://nginx.org/packages/debian/ xenial nginx
deb-src http://nginx.org/packages/debian/ xenial nginx
echo "deb http://nginx.org/packages/debian/ xenial nginx
deb-src http://nginx.org/packages/debian/ xenial nginx" > /etc/apt/sources.list.d/nginx.list
sudo apt-get update
sudo apt install nginx -y
nginx conf¶
Contents:
nginx-conf-server_name¶
下面的配置，是 蛟龙出海 阿里云机器 的 nginx 配置。
注意看下面的server_name
[tom@jlch_web_001 ~]$ cd /etc/nginx/conf.d/
[tom@jlch_web_001 conf.d]$ ls
jiaolongchuhai.conf mjiaolongchuhai.conf zhangtl.conf
[tom@jlch_web_001 conf.d]$ cat zhangtl.conf
server {
listen 80;
server_name www.zhangtl.com www.zhangtl.cn;
root /home/jlch/zhangtl;
index index.html index.htm;
location / {
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
[tom@jlch_web_001 conf.d]$ cat mjiaolongchuhai.conf
server {
listen 80;
server_name m.jiaolongchuhai.com;
root /home/jlch/mjiaolongchuhai;
index index.html index.htm;
location / {
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
[tom@jlch_web_001 conf.d]$ cat jiaolongchuhai.conf
server {
listen 80;
server_name www.jiaolongchuhai.com;
root /home/jlch/jiaolongchuhai;
index index.html index.htm;
location / {
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
[tom@jlch_web_001 conf.d]$
goaccess¶
Contents:
goaccess 安装 使用¶
reference¶
http://www.itread01.com/articles/1476744943.html
http://www.jianshu.com/p/33e2e79d0373
http://chenx1242.blog.51cto.com/10430133/1882643
192.168.31.240:/opt/goaccess-1.2/
goaccess 安装¶
ps -ef | grep goacc*
sudo yum install glib2 glib2-devel GeoIP-devel ncurses-devel zlib zlib-devel -y
cd /opt/
ls
sudo wget http://tar.goaccess.io/goaccess-1.2.tar.gz
sudo tar -xzvf goaccess-1.2.tar.gz
cd goaccess-1.2/
sudo ./configure --enable-utf8 --enable-geoip=legacy
sudo make
sudo make install
ls
ls goaccess*
sudo cp /usr/local/etc/goaccess.conf .
cat >>./goaccess.conf<<EOF
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u"
EOF
使用：
M1
ls /var/log/nginx/access.log
tail -f /var/log/nginx/access.log
sudo goaccess -a -d -f /var/log/nginx/access.log -p ./goaccess.conf >
sudo goaccess -a -d -f /var/log/nginx/access.log -p ./goaccess.conf
/home/tom/index.html
M2
[root@qlw goaccess-1.2]# goaccess -a -d -f /var/log/nginx/8000.access.log -p ./goaccess.conf -o /usr/share/nginx/html/qstation/statistics/report.html –real-time-html
WebSocket server ready to accept new client connections
M3
tail -f -n +0 /var/log/nginx/8000.access.log | grep --color=auto "factor-zx" | goaccess -p ./goaccess.conf -o /usr/share/nginx/html/qstation/statistics/report-factor-zx.html --real-time-html &
tail -f -n +0 /var/log/nginx/8000.access.log | grep --color=auto "GET /factor-zx/ " | goaccess -p ./goaccess.conf -o /usr/share/nginx/html/qstation/statistics/report-factor-zx.html --real-time-html &
tail -f -n +0 /var/log/nginx/8002.access.log | goaccess -p ./goaccess.conf -o /usr/share/nginx/html/qstation/statistics/report-factorzx.html --real-time-html
tail -f -n +0 /var/log/nginx/443.access.log | goaccess -p ./goaccess.conf -o /data/nginx/html/statistics/reports.html --real-time-html
实践：
root@10.10.11.10
mkdir -p /data/nginx/html/statistics/
tail -f -n +0 /var/log/nginx/443.access.log | grep "citics" | goaccess -p ./goaccess.conf -o /data/nginx/html/statistics/reports-citics.html --real-time-html
nginx-url-rewrite¶
Contents:
nginx url rewrite¶
http://www.cnblogs.com/czlun/articles/7010604.html
env¶
192.168.31.240
step¶
获取访问结果文件¶
先把常用的 url 用 wget, curl 把文件取下来。
wget --header="accept:application/json" -d "http://192.168.31.181:3062/stocks/600006.SH/factor_analysis?type=1&cycle=2" # -d 输出调试信息
curl -v -X GET "http://192.168.31.181:3062/stocks/600006.SH/factor_analysis?type=1&cycle=2" -H "accept:application/json" # -v 输出调试信息
这些文件，放在某个固定的位置下，
配置 nginx¶
nginx-url-rewrite-conf¶
先放一个示例。
server {
listen 5112;
server_name factorapi;
#charset koi8-r;
access_log /var/log/nginx/log/5111.access.log main;
rewrite_log on;
charset utf-8;
charset_types text/html application/json;
location / {
root /usr/share/nginx/html/rewrite;
index index.html index.htm;
# rewrite /a.json /b.json permanent;
rewrite /a.json /b.json ;
}
location /factor-api {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# 如果参数中有 type=xx 则使用该值
if ($args ~* type=(\d+?)(&|$)) {
set $type $1;
}
# 如果参数中有 type=xx 则使用该值
if ($args ~* cycle=(\d+?)(&|$)) {
set $cycle $1;
}
# 取 request_uri 中的数据
if ( $request_uri ~* '^/factor-api\/stocks\/(\d{6}\.S[H|Z])\/factor_analysis' ) {
set $stockcode $1;
}
if (-e /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json){
rewrite ^/(.*)$ /data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json last;
}
proxy_pass http://192.168.31.181:3062/;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
上面这个示例，是参考 https://stackoverflow.com/questions/43284560/nginx-try-files-rewrite-if-exists-proxy-if-doesnt 而得来的。比较不错。
server {
listen 5112;
server_name factorapi;
#charset koi8-r;
access_log /var/log/nginx/log/5111.access.log main;
rewrite_log on;
charset utf-8;
charset_types text/html application/json;
location / {
root /usr/share/nginx/html/rewrite;
index index.html index.htm;
# rewrite /a.json /b.json permanent;
rewrite /a.json /b.json ;
}
location /factor-api {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# 如果参数中有 type=xx 则使用该值
if ($args ~* type=(\d+?)(&|$)) {
set $type $1;
}
# 如果参数中有 type=xx 则使用该值
if ($args ~* cycle=(\d+?)(&|$)) {
set $cycle $1;
}
# 取 request_uri 中的数据
if ( $request_uri ~* '^/factor-api\/stocks\/(\d{6}\.S[H|Z])\/factor_analysis' ) {
set $stockcode $1;
}
rewrite '^/factor-api\/stocks\/(\d{6}\.S[H|Z])\/factor_analysis' /data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json ; # 注意，这里不能用 _$cycle ，所以只能用 .$cycle 来连接了。
if (!-e /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json ) {
# return 400;
rewrite ^/(.+)$ /index.html last;
}
proxy_pass http://192.168.31.181:3062/; # 基本上这条规则，已失效了。
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
下面是 try_files,
注意，try_files 还不能 与 proxy_pass 配合使用。
而且，在没有完全弄明白 try_files 之前，还是先不要使用了。
server {
listen 5112;
server_name factorapi;
#charset koi8-r;
access_log /var/log/nginx/log/5111.access.log main;
rewrite_log on;
charset utf-8;
charset_types text/html application/json;
location / {
root /usr/share/nginx/html/rewrite;
index index.html index.htm;
# rewrite /a.json /b.json permanent;
rewrite /a.json /b.json ;
}
location /factor-api {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# 如果参数中有 type=xx 则使用该值
if ($args ~* type=(\d+?)(&|$)) {
set $type $1;
}
# 如果参数中有 type=xx 则使用该值
if ($args ~* cycle=(\d+?)(&|$)) {
set $cycle $1;
}
if ( $request_uri ~* '^/factor-api\/stocks\/(\d{6}\.S[H|Z])\/factor_analysis' ) {
set $stockcode $1;
}
try_files /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json @qwe;
proxy_pass http://192.168.31.181:3062/; # 基本上这条规则，已失效了。
}
location @qwe {
rewrite ^/(.+)$ /index.html last;
}
# try_files /data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json $request_uri;
# try_files "http://192.168.31.181:5112/data/factor/20171107/600006.SH/c.json" $request_uri;
# try_files /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json $request_uri;
# }
# location @qwe {
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_pass http://192.168.31.181:3062/; # 基本上这条规则，已失效了。
# }
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
上面是 精简版本 测试的过程中，有下面这些
测试 正则表达式
# ——- not ok ———– # rewrite ^/factor-api/stocks/(d{6}.S[H|Z])/factor_analysis /data/factor/20171107/$1/c.json permanent; # 这里有 大括号，要用 单引号 引起来。 # —— ok ——— # rewrite ‘^/factor-api/stocks/(d{6}.S[H|Z])/factor_analysis’ /data/factor/20171107/$1/c.json permanent; # rewrite ^/factor-api/stocks/([1-9]d*.S[H|Z])/factor_analysis /data/factor/20171107/$1/c.json permanent; # rewrite ^/factor-api/stocks/(.*)/factor_analysis /data/factor/20171107/$1/c.json permanent; # rewrite ^/factor-api/stocks/(d*.S[H|Z])/factor_analysis /data/factor/20171107/$1/c.json permanent;
# ——— if ok —————– # rewrite ‘^/factor-api/stocks/(d{6}.S[H|Z])/factor_analysis’ /data/factor/20171107/$1/c.json ; # set $stockcode $1; # if (!-e /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/c.json ) { # return 400; # }
# rewrite ‘^/factor-api/stocks/(d{6}.S[H|Z])/factor_analysis’ /data/factor/20171107/$1/factor_analysis_type_1_cycle_2.json ; # set $stockcode $1; # if (!-e /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/factor_analysis_type_1_cycle_2.json ) { # return 400; # }
# rewrite ‘^/factor-api/stocks/(d{6}.S[H|Z])/factor_analysis’ /data/factor/20171107/$1/$cycle.json ; # set $stockcode $1; # if (!-e /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/factor_analysis_type_1_cycle_2.json ) { # return 400; # }
- if ( $request_uri ~* ‘^/factor-api/stocks/(d{6}.S[H|Z])/factor_analysis’ ) {
- set $stockcode $1;
} try_files /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json @qwe;
# rewrite ‘^/factor-api/stocks/(d{6}.S[H|Z])/factor_analysis’ /data/factor/20171107/$1/factor_analysis.type.$type.cycle.$cycle.json ; # 注意，这里不能用 _$cycle ，所以只能用 .$cycle 来连接了>。 # set $stockcode $1; # try_files /usr/share/nginx/html/rewrite/data/factor/20171107/$stockcode/factor_analysis.type.$type.cycle.$cycle.json @qwe;
}
- location @qwe {
- rewrite ^/(.+)$ /index.html last; #proxy_pass http://192.168.31.181:3062/; # 基本上这条规则，已失效了。
}
big-data-1g-download-failure¶
蛟龙-海通，通讯程序，超过1G的数据，就出错了。
那么百度了一下，
https://www.baidu.com/s?ie=utf-8&f=8&rsv_bp=0&rsv_idx=1&tn=baidu&wd=nginx%20%E4%B8%8B%E8%BD%BD%201G%20%E6%96%AD%E5%BC%80&rsv_pq=80215161000085a1&rsv_t=0e99I%2FzMf%2BHB7Umakbrg9zOn08tQ2h6Nc%2FAkVeEmCSu43QW%2B%2Ffyfn%2BD2zaY&rqlang=cn&rsv_enter=1&rsv_sug3=23&rsv_sug1=10&rsv_sug7=100&rsv_sug2=0&inputT=11519&rsv_sug4=11519 | nginx 下载 1G 断开_百度搜索 https://segmentfault.com/q/1010000000383415 | nginx 反向代理，遇到大文件无法下载 - SegmentFault http://blog.chinaunix.net/uid-20332519-id-5755724.html | Nginx反向代理导致大文件下载失败-lsstarboy-ChinaUnix博客 http://www.ttlsa.com/nginx/optimizing-nginx-bigger-file/ | nginx大文件下载优化 – 运维生存时间 http://www.cnblogs.com/breezey/p/6740229.html | nginx反向代理下载文件失败处理 - breezey - 博客园
proxy_send_timeout 120;
proxy_read_timeout 120;
proxy_max_temp_file_size 4096m;
重启了一下 nginx, 看一下
[jlch@qlw ~]$ sudo /usr/sbin/nginx -s reload
nginx: [warn] duplicate MIME type "text/html" in /etc/nginx/conf.d/8000.qstation.conf:11
nginx: [warn] duplicate MIME type "text/html" in /etc/nginx/conf.d/8002.factor-zx.conf:10
[jlch@qlw ~]$ sudo systemctl restart nginx
等行痴的反馈吧。
nginx-install-by-rpm¶
nginx location¶
location 的 匹配规则
http://blog.csdn.net/fay462298322/article/details/54666636
Nginx location的匹配规则
~ 波浪线表示执行一个正则匹配，区分大小写
~* 表示执行一个正则匹配，不区分大小写
^~ ^~表示普通字符匹配，如果该选项匹配，只匹配该选项，不匹配别的选项，一般用来匹配目录
= 进行普通字符精确匹配
@ #"@" 定义一个命名的 location，使用在内部定向时，例如 error_page, try_files
location 匹配优先级
= 精确匹配会第一个被处理。如果发现精确匹配，nginx停止搜索其他匹配。
普通字符匹配，正则表达式规则和长的块规则将被优先和查询匹配，也就是说如果该项匹配还需去看有没有正则表达式匹配和更长的匹配。
^~ 则只匹配该规则，nginx停止搜索其他匹配，否则nginx会继续处理其他location指令。
最后匹配理带有"~"和"~*"的指令，如果找到相应的匹配，则nginx停止搜索其他匹配；当没有正则表达式或者没有正则表达式被匹配的情况下，那么匹配程度最高的逐字匹配指令会被使用。
示例
location = / {
# 只匹配"/".
[ configuration A ]
}
location / {
# 匹配任何请求，因为所有请求都是以"/"开始
# 但是更长字符匹配或者正则表达式匹配会优先匹配
[ configuration B ]
}
location ^~ /images/ {
# 匹配任何以 /images/ 开始的请求，并停止匹配 其它location
[ configuration C ]
}
location ~* \.(gif|jpg|jpeg)$ {
# 匹配以 gif, jpg, or jpeg结尾的请求.
# 但是所有 /images/ 目录的请求将由 [Configuration C]处理.
[ configuration D ]
}
mysql¶
Contents:
mysqladmin¶
Contents:
mysqladmin flush-hosts¶
Database ErrorHost ‘adonis.havehost.com’ is blocked because of many connection errors; unblock with ‘mysqladmin flush-hosts’
https://stackoverflow.com/questions/21519997/how-to-do-mysqladmin-flush-hosts-on-server
mysql -u root -p -e 'flush hosts'
mysqldump¶
Contents:
mysqldump jlch 表结构¶
mysqldump –all-databases –no-data > test.dump
jlch¶
使用 mysqldump.sh 吧
[jlch@check mariadb.dump]$ cat mysqldump.sh
#!/bin/bash
mysqldump -h $1 -u root -p --all-databases --no-data > $1.mariadb.dump
[jlch@check mariadb.dump]$
必须本机完成¶
10.10.13.100, 密码1
跑起来
[jlch@mysql1 ~]$ mysqldump -u root -p --all-databases --no-data > 10.10.13.100.mariadb.dump
Enter password:
[jlch@mysql1 ~]$
可在 jlch@192.168.31.181 完成¶
192.168.31.181.mariadb.dump, 密码1
192.168.31.240.mariadb.dump, 密码1
192.168.31.11.mariadb.dump, 密码1
10.10.12.13.mariadb.dump, 密码1
192.168.31.99.mariadb.dump, 密码2
192.168.31.249.mariadb.dump, 密码2
192.168.31.105.mariadb.dump, 密码2
192.168.31.201.mariadb.dump, 密码2
跑起来
[jlch@check ~]$ mysqldump -h 10.10.12.13 -u fish -p --all-databases --no-data > 10.10.12.13.mariadb.dump
Enter password:
[jlch@check ~]$
optimize-a-mysqldump-of-a-large-database¶
https://dba.stackexchange.com/questions/20/how-can-i-optimize-a-mysqldump-of-a-large-database
mysqldump -h… -u… -p… –hex-blob –routines –triggers –all-databases | gzip > MySQLData.sql.gz
mysql FAQ¶
Contents:
mysql could not be resolved: Name or service not known¶
开发人员说,数据库连接不上…
第一时间, tail -f 打开日志, 发现
错误日志有类似警告：
1.120119 16:26:04 [Warning] IP address '192.168.1.10' could not be resolved: Name or service not known
2.120119 16:26:04 [Warning] IP address '192.168.1.14' could not be resolved: Name or service not known
3.120119 16:26:04 [Warning] IP address '192.168.1.17' could not be resolved: Name or service not known
通过show processlist发现大量类似如下的连接：
1.|592|unauthenticated user|192.168.1.10:35320|NULL|Connect| |login|NULL|
2.|593|unauthenticated user|192.168.1.14:35321|NULL|Connect| |login|NULL|
3.|594|unauthenticated user|192.168.1.17:35322|NULL|Connect| |login|NULL|
先是参考 http://www.jb51.net/article/70893.htm
skip-name-resolve 参数的作用：不再进行反解析（ip不反解成域名），这样可以加快数据库的反应时间
[mysqld]
skip-name-resolve
设置后, 重启, 没有生效.
然后查看系统
top
发现, 负载很高呀…
回忆了一下, 是刚刚之前, 做了一个大数据的 mv 操作.
然后, 我把这个进程 kill 后, 再重启数据库, 就一切正常了.
我的天啊, 原来, 数据库这台机器, 的负载不能太高呀…..
mysql-takes-too-long-on-checking-permissions¶
https://dba.stackexchange.com/questions/66215/mysql-takes-too-long-on-checking-permissions https://serverfault.com/questions/233963/mysql-checking-permission-takes-a-long-time
主要出这个错误，是
mysql config¶
Contents:
181 config¶
[root@check my.cnf.d]# cat /etc/my.cnf.d/server.cnf
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
datadir=/data/mariadb/mysql
socket=/var/lib/mysql/mysql.sock
#default-character-set=utf8
character_set_server=utf8
log-bin=mysql.server
slow_query_log=on
slow_query_log_file=/data/mariadb/mysql/slow_query_log.log
long_query_time=2
event_scheduler=on
max_connections = 1000
max_allowed_packet = 8M
thread_cache_size=64
log_slow_admin_statements=1
log_slow_slave_statements=1
sort_buffer_size = 80M
read_buffer_size = 80M
join_buffer_size = 80M
max_length_for_sort_data=9196
tmp_table_size = 57108864
innodb_log_file_size= 300M
innodb_log_files_in_group=6
innodb_log_buffer_size=450M
innodb_write_io_threads = 8
innodb_read_io_threads = 8
innodb_buffer_pool_size=4G
innodb_buffer_pool_instances=6
log_bin_trust_function_creators = 1
# this is only for embedded server
[embedded]
# This group is only read by MariaDB-5.5 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mysqld-5.5]
# These two groups are only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
[mariadb-5.5]
[root@check my.cnf.d]#
p1¶
innodb_buffer_pool_size=4G
表示占用内存4G, 181这台机器内存共16G. 如果这个地方设置成 8G, 则会真实使用到 12.8G(16*0.8) 以上，导致监控报警。
p2¶
innodb_log_file_size= 300M
innodb_log_files_in_group=6
表示，日志文件大小，如果这个太小了，会导致mysql 写入速度变慢。 默认是 2组，每组 5M。太小了。 现在就是 6组，每组 300M。
在如果是第一次配置，请按下面进行：
1. stop mysql
2. 将 mysql 数据目录(/data/mariadb/mysql/)下的 ib_logfile* 文件转移出去。
3. 修改 server.cnf 文件将 配置加入
4. start mysql
p3¶
log_bin_trust_function_creators = 1
这个参数相当于： 在你的会话级运行一下这个语句：SET GLOBAL log_bin_trust_function_creators = 1;
[jlch@check ~]$ mysql -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 5482
Server version: 5.5.44-MariaDB-log MariaDB Server
Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> SET GLOBAL log_bin_trust_function_creators = 1;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> show variables like '%log_bin_trust_function_creators%';
+---------------------------------+-------+
| Variable_name | Value |
+---------------------------------+-------+
| log_bin_trust_function_creators | ON |
+---------------------------------+-------+
1 row in set (0.00 sec)
MariaDB [(none)]> exit
Bye
[jlch@check ~]$
好了，看到上面是 on 了。哈哈。
p4¶
log-bin¶
配置 log-bin¶
在 数据库 中配置添加 log-bin=mysql.server 完成后 ，重启，
[root@check my.cnf.d]# ls /data/mariadb/mysql/
aria_log.00000001 cnpmjs_test ibdata1 ib_logfile1 ib_logfile3 ib_logfile5 openquote product q_station slow_query_log.log stock20151029 stock_sqlserver_df sync testhufei zabbix
aria_log_control haitong ib_logfile0 ib_logfile2 ib_logfile4 mysql performance_schema program rates stock stock_quotation stock_test test xiaozhao zhongtai2
[root@check my.cnf.d]# sudo systemctl start mariadb.service
[root@check my.cnf.d]# sudo systemctl status mariadb.service
● mariadb.service - MariaDB database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
Active: active (running) since 二 2017-10-31 11:48:11 CST; 5s ago
Process: 15315 ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID (code=exited, status=0/SUCCESS)
Process: 15269 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n (code=exited, status=0/SUCCESS)
Main PID: 15314 (mysqld_safe)
Memory: 854.0M
CGroup: /system.slice/mariadb.service
├─15314 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
└─15825 /usr/libexec/mysqld --basedir=/usr --datadir=/data/mariadb/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mariadb/mariadb.log --pid-file=/var/run/mariadb/mariadb.pid --socket=/var/lib/mysql/mysql.sock
10月 31 11:48:08 check systemd[1]: Starting MariaDB database server...
10月 31 11:48:08 check mysqld_safe[15314]: 171031 11:48:08 mysqld_safe Logging to '/var/log/mariadb/mariadb.log'.
10月 31 11:48:08 check mysqld_safe[15314]: 171031 11:48:08 mysqld_safe Starting mysqld daemon with databases from /data/mariadb/mysql
10月 31 11:48:11 check systemd[1]: Started MariaDB database server.
[root@check my.cnf.d]# ls /data/mariadb/mysql/
aria_log.00000001 cnpmjs_test ibdata1 ib_logfile1 ib_logfile3 ib_logfile5 mysql.000001 openquote product q_station slow_query_log.log stock20151029 stock_sqlserver_df sync testhufei zabbix
aria_log_control haitong ib_logfile0 ib_logfile2 ib_logfile4 mysql mysql.index performance_schema program rates stock stock_quotation stock_test test xiaozhao zhongtai2
[root@check my.cnf.d]#
如上，在 mysql 文件（/data/mariadb/mysql/）中，会新增 mysql.index， mysql.000001 这2个文件。
TechEmpower-my.cnf¶
TechEmpower 下推荐的一个配置, 可以做一个参考
mysql use¶
mysql -h 192.168.31.181 -u root -p stock -e "select * from baidu_20_quar ;" -N -s | sed -e 's/^/"/g;s/$/"\n/g' > ./baidu_20_quar.csv
## ok
mysql -h 192.168.31.181 -u root -p stock -e "select * from baidu_20_quar ;" | sed 's/\t/","/g;s/^/"/;s/$/"/;s/\n//g' > ./baidu_20_quar.csv
mysql -h 192.168.31.181 -u root -p stock -ss -e "select * from baidu_20_quar ;" | sed 's/\t/","/g;s/^/"/;s/$/"/;s/\n//g' > ./2.csv
mysql -h 192.168.31.181 -u root -p stock -ss -e "select * from baidu_20_quar ;" > ./3.csv
##
mysql -h 192.168.31.181 -u root -p stock --execute='SELECT `FIELD`, `FIELD` FROM `TABLE` LIMIT 0, 10000 ' -X > file.csv
##
mysqldump -h 192.168.31.181 -u root -p -t -T /home/tom/draft/mysql-20170619 stock baidu_20_quar --fields-terminated-by=','
mysqldump -h 192.168.31.181 -u root -p -T/path/to/directory dbname table_name --fields-terminated-by=','
##
SELECT * FROM mytable INTO OUTFILE '/tmp/mytable.csv' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n';
mysqldump -h 192.168.31.181 -u root -p stock czs_dxg --tab=/home/tom/draft2/ --fields-terminated-by=','
select * from table into outfile '/tmp/table.csv' fields terminated by ',' optionally enclosed by '"' lines terminated by '\r\n';
##
mysqldump -u root -p haitong factor_info | mysql -h 192.168.31.240 test
mysqldump -u root -p haitong factor_info | mysql -h 192.168.31.240 -u root -p test
mysqldump -u root -p -T/tmp/ test factor_info --fields-terminated-by=','
查 hq_online_cjmx库 中表的个数
SELECT count(TABLE_NAME) FROM information_schema.TABLES WHERE TABLE_SCHEMA='hq_online_cjmx';
Meituan-SQLadvisor¶
git clone https://github.com/Meituan-Dianping/SQLadvisor.git
mongodb¶
Contents:
mongoexport¶
Contents:
mongoexport¶
[tom@check memi]$ cat field_KLine_1Min_b.txt
_id
Type
Sign
ID
TrdDt
TrdTm
PrevClsPx
LastPx
TM
[tom@check memi]$ cat signals_1min_to_csv.sh
#!/bin/bash
# mongoexport -h 192.168.31.240 -d openQuoteTmp -c Signals_1Min -q "{TrdDt:20171101}" --fieldFile=field_KLine_1Min_b.txt --type=csv -o Signals_1Min_csv/20171101.csv
for dt in `cat dt_signals_1min.txt`
do
echo "$dt"
mongoexport -h 192.168.31.240 -d openQuoteTmp -c Signals_1Min -q "{TrdDt:$dt}" --fieldFile=field_KLine_1Min_b.txt --type=csv -o Signals_1Min_csv/$dt.csv
done
echo "game over"
[tom@check memi]$
draft¶
[tom@check memi]$ mongoexport -h 192.168.31.240 -d openQuote20171106 -c Signals_1Min -q "{TrdDt:20171107}" --fieldFile=field_KLine_1Min_b.txt --type=csv -o Signals_1Min_csv/20171107.csv
2017-11-07T15:29:15.446+0800 connected to: 192.168.31.240
2017-11-07T15:29:19.423+0800 exported 92421 records
[tom@check memi]$ mongoexport -h 192.168.31.240 -d openQuote20171106 -c Signals_1Min -q "{TrdDt:20171106}" --fieldFile=field_KLine_1Min_b.txt --type=csv -o Signals_1Min_csv/20171106.csv
2017-11-07T15:29:39.203+0800 connected to: 192.168.31.240
2017-11-07T15:29:42.738+0800 exported 82241 records
[tom@check memi]$
如果说要转移到共享盘，则
sudo cp crontab/memi/Signals_1Min_csv/20171107.csv /misc/share/Y-运维组/data/tmp/Signals_1Min_csv/
mongodb¶
Contents:
mongodb gui resource¶
studio-3t
相比于 robo-3t, 优先使用 studio-3t, 因为, studio-3t 收购了 robo-3t
robo-3t
这个就是原来的 robomongo
MongoDB的客户端robo3t在，Ubuntu 16.04中启动时报一个QT的错误¶
ref¶
https://www.cnblogs.com/hexin0614/p/7487191.html
原帖路径：https://askubuntu.com/questions/941383/cannot-run-robo-3t-qt-app
env¶
【问题】
MongoDB的客户端robo3t在，Ubuntu 16.04中启动时报一个QT的错误：
复制代码 1 This application failed to start because it could not find or load the Qt platform plugin “xcb” 2 in “”. 3 4 Available platform plugins are: xcb. 5 6 Reinstalling the application may fix this problem. 7 Aborted (core dumped) 复制代码
step¶
【解决方法】
https://www.cnblogs.com/hexin0614/p/7487191.html
在参考这个URL之前,我参考了 https://www.zhihu.com/question/27387909 , 运行了
sudo apt-get install qt5-default qtcreator
但是我觉得应该与这条语句无关..所以如果有机会再次遇到这个问题, 先不执行上面这句, 然后看一下效果…
1 mkdir ~/robo-backup 2 mv robo3t-1.1.1-linux-x86_64-c93c6b0/lib/libstdc++* ~/robo-backup/ 3 robo3t-1.1.1-linux-x86_64-c93c6b0/bin/robo3t
果然是成功了….
html¶
Contents:
npm-run¶
Contents:
npm run build 出错情况1¶
no such file or directory¶
[jlch@check ktraining]$ npm run build
> line@1.0.0 build /home/jlch/registry/ktraining
> node build/build.js
⠸ building for production...
Starting to optimize CSS...
Processing static/css/app.d0d16c3b538edb8384cf55d5f92bcddd.css...
Processed static/css/app.d0d16c3b538edb8384cf55d5f92bcddd.css, before: 74052, after: 71485, ratio: 96.53%
Hash: 004d38f2752eb4dd3ca6
Version: webpack 2.7.0
Time: 56052ms
Asset Size Chunks Chunk Names
static/iconfont.js 38.9 kB [emitted]
static/img/bg.54503c9.png 99.2 kB [emitted]
static/js/app.1302bf22af6be784678a.js 69.7 kB 1 [emitted] app
static/js/manifest.83d402e7507a68529b24.js 1.51 kB 2 [emitted] manifest
static/css/app.d0d16c3b538edb8384cf55d5f92bcddd.css 71.5 kB 1 [emitted] app
static/js/vendor.4e94ca5aec8a4e524a18.js.map 7.98 MB 0 [emitted] vendor
static/js/app.1302bf22af6be784678a.js.map 393 kB 1 [emitted] app
static/css/app.d0d16c3b538edb8384cf55d5f92bcddd.css.map 92.3 kB 1 [emitted] app
static/js/manifest.83d402e7507a68529b24.js.map 14.6 kB 2 [emitted] manifest
index.html 820 bytes [emitted]
static/address.js 243 bytes [emitted]
static/flexible.js 3.49 kB [emitted]
static/js/vendor.4e94ca5aec8a4e524a18.js 924 kB 0 [emitted] [big] vendor
static/css/clearAd.css 109 bytes [emitted]
static/img/fsRank.png 2.82 kB [emitted]
static/img/bg.png 99.2 kB [emitted]
static/img/kline.png 19.2 kB [emitted]
static/img/kLine.png 3.75 kB [emitted]
static/img/kRank.png 2.42 kB [emitted]
static/img/records.png 3.25 kB [emitted]
static/img/record.png 19.2 kB [emitted]
static/img/share.png 19.7 kB [emitted]
static/img/timeShare.png 3.12 kB [emitted]
static/users.js 44 bytes [emitted]
ERROR in ./~/_css-loader@0.28.7@css-loader?{"minimize":true,"sourceMap":true}!./~/_sass-loader@6.0.6@sass-loader/lib/loader.js?{"sourceMap":true}!./src/css/base.scss
Module build failed: Error: ENOENT: no such file or directory, scandir '/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/vendor'
at Object.fs.readdirSync (fs.js:862:18)
at Object.getInstalledBinaries (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/extensions.js:124:13)
at foundBinariesList (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/errors.js:20:15)
at foundBinaries (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/errors.js:15:5)
at Object.module.exports.missingBinary (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/errors.js:45:5)
at module.exports (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/binding.js:15:30)
at Object.<anonymous> (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/index.js:14:35)
at Module._compile (module.js:573:32)
at Object.Module._extensions..js (module.js:582:10)
at Module.load (module.js:490:32)
at tryModuleLoad (module.js:449:12)
at Function.Module._load (module.js:441:3)
at Module.require (module.js:500:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/home/jlch/registry/ktraining/node_modules/_sass-loader@6.0.6@sass-loader/lib/loader.js:3:14)
at Module._compile (module.js:573:32)
@ ./src/css/base.scss 4:14-180
@ ./src/main.js
ERROR in ./src/css/base.scss
Module build failed: ModuleBuildError: Module build failed: Error: ENOENT: no such file or directory, scandir '/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/vendor'
at Object.fs.readdirSync (fs.js:862:18)
at Object.getInstalledBinaries (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/extensions.js:124:13)
at foundBinariesList (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/errors.js:20:15)
at foundBinaries (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/errors.js:15:5)
at Object.module.exports.missingBinary (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/errors.js:45:5)
at module.exports (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/binding.js:15:30)
at Object.<anonymous> (/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/lib/index.js:14:35)
at Module._compile (module.js:573:32)
at Object.Module._extensions..js (module.js:582:10)
at Module.load (module.js:490:32)
at tryModuleLoad (module.js:449:12)
at Function.Module._load (module.js:441:3)
at Module.require (module.js:500:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/home/jlch/registry/ktraining/node_modules/_sass-loader@6.0.6@sass-loader/lib/loader.js:3:14)
at Module._compile (module.js:573:32)
at runLoaders (/home/jlch/registry/ktraining/node_modules/_webpack@2.7.0@webpack/lib/NormalModule.js:192:19)
at /home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/LoaderRunner.js:364:11
at /home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/LoaderRunner.js:170:18
at loadLoader (/home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/loadLoader.js:27:11)
at iteratePitchingLoaders (/home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/LoaderRunner.js:169:2)
at iteratePitchingLoaders (/home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/LoaderRunner.js:165:10)
at /home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/LoaderRunner.js:173:18
at loadLoader (/home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/loadLoader.js:36:3)
at iteratePitchingLoaders (/home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/LoaderRunner.js:169:2)
at runLoaders (/home/jlch/registry/ktraining/node_modules/_loader-runner@2.3.0@loader-runner/lib/LoaderRunner.js:362:2)
at NormalModule.doBuild (/home/jlch/registry/ktraining/node_modules/_webpack@2.7.0@webpack/lib/NormalModule.js:179:3)
at NormalModule.build (/home/jlch/registry/ktraining/node_modules/_webpack@2.7.0@webpack/lib/NormalModule.js:268:15)
at Compilation.buildModule (/home/jlch/registry/ktraining/node_modules/_webpack@2.7.0@webpack/lib/Compilation.js:146:10)
at moduleFactory.create (/home/jlch/registry/ktraining/node_modules/_webpack@2.7.0@webpack/lib/Compilation.js:433:9)
at factory (/home/jlch/registry/ktraining/node_modules/_webpack@2.7.0@webpack/lib/NormalModuleFactory.js:253:5)
at applyPluginsAsyncWaterfall (/home/jlch/registry/ktraining/node_modules/_webpack@2.7.0@webpack/lib/NormalModuleFactory.js:99:14)
Build complete.
Tip: built files are meant to be served over an HTTP server.
Opening index.html over file:// won't work.
[jlch@check ktraining]$
解决
这个看出来是 Error: ENOENT: no such file or directory, scandir ‘/home/jlch/registry/ktraining/node_modules/_node-sass@4.5.3@node-sass/vendor’, node-sass 少了 vendor, 那就是说，我们要把这个 node-sass 安装完整才行。
那就来吧
cd node_modules/
ls node-sass/
cd node-sass/
ls
npm test ## 这里先做一个 测试，应该来说，此时是会报错的。
npm install
npm test ## 好了，安装成功后，此时不应该报错。
cd ../../../ktraining/ ## 回头重新 build 吧。
npm run build
这下子成功啦。
log¶
Contents:
logstash¶
Contents:
getting-started-with-logstash¶
http://blog.csdn.net/wp500/article/details/41040213
http://logstash.net/docs/1.4.2/tutorials/getting-started-with-logstash
env¶
192.168.31.149(192.168.31.193)
logstash plugin configuration¶
Inputs¶
input 及输入是指日志数据传输到Logstash中。其中常见的配置如下：
file：从文件系统中读取一个文件，很像UNIX命令 “tail -0a” syslog：监听514端口，按照RFC3164标准解析日志数据 redis：从redis服务器读取数据，支持channel(发布订阅)和list模式。redis一般在Logstash消费集群中作为”broker”角色，保存events队列共Logstash消费。 lumberjack：使用lumberjack协议来接收数据，目前已经改为 logstash-forwarder。 tcp:
Filters¶
Fillters 在Logstash处理链中担任中间处理组件。他们经常被组合起来实现一些特定的行为来，处理匹配特定规则的事件流。常见的filters如下：
grok：解析无规则的文字并转化为有结构的格式。Grok 是目前最好的方式来将无结构的数据转换为有结构可查询的数据。有120多种匹配规则，会有一种满足你的需要。 mutate：mutate filter 允许改变输入的文档，你可以从命名，删除，移动或者修改字段在处理事件的过程中。
gsub: 仅对字条串类型字段有效 split: 分割 join: 粘合 merge: 合并 rename: 重命名 update: 更新 replace: 更新或添加 filter/mutate 内部是有执行次序的.drop：丢弃一部分events不进行处理，例如：debug events。 clone：拷贝 event，这个过程中也可以添加或移除字段。 geoip：添加地理信息(为前台kibana图形化展示使用) date: 转换日志记录中的时间字符串 dissect: 解析简单规则的文字. json: 处理json格式 kv: 处理key-value格式日志 metrics: 数值统计. meter(速率阀值检测), timer(box and whisker 异常检测) ruby: 随心所欲的Ruby处理 split: 拆分事件 elapsed:
Outputs¶
outputs是logstash处理管道的最末端组件。一个event可以在处理过程中经过多重输出，但是一旦所有的outputs都执行结束，这个event也就完成生命周期。一些常用的outputs包括：
- elasticsearch：如果你计划将高效的保存数据，并且能够方便和简单的进行查询…Elasticsearch是一个好的方式。是的，此处有做广告的嫌疑,呵呵。
- flush_size: 事件间隔条数,默认500条 idle_flush_time: idle flush 间隔时间, 默认1秒 以上2个参数, 只要满足一个条件就促发.
- file：将event数据保存到文件中。
- 在 file 下使用 codec, codec 设置无效. gzip: 压缩
email: 发送邮件. exec: 调用命令执行. 比如, 发短信. nagios: 发送数据给本机的 nagios.cmd 管道命令文件 nagios_nsca: 调用 send_nsca 命令以NSCA协议格式把数据发送给 nagios 服务器(远端或本地皆可) zabbix: 调用本机的 zabbix_sender 命令发送 ganglia: 通过 UDP 协议, 发送gmetric型数据给本机/远端的 gmond 或者 gmetad graphite：将event数据发送到图形化组件中，一个很流行的开源存储图形化展示的组件。http://graphite.wikidot.com/。 statsd：statsd是一个统计服务，比如技术和时间统计，通过udp通讯，聚合一个或者多个后台服务，如果你已经开始使用statsd，该选项对你应该很有用. stdout: 最基础和最简单的输出插件.
codec: 一般可以省略掉后面的配置区段 workers: 多少个线程 加上命令行参数 -vv 运行, 查看更多详细调试信息.tcp: 发送网络数据. 不建议使用. hadoop_webhdfs: Post data to WebHDFS. hdfs: it import java classes like “org.apache.hadoop.fs.FileSystem” etc. redis: 输出到redis juggernaut: 基于redis服务器和socker.io框架构建的, 利用它, logstash 可以直接向 webkit等支持 socket.io 的浏览器推送告警信息.
Codecs¶
codecs 是基于数据流的过滤器，它可以作为input，output的一部分配置。Codecs可以帮助你轻松的分割发送过来已经被序列化的数据。流行的codecs包括 json,msgpack,plain(text)。
json：使用json格式对数据进行编码/解码 multiline：将汇多个事件中数据汇总为一个单一的行。比如：java异常信息和堆栈信息 log4j: 应用日志 collectd: 收集系统性能, 在系统运行和存储信息时周期性的统计系统的相关统计信息 netflow:
获取完整的配置信息，请参考 Logstash文档中 “plugin configuration”部分。
rudiments-logstash-elasticsearch-kibana¶
【入门篇】logstash、elasticsearch、kibana搭建日志平台
http://blog.csdn.net/pistolove/article/details/53634433
安装elasticsearch的时候, 感觉好烦, 配置来配置去的…直接用 docker 了..
sudo docker pull logstash:5.6.5
sudo docker pull elasticsearch:5.6.5
sudo docker pull kibana:5.6.5
启动elasticsearch
sudo docker run -d -p 9200:9200 --name="es" elasticsearch:5.6.5
启动kibana
[jlch@udvbct-ekl-a elasticsearch-6.1.1]$ sudo docker run -d -p 5601:5601 --name="kibana" kibana:5.6.5
[sudo] password for jlch:
14a4381dbfcda13a232d8f066d3af394c0cdb486e8bf93276e9b2b7e6721ca8d
[jlch@udvbct-ekl-a elasticsearch-6.1.1]$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
14a4381dbfcd kibana:5.6.5 "/docker-entrypoint.…" 6 seconds ago Up 3 seconds 0.0.0.0:5601->5601/tcp kibana
5cfd6dac25c2 elasticsearch:5.6.5 "/docker-entrypoint.…" 20 minutes ago Up 20 minutes 0.0.0.0:9200->9200/tcp, 9300/tcp es
开放端口
[jlch@udvbct-ekl-a elasticsearch-6.1.1]$ sudo firewall-cmd --permanent --add-port=9200/tcp
success
[jlch@udvbct-ekl-a elasticsearch-6.1.1]$ sudo firewall-cmd --permanent --add-port=5601/tcp
success
[jlch@udvbct-ekl-a elasticsearch-6.1.1]$ sudo firewall-cmd --reload
success
[jlch@udvbct-ekl-a elasticsearch-6.1.1]$
这样浏览器中打开, 就可以看到了.
beats¶
Contents:
Beats 基础¶
http://soft.dog/2015/12/24/beats-basic/#section
env¶
192.168.31.149(192.168.31.193) centos7
filebeat¶
Contents:
logstash 和filebeat 是什么关系¶
https://www.zhihu.com/question/54058964
logstash 和filebeat都具有日志收集功能，filebeat更轻量，占用资源更少，但logstash 具有filter功能，能过滤分析日志。一般结构都是filebeat采集日志，然后发送到消息队列，redis，kafaka。然后logstash去获取，利用filter功能过滤分析，然后存储到elasticsearch中
Filebeat中文指南¶
https://www.cnblogs.com/kerwinC/p/6227768.html
Filebeat中文指南 翻译自：https://www.elastic.co/guide/en/beats/filebeat/current/index.html
power by kerwin
elasticsearch¶
Contents:
全文搜索引擎 Elasticsearch 入门教程¶
env¶
192.168.31.149(192.168.31.193) centos7
step¶
[jlch@udvbct-ekl-a elasticsearch-5.6.5]$ curl -X GET 'http://localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
[jlch@udvbct-ekl-a elasticsearch-5.6.5]$ curl 'localhost:9200/_mapping?pretty=true'
{ }
[jlch@udvbct-ekl-a elasticsearch-5.6.5]$ curl -X PUT 'localhost:9200/weather'
{"acknowledged":true,"shards_acknowledged":true,"index":"weather"}
[jlch@udvbct-ekl-a elasticsearch-5.6.5]$ curl -X GET 'http://localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open weather -58xfoleTr6sI58ZPZzLKA 5 1 0 0 324b 324b
[jlch@udvbct-ekl-a elasticsearch-5.6.5]$ curl -X DELETE 'localhost:9200/weather'
{"acknowledged":true}
[jlch@udvbct-ekl-a elasticsearch-5.6.5]$ curl -X GET 'http://localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
[jlch@udvbct-ekl-a elasticsearch-5.6.5]$
有一个思维导图,见:
http://naotu.baidu.com/file/e7feaf5c7ab075150dcb22fb3977128d
Elasticsearch 基础入门¶
env¶
elasticsearch: 192.168.31.149(192.168.31.193) centos7
step¶
2.5 安装插件
修改 Elasticsearch 的配置文件：
cd elasticsearch-5.6.1
vim config/elasticsearch.yml
# 在文件末尾添加 2 段配置
http.cors.enabled: true
http.cors.allow-origin: "*"
安装插件 mobz/elasticsearch-head
这里我是去 官网, https://github.com/mobz/elasticsearch-head, 用docker运行的, 很方便
sudo docker run -p 9100:9100 mobz/elasticsearch-head:5
运行完成后, 在浏览器输入
然后在 elasticsearch 集群输入框中, 输入elasticsearch 集群IP地址:
马上就有了内容, 很方便..
三、使用
准备 postman 工具
http://blog.csdn.net/water_0815/article/details/53263643
使用 POSTMAN 工具进行接口的请求。
draft, 期间用到的一些 json 文字¶
{
"settings": {
"index": {
"number_of_shards": "3",
"number_of_replicas": "1"
}
},
"mappings": {
"apple": {
"properties": {
"color": {
"type": "text"
},
"createTime": {
"format": "yyyy-MM-dd HH:mm:ss || yyyy-MM-dd || epoch_millis",
"type": "date"
},
"weight": {
"type": "integer"
}
}
},
"orange": {}
}
}
{
"color": "red",
"createTime":"2017-12-27",
"weight": 2
}
{"doc":{"color":"black"}}
{
"query": {
"match":{
"color": "green"
}
}
}
Elasticsearch plugin¶
mobz/elasticsearch-head
kibana¶
Contents:
Kibana 基础入门¶
env¶
192.168.31.149(192.168.31.193) centos7
step¶
在操作前, 必须先把 rudiments-elasticsearch.rst 的内容先走一遍.
2.2 修改配置文件
vim config/kibana.yml
# 将默认配置改成如下：
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://192.168.31.149:9200"
kibana.index: ".kibana"
2.3 启动
bin/kibana 启动后打开浏览器访问 http://192.168.2.43:5601 浏览 kibana 界面：
三、演示 上图中，提示不能获取映射，即 Elasticsearch 中的索引。我们需要手动配置。在 Index Pattern 下边的输入框中输入 access-*(好了,因为我们之前的index是 accounts, 所以这里就写 accounts)，它是 Elasticsearch 中的一个索引名称开头。
Kibana 会自动检测在 Elasticsearch 中是否存在该索引名称，如果有，则下边出现 “Create” 按钮，我们点击进行创建并来到如下界面：
“Discovery” 菜单界面主要用于通过搜索请求，过滤结果，查看文档数据。可以查询搜索请求的文档总数，获取字段值的统计情况并通过柱状图进行展示。
点击左侧 “Discovery” 菜单，来到如下界面：
nfs¶
Contents:
nfs-server¶
Contents:
nfs-erver.install.ubuntu16.04¶
Ubuntu 16.04安装NFS server
环境¶
192.168.31.232 nfs-server
192.168.31.197 nfs-client, 测试
step¶
安装包
sudo apt-get -y install nfs-kernel-server
sudo apt-get -y install nfs-common
设置存储位置
mkdir -p /opt/share
chmod -R 777 /opt/share
配置 /etc/exports
root@ubuntu:/opt/share# vi /etc/exports
root@ubuntu:/opt/share# cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
/opt/share *(rw,no_root_squash)
root@ubuntu:/opt/share#
重启相关服务
sudo service rpcbind restart
sudo service portmap restart
sudo service nfs-kernel-server restart
到 nfs-client 机器中，创建 a.txt , 然后回来
cd /opt/share/
ls
cat a.txt
如果有数据，那就是OK了。
over¶
nfs-client¶
echo “deb http://ftp.cn.debian.org/debian jessie main” >> sources.list
apt update
apt-get install nfs-common
ping 192.168.31.232
network-security¶
Contents:
network-security resource¶
http://www.owasp.org.cn/owasp-project/OWASPTop102017v1.02.pdf
ceph¶
Contents:
cephfs¶
Contents:
cephfs¶
环境：¶
192.168.31.115 192.168.31.114 192.168.31.113
line1 官网¶
我的安装设计是这样的：
admin-node, deploy-node(ceph-deploy)：192.168.31.115 cephfs5
mon.0: 192.168.31.114 cephfs4
osd.0: 192.168.31.113 cephfs3
osd.1: 192.168.31.115 cephfs5
mds.0: 192.168.31.113 cephfs3
mds.1: 192.168.31.114 cephfs4
over¶
ceph auth¶
cephu@cephadmin:~/my-cluster$ ceph auth ls
installed auth entries:
mds.node1
key: AQADpu5ZJgEAAhAA6Om+UrlNn3s4v728UGIuTQ==
caps: [mds] allow
caps: [mon] allow profile mds
caps: [osd] allow rwx
osd.0
key: AQCAou5ZbAFBCRAANHUYj3trYJm+bbpzM68Czw==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.1
key: AQCNou5Z0AEiChAAoXaQS1KsqnygVzIdK4LeBQ==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.2
key: AQCaou5ZckceFxAArxSTMrgL94Pe9wGqhT2tGA==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
client.admin
key: AQCtj+5ZnNTvGRAA2RxAGcIQZJnaJSPEz4jdGw==
caps: [mds] allow *
caps: [mgr] allow *
caps: [mon] allow *
caps: [osd] allow *
client.bootstrap-mds
key: AQCuj+5ZL4meKBAAQ/AJ/63mw4XD3HIcx1+ojw==
caps: [mon] allow profile bootstrap-mds
client.bootstrap-mgr
key: AQDYne5Zf3L3BxAAt/PZfcXS1b0ioDrgZX8v8w==
caps: [mon] allow profile bootstrap-mgr
client.bootstrap-osd
key: AQCtj+5ZEe+yMhAAX8vyaP2g7cBznN5dk48mBQ==
caps: [mon] allow profile bootstrap-osd
client.bootstrap-rbd
key: AQCvj+5Zpj97BRAAeBH22SI5YQ8+iI4+V+Kopg==
caps: [mon] allow profile bootstrap-rbd
client.bootstrap-rgw
key: AQCuj+5ZRqQKEBAABXu387mLGQGPwkvcVBhPyg==
caps: [mon] allow profile bootstrap-rgw
client.kubernetes-dynamic-user-02edf53a-bec3-11e7-bda7-000c299a346f
key: AQBTV/lZenwMKhAAFI7qG14jcFj6kScyg65DsQ==
caps: [mds] allow r
caps: [mon] allow r
caps: [osd]
client.kubernetes-dynamic-user-34d65a6c-bed2-11e7-bda7-000c299a346f
key: AQDRcPlZ9MBSNBAAsDbLR8rEWslQmq9fhcsZrw==
caps: [mds] allow r,allow rw path=/volumes/kubernetes/kubernetes-dynamic-pvc-34d659ff-bed2-11e7-bda7-000c299a346f
caps: [mon] allow r
caps: [osd] allow rw pool=cephfs_data namespace=fsvolumens_kubernetes-dynamic-pvc-34d659ff-bed2-11e7-bda7-000c299a346f
client.kubernetes-dynamic-user-34fb64a9-be15-11e7-8636-000c299a346f
key: AQC7M/hZxmS/DhAAZR/ybQ7WSBEvmF3ZYMarRw==
caps: [mds] allow r
caps: [mon] allow r
caps: [osd]
client.kubernetes-dynamic-user-612fd2ba-be13-11e7-8636-000c299a346f
key: AQCqMPhZ0fWTFxAAN5nMfYyNHjMqVamVjmQsCA==
caps: [mds] allow r
caps: [mon] allow r
caps: [osd]
client.rgw.ceph-client
key: AQDoM/BZiyLcHBAASXop8G7HJPJQfVYDrA5BkQ==
caps: [mon] allow rw
caps: [osd] allow rwx
client.rgw.node1
key: AQCGr+5ZQDQRLRAAw9WB8JbSUSeUkJ7k7+cCAg==
caps: [mon] allow rw
caps: [osd] allow rwx
mgr.mon1
key: AQDvne5ZbU0JOhAAotdp7L9xiVaL54awr26lcA==
caps: [mds] allow *
caps: [mon] allow profile mgr
caps: [osd] allow *
mgr.node2
key: AQBhqO5ZEy2gMxAAfG496CAtCeVdApXfffwf+Q==
caps: [mds] allow *
caps: [mon] allow profile mgr
caps: [osd] allow *
mgr.node3
key: AQBjqO5ZM0l9HBAALhWmqYNbpOFUvPfHv4aOug==
caps: [mds] allow *
caps: [mon] allow profile mgr
caps: [osd] allow *
cephu@cephadmin:~/my-cluster$ ceph auth get client.kubernetes-dynamic-user-02edf53a-bec3-11e7-bda7-000c299a346f
exported keyring for client.kubernetes-dynamic-user-02edf53a-bec3-11e7-bda7-000c299a346f
[client.kubernetes-dynamic-user-02edf53a-bec3-11e7-bda7-000c299a346f]
key = AQBTV/lZenwMKhAAFI7qG14jcFj6kScyg65DsQ==
caps mds = "allow r"
caps mon = "allow r"
caps osd = ""
cephu@cephadmin:~/my-cluster$ ceph auth ls | grep kubernetes-dynamic-pvc-34d659ff-bed2-11e7-bda7-000c299a346f
installed auth entries:
caps: [mds] allow r,allow rw path=/volumes/kubernetes/kubernetes-dynamic-pvc-34d659ff-bed2-11e7-bda7-000c299a346f
caps: [osd] allow rw pool=cephfs_data namespace=fsvolumens_kubernetes-dynamic-pvc-34d659ff-bed2-11e7-bda7-000c299a346f
cephu@cephadmin:~/my-cluster$
ceph install¶
Contents:
ceph install by docker¶
install FAQ¶
Q1¶
ceph-deploy install ceph-admin ceph-osd1 ceph-osd2 mon1
...
[ceph-osd1][DEBUG ] Get:40 https://download.ceph.com/debian-jewel xenial/main amd64 python-rbd amd64 10.2.10-1xenial [1,449 kB]
[ceph-osd1][DEBUG ] Get:41 https://download.ceph.com/debian-jewel xenial/main amd64 ceph-common amd64 10.2.10-1xenial [15.5 MB]
[ceph-osd1][DEBUG ] Get:42 https://download.ceph.com/debian-jewel xenial/main amd64 ceph-base amd64 10.2.10-1xenial [50.5 MB]
[ceph-osd1][WARNIN] No data was received after 300 seconds, disconnecting...
[ceph-osd1][INFO ] Running command: sudo ceph --version
[ceph-osd1][ERROR ] Traceback (most recent call last):
[ceph-osd1][ERROR ] File "/usr/lib/python2.7/dist-packages/ceph_deploy/lib/vendor/remoto/process.py", line 119, in run
[ceph-osd1][ERROR ] reporting(conn, result, timeout)
[ceph-osd1][ERROR ] File "/usr/lib/python2.7/dist-packages/ceph_deploy/lib/vendor/remoto/log.py", line 13, in reporting
[ceph-osd1][ERROR ] received = result.receive(timeout)
[ceph-osd1][ERROR ] File "/usr/lib/python2.7/dist-packages/ceph_deploy/lib/vendor/remoto/lib/vendor/execnet/gateway_base.py", line 704, in receive
[ceph-osd1][ERROR ] raise self._getremoteerror() or EOFError()
[ceph-osd1][ERROR ] RemoteError: Traceback (most recent call last):
[ceph-osd1][ERROR ] File "<string>", line 1036, in executetask
[ceph-osd1][ERROR ] File "<remote exec>", line 12, in _remote_run
[ceph-osd1][ERROR ] File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
[ceph-osd1][ERROR ] errread, errwrite)
[ceph-osd1][ERROR ] File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
[ceph-osd1][ERROR ] raise child_exception
[ceph-osd1][ERROR ] OSError: [Errno 2] No such file or directory
[ceph-osd1][ERROR ]
[ceph-osd1][ERROR ]
[ceph_deploy][ERROR ] RuntimeError: Failed to execute command: ceph --version
cephuser@cephfs5:~/my-cluster$
https://blog.54im.com/2016/12/15/centos-install-ceph-doc/ 超时问题
Q2¶
https://serverfault.com/questions/659308/unable-to-add-initial-monitor-to-ceph-in-ubuntu
[global]
fsid = 33cb5c76-a685-469e-8cdd-fee7c98c3f4d
mon_initial_members = ceph1,ceph2
mon_host = 192.168.61.39,192.168.61.40
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true
public_network = 192.168.61.0/24
And the running the command:
$ ceph-deploy --overwrite-conf mon create <ceph-node>
ceph install by ubuntu¶
Contents:
README (install-by-ubuntu)¶
本次安装，完全按照 官方文档 http://docs.ceph.com/docs/master/start/ 进行。
env¶
192.168.31.115 cephadmin # 这个是admin节点
192.168.31.114 mon1 # 存储节点
192.168.31.113 cephfsn2 # 存储节点
192.168.31.173 cephfsn3 # 存储节点
192.168.31.172 ceph-client # 客户端节点
上面这几个，可以全部写进 /etc/hosts 中。且各节点的 hostname 必须对应相同（否则安装容易出错）。
Preflight¶
# http://docs.ceph.com/docs/master/start/quick-start-preflight/#
这里的配置注意点如下：
ceph-admin 节点：¶
安装时， with a stable Ceph release (e.g., luminous.) ，安装 luminous 或以上版本。
cephu@cephadmin:~/my-cluster$ sudo cat /etc/apt/sources.list.d/ceph.list
deb https://download.ceph.com/debian-luminous/ xenial main
cephu@cephadmin:~/my-cluster$
配置 ~/.ssh/config 方便后续安装
cephu@cephadmin:~/my-cluster$ cat ~/.ssh/config
Host node0
Hostname cephadmin
User cephu
Host node1
Hostname mon1
User cephu
Host node2
Hostname cephfsn2
User cephu
Host node3
Hostname cephfsn3
User cephu
cephu@cephadmin:~/my-cluster$
这里，我后来想了一下，应该把这个地方，修改成 全用 node0, node1, node2, node3, 然后，各节点的 Hostname 也修改成这些，这样子才是真的方便安装。如下：
cephu@cephadmin:~/my-cluster$ cat ~/.ssh/config
Host node0
Hostname node0
User cephu
Host node1
Hostname node1
User cephu
Host node2
Hostname node2
User cephu
Host node3
Hostname node3
User cephu
cephu@cephadmin:~/my-cluster$
存储节点：¶
cephu@cephadmin:~/my-cluster$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 ubuntu
#127.0.0.1 cephfs5
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
192.168.31.115 cephadmin
192.168.31.114 mon1
192.168.31.113 cephfsn2
192.168.31.173 cephfsn3
Storage Cluster Quick Start¶
# http://docs.ceph.com/docs/master/start/quick-ceph-deploy/
2.1 Create the cluster.
ceph-deploy new node1 不生效的，要写成 ceph-deploy new mon1
cephu@cephadmin:~/my-cluster$ ceph-deploy new mon1
2.2 - 2.3
cephu@cephadmin:~/my-cluster$ cat ~/my-cluster/ceph.conf
[global]
fsid = d0aa5af1-4f8e-4953-9448-7f1b2448b8a5
mon_initial_members = mon1
mon_host = 192.168.31.114
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
public network = 192.168.31.1/24
ms bind ipv6 = true
cephu@cephadmin:~/my-cluster$
2.4
cephu@cephadmin:~/my-cluster$ ceph-deploy install node1 node2 node3
2.5
cephu@cephadmin:~/my-cluster$ ceph-deploy mon create-initial
报错，然后，经过 农总在 https://my.oschina.net/u/2475751/blog/647777 查到要运行下面
cephu@cephadmin:~/my-cluster$ ssh mon1 sudo ceph-create-keys --id mon1
再来一次
cephu@cephadmin:~/my-cluster$ ceph-deploy mon create-initial
3.1
cephu@cephadmin:~/my-cluster$ ceph-deploy --overwrite-conf admin node1 node2 node3
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.32): /usr/bin/ceph-deploy --overwrite-conf admin node1 node2 node3
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f2c1fbe2440>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] client : ['node1', 'node2', 'node3']
[ceph_deploy.cli][INFO ] func : <function admin at 0x7f2c20489b18>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.admin][DEBUG ] Pushing admin keys and conf to node1
[node1][DEBUG ] connection detected need for sudo
[node1][DEBUG ] connected to host: node1
[node1][DEBUG ] detect platform information from remote host
[node1][DEBUG ] detect machine type
[node1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.admin][DEBUG ] Pushing admin keys and conf to node2
[node2][DEBUG ] connection detected need for sudo
[node2][DEBUG ] connected to host: node2
[node2][DEBUG ] detect platform information from remote host
[node2][DEBUG ] detect machine type
[node2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.admin][DEBUG ] Pushing admin keys and conf to node3
[node3][DEBUG ] connection detected need for sudo
[node3][DEBUG ] connected to host: node3
[node3][DEBUG ] detect platform information from remote host
[node3][DEBUG ] detect machine type
[node3][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
cephu@cephadmin:~/my-cluster$
3.2
cephu@cephadmin:~/my-cluster$ ceph-deploy mgr create node1
usage: ceph-deploy [-h] [-v | -q] [--version] [--username USERNAME]
[--overwrite-conf] [--cluster NAME] [--ceph-conf CEPH_CONF]
COMMAND ...
ceph-deploy: error: argument COMMAND: invalid choice: 'mgr' (choose from 'new', 'install', 'rgw', 'mon', 'mds', ' gatherkeys', 'disk', 'osd', 'admin', 'repo', 'config', 'uninstall', 'purge', 'purgedata', 'calamari', 'forgetkeys ', 'pkg')
cephu@cephadmin:~/my-cluster$
报错
cephu@cephadmin:~/my-cluster$ ceph-deploy -h
usage: ceph-deploy [-h] [-v | -q] [--version] [--username USERNAME]
[--overwrite-conf] [--cluster NAME] [--ceph-conf CEPH_CONF]
COMMAND ...
Easy Ceph deployment
-^-
/ \
|O o| ceph-deploy v1.5.32
).-.(
'/|||\`
| '|` |
'|`
Full documentation can be found at: http://ceph.com/ceph-deploy/docs
optional arguments:
-h, --help show this help message and exit
-v, --verbose be more verbose
-q, --quiet be less verbose
--version the current installed version of ceph-deploy
--username USERNAME the username to connect to the remote host
--overwrite-conf overwrite an existing conf file on remote host (if
present)
--cluster NAME name of the cluster
--ceph-conf CEPH_CONF
use (or reuse) a given ceph.conf file
commands:
COMMAND description
new Start deploying a new cluster, and write a
CLUSTER.conf and keyring for it.
install Install Ceph packages on remote hosts.
rgw Ceph RGW daemon management
mon Ceph MON Daemon management
mds Ceph MDS daemon management
gatherkeys Gather authentication keys for provisioning new nodes.
disk Manage disks on a remote host.
osd Prepare a data disk on remote host.
admin Push configuration and client.admin key to a remote
host.
repo Repo definition management
config Copy ceph.conf to/from remote host(s)
uninstall Remove Ceph packages from remote hosts.
purge Remove Ceph packages from remote hosts and purge all
data.
purgedata Purge (delete, destroy, discard, shred) any Ceph data
from /var/lib/ceph
calamari Install and configure Calamari nodes. Assumes that a
repository with Calamari packages is already
configured. Refer to the docs for examples
(http://ceph.com/ceph-deploy/docs/conf.html)
forgetkeys Remove authentication keys from the local directory.
pkg Manage packages on remote hosts.
cephu@cephadmin:~/my-cluster$
我去，真的没有 mgr 命令。
版本升级吧
cephu@cephadmin:~/my-cluster$ pip install ceph-deploy
Collecting ceph-deploy
Downloading ceph-deploy-1.5.39.tar.gz (114kB)
100% |████████████████████████████████| 122kB 292kB/s
Collecting setuptools (from ceph-deploy)
Downloading setuptools-36.6.0-py2.py3-none-any.whl (481kB)
100% |████████████████████████████████| 481kB 968kB/s
Building wheels for collected packages: ceph-deploy
Running setup.py bdist_wheel for ceph-deploy ... done
Stored in directory: /home/cephu/.cache/pip/wheels/5e/4a/c5/5759b04fedf1eaa17d4453b562ab28a2142dbf93ced0c37e5d
Successfully built ceph-deploy
Installing collected packages: setuptools, ceph-deploy
Successfully installed ceph-deploy-1.5.32 setuptools-20.7.0
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
cephu@cephadmin:~/my-cluster$ ceph-deploy --version
1.5.39
cephu@cephadmin:~/my-cluster$ ceph-deploy -h
这下有了。重新来一下
cephu@cephadmin:~/my-cluster$ ceph-deploy mgr create node1
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /usr/bin/ceph-deploy mgr create node1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] mgr : [('node1', 'node1')]
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fa24a321ab8>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function mgr at 0x7fa24a993578>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.mgr][DEBUG ] Deploying mgr, cluster ceph hosts node1:node1
[ceph_deploy][ERROR ] RuntimeError: bootstrap-mgr keyring not found; run 'gatherkeys'
哇，又来错误。
提示了 run ‘gatherkeys’ 。
cephu@cephadmin:~/my-cluster$ ceph-deploy gatherkeys mon1
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /usr/bin/ceph-deploy gatherkeys mon1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fbfda5f8a70>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] mon : ['mon1']
[ceph_deploy.cli][INFO ] func : <function gatherkeys at 0x7fbfda8570c8>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.gatherkeys][INFO ] Storing keys in temp directory /tmp/tmpZ8THzr
[mon1][DEBUG ] connection detected need for sudo
[mon1][DEBUG ] connected to host: mon1
[mon1][DEBUG ] detect platform information from remote host
[mon1][DEBUG ] detect machine type
[mon1][DEBUG ] get remote short hostname
[mon1][DEBUG ] fetch remote file
[mon1][INFO ] Running command: sudo /usr/bin/ceph --connect-timeout=25 --cluster=ceph --admin-daemon=/var/run/ceph/ceph-mon.mon1.asok mon_status
[mon1][INFO ] Running command: sudo /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-mon1/keyring auth get client.admin
[mon1][INFO ] Running command: sudo /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-mon1/keyring auth get client.bootstrap-mds
[mon1][INFO ] Running command: sudo /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-mon1/keyring auth get client.bootstrap-mgr
[mon1][INFO ] Running command: sudo /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-mon1/keyring auth get-or-create client.bootstrap-mgr mon allow profile bootstrap-mgr
[mon1][INFO ] Running command: sudo /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-mon1/keyring auth get client.bootstrap-osd
[mon1][INFO ] Running command: sudo /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-mon1/keyring auth get client.bootstrap-rgw
[ceph_deploy.gatherkeys][INFO ] keyring 'ceph.client.admin.keyring' already exists
[ceph_deploy.gatherkeys][INFO ] keyring 'ceph.bootstrap-mds.keyring' already exists
[ceph_deploy.gatherkeys][INFO ] Storing ceph.bootstrap-mgr.keyring
[ceph_deploy.gatherkeys][INFO ] keyring 'ceph.mon.keyring' already exists
[ceph_deploy.gatherkeys][INFO ] keyring 'ceph.bootstrap-osd.keyring' already exists
[ceph_deploy.gatherkeys][INFO ] keyring 'ceph.bootstrap-rgw.keyring' already exists
[ceph_deploy.gatherkeys][INFO ] Destroy temp directory /tmp/tmpZ8THzr
cephu@cephadmin:~/my-cluster$
再走一个
cephu@cephadmin:~/my-cluster$ ceph-deploy mgr create node1
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /usr/bin/ceph-deploy mgr create node1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] mgr : [('node1', 'node1')]
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f5d02a1bab8>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function mgr at 0x7f5d0308d578>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.mgr][DEBUG ] Deploying mgr, cluster ceph hosts node1:node1
[node1][DEBUG ] connection detected need for sudo
[node1][DEBUG ] connected to host: node1
[node1][DEBUG ] detect platform information from remote host
[node1][DEBUG ] detect machine type
[ceph_deploy.mgr][INFO ] Distro info: Ubuntu 16.04 xenial
[ceph_deploy.mgr][DEBUG ] remote host will use systemd
[ceph_deploy.mgr][DEBUG ] deploying mgr bootstrap to node1
[node1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.mgr][ERROR ] RuntimeError: config file /etc/ceph/ceph.conf exists with different content; use --overwrite-conf to overwrite
[ceph_deploy][ERROR ] GenericError: Failed to create 1 MGRs
提示了 use –overwrite-conf
再来
cephu@cephadmin:~/my-cluster$ ceph-deploy --overwrite-conf mgr create mon1
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /usr/bin/ceph-deploy --overwrite-conf mgr create mon1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] mgr : [('mon1', 'mon1')]
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fe415166ab8>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function mgr at 0x7fe4157d8578>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.mgr][DEBUG ] Deploying mgr, cluster ceph hosts mon1:mon1
[mon1][DEBUG ] connection detected need for sudo
[mon1][DEBUG ] connected to host: mon1
[mon1][DEBUG ] detect platform information from remote host
[mon1][DEBUG ] detect machine type
[ceph_deploy.mgr][INFO ] Distro info: Ubuntu 16.04 xenial
[ceph_deploy.mgr][DEBUG ] remote host will use systemd
[ceph_deploy.mgr][DEBUG ] deploying mgr bootstrap to mon1
[mon1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[mon1][WARNIN] mgr keyring does not exist yet, creating one
[mon1][DEBUG ] create a keyring file
[mon1][DEBUG ] create path if it doesn't exist
[mon1][INFO ] Running command: sudo ceph --cluster ceph --name client.bootstrap-mgr --keyring /var/lib/ceph/bootstrap-mgr/ceph.keyring auth get-or-create mgr.mon1 mon allow profile mgr osd allow * mds allow * -o /var/lib/ceph/mgr/ceph-mon1/keyring
[mon1][INFO ] Running command: sudo systemctl enable ceph-mgr@mon1
[mon1][WARNIN] Created symlink from /etc/systemd/system/ceph-mgr.target.wants/ceph-mgr@mon1.service to /lib/systemd/system/ceph-mgr@.service.
[mon1][INFO ] Running command: sudo systemctl start ceph-mgr@mon1
[mon1][INFO ] Running command: sudo systemctl enable ceph.target
cephu@cephadmin:~/my-cluster$
终于成功了。
3.3
cephu@cephadmin:~/my-cluster$ ceph-deploy --overwrite-conf osd create node1:sdb node2:sdb node3:sdc
...
[node3][DEBUG ] Warning: The kernel is still using the old partition table.
[node3][DEBUG ] The new table will be used at the next reboot or after you
[node3][DEBUG ] run partprobe(8) or kpartx(8)
[node3][DEBUG ] The operation has completed successfully.
[node3][WARNIN] update_partition: Calling partprobe on prepared device /dev/sdc
[node3][WARNIN] command_check_call: Running command: /sbin/udevadm settle --timeout=600
[node3][WARNIN] command: Running command: /usr/bin/flock -s /dev/sdc /sbin/partprobe /dev/sdc
[node3][WARNIN] command_check_call: Running command: /sbin/udevadm settle --timeout=600
[node3][WARNIN] command_check_call: Running command: /sbin/udevadm trigger --action=add --sysname-match sdc1
[node3][INFO ] Running command: sudo systemctl enable ceph.target
[node3][INFO ] checking OSD status...
[node3][DEBUG ] find the location of an executable
[node3][INFO ] Running command: sudo /usr/bin/ceph --cluster=ceph osd stat --format=json
[node3][WARNIN] there is 1 OSD down
[node3][WARNIN] there is 1 OSD out
[ceph_deploy.osd][DEBUG ] Host node3 is now ready for osd use.
cephu@cephadmin:~/my-cluster$
成功
3.4
cephu@cephadmin:~/my-cluster$ ssh node1 sudo ceph health
HEALTH_OK
cephu@cephadmin:~/my-cluster$ ssh node2 sudo ceph health
HEALTH_OK
cephu@cephadmin:~/my-cluster$ ssh node3 sudo ceph health
HEALTH_OK
cephu@cephadmin:~/my-cluster$ ssh node3 sudo ceph -s
cluster:
id: d0aa5af1-4f8e-4953-9448-7f1b2448b8a5
health: HEALTH_OK
services:
mon: 1 daemons, quorum mon1
mgr: mon1(active)
osd: 3 osds: 3 up, 3 in
data:
pools: 0 pools, 0 pgs
objects: 0 objects, 0 bytes
usage: 3164 MB used, 80500 MB / 83664 MB avail
pgs:
cephu@cephadmin:~/my-cluster$
到这里，我们应该是完成了基础的搭建
附加项：expanding your cluster¶
ceph-deploy mds create node1
ceph-deploy mon add node2 node3 会报错
换成：
cephu@cephadmin:~/my-cluster$ ceph-deploy mon add cephfsn2
cephu@cephadmin:~/my-cluster$ ceph-deploy mon add cephfsn3
验证
cephu@cephadmin:~/my-cluster$ ssh cephfsn2 ceph quorum_status --format json-pretty
2017-10-24 10:40:24.959942 7f261141a700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory
2017-10-24 10:40:24.959973 7f261141a700 -1 monclient: ERROR: missing keyring, cannot use cephx for authentication
2017-10-24 10:40:24.959975 7f261141a700 0 librados: client.admin initialization error (2) No such file or directory
[errno 2] error connecting to the cluster
cephu@cephadmin:~/my-cluster$ ssh cephfsn2 sudo ceph quorum_status --format json-pretty
{
"election_epoch": 14,
"quorum": [
0,
1,
2
],
"quorum_names": [
"cephfsn2",
"mon1",
"cephfsn3"
],
"quorum_leader_name": "cephfsn2",
"monmap": {
"epoch": 3,
"fsid": "d0aa5af1-4f8e-4953-9448-7f1b2448b8a5",
"modified": "2017-10-24 10:32:19.273831",
"created": "2017-10-23 15:22:32.766470",
"features": {
"persistent": [
"kraken",
"luminous"
],
"optional": []
},
"mons": [
{
"rank": 0,
"name": "cephfsn2",
"addr": "192.168.31.113:6789/0",
"public_addr": "192.168.31.113:6789/0"
},
{
"rank": 1,
"name": "mon1",
"addr": "192.168.31.114:6789/0",
"public_addr": "192.168.31.114:6789/0"
},
{
"rank": 2,
"name": "cephfsn3",
"addr": "192.168.31.173:6789/0",
"public_addr": "192.168.31.173:6789/0"
}
]
}
}
cephu@cephadmin:~/my-cluster$
cephu@cephadmin:~/my-cluster$ ceph-deploy mgr create node2 node3
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /usr/bin/ceph-deploy mgr create node2 node3
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] mgr : [('node2', 'node2'), ('node3', 'node3')]
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fb9043a4ab8>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function mgr at 0x7fb904a16578>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.mgr][DEBUG ] Deploying mgr, cluster ceph hosts node2:node2 node3:node3
[node2][DEBUG ] connection detected need for sudo
[node2][DEBUG ] connected to host: node2
[node2][DEBUG ] detect platform information from remote host
[node2][DEBUG ] detect machine type
[ceph_deploy.mgr][INFO ] Distro info: Ubuntu 16.04 xenial
[ceph_deploy.mgr][DEBUG ] remote host will use systemd
[ceph_deploy.mgr][DEBUG ] deploying mgr bootstrap to node2
[node2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[node2][WARNIN] mgr keyring does not exist yet, creating one
[node2][DEBUG ] create a keyring file
[node2][DEBUG ] create path if it doesn't exist
[node2][INFO ] Running command: sudo ceph --cluster ceph --name client.bootstrap-mgr --keyring /var/lib/ceph/bootstrap-mgr/ceph.keyring auth get-or-create mgr.node2 mon allow profile mgr osd allow * mds allow * -o /var/lib/ceph/mgr/ceph-node2/keyring
[node2][INFO ] Running command: sudo systemctl enable ceph-mgr@node2
[node2][WARNIN] Created symlink from /etc/systemd/system/ceph-mgr.target.wants/ceph-mgr@node2.service to /lib/systemd/system/ceph-mgr@.service.
[node2][INFO ] Running command: sudo systemctl start ceph-mgr@node2
[node2][INFO ] Running command: sudo systemctl enable ceph.target
[node3][DEBUG ] connection detected need for sudo
[node3][DEBUG ] connected to host: node3
[node3][DEBUG ] detect platform information from remote host
[node3][DEBUG ] detect machine type
[ceph_deploy.mgr][INFO ] Distro info: Ubuntu 16.04 xenial
[ceph_deploy.mgr][DEBUG ] remote host will use systemd
[ceph_deploy.mgr][DEBUG ] deploying mgr bootstrap to node3
[node3][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[node3][WARNIN] mgr keyring does not exist yet, creating one
[node3][DEBUG ] create a keyring file
[node3][DEBUG ] create path if it doesn't exist
[node3][INFO ] Running command: sudo ceph --cluster ceph --name client.bootstrap-mgr --keyring /var/lib/ceph/bootstrap-mgr/ceph.keyring auth get-or-create mgr.node3 mon allow profile mgr osd allow * mds allow * -o /var/lib/ceph/mgr/ceph-node3/keyring
[node3][INFO ] Running command: sudo systemctl enable ceph-mgr@node3
[node3][WARNIN] Created symlink from /etc/systemd/system/ceph-mgr.target.wants/ceph-mgr@node3.service to /lib/systemd/system/ceph-mgr@.service.
[node3][INFO ] Running command: sudo systemctl start ceph-mgr@node3
[node3][INFO ] Running command: sudo systemctl enable ceph.target
cephu@cephadmin:~/my-cluster$
验证
cephu@cephadmin:~/my-cluster$ ssh node1 sudo ceph -s
cluster:
id: d0aa5af1-4f8e-4953-9448-7f1b2448b8a5
health: HEALTH_WARN
clock skew detected on mon.cephfsn3
services:
mon: 3 daemons, quorum cephfsn2,mon1,cephfsn3
mgr: mon1(active), standbys: node2, node3
osd: 3 osds: 3 up, 3 in
data:
pools: 0 pools, 0 pgs
objects: 0 objects, 0 bytes
usage: 3164 MB used, 80500 MB / 83664 MB avail
pgs:
cephu@cephadmin:~/my-cluster$ ssh node3 sudo ceph -s
cluster:
id: d0aa5af1-4f8e-4953-9448-7f1b2448b8a5
health: HEALTH_WARN
clock skew detected on mon.cephfsn3
services:
mon: 3 daemons, quorum cephfsn2,mon1,cephfsn3
mgr: mon1(active), standbys: node2, node3
osd: 3 osds: 3 up, 3 in
data:
pools: 0 pools, 0 pgs
objects: 0 objects, 0 bytes
usage: 3164 MB used, 80500 MB / 83664 MB avail
pgs:
cephu@cephadmin:~/my-cluster$
STORING/RETRIEVING OBJECT DATA¶
这一个小节，现在先不动。
算了，还是动吧。
cephu@cephadmin:~/my-cluster$ sudo apt install ceph-common -y
cephu@cephadmin:~/my-cluster$ ls /etc/ceph/*
cephu@cephadmin:~/my-cluster$ ceph-deploy admin cephadmin
cephu@cephadmin:~/my-cluster$ ls /etc/ceph/* ## 这里可以看到新增3个文件
cephu@cephadmin:~/my-cluster$ ceph-deploy gatherkeys cephadmin
在 admin 与 ceph-client 节点，都运行一下。
jlch@k-m:/etc/apt/sources.list.d$ sudo chmod +r /etc/ceph/ceph.client.admin.keyring
现在可以开始执行命令了
jlch@k-m:/etc/apt/sources.list.d$ echo {Test-data} > testfile.txt
jlch@k-m:/etc/apt/sources.list.d$ ceph osd pool create mytest 8
jlch@k-m:/etc/apt/sources.list.d$ rados put test-object-1 testfile.txt --pool=mytest
jlch@k-m:/etc/apt/sources.list.d$ rados -p mytest ls
jlch@k-m:/etc/apt/sources.list.d$ ceph osd map mytest test-object-1
Block Device Quick Start¶
INSTALL CEPH¶
admin 节点：
ceph-deploy install ceph-client
ceph-deploy admin ceph-client
ceph-client 节点：
sudo chmod +r /etc/ceph/ceph.client.admin.keyring
CREATE A BLOCK DEVICE POOL¶
admin 节点：
原来在 admin 节点，是没有这个 rbd 命令的。
要通过 sudo apt install ceph-common 之后，才会有。所以先运行一下。
cephu@cephadmin:~/my-cluster$ sudo apt install ceph-common -y
cephu@cephadmin:~/my-cluster$ rbd pool init jlch
2017-10-24 15:57:45.951917 7fde4fa6e0c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory
2017-10-24 15:57:45.951937 7fde4fa6e0c0 -1 monclient: ERROR: missing keyring, cannot use cephx for authentication
2017-10-24 15:57:45.951942 7fde4fa6e0c0 0 librados: client.admin initialization error (2) No such file or directory
rbd: couldn't connect to the cluster!
cephu@cephadmin:~/my-cluster$
ceph-client 节点：
jlch@k-m:/etc/apt/sources.list.d$ sudo ls /etc/ceph/* -l
[sudo] password for jlch:
-rw-r--r-- 1 root root 63 Oct 24 15:32 /etc/ceph/ceph.client.admin.keyring
-rw-r--r-- 1 root root 249 Oct 24 15:32 /etc/ceph/ceph.conf
-rw-r--r-- 1 root root 92 Apr 21 2017 /etc/ceph/rbdmap
-rw------- 1 root root 0 Oct 24 11:20 /etc/ceph/tmp2IJh4C
jlch@k-m:/etc/apt/sources.list.d$
我去，明明有的，为什么说 unable to find a keyring on ***，哪里出的问题?
哈哈，这个问题的原因找到了。需要把 上一小节的 最后一个部分，执行一下。
而且，要按上一小节的方式，osd create 一个 pool 出来，才能在这里 pool init .
比如之前就是创建了 pool mytest. 这里就是
rbd pool init mytest
CONFIGURE A BLOCK DEVICE¶
On the ceph-client node
cephu@ceph-client:~$ rbd create mytest --size 4096 -m mon1 -k /etc/ceph/ceph.client.admin.keyring
rbd: error opening default pool 'rbd'
Ensure that the default pool has been created or specify an alternate pool name.
cephu@ceph-client:~$
报错，因为，这里说了，opening default pool ‘rbd’ 出错，为啥？因为之前没有建立过 pool rbd 呀，之前，只有 pool mytest 呀。 怎么办？2个方法。1）重新新建立一个 pool rbd, 2）指向到已建立的pool, 如 pool mytest 。
我们走 方法2.
方法1
回 ceph-admin 节点：
cephu@cephadmin:~/my-cluster$ ceph osd pool create rbd 8
pool 'rbd' created
cephu@cephadmin:~/my-cluster$ rbd pool init rbd
cephu@cephadmin:~/my-cluster$
回 ceph-client 节点：
cephu@ceph-client:~$ rbd create foo --size 4096 -m mon1 -k /etc/ceph/ceph.client.admin.keyring
方法2
cephu@ceph-client:~$ rbd help create # 这里会发现是加 -p 参数来指定 pool
cephu@ceph-client:~$ rbd create foo --size 4096 -m mon1 -k /etc/ceph/ceph.client.admin.keyring -p mytest
cephu@ceph-client:~$
好了，可以走下一步了。 我们继续。
On the ceph-client node, map the image to a block device.
cephu@ceph-client:~$ sudo rbd map foo --image client.admin -m mon1 -p mytest
rbd: sysfs write failed
rbd: error opening image client.admin: (2) No such file or directory
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (110) Connection timed out
cephu@ceph-client:~$
报错了。
cephu@ceph-client:~$ dmesg | tail -n 100
...
[692522.117250] libceph: mon0 192.168.31.114:6789 missing required protocol features
[692532.096436] libceph: mon0 192.168.31.114:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400000000000000
[692532.099897] libceph: mon0 192.168.31.114:6789 missing required protocol features
[692542.111938] libceph: mon0 192.168.31.114:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400000000000000
[692542.115603] libceph: mon0 192.168.31.114:6789 missing required protocol features
http://www.hl10502.com/2017/08/01/ceph-rbdmap-error-1/
上面有一些详细的原理
又要农总出马了。
http://blog.csdn.net/lk142500/article/details/78275910
这里指出了解决方法。来吧。
admin 节点
cephu@cephadmin:~/my-cluster$ ceph -v
ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
cephu@cephadmin:~/my-cluster$ ceph osd crush tunables optimal
adjusted tunables profile to optimal
cephu@cephadmin:~/my-cluster$ ceph osd crush rule ls
replicated_rule
cephu@cephadmin:~/my-cluster$ ceph osd crush rule dump
[
{
"rule_id": 0,
"rule_name": "replicated_rule",
"ruleset": 0,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{
"op": "take",
"item": -1,
"item_name": "default"
},
{
"op": "chooseleaf_firstn",
"num": 0,
"type": "host"
},
{
"op": "emit"
}
]
}
]
cephu@cephadmin:~/my-cluster$ ceph osd crush show-tunables
{
"choose_local_tries": 0,
"choose_local_fallback_tries": 0,
"choose_total_tries": 50,
"chooseleaf_descend_once": 1,
"chooseleaf_vary_r": 1,
"chooseleaf_stable": 1,
"straw_calc_version": 1,
"allowed_bucket_algs": 54,
"profile": "jewel",
"optimal_tunables": 1,
"legacy_tunables": 0,
"minimum_required_version": "jewel",
"require_feature_tunables": 1,
"require_feature_tunables2": 1,
"has_v2_rules": 0,
"require_feature_tunables3": 1,
"has_v3_rules": 0,
"has_v4_buckets": 1,
"require_feature_tunables5": 1,
"has_v5_rules": 0
}
cephu@cephadmin:~/my-cluster$ ceph osd crush -h
General usage:
==============
usage: ceph [-h] [-c CEPHCONF] [-i INPUT_FILE] [-o OUTPUT_FILE]
[--id CLIENT_ID] [--name CLIENT_NAME] [--cluster CLUSTER]
[--admin-daemon ADMIN_SOCKET] [-s] [-w] [--watch-debug]
[--watch-info] [--watch-sec] [--watch-warn] [--watch-error]
[--watch-channel WATCH_CHANNEL] [--version] [--verbose]
[--concise] [-f {json,json-pretty,xml,xml-pretty,plain}]
[--connect-timeout CLUSTER_TIMEOUT]
Ceph administration tool
optional arguments:
-h, --help request mon help
-c CEPHCONF, --conf CEPHCONF
ceph configuration file
-i INPUT_FILE, --in-file INPUT_FILE
input file, or "-" for stdin
-o OUTPUT_FILE, --out-file OUTPUT_FILE
output file, or "-" for stdout
--id CLIENT_ID, --user CLIENT_ID
client id for authentication
--name CLIENT_NAME, -n CLIENT_NAME
client name for authentication
--cluster CLUSTER cluster name
--admin-daemon ADMIN_SOCKET
submit admin-socket commands ("help" for help
-s, --status show cluster status
-w, --watch watch live cluster changes
--watch-debug watch debug events
--watch-info watch info events
--watch-sec watch security events
--watch-warn watch warn events
--watch-error watch error events
--watch-channel WATCH_CHANNEL
which log channel to follow when using -w/--watch. One
of ['cluster', 'audit', '*'
--version, -v display version
--verbose make verbose
--concise make less verbose
-f {json,json-pretty,xml,xml-pretty,plain}, --format {json,json-pretty,xml,xml-pretty,plain}
--connect-timeout CLUSTER_TIMEOUT
set a timeout for connecting to the cluster
Local commands:
===============
ping <mon.id> Send simple presence/life test to a mon
<mon.id> may be 'mon.*' for all mons
daemon {type.id|path} <cmd>
Same as --admin-daemon, but auto-find admin socket
daemonperf {type.id | path} [stat-pats] [priority] [<interval>] [<count>]
daemonperf {type.id | path} list|ls [stat-pats] [priority]
Get selected perf stats from daemon/admin socket
Optional shell-glob comma-delim match string stat-pats
Optional selection priority (can abbreviate name):
critical, interesting, useful, noninteresting, debug
List shows a table of all available stats
Run <count> times (default forever),
once per <interval> seconds (default 1)
Monitor commands:
=================
osd crush add <osdname (id|osd.id)> <float[0.0-]> <args> [<args>...] add or update crushmap position and weight for <name> with <weight> and location <args>
osd crush add-bucket <name> <type> add no-parent (probably root) crush bucket <name> of type <type>
osd crush class ls list all crush device classes
osd crush class ls-osd <class> list all osds belonging to the specific <class>
osd crush class rename <srcname> <dstname> rename crush device class <srcname> to <dstname>
osd crush create-or-move <osdname (id|osd.id)> <float[0.0-]> <args> [<args>...] create entry or move existing entry for <name> <weight> at/to location <args>
osd crush dump dump crush map
osd crush get-tunable straw_calc_version get crush tunable <tunable>
osd crush link <name> <args> [<args>...] link existing entry for <name> under location <args>
osd crush ls <node> list items beneath a node in the CRUSH tree
osd crush move <name> <args> [<args>...] move existing entry for <name> to location <args>
osd crush remove <name> {<ancestor>} remove <name> from crush map (everywhere, or just at <ancestor>)
osd crush rename-bucket <srcname> <dstname> rename bucket <srcname> to <dstname>
osd crush reweight <name> <float[0.0-]> change <name>'s weight to <weight> in crush map
osd crush reweight-all recalculate the weights for the tree to ensure they sum correctly
osd crush reweight-subtree <name> <float[0.0-]> change all leaf items beneath <name> to <weight> in crush map
osd crush rm <name> {<ancestor>} remove <name> from crush map (everywhere, or just at <ancestor>)
osd crush rm-device-class <ids> [<ids>...] remove class of the osd(s) <id> [<id>...],or use <all|any|*> to remove all.
osd crush rule create-erasure <name> {<profile>} create crush rule <name> for erasure coded pool created with <profile> (default default)
osd crush rule create-replicated <name> <root> <type> {<class>} create crush rule <name> for replicated pool to start from <root>, replicate across buckets of type <type>, using a
choose mode of <firstn|indep> (default firstn; indep best for erasure pools)
osd crush rule create-simple <name> <root> <type> {firstn|indep} create crush rule <name> to start from <root>, replicate across buckets of type <type>, using a choose mode of
<firstn|indep> (default firstn; indep best for erasure pools)
osd crush rule dump {<name>} dump crush rule <name> (default all)
osd crush rule ls list crush rules
osd crush rule ls-by-class <class> list all crush rules that reference the same <class>
osd crush rule rename <srcname> <dstname> rename crush rule <srcname> to <dstname>
osd crush rule rm <name> remove crush rule <name>
osd crush set <osdname (id|osd.id)> <float[0.0-]> <args> [<args>...] update crushmap position and weight for <name> to <weight> with location <args>
osd crush set {<int>} set crush map from input file
osd crush set-device-class <class> <ids> [<ids>...] set the <class> of the osd(s) <id> [<id>...],or use <all|any|*> to set all.
osd crush set-tunable straw_calc_version <int> set crush tunable <tunable> to <value>
osd crush show-tunables show current crush tunables
osd crush swap-bucket <source> <dest> {--yes-i-really-mean-it} swap existing bucket contents from (orphan) bucket <source> and <target>
osd crush tree {--show-shadow} dump crush buckets and items in a tree view
osd crush tunables legacy|argonaut|bobtail|firefly|hammer|jewel|optimal|default set crush tunables values to <profile>
osd crush unlink <name> {<ancestor>} unlink <name> from crush map (everywhere, or just at <ancestor>)
osd crush weight-set create <poolname> flat|positional create a weight-set for a given pool
osd crush weight-set create-compat create a default backward-compatible weight-set
osd crush weight-set dump dump crush weight sets
osd crush weight-set ls list crush weight sets
osd crush weight-set reweight <poolname> <item> <float[0.0-]> [<float[0.0-]>...] set weight for an item (bucket or osd) in a pool's weight-set
osd crush weight-set reweight-compat <item> <float[0.0-]> [<float[0.0-]>...] set weight for an item (bucket or osd) in the backward-compatible weight-set
osd crush weight-set rm <poolname> remove the weight-set for a given pool
osd crush weight-set rm-compat remove the backward-compatible weight-set
cephu@cephadmin:~/my-cluster$ ceph osd crush tunables hammer
adjusted tunables profile to hammer
cephu@cephadmin:~/my-cluster$ ceph osd crush show-tunables
{
"choose_local_tries": 0,
"choose_local_fallback_tries": 0,
"choose_total_tries": 50,
"chooseleaf_descend_once": 1,
"chooseleaf_vary_r": 1,
"chooseleaf_stable": 0,
"straw_calc_version": 1,
"allowed_bucket_algs": 54,
"profile": "hammer",
"optimal_tunables": 0,
"legacy_tunables": 0,
"minimum_required_version": "hammer",
"require_feature_tunables": 1,
"require_feature_tunables2": 1,
"has_v2_rules": 0,
"require_feature_tunables3": 1,
"has_v3_rules": 0,
"has_v4_buckets": 1,
"require_feature_tunables5": 0,
"has_v5_rules": 0
}
cephu@cephadmin:~/my-cluster$
回 ceph-client 节点：
cephu@ceph-client:~$ ceph -v
ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
cephu@ceph-client:~$ rbd ls
foo
cephu@ceph-client:~$ sudo rbd map foo --name client.admin
/dev/rbd0
cephu@ceph-client:~$ ls /dev/rbd
rbd/ rbd0
cephu@ceph-client:~$ ls /dev/rbd/rbd/foo
/dev/rbd/rbd/foo
cephu@ceph-client:~$ ls /dev/rbd0
/dev/rbd0
cephu@ceph-client:~$ ls /dev/rbd0 -l
brw-rw---- 1 root disk 251, 0 Oct 25 12:03 /dev/rbd0
cephu@ceph-client:~$
成功了。。继续。。
Use the block device by creating a file system on the ceph-client node.
cephu@ceph-client:~$ sudo mkfs.ext4 -m0 /dev/rbd/rbd/foo
mke2fs 1.42.13 (17-May-2015)
Discarding device blocks: done
Creating filesystem with 1048576 4k blocks and 262144 inodes
Filesystem UUID: d83ebc8d-1956-4d81-b9db-391f939634ac
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
cephu@ceph-client:~$
Mount the file system on the ceph-client node.
cephu@ceph-client:~$ sudo mkdir /mnt/ceph-block-device
cephu@ceph-client:~$ sudo mount /dev/rbd/rbd/foo /mnt/ceph-block-device
cephu@ceph-client:~$ cd /mnt/ceph-block-device
cephu@ceph-client:/mnt/ceph-block-device$ ls
lost+found
cephu@ceph-client:/mnt/ceph-block-device$
好了，这小一节，结束了。
Filesystem Quick Start¶
http://docs.ceph.com/docs/master/start/quick-cephfs/
PREREQUISITES¶
admin 节点：
ceph -s
看一下，是不是都正常。 这里是看不到 mds 的，要看，要去 moniter 节点
moniter 节点： 我们环境的是 192.168.31.114
cephu@mon1:~$ ps -ef | grep ceph
root 7541 7227 0 Oct23 pts/0 00:00:00 su - cephu
cephu 7542 7541 0 Oct23 pts/0 00:00:00 -su
cephu 20793 7542 0 12:39 pts/0 00:00:00 ps -ef
cephu 20794 7542 0 12:39 pts/0 00:00:00 grep --color=auto ceph
ceph 21275 1 0 Oct23 ? 00:07:50 /usr/bin/ceph-mon -f --cluster ceph --id mon1 --setuser ceph --setgroup ceph
ceph 22756 1 0 Oct24 ? 00:03:30 /usr/bin/ceph-mgr -f --cluster ceph --id mon1 --setuser ceph --setgroup ceph
ceph 25536 1 0 Oct24 ? 00:04:25 /usr/bin/ceph-osd -f --cluster ceph --id 0 --setuser ceph --setgroup ceph
ceph 26870 1 0 Oct24 ? 00:00:30 /usr/bin/ceph-mds -f --cluster ceph --id node1 --setuser ceph --setgroup ceph
ceph 29545 1 0 Oct24 ? 00:03:06 /usr/bin/radosgw -f --cluster ceph --name client.rgw.node1 --setuser ceph --setgroup ceph
cephu@mon1:~$
cephu@mon1:~$ ps -ef | grep ceph | grep mds
cephu 21152 7542 0 12:45 pts/0 00:00:00 grep --color=auto mds
ceph 26870 1 0 Oct24 ? 00:00:30 /usr/bin/ceph-mds -f --cluster ceph --id node1 --setuser ceph --setgroup ceph
cephu@mon1:~$
CREATE A FILESYSTEM¶
admin 节点：
cephu@cephadmin:~/my-cluster$ ceph osd pool create cephfs_data 8
pool 'cephfs_data' created
cephu@cephadmin:~/my-cluster$ ceph osd pool create cephfs_metadata 8
pool 'cephfs_metadata' created
cephu@cephadmin:~/my-cluster$ ceph fs new cephfs-jlch cephfs_metadata cephfs_data
new fs with metadata pool 8 and data pool 7
cephu@cephadmin:~/my-cluster$ ceph fs ls
name: cephfs-jlch, metadata pool: cephfs_metadata, data pools: [cephfs_data ]
cephu@cephadmin:~/my-cluster$
cephu@cephadmin:~/my-cluster$ ceph mds stat
cephfs-jlch-1/1/1 up {0=node1=up:active}
cephu@cephadmin:~/my-cluster$
CREATE A SECRET FILE¶
admin 节点：
cephu@cephadmin:~/my-cluster$ cat ceph.client.admin.keyring
[client.admin]
key = AQCtj+5ZnNTvGRAA2RxAGcIQZJnaJSPEz4jdGw==
cephu@cephadmin:~/my-cluster$ echo "AQCtj+5ZnNTvGRAA2RxAGcIQZJnaJSPEz4jdGw==" > admin.secret
cephu@cephadmin:~/my-cluster$ ls
admin.secret ceph.bootstrap-osd.keyring ceph.conf testfile.txt
ceph.bootstrap-mds.keyring ceph.bootstrap-rgw.keyring ceph-deploy-ceph.log
ceph.bootstrap-mgr.keyring ceph.client.admin.keyring ceph.mon.keyring
cephu@cephadmin:~/my-cluster$ cat admin.secret
AQCtj+5ZnNTvGRAA2RxAGcIQZJnaJSPEz4jdGw==
cephu@cephadmin:~/my-cluster$
KERNEL DRIVER¶
cephu@cephadmin:~/my-cluster$ sudo mkdir /mnt/mycephfs
cephu@cephadmin:~/my-cluster$ sudo mount -t ceph 192.168.31.114:6789:/ /mnt/mycephfs
mount error 22 = Invalid argument
cephu@cephadmin:~/my-cluster$ sudo mount -t ceph 192.168.31.114:6789:/ /mnt/mycephfs -o name=admin,secretfile=admin.secret
cephu@cephadmin:~/my-cluster$
如下可以看出， cephfs 将3个物理节点上的磁盘全部空间（82G = 16G + 16G + 50G）作为了自己的空间。
cephu@cephadmin:~/my-cluster$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 799M 79M 720M 10% /run
/dev/mapper/ubuntu--vg-root 71G 4.7G 62G 8% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/loop0 84M 84M 0 100% /snap/core/3017
/dev/loop2 82M 82M 0 100% /snap/core/2898
/dev/sda1 472M 58M 391M 13% /boot
tmpfs 799M 0 799M 0% /run/user/1000
/dev/loop3 84M 84M 0 100% /snap/core/3247
192.168.31.114:6789:/ 82G 4.2G 78G 6% /mnt/mycephfs
cephu@cephadmin:~/my-cluster$
FILESYSTEM IN USER SPACE (FUSE)¶
cephu@cephadmin:~/my-cluster$ sudo ceph-fuse -k ./ceph.client.admin.keyring -m 192.168.31.114:6789 ~/mycephfs
sudo: ceph-fuse: command not found
cephu@cephadmin:~/my-cluster$
没有这个命令。
农总这个时候，让我去 ceph-client 节点
ceph-client 节点：
cephu@ceph-client:~$ which ceph-fuse
cephu@ceph-client:~$ apt search ceph-fuse
Sorting... Done
Full Text Search... Done
ceph-fuse/stable 10.2.10-1xenial amd64
FUSE-based client for the Ceph distributed file system
ceph-fuse-dbg/stable 10.2.10-1xenial amd64
debugging symbols for ceph-fuse
cephu@ceph-client:~$ sudo apt install ceph-fuse
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libboost-program-options1.58.0 libboost-random1.58.0 libboost-regex1.58.0 libcephfs1 libfcgi0ldbl libllvm3.8
libmircommon5 linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-image-4.4.0-31-generic
linux-image-extra-4.4.0-31-generic
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
ceph-fuse
0 upgraded, 1 newly installed, 0 to remove and 48 not upgraded.
Need to get 2,926 kB of archives.
After this operation, 7,873 kB of additional disk space will be used.
Get:1 https://download.ceph.com/debian-jewel xenial/main amd64 ceph-fuse amd64 10.2.10-1xenial [2,926 kB]
Fetched 2,926 kB in 20s (145 kB/s)
Selecting previously unselected package ceph-fuse.
(Reading database ... 152232 files and directories currently installed.)
Preparing to unpack .../ceph-fuse_10.2.10-1xenial_amd64.deb ...
Unpacking ceph-fuse (10.2.10-1xenial) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up ceph-fuse (10.2.10-1xenial) ...
cephu@ceph-client:~$ sudo mkdir ~/mycephfs
cephu@ceph-client:~$ sudo ceph-fuse -m 192.168.31.114:6789 ~/mycephfs
ceph-fuse[28422]: starting ceph client
2017-10-25 14:10:35.041539 7fb1f6430f00 -1 init, newargv = 0x5608ad4acf60 newargc=11
ceph-fuse[28422]: starting fuse
cephu@ceph-client:~$
可以了，成功了。
看一下状态
cephu@ceph-client:~$ ceph mds stat
cephfs-jlch-1/1/1 up {0=node1=up:active}
cephu@ceph-client:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 799M 9.0M 790M 2% /run
/dev/mapper/ubuntu--vg-root 35G 4.2G 29G 13% /
tmpfs 3.9G 528K 3.9G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/sda1 472M 153M 295M 35% /boot
tmpfs 799M 0 799M 0% /run/user/113
tmpfs 799M 0 799M 0% /run/user/1000
/dev/rbd0 3.9G 8.0M 3.8G 1% /mnt/ceph-block-device
ceph-fuse 82G 4.2G 78G 6% /home/cephu/mycephfs
cephu@ceph-client:~$
ADDITIONAL INFORMATION¶
这里没有什么了。
测试一下¶
回到 admin 节点吧。
cephu@cephadmin:~/my-cluster$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 799M 79M 720M 10% /run
/dev/mapper/ubuntu--vg-root 71G 4.7G 62G 8% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/loop0 84M 84M 0 100% /snap/core/3017
/dev/loop2 82M 82M 0 100% /snap/core/2898
/dev/sda1 472M 58M 391M 13% /boot
tmpfs 799M 0 799M 0% /run/user/1000
/dev/loop3 84M 84M 0 100% /snap/core/3247
192.168.31.114:6789:/ 82G 4.2G 78G 6% /mnt/mycephfs
cephu@cephadmin:~/my-cluster$ touch /mnt/mycephfs/h.txt
touch: cannot touch '/mnt/mycephfs/h.txt': Permission denied
cephu@cephadmin:~/my-cluster$
为什么这样子？？？ Permission denied 这不是玩我么？
cephu@cephadmin:~/my-cluster$ sudo touch /mnt/mycephfs/h.txt
cephu@cephadmin:~/my-cluster$ sudo vi /mnt/mycephfs/h.txt
cephu@cephadmin:~/my-cluster$ sudo ls -l /mnt/mycephfs/
total 1
-rw-r--r-- 1 root root 14 Oct 25 14:33 h.txt
那直接修改目录权限。
cephu@cephadmin:~/my-cluster$ sudo chown -R cephu:cephu /mnt/mycephfs/
cephu@cephadmin:~/my-cluster$ touch /mnt/mycephfs/a.txt
cephu@cephadmin:~/my-cluster$ echo "hello, world" > /mnt/mycephfs/b.txt
cephu@cephadmin:~/my-cluster$ cat /mnt/mycephfs/b.txt
hello, world
cephu@cephadmin:~/my-cluster$ sudo ls -l /mnt/mycephfs/
total 1
-rw-rw-r-- 1 cephu cephu 0 Oct 25 14:34 a.txt
-rw-rw-r-- 1 cephu cephu 2 Oct 25 14:35 b.txt
-rw-r--r-- 1 cephu cephu 14 Oct 25 14:33 h.txt
cephu@cephadmin:~/my-cluster$
好了。这下成功了。
再回到 ceph-client 下测试吧（因为其下的 mount 方式不同，是 ceph-fuse，也测试一下咯）。
ceph-client节点：
cephu@ceph-client:~$ ls mycephfs/
a.txt b.txt h.txt
文件是看到了。 创建一下。
cephu@ceph-client:~$ echo "c" > mycephfs/b.txt
cephu@ceph-client:~$ cat mycephfs/b.txt
c
cephu@ceph-client:~$
OK。成功。
umount¶
到我们这个文档结束的时候，记得要 umount 一下哟。（这里只写 admin节点了，ceph-client节点是一样样的。）
admin:
cephu@cephadmin:~/my-cluster$ sudo umount /mnt/mycephfs
检查一下：
cephu@cephadmin:~/my-cluster$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=4067352k,nr_inodes=1016838,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=817512k,mode=755)
/dev/mapper/ubuntu--vg-root on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
/var/lib/snapd/snaps/core_3017.snap on /snap/core/3017 type squashfs (ro,nodev,relatime)
/var/lib/snapd/snaps/core_2898.snap on /snap/core/2898 type squashfs (ro,nodev,relatime)
/dev/sda1 on /boot type ext2 (rw,relatime,block_validity,barrier,user_xattr,acl)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
/dev/mapper/ubuntu--vg-root on /var/lib/docker/aufs type ext4 (rw,relatime,errors=remount-ro,data=ordered)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=817512k,mode=700,uid=1000,gid=1000)
/var/lib/snapd/snaps/core_3247.snap on /snap/core/3247 type squashfs (ro,nodev,relatime)
tmpfs on /run/snapd/ns type tmpfs (rw,nosuid,noexec,relatime,size=817512k,mode=755)
nsfs on /run/snapd/ns/core.mnt type nsfs (rw)
cephu@cephadmin:~/my-cluster$
确实没有了。安心。
到此，应该这一小节结束。 喝杯水，压压惊！~~~~~~~~~~
Object Storage Quick Start¶
INSTALLING CEPH OBJECT GATEWAY¶
ceph-client：
cephu@ceph-client:~$ sudo ufw allow 7480
Rule added
Rule added (v6)
cephu@ceph-client:~$ sudo ufw status
Status: active
To Action From
-- ------ ----
22 ALLOW Anywhere
7480 ALLOW Anywhere
22 (v6) ALLOW Anywhere (v6)
7480 (v6) ALLOW Anywhere (v6)
cephu@ceph-client:~$
ceph-admin:
cephu@cephadmin:~/my-cluster$ ping ceph-client
PING ceph-client (192.168.31.172) 56(84) bytes of data.
64 bytes from ceph-client (192.168.31.172): icmp_seq=1 ttl=64 time=0.558 ms
^C
--- ceph-client ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.558/0.558/0.558/0.000 ms
cephu@cephadmin:~/my-cluster$ ceph-deploy install --rgw ceph-client
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /home/cephu/.local/bin/ceph-deploy install --rgw ceph-client
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] testing : None
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f61e87c4908>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] dev_commit : None
[ceph_deploy.cli][INFO ] install_mds : False
[ceph_deploy.cli][INFO ] stable : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] adjust_repos : True
[ceph_deploy.cli][INFO ] func : <function install at 0x7f61e8e9b848>
[ceph_deploy.cli][INFO ] install_mgr : False
[ceph_deploy.cli][INFO ] install_all : False
[ceph_deploy.cli][INFO ] repo : False
[ceph_deploy.cli][INFO ] host : ['ceph-client']
[ceph_deploy.cli][INFO ] install_rgw : True
[ceph_deploy.cli][INFO ] install_tests : False
[ceph_deploy.cli][INFO ] repo_url : None
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] install_osd : False
[ceph_deploy.cli][INFO ] version_kind : stable
[ceph_deploy.cli][INFO ] install_common : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] dev : master
[ceph_deploy.cli][INFO ] nogpgcheck : False
[ceph_deploy.cli][INFO ] local_mirror : None
[ceph_deploy.cli][INFO ] release : None
[ceph_deploy.cli][INFO ] install_mon : False
[ceph_deploy.cli][INFO ] gpg_url : None
[ceph_deploy.install][DEBUG ] Installing stable version jewel on cluster ceph hosts ceph-client
[ceph_deploy.install][DEBUG ] Detecting platform for host ceph-client ...
[ceph-client][DEBUG ] connection detected need for sudo
[ceph-client][DEBUG ] connected to host: ceph-client
[ceph-client][DEBUG ] detect platform information from remote host
[ceph-client][DEBUG ] detect machine type
[ceph_deploy.install][INFO ] Distro info: Ubuntu 16.04 xenial
[ceph-client][INFO ] installing Ceph on ceph-client
[ceph-client][INFO ] Running command: sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get --assume-yes -q --no-install-recommends install ca-certificates apt-transport-https
[ceph-client][DEBUG ] Reading package lists...
[ceph-client][DEBUG ] Building dependency tree...
[ceph-client][DEBUG ] Reading state information...
[ceph-client][DEBUG ] apt-transport-https is already the newest version (1.2.24).
[ceph-client][DEBUG ] ca-certificates is already the newest version (20170717~16.04.1).
[ceph-client][DEBUG ] The following packages were automatically installed and are no longer required:
[ceph-client][DEBUG ] libboost-program-options1.58.0 libboost-random1.58.0 libboost-regex1.58.0
[ceph-client][DEBUG ] libcephfs1 libfcgi0ldbl libllvm3.8 libmircommon5 linux-headers-4.4.0-31
[ceph-client][DEBUG ] linux-headers-4.4.0-31-generic linux-image-4.4.0-31-generic
[ceph-client][DEBUG ] linux-image-extra-4.4.0-31-generic
[ceph-client][DEBUG ] Use 'sudo apt autoremove' to remove them.
[ceph-client][DEBUG ] 0 upgraded, 0 newly installed, 0 to remove and 48 not upgraded.
[ceph-client][INFO ] Running command: sudo wget -O release.asc https://download.ceph.com/keys/release.asc
[ceph-client][WARNIN] --2017-10-25 14:45:28-- https://download.ceph.com/keys/release.asc
[ceph-client][WARNIN] Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1
[ceph-client][WARNIN] Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:443... connected.
[ceph-client][WARNIN] HTTP request sent, awaiting response... 200 OK
[ceph-client][WARNIN] Length: 1645 (1.6K) [application/octet-stream]
[ceph-client][WARNIN] Saving to: ‘release.asc’
[ceph-client][WARNIN]
[ceph-client][WARNIN] 0K . 100% 233M=0s
[ceph-client][WARNIN]
[ceph-client][WARNIN] 2017-10-25 14:45:35 (233 MB/s) - ‘release.asc’ saved [1645/1645]
[ceph-client][WARNIN]
[ceph-client][INFO ] Running command: sudo apt-key add release.asc
[ceph-client][DEBUG ] OK
[ceph-client][DEBUG ] add deb repo to /etc/apt/sources.list.d/
[ceph-client][INFO ] Running command: sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get --assume-yes -q update
[ceph-client][DEBUG ] Hit:1 http://cn.archive.ubuntu.com/ubuntu xenial InRelease
[ceph-client][DEBUG ] Hit:2 http://cn.archive.ubuntu.com/ubuntu xenial-updates InRelease
[ceph-client][DEBUG ] Hit:3 http://cn.archive.ubuntu.com/ubuntu xenial-backports InRelease
[ceph-client][DEBUG ] Hit:4 http://www.rabbitmq.com/debian testing InRelease
[ceph-client][DEBUG ] Get:5 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
[ceph-client][DEBUG ] Hit:6 https://download.docker.com/linux/ubuntu xenial InRelease
[ceph-client][DEBUG ] Get:7 http://security.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [7,472 B]
[ceph-client][DEBUG ] Get:8 http://security.ubuntu.com/ubuntu xenial-security/restricted i386 Packages [7,472 B]
[ceph-client][DEBUG ] Get:9 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [3,208 B]
[ceph-client][DEBUG ] Get:10 http://security.ubuntu.com/ubuntu xenial-security/multiverse i386 Packages [3,384 B]
[ceph-client][DEBUG ] Hit:11 https://download.ceph.com/debian-jewel xenial InRelease
[ceph-client][DEBUG ] Fetched 124 kB in 7s (16.8 kB/s)
[ceph-client][DEBUG ] Reading package lists...
[ceph-client][INFO ] Running command: sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get --assume-yes -q --no-install-recommends install -o Dpkg::Options::=--force-confnew radosgw
[ceph-client][DEBUG ] Reading package lists...
[ceph-client][DEBUG ] Building dependency tree...
[ceph-client][DEBUG ] Reading state information...
[ceph-client][DEBUG ] radosgw is already the newest version (12.2.1-1xenial).
[ceph-client][DEBUG ] The following packages were automatically installed and are no longer required:
[ceph-client][DEBUG ] libboost-program-options1.58.0 libboost-random1.58.0 libboost-regex1.58.0
[ceph-client][DEBUG ] libcephfs1 libfcgi0ldbl libllvm3.8 libmircommon5 linux-headers-4.4.0-31
[ceph-client][DEBUG ] linux-headers-4.4.0-31-generic linux-image-4.4.0-31-generic
[ceph-client][DEBUG ] linux-image-extra-4.4.0-31-generic
[ceph-client][DEBUG ] Use 'sudo apt autoremove' to remove them.
[ceph-client][DEBUG ] 0 upgraded, 0 newly installed, 0 to remove and 48 not upgraded.
[ceph-client][INFO ] Running command: sudo ceph --version
[ceph-client][DEBUG ] ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
cephu@cephadmin:~/my-cluster$
看状态
cephu@cephadmin:~/my-cluster$ ceph -s
cluster:
id: d0aa5af1-4f8e-4953-9448-7f1b2448b8a5
health: HEALTH_OK
services:
mon: 3 daemons, quorum cephfsn2,mon1,cephfsn3
mgr: mon1(active), standbys: node2, node3
mds: cephfs-jlch-1/1/1 up {0=node1=up:active}
osd: 3 osds: 3 up, 3 in
rgw: 1 daemon active
data:
pools: 8 pools, 64 pgs
objects: 296 objects, 134 MB
usage: 4286 MB used, 79378 MB / 83664 MB avail
pgs: 64 active+clean
cephu@cephadmin:~/my-cluster$
CREATING THE CEPH OBJECT GATEWAY INSTANCE¶
admin:
cephu@cephadmin:~/my-cluster$ ceph-deploy rgw create ceph-client
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /home/cephu/.local/bin/ceph-deploy rgw create ceph-client
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] rgw : [('ceph-client', 'rgw.ceph-client')]
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f1dc4631bd8>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function rgw at 0x7f1dc4c7e140>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.rgw][DEBUG ] Deploying rgw, cluster ceph hosts ceph-client:rgw.ceph-client
[ceph-client][DEBUG ] connection detected need for sudo
[ceph-client][DEBUG ] connected to host: ceph-client
[ceph-client][DEBUG ] detect platform information from remote host
[ceph-client][DEBUG ] detect machine type
[ceph_deploy.rgw][INFO ] Distro info: Ubuntu 16.04 xenial
[ceph_deploy.rgw][DEBUG ] remote host will use systemd
[ceph_deploy.rgw][DEBUG ] deploying rgw bootstrap to ceph-client
[ceph-client][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.rgw][ERROR ] RuntimeError: config file /etc/ceph/ceph.conf exists with different content; use --overwrite-conf to overwrite
[ceph_deploy][ERROR ] GenericError: Failed to create 1 RGWs
cephu@cephadmin:~/my-cluster$ ceph-deploy --overwrite-conf rgw create ceph-client
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephu/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /home/cephu/.local/bin/ceph-deploy --overwrite-conf rgw create ceph-client
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] rgw : [('ceph-client', 'rgw.ceph-client')]
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fb62d844bd8>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function rgw at 0x7fb62de91140>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.rgw][DEBUG ] Deploying rgw, cluster ceph hosts ceph-client:rgw.ceph-client
[ceph-client][DEBUG ] connection detected need for sudo
[ceph-client][DEBUG ] connected to host: ceph-client
[ceph-client][DEBUG ] detect platform information from remote host
[ceph-client][DEBUG ] detect machine type
[ceph_deploy.rgw][INFO ] Distro info: Ubuntu 16.04 xenial
[ceph_deploy.rgw][DEBUG ] remote host will use systemd
[ceph_deploy.rgw][DEBUG ] deploying rgw bootstrap to ceph-client
[ceph-client][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph-client][WARNIN] rgw keyring does not exist yet, creating one
[ceph-client][DEBUG ] create a keyring file
[ceph-client][DEBUG ] create path recursively if it doesn't exist
[ceph-client][INFO ] Running command: sudo ceph --cluster ceph --name client.bootstrap-rgw --keyring /var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create client.rgw.ceph-client osd allow rwx mon allow rw -o /var/lib/ceph/radosgw/ceph-rgw.ceph-client/keyring
[ceph-client][INFO ] Running command: sudo systemctl enable ceph-radosgw@rgw.ceph-client
[ceph-client][WARNIN] Created symlink from /etc/systemd/system/ceph-radosgw.target.wants/ceph-radosgw@rgw.ceph-client.service to /lib/systemd/system/ceph-radosgw@.service.
[ceph-client][INFO ] Running command: sudo systemctl start ceph-radosgw@rgw.ceph-client
[ceph-client][INFO ] Running command: sudo systemctl enable ceph.target
[ceph_deploy.rgw][INFO ] The Ceph Object Gateway (RGW) is now running on host ceph-client and default port 7480
cephu@cephadmin:~/my-cluster$
CONFIGURING THE CEPH OBJECT GATEWAY INSTANCE¶
这里是配置web端口成 80 .
这里我就不做了，我没必要。修改也简单，看一下就明白了。
我的 ceph-client 的IP是 192.168.31.172，则： 我就直接到浏览器下打开 http://192.168.31.172:7480/
输出大体如下：
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Owner>
<ID>anonymous</ID>
<DisplayName/>
</Owner>
<Buckets/>
</ListAllMyBucketsResult>
好了，这一小节就到这里了。
vs code¶
Contents:
vs code 调试 nodejs fork¶
如 OpenX/OpenQuote2/tick_signaller.js 中，用了 fork 子进程。这个时候要使用 如下的方式。
首次调试¶
父进程加参数¶
this._procChild = cp.fork(__dirname + this.TSFile);
// this._procChild = cp.fork(__dirname + this.TSFile, [], {execArgv: ['--debug=5859']});
启动父进程¶
然后 终端上 node tick_signaller.js SZ
launch.json添加配置¶
然后，在 vs code 上, 打开 launch.json 后选择，调试/添加配置/ ，选择 “nodejs: 附加”, 然后在 添加配置
- ::
- “configurations”: [
- {
- “type”: “node”, “request”: “attach”, “name”: “Attach”, “port”: 5859 # 这里的 5859 必须是上面的 –debug=5859 一致。
},
F5 调试¶
然后 F5 调试吧。
再次调试¶
调试线束后，再次调试需要先开 父进程。
ubuntu¶
Contents:
update-kernel¶
Contents:
ubuntu16.04 update-kernel¶
env¶
192.168.31.118
192.168.31.119
step¶
浏览器打开 http://kernel.ubuntu.com/~kernel-ppa/mainline/
找到适合的 内核版本（这时v4.12), 进入，
找到合适的内核文件(linux-image-4.12.0-041200-generic_4.12.0-041200.201707022031_amd64.deb)
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.12/linux-image-4.12.0-041200-generic_4.12.0-041200.201707022031_amd64.deb
然后安装就可以了。
how-to-install-certificates-for-command-line¶
https://askubuntu.com/questions/645818/how-to-install-certificates-for-command-line
env¶
192.168.31.120
step¶
尝试直接 apt install
jlch@km:~$ sudo apt install ./CA.crt
[sudo] password for jlch:
Reading package lists... Done
E: Unsupported file ./CA.crt given on commandline
jlch@km:~$
不行，那尝试一下
jlch@km:~$ dpkg-query -L ca-certificates
/.
/etc
/etc/ssl
/etc/ssl/certs
/etc/ca-certificates
/etc/ca-certificates/update.d
/usr
/usr/sbin
/usr/sbin/update-ca-certificates
/usr/share
/usr/share/ca-certificates
/usr/share/ca-certificates/mozilla
... # 看到了好多 mozilla 的证书相关信息呀。
/usr/share/ca-certificates/mozilla/DigiCert_Global_Root_CA.crt
/usr/share/doc
/usr/share/doc/ca-certificates
/usr/share/doc/ca-certificates/examples
... # 好多 examples
/usr/share/doc/ca-certificates/examples/ca-certificates-local/README
/usr/share/doc/ca-certificates/changelog.gz
/usr/share/doc/ca-certificates/copyright
/usr/share/doc/ca-certificates/README.Debian
/usr/share/doc/ca-certificates/NEWS.Debian.gz
/usr/share/man
/usr/share/man/man8
/usr/share/man/man8/update-ca-certificates.8.gz
jlch@km:~$
那查一下 update-ca-certificates
jlch@km:~$ which update-ca-certificates
jlch@km:~$ sudo which update-ca-certificates
[sudo] password for jlch:
/usr/sbin/update-ca-certificates
jlch@km:~$
好了, 根据 https://askubuntu.com/questions/645818/how-to-install-certificates-for-command-line 来更新吧
jlch@km:~$ cd /usr/sbin/update-ca-certificates
jlch@km:/usr/share/ca-certificates/xxnet$ sudo cp /home/jlch/CA.crt .
jlch@km:/usr/share/ca-certificates/xxnet$ sudo /usr/sbin/update-ca-certificates
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
jlch@km:/usr/share/ca-certificates/xxnet$
好了，CA证书已经更新成功了。
测试¶
jlch@km:~$ export http_proxy=http://192.168.31.10:8087
jlch@km:~$ curl -sSL https://dl.k8s.io/release/stable.txt
jlch@km:~$
ubuntu desktop¶
Contents:
network¶
Contents:
sar宽带限制测试¶
蛟龙网络策略是保障2M带宽，最大10带宽。
但是我们要测试一下。
https://www.2cto.com/kf/201611/568876.html
env¶
A侧主机： 阿里云
B侧主机： 蛟龙
step¶
主要思路：用阿里云主机作带宽测试，A侧的接收速率rxkB/s应该约等于B侧出带宽的限制；
B侧主机 提供一个大文件供下载¶
A侧主机：terminal 1，模拟用户不断发送请求¶
负责下载
for((i=0;i<10;i++)); do (curl B侧IP/CentOS-7-x86_64-DVD-1511.iso > /dev/null 2>&1 &); done（模拟用户不断发送请求）
[tom@jlch_web_001 ~]$ curl http://api-cm.zhangtl.com:8000/draft/CentOS-7-x86_64-Minimal-1611.iso
A侧主机：terminal 2，查看网络设备的状态信息¶
sar -n DEV 2（每2秒查看网络设备的状态信息）
[tom@jlch_web_001 ~]$ sar -n DEV 2
10:57:27 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
10:57:29 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:29 AM eth1 659.50 280.00 268.49 162.34 0.00 0.00 0.00
10:57:29 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:29 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
10:57:31 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:31 AM eth1 644.00 290.00 207.79 173.37 0.00 0.00 0.00
10:57:31 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:31 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
10:57:33 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:33 AM eth1 831.50 361.00 198.42 172.36 0.00 0.00 0.00
10:57:33 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:33 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
10:57:35 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:35 AM eth1 1096.00 489.50 133.77 178.13 0.00 0.00 0.00
10:57:35 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:35 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
10:57:37 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10:57:37 AM eth1 443.50 178.00 273.62 170.70 0.00 0.00 0.00
10:57:37 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
防火墙把 403 转成 404¶
关闭规则¶
这个是原始错误
[tom@jlch_web_001 ~]$ curl -X POST "http://www.qliang.com.cn/transmitter-api/session" -H "accept: application/json" -H "content-type: application/json" -d "{ \"login_name\": \"htqilei\", \"login_pwd\": \"123456\"}"
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.8.0</center>
</body>
</html>
[tom@jlch_web_001 ~]$
开启规则¶
浏览器 打开 http://120.234.23.98:8000/transmitter-api-v2/session ， 则返回
Sorry, Page Not Found
从内网取
[root@test_240 conf.d]# curl -X POST "http://10.10.11.20:8000/transmitter-api-v2/session" -H "accept: application/json" -H "content-type: application/json" -d "{ \"login_name\": \"xiaozhao\", \"login_pwd\": \"wdxiaozhao\"}" -v
* About to connect() to 10.10.11.20 port 8000 (#0)
* Trying 10.10.11.20...
* Connected to 10.10.11.20 (10.10.11.20) port 8000 (#0)
> POST /transmitter-api-v2/session HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.10.11.20:8000
> accept: application/json
> content-type: application/json
> Content-Length: 54
>
* upload completely sent off: 54 out of 54 bytes
< HTTP/1.1 404 Not Found
< Server: nginx/1.8.0
< Date: Fri, 08 Dec 2017 06:47:01 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 63
< Connection: keep-alive
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, POST, PUT, UPDATE, DELETE, PATCH, OPTIONS
< Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, x-access-token
<
* Connection #0 to host 10.10.11.20 left intact
{"code":"ResourceNotFound","message":"/session does not exist"}
从阿里云过外网取
[tom@jlch_web_001 ~]$ curl -X POST "http://120.234.23.98:8000/transmitter-api-v2/session" -H "accept: application/json" -H "content-type: applicatin_name\": \"xiaozhao\", \"login_pwd\": \"wdxiaozhao\"}" -v
* About to connect() to 120.234.23.98 port 8000 (#0)
* Trying 120.234.23.98...
* Connected to 120.234.23.98 (120.234.23.98) port 8000 (#0)
> POST /transmitter-api-v2/session HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 120.234.23.98:8000
> accept: application/json
> content-type: application/json
> Content-Length: 54
>
* upload completely sent off: 54 out of 54 bytes
< HTTP/1.1 404 Not Found
< Content-Type: text/html
< Expires: 0
< Cache-control: private
< Content-Length: 334
<
* Connection #0 to host 120.234.23.98 left intact
Sorry, Page Not Found
你会发现，这里
< HTTP/1.1 404 Not Found
< Content-Type: text/html
< Expires: 0
< Cache-control: private
< Content-Length: 334
<
* Connection #0 to host 120.234.23.98 left intact
Sorry, Page Not Found
是防火墙篡改了
< Server: nginx/1.8.0
< Date: Fri, 08 Dec 2017 06:47:12 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 63
< Connection: keep-alive
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, POST, PUT, UPDATE, DELETE, PATCH, OPTIONS
< Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, x-access-token
哈哈，防火墙，还是很666的…..
来一个正常的正确的¶
[tom@jlch_web_001 ~]$ curl -X POST "http://120.234.23.98:8000/transmitter-api-v2/v2/session" -H "accept: application/json" -H "content-type: application/json" -d "{ \"login_name\": \"xiaozhao\", \"login_pwd\": \"wdxiaozhao\"}"
{"token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJ4aWFvemhhbyIsImV4cCI6MTUxMjcyODYyMjM1MywiY2xpZW50Ijoid2QiLCJ1c2VyX2lkIjoyLCJ1c2VyX25hbWUiOiLlsI_mmK0ifQ.Hr7HxvkdlDm_qg7DU1pI3eJ4-2dCQJF9GZr3BFkWUS4","client_code":"wd","user_name":"å°æ","user_id":2}[tom@jlch_web_001 ~]$
[tom@jlch_web_001 ~]$
https status code¶
Hypertext Transfer Protocol (HTTP) Status Code Registry
https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
内网穿透，可以让全世界访问家用电脑里的网站¶
内网穿透，可以让全世界访问家用电脑里的网站
https://gitee.com/wapai/chuantou
env¶
192.168.1.104 client, home
120.25.204.216 server, aliyun
step¶
server¶
sudo systemctl status firewalld.service
export PATH=$PATH:/home/tom/go/bin
git clone https://gitee.com/wapai/chuantou
cd chuantou/
ls
go run server.go -localPort 3002 -remotePort 20012
sudo firewall-cmd --permanent --add-port=3002/tcp
sudo firewall-cmd --reload
go run server.go -localPort 3002 -remotePort 20012
client¶
export PATH=$PATH:/home/jlch/go/bin
git clone https://gitee.com/wapai/chuantou
cd chuantou/
ls
go run client.go -host 120.25.204.216 -localPort 80 -remotePort 20012
这样, 在浏览器中, 通过 serverIP:3002 可以得到 clientIP:80 的页面.
通过 wget 获得本机IP¶
有的时候需要知道自己的通过哪个外网IP出去
env¶
与 中泰张锐 远程, 发现对方机器访问不了我们的服务, 那么, 想通过我们的机器来抓取中泰发过来的数据包, 那么就需要中泰过来的IP.
step¶
方法有下列几种:
- 远程ssh登陆, 看ssh日志得IP
- 访问服务, 通过其它日志(防火墙, nginx日志)得到IP
- 通过访问其它服务(如 返回IP服务), 得IP
现在我们介绍 “通过访问其它服务” 的方法
因为我们正常访问 http://ip.chinaz.com/ 可以直接得到我们自己的外网IP, 所以, 我们可以在机器上 wget 得到这个IP.
[tom@jlch_web_001 ~]$ wget ip.chinaz.com
--2018-01-16 15:12:07-- http://ip.chinaz.com/
Resolving ip.chinaz.com (ip.chinaz.com)... 122.228.242.37, 117.25.139.23
Connecting to ip.chinaz.com (ip.chinaz.com)|122.228.242.37|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 33569 (33K) [text/html]
Saving to: ‘index.html.4’
100%[==================================================================================================================================================================>] 33,569 178KB/s in 0.2s
2018-01-16 15:12:07 (178 KB/s) - ‘index.html.4’ saved [33569/33569]
[tom@jlch_web_001 ~]$ cat index.html.4 | grep fz24
<p class="getlist pl10"><span>您来自：</span>120.25.204.216 <span class="pl10">所在区域：</span>浙江省杭州市 阿里云BGP数据中心<a href="http://tool.chinaz.com/contact" target="_blank" class="col-blue02 pl5">(纠错)</a></p>
<dd class="fz24">120.25.204.216</dd>
<p class="info"><span>© CopyRight 2002-2018, CHINAZ.COM, Inc.All Rights Reserved.</span><span>闽ICP备08105208号</span><span>增值电信业务经营许可证闽B2-20120007号</span><a href="http://www.wy.cn" rel="nofollow" target="_blank" class="col-gray02">服务器资源由唯一网络赞助</a></p>
[tom@jlch_web_001 ~]$
完美, 这样我们就得到了IP.
java¶
Contents:
java install¶
http://blog.csdn.net/wp500/article/details/41040213
可以获取开源版本的jre在：http://openjdk.java.net 或者你可以在官网下载Oracle jdk版本：http://www.oracle.com/technetwork/java/index.html 一旦jre已经成功在你的系统中安装完成，我们就可以继续了
tools¶
Contents:
windows tools¶
Contents:
PostgreSQL¶
marmelab.com 里有 postgresql UI 管理工具
Jolt¶
Contents:
Jolt获奖图书¶
https://www.douban.com/doulist/41493715/
https://www.douban.com/doulist/13728837/
Jolt大奖素有“软件业界的奥斯卡”之美誉，共设通用类图书、技术类图书、语言和开发环境、框架库和组件、开发者网站等十余个分类，每个分类设有一个“震撼奖”（Jolt Award）和三个“生产力奖”（Productivity Award）。 一项技术产品只有在获得了Jolt奖之后才能真正成为行业的主流，一本技术书籍只有在获得了Jolt奖之后才能真正奠定经典的地位
赞助商Jolt可乐的广告词是“震撼全世界”，Jolt奖就让我们看到，是谁在震撼着我们今天的世界。虽然Jolt并不起决定作用，但代表了某种技术趋势与潮流。
官网： http://www.drdobbs.com/joltawards
维基： https://en.wikipedia.org/wiki/Jolt_Awards
二十多年来没有间断的Jolt大奖涵盖了年度经典计算机图书。
从1991年至2014年每届至少评出一个“震撼奖”（Jolt Award）、 三个“生产力奖”（Productivity Award）， 2003年至2009年几届按“通用类”（General）和“技术类”（Technical）两大类评奖， 后期几年名单里包含了“提名奖”（Finalist Award）。
推荐语：
一项技术产品只有在获得了Jolt奖之后才能真正成为行业的主流， 一本技术书籍只有在获得了Jolt奖之后才能真正奠定经典的地位。 赞助商Jolt可乐的广告词是“震撼全世界”，Jolt奖就让我们看到，是谁在震撼着我们今天的世界。虽然Jolt并不起决定作用，但代表了某种技术趋势与潮流。
科学上网¶
Contents:
科学上网¶
free¶
https://github.com/XX-net/XX-Net
https://github.com/Alvin9999/new-pac/wiki
https://blog.phpgao.com/slip_across_gfw.html
google 搜索¶
里面有google 搜索 https://freessr.win/
收费¶
line 1¶
翻墙软件排行榜
商家名称 免费套餐 付费套餐 官方网站 1、加速度 注册后可申请2元试用，可同时使用2条线路，24小时不满意退款 (推荐 ) 最低￥10/月 官网直达 2、socketpro 注册后可申请2元试用，有效时间1天 (推荐 ) 最低￥12.5/月 官网直达 3、妙速 注册可获得500M流量，每个新用户均送10元余额, 最低￥18/月 官网直达 4、云影SS 3G流量，有效期7天 最低￥20/月 官网直达
我基于这个 加速度 买了。哈哈。还不错哟。 http://center.36fy.com/clientarea.php?action=productdetails&id=122802
使用加速度科学上网¶
env¶
192.168.31.239 windows10
注册缴费¶
下载客户端，安装，把
try01.jiasudu.biz 23479 mima
写好。
然后到常见问题，一步步来就行了。
这里的客户端是写死了 127.0.0.1 的。所以我们 telnet 127.0.0.1 1080 是OK的，但是 telnet 192.168.31.239 1080 是不行的。
那么怎么让其它用户使用呢？
其实就是要实现 将socks5代理转为http代理 。好吧，百度咯。
使用Privoxy将socks5代理转为http代理¶
详见 privoxy-socks5-http.rst
使用Privoxy将socks5代理转为http代理¶
https://blog.phpgao.com/privoxy-shadowsocks.html
https://tzrgaga.github.io/2017/04/12/forward-socks-by-privoxy/
修改配置¶
找到Privoxy的配置文件位置，默认是在C:Program Files (x86)Privoxyconfig.txt
然后使用编辑器来编辑它，底部加入一行forward-socks5 / 127.0.0.1:1080 .
加
forward-socks5 / 127.0.0.1:1080 .
修改 绑定IP
#listen-address 127.0.0.1:8118
# 0:8118， *:8118, [::1]:8118, 都无效。
listen-address 0.0.0.0:8118
测试¶
本机确实是有在 LISTENING, 看：
C:\Users\k>netstat -anto | grep 8118
TCP 127.0.0.1:8118 0.0.0.0:0 LISTENING 9648 InHost
TCP 127.0.0.1:8118 127.0.0.1:54831 ESTABLISHED 9648 InHost
TCP 127.0.0.1:54831 127.0.0.1:8118 ESTABLISHED 15868 InHost
TCP [::1]:8118 [::]:0 LISTENING 16524 InHost
C:\Users\k>
使用另一台机器，如 linux
export http_proxy=http://192.168.31.239:8118
export https_proxy=http://92.168.31.239:8118
wget www.google.com
curl -sSL https://dl.k8s.io/release/stable.txt
linux下使用privoxy将socks转为http代理¶
linux下使用privoxy将socks转为http代理
https://www.cnblogs.com/liuxuzzz/p/5324749.html
环境：ubuntu 15.04 工具：shadowsocks（socks5），privoxy（http） privoxy有将socks代理转为http代理的功能。 1.开启shadowsocks，socks代理地址为127.0.0.1:1080。 2.安装privoxy。
$ sudo apt-get install privoxy
3.更改provoxy配置，位置在“/etc/privoxy/config”。 $ sudo vim /etc/privoxy/config 在里面添加一条：
# 在 froward-socks4下面添加一条socks5的，因为shadowsocks为socks5，
# 地址是127.0.0.1:1080。注意他们最后有一个“.”
# forward-socks4 / socks-gw.example.com:1080 .
forward-socks5 / 127.0.0.1:1080 .
# 下面还存在以下一条配置，表示privoxy监听本机8118端口，
# 把它作为http代理，代理地址为 http://localhost.8118/ 。
# 可以把地址改为 0.0.0.0:8118，表示外网也可以通过本机IP作http代理。
# 这样，你的外网IP为1.2.3.4，别人就可以设置 http://1.2.3.4:8118/ 为http代理。
listen-address localhost:8118
4.然后重启privoxy。
$ sudo systemctl restart privoxy.serivce
5.现在你就可以使用http代理了，如果你要给系统设置http代理，就在~/.bashrc里添加一条http_proxy配置。
$ vim ~/.bashrc
添加：
export http_proxy=http://127.0.0.1:8118/
然后使用source是它立刻生效。
$ source ~/.bashrc
ubuntu install shadowsocks¶
http://abc.36fy.com/knowledgebase/9/Linux.html
本文将以Ubuntu系统为例，介绍下Linux系统设置使用加速度帐号教程； 更多linux操作系统使用教程请参考： https://github.com/shadowsocks/shadowsocks-qt5/wiki 第1步：下载安装客户端shadowsocks-qt5： 客户端需要通过PPA源安装，仅支持Ubuntu 14.04或更高版本； sudo add-apt-repository ppa:hzwhuang/ss-qt5 sudo apt-get update sudo apt-get install shadowsocks-qt5 安装过程如果遇到 libappindicator1 依赖问题（dependency problems），而 libappindicator1 又遇到 libindicator7 依赖的解决办法。一并安装 libappindicator1 libindicator7 依赖，再重新安装shadowsocks-qt5； sudo apt-get-f install libappindicator1 libindicator7
第2步：配置客户端： 安装完成后，我们可以在应用程序里面找到 ShadowSocks-Qt5客户端，启动，在弹出的界面空白处右键add添加配置你的帐号信息，如下图：
配置方法与windows客户端配置是差不多的； Server Address填写服务器ip地址；Server Port填写服务端口；Password填写连接密码；Encryption Method选择AES-256-CFB，其他选项均保持默认不变。 说明：服务器IP地址、服务端口、连接密码信息，购买了加速度帐号后由加速度提供！ 注：有朋友反映安装后不知如何启动客户端，你可以直接在用命令行启动客户端： sudo ss-qt5
或者试试直接win键呼出应用搜索，输入Shadowsocks就出来了。 第3步：配置下浏览器代理： 以大家常用的浏览器为例： 谷歌浏览器 有2种方案，我们更推荐方案2， 方案1： a：打开“局域网LAN设置”选项（即：打开浏览器，依次点击设置 ，显示高级设置 ，更改代理服务器设置，在弹出的Internet选项窗口中，依次点击连接，局域网设置，弹出局域网LAN设置）； b：在弹出的“局域网LAN设置”选项中，代理服务器的复选框打勾，设置地址：127.0.0.1，设置端口：1080，点击确定； 完成上面的a，b步骤，我们就可以安全愉快的上网了； 方案2： 方案1只能使用全局模式，没有pac模式，如果要实现pac模式和全局模式的切换，你可以使用谷歌浏览器+扩展的方法，具体教程请查看： http://abc.36fy.com/knowledgebase/22/plus.html，这篇教程是windows下的教程，但和linux使用教程是完全一样的，自行下载linux系统的chrome浏览器即可，扩展插件是通用的。 firefox 浏览器： 有2种方案，我们同样推荐方案2， 方案1： a：打开firefox浏览器代理连接设置选项（即：打开 firefox 浏览器，依次点击 编辑 ，首选项 ，高级 ，网络 ，设置）； b：在打开的窗口中选择【手动配置代理】，默认有个http代理，这个一定要清空，在 socks主机中填写 127.0.0.1 ，端口填写1080，选择socks v5， 然后点击确定，如下图：
完成上面的a，b步骤，我们就可以安全愉快的上网了； 方案2： 方案1只能使用全局模式，没有pac模式，如果要实现pac模式和全局模式的切换，你可以使用Firefox+扩展的方法，具体教程请查看： http://abc.36fy.com/knowledgebase/38/Firefoxplus.html，这篇教程是windows下的教程，但和linux使用教程是完全一样的，自行下载linux系统的Firefox浏览器即可，扩展插件是通用的。 其他浏览器 与谷歌浏览器的方案1是一样的： 打开“局域网LAN设置”选项，在弹出的“局域网LAN设置”选项中，代理服务器的复选框打勾，设置地址：127.0.0.1，设置端口：1080，点击确定即可，仅支持全局模式。
postman¶
Contents:
markdown¶
reStructuredText(rst)快速入门语法说明¶
https://www.cnblogs.com/seayxu/p/5603876.html
http://zh-sphinx-doc.readthedocs.io/en/latest/rest.html#id78
Træfik¶
Træfik
Træfik (pronounced like traffic) is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease.
OSSEC and OSSIM¶
https://www.alienvault.com/forums/discussion/1335/what-is-the-difference-between-ossec-and-ossim
meteor¶
THE FASTEST WAY TO BUILD JAVASCRIPT APPS
Meteor is an open source platform for web, mobile, and desktop.
DSL编程技术的介绍¶
wework¶
【36氪新闻】来自十家独角兽公司的十条创业经验
https://36kr.com/p/5108940.html
联合办公空间 wework
atlassian 团队协作软件商
lyft 又一个 UBER
twillio 云通讯平台
New Relic 监控工具, 为软件工程师所用
HubSpot 中小企业数字营销工具
Palantir 侦测犯罪行为公司
利用 new relic 监控 express 项目¶
gist¶
https://www.zhihu.com/question/21343711
http://www.roughdraft.io/
GistBox
Lepton, 开源, Gist desktop client
gisto
zabbix¶
Contents:
zabbix 常用命令¶
安装：
sudo rpm -ivh http://repo.zabbix.com/zabbix/3.4/rhel/7/x86_64/zabbix-release-3.4-1.el7.centos.noarch.rpm
sudo yum install -y zabbix-agent
sudo systemctl start zabbix-agent.service
sudo systemctl restart zabbix-agent.service
sudo systemctl status zabbix-agent.service
开放端口：
sudo firewall-cmd --permanent --add-port=10050/tcp; sudo firewall-cmd --reload; sudo firewall-cmd --permanent --list-port
sudo ufw allow 10050
安装插件：
sudo yum install -y sysstat
sudo systemctl status sysstat
sudo systemctl enable sysstat
修改配置：
vi /etc/zabbix/zabbix-agent.conf
## 修改下列配置
# Server=10.10.11.20
Server=zabbix_server
ServerActive=10.10.11.20
Hostname=本机IP
## 新增下列配置
##Linux磁盘IOPS
##---------------------------------
#每秒合并读操作的次数，读相邻块(block)的两个请求被合并成一个IO
UserParameter=disk.rrqm[*],iostat -dx | sed -n '4p' | awk '{print $$2}'
#每秒合并写操作的次数
UserParameter=disk.wrqm[*],iostat -dx | sed -n '4p' | awk '{print $$3}'
#每秒读操作的次数
UserParameter=disk.rps[*],iostat -dx | sed -n '4p' | awk '{print $$4}'
#每秒写操作次数
UserParameter=disk.wps[*],iostat -dx | sed -n '4p' | awk '{print $$5}'
#每秒读写次数之和
UserParameter=disk.iops[*],iostat -d | sed -n '4p' | awk '{print $$2}'
#每秒读的数据量(MB为 单位)
UserParameter=disk.rMBps[*],iostat -dxm | sed -n '4p' | awk '{print $$6}'
#每秒写的数据量(MB为单位)
UserParameter=disk.wMBps[*],iostat -dxm | sed -n '4p' | awk '{print $$7}'
#每个I/O平均所需的时间(包括等待时间和处理时间，毫秒为单位)
UserParameter=disk.await[*],iostat -dx | sed -n '4p' | awk '{print $$10}'
#每个读操作平均所需的时间包括等待时间和处理时间，毫秒为单位)
UserParameter=disk.rawait[*],iostat -dx | sed -n '4p' | awk '{print $$11}'
#每个写操作平均所需的时间(包括等待时间和处理时间，毫秒为单位)
UserParameter=disk.wawait[*],iostat -dx | sed -n '4p' | awk '{print $$12}'
#每个IO请求的平均处理时间(毫秒为单位)
UserParameter=disk.svctm[*],iostat -dx | sed -n '4p' | awk '{print $$13}'
#采样周期内用于IO操作的时间比率，即IO队列非空的时间比率
UserParameter=disk.util[*],iostat -dx | sed -n '4p' | awk '{print $$14}'
##---------------------------------
以下机器全都安装好了。
## centos
192.168.31.249
10.10.15.181
10.10.15.240
10.10.12.17
10.10.12.18
10.10.12.16
10.10.13.100
10.10.13.101
10.10.13.12
10.10.13.11
10.10.13.110
10.10.12.10
10.10.11.20
10.10.13.10
## ubuntu
12.19
13.103
## over
zabbix 配置 debug level¶
在 /etc/zabbix/zabbix_agentd.conf 中设置 DebugLevel=3
[root@mariadb-node-0 ~]# cat /etc/zabbix/zabbix_agentd.conf | grep Debug
### Option: DebugLevel
DebugLevel=3
[root@mariadb-node-0 ~]#
DebugLevel=5 则输出下面这样的日志：
[root@mariadb-node-0 ~]# tail -f /var/log/zabbix/zabbix_agentd.log
148519:20171010:103802.537 __zbx_zbx_setproctitle() title:'collector [processing data]'
148519:20171010:103802.537 In update_cpustats()
148519:20171010:103802.538 End of update_cpustats()
148519:20171010:103802.538 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
148520:20171010:103802.563 __zbx_zbx_setproctitle() title:'listener #1 [processing request]'
148520:20171010:103802.563 Requested [disk.rawait[sda]]
148520:20171010:103802.563 In zbx_popen() command:'iostat -dx | sed -n '4p' | awk '{print $11}''
148520:20171010:103802.564 End of zbx_popen():7
148997:20171010:103802.564 zbx_popen(): executing script
148520:20171010:103802.571 In zbx_waitpid()
148520:20171010:103802.571 zbx_waitpid() exited, status:0
148520:20171010:103802.571 End of zbx_waitpid():148997
148520:20171010:103802.571 EXECUTE_STR() command:'iostat -dx | sed -n '4p' | awk '{print $11}'' len:4 cmd_result:'3.18'
148520:20171010:103802.571 Sending back [3.18]
148520:20171010:103802.571 __zbx_zbx_setproctitle() title:'listener #1 [waiting for connection]'
148525:20171010:103802.581 In send_buffer() host:'zabbix_server' port:10051 entries:0/100
148525:20171010:103802.581 End of send_buffer():SUCCEED
148525:20171010:103802.581 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
^C^C
DebugLevel=3 则输出下面这样的日志：
[root@mariadb-node-0 ~]# tail -f /var/log/zabbix/zabbix_agentd.log
150001:20171010:103908.063 IPv6 support: YES
150001:20171010:103908.063 TLS support: YES
150001:20171010:103908.063 **************************
150001:20171010:103908.063 using configuration file: /etc/zabbix/zabbix_agentd.conf
150001:20171010:103908.064 agent #0 started [main process]
150002:20171010:103908.065 agent #1 started [collector]
150003:20171010:103908.065 agent #2 started [listener #1]
150004:20171010:103908.065 agent #3 started [listener #2]
150005:20171010:103908.066 agent #4 started [listener #3]
150006:20171010:103908.066 agent #5 started [active checks #1]
^C
[root@mariadb-node-0 ~]#
zabbix-debug-1¶
在 10.10.12.17 这台机器上，zabbix-agent 服务报错了
[root@mariadb-node-0 zabbix]# systemctl start zabbix-agent.service
Job for zabbix-agent.service failed because a configured resource limit was exceeded. See "systemctl status zabbix-agent.service" and "journalctl -xe" for details.
[root@mariadb-node-0 zabbix]# journalctl -xe
10月 10 10:04:08 mariadb-node-0 systemd[1]: Starting Zabbix Agent...
-- Subject: Unit zabbix-agent.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit zabbix-agent.service has begun starting up.
10月 10 10:04:08 mariadb-node-0 systemd[1]: PID file /var/run/zabbix/zabbix_agentd.pid not readable (yet?) after start.
10月 10 10:04:08 mariadb-node-0 systemd[1]: zabbix-agent.service: Supervising process 124777 which is not our child. We'll most like
10月 10 10:04:08 mariadb-node-0 systemd[1]: zabbix-agent.service: main process exited, code=exited, status=1/FAILURE
10月 10 10:04:08 mariadb-node-0 kill[124779]: kill: 向 124777 发送信号失败: 没有那个进程
10月 10 10:04:08 mariadb-node-0 systemd[1]: zabbix-agent.service: control process exited, code=exited status=1
10月 10 10:04:08 mariadb-node-0 systemd[1]: Failed to start Zabbix Agent.
-- Subject: Unit zabbix-agent.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit zabbix-agent.service has failed.
--
-- The result is failed.
10月 10 10:04:08 mariadb-node-0 systemd[1]: Unit zabbix-agent.service entered failed state.
10月 10 10:04:08 mariadb-node-0 systemd[1]: zabbix-agent.service failed.
10月 10 10:04:18 mariadb-node-0 systemd[1]: zabbix-agent.service holdoff time over, scheduling restart.
10月 10 10:04:18 mariadb-node-0 systemd[1]: Starting Zabbix Agent...
-- Subject: Unit zabbix-agent.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit zabbix-agent.service has begun starting up.
10月 10 10:04:18 mariadb-node-0 systemd[1]: PID file /var/run/zabbix/zabbix_agentd.pid not readable (yet?) after start.
10月 10 10:04:18 mariadb-node-0 systemd[1]: zabbix-agent.service: Supervising process 125011 which is not our child. We'll most like
10月 10 10:04:18 mariadb-node-0 systemd[1]: Started Zabbix Agent.
-- Subject: Unit zabbix-agent.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit zabbix-agent.service has finished starting up.
--
-- The start-up result is done.
10月 10 10:04:18 mariadb-node-0 systemd[1]: zabbix-agent.service: main process exited, code=exited, status=1/FAILURE
10月 10 10:04:18 mariadb-node-0 kill[125014]: 用法：
10月 10 10:04:18 mariadb-node-0 kill[125014]: kill [选项] <pid|名称> [...]
10月 10 10:04:18 mariadb-node-0 kill[125014]: 选项：
10月 10 10:04:18 mariadb-node-0 kill[125014]: -a, --all 不限于只对和当前进程的用户 id 相同的进程进行
10月 10 10:04:18 mariadb-node-0 kill[125014]: 名称-进程id 转换
10月 10 10:04:18 mariadb-node-0 kill[125014]: -s, --signal <信号> 发送指定的信号
10月 10 10:04:18 mariadb-node-0 kill[125014]: -q, --queue <信号> 使用 sigqueue(2) 代替 kill(2)
10月 10 10:04:18 mariadb-node-0 systemd[1]: zabbix-agent.service: control process exited, code=exited status=1
10月 10 10:04:18 mariadb-node-0 kill[125014]: -p, --pid 打印 pid 而不向它们发送信号
10月 10 10:04:18 mariadb-node-0 kill[125014]: -l, --list [=<信号>] 列出信号名，或将一个信号转换为名称
10月 10 10:04:18 mariadb-node-0 kill[125014]: -L, --table 列出信号名和数值
10月 10 10:04:18 mariadb-node-0 kill[125014]: -h, --help 显示此帮助并退出
10月 10 10:04:18 mariadb-node-0 kill[125014]: -V, --version 输出版本信息并退出
10月 10 10:04:18 mariadb-node-0 kill[125014]: 更多信息请参阅 kill(1)。
10月 10 10:04:18 mariadb-node-0 systemd[1]: Unit zabbix-agent.service entered failed state.
10月 10 10:04:18 mariadb-node-0 systemd[1]: zabbix-agent.service failed.
[root@mariadb-node-0 zabbix]#
查进程，查日志
[root@mariadb-node-0 zabbix]# ps -ef | grep zabbix
root 132576 131957 0 10:15 pts/2 00:00:00 grep --color=auto zabbix
[root@mariadb-node-0 zabbix]# tail -f /var/log/zabbix/zabbix_agentd.log
zabbix_agentd [131649]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [131712]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [131939]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [131948]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [132150]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [132279]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [132288]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [132456]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [132579]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [132586]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_agentd [132875]: cannot open log: cannot create semaphore set: [28] No space left on device
^C
[root@mariadb-node-0 zabbix]# tail -f /var/log/messages
Oct 10 10:15:48 mariadb-node-0 kill: -q, --queue <信号> 使用 sigqueue(2) 代替 kill(2)
Oct 10 10:15:48 mariadb-node-0 systemd: zabbix-agent.service: control process exited, code=exited status=1
Oct 10 10:15:48 mariadb-node-0 kill: -p, --pid 打印 pid 而不向它们发送信号
Oct 10 10:15:48 mariadb-node-0 kill: -l, --list [=<信号>] 列出信号名，或将一个信号转换为名称
Oct 10 10:15:48 mariadb-node-0 kill: -L, --table 列出信号名和数值
Oct 10 10:15:48 mariadb-node-0 kill: -h, --help 显示此帮助并退出
Oct 10 10:15:48 mariadb-node-0 kill: -V, --version 输出版本信息并退出
Oct 10 10:15:48 mariadb-node-0 kill: 更多信息请参阅 kill(1)。
Oct 10 10:15:48 mariadb-node-0 systemd: Unit zabbix-agent.service entered failed state.
Oct 10 10:15:48 mariadb-node-0 systemd: zabbix-agent.service failed.
Oct 10 10:15:59 mariadb-node-0 systemd: zabbix-agent.service holdoff time over, scheduling restart.
Oct 10 10:15:59 mariadb-node-0 systemd: Starting Zabbix Agent...
Oct 10 10:15:59 mariadb-node-0 systemd: PID file /run/zabbix/zabbix_agentd.pid not readable (yet?) after start.
Oct 10 10:15:59 mariadb-node-0 systemd: zabbix-agent.service: Supervising process 132893 which is not our child. We'll most likely not notice when it exits.
Oct 10 10:15:59 mariadb-node-0 systemd: Started Zabbix Agent.
Oct 10 10:15:59 mariadb-node-0 systemd: zabbix-agent.service: main process exited, code=exited, status=1/FAILURE
Oct 10 10:15:59 mariadb-node-0 kill: 用法：
Oct 10 10:15:59 mariadb-node-0 kill: kill [选项] <pid|名称> [...]
Oct 10 10:15:59 mariadb-node-0 kill: 选项：
Oct 10 10:15:59 mariadb-node-0 kill: -a, --all 不限于只对和当前进程的用户 id 相同的进程进行
Oct 10 10:15:59 mariadb-node-0 kill: 名称-进程id 转换
Oct 10 10:15:59 mariadb-node-0 kill: -s, --signal <信号> 发送指定的信号
Oct 10 10:15:59 mariadb-node-0 kill: -q, --queue <信号> 使用 sigqueue(2) 代替 kill(2)
Oct 10 10:15:59 mariadb-node-0 kill: -p, --pid 打印 pid 而不向它们发送信号
Oct 10 10:15:59 mariadb-node-0 kill: -l, --list [=<信号>] 列出信号名，或将一个信号转换为名称
Oct 10 10:15:59 mariadb-node-0 kill: -L, --table 列出信号名和数值
Oct 10 10:15:59 mariadb-node-0 kill: -h, --help 显示此帮助并退出
Oct 10 10:15:59 mariadb-node-0 kill: -V, --version 输出版本信息并退出
Oct 10 10:15:59 mariadb-node-0 kill: 更多信息请参阅 kill(1)。
Oct 10 10:15:59 mariadb-node-0 systemd: zabbix-agent.service: control process exited, code=exited status=1
Oct 10 10:15:59 mariadb-node-0 systemd: Unit zabbix-agent.service entered failed state.
Oct 10 10:15:59 mariadb-node-0 systemd: zabbix-agent.service failed.
^C
[root@mariadb-node-0 zabbix]#
出现这种错误，我们要去查一下，到底为什么会
zabbix_agentd [132875]: cannot open log: cannot create semaphore set: [28] No space left on device和
Oct 10 10:15:59 mariadb-node-0 systemd: PID file /run/zabbix/zabbix_agentd.pid not readable (yet?) after start.查一下硬盘
df -h
发现是OK的。
- 检查，是不是真的
No space left on device
我们用root 用户，在 /run/zabbix/ 下 创建文件，
sudo -u zabbix echo 'hello' > /run/zabbix/zabbix_agentd.pid ## 这里的命令是 sudo -u zabbix, 不是 sudo su - zabbix
cat /run/zabbix/zabbix_agentd.pid
cd /run/zabbix/
ls
ls -la ## 过了一会，发现消失了。
sudo -u root echo 'hello' > /run/zabbix/zabbix_agentd.pid ## 切成 root
cat zabbix_agentd.pid
ls
ls ## ## 过了一会，发现消失了。
sudo -u root echo 'hello' > /run/zabbix/a.txt ## 切成 root
cat a.txt
ls
ls ## ## 过了一会，发现消失了。
到底是什么自动删除了它呢？
- google了一下，找到，原来还真的是内存文件系统，具体网址找不到了。
命令 ipcs 查看 命令 ipcrm 删除
[jlch@mariadb-node-0 zabbix]$ which ipcs
/usr/bin/ipcs
[jlch@mariadb-node-0 zabbix]$
[root@mariadb-node-0 zabbix]# ipcs
--------- 消息队列 -----------
键 msqid 拥有者 权限 已用字节数 消息
------------ 共享内存段 --------------
键 shmid 拥有者 权限 字节 nattch 状态
0x00000000 65536 tom 600 4194304 2 目标
0x00000000 163841 tom 600 4194304 2 目标
0x00000000 262146 tom 600 1048576 2 目标
0x00000000 360451 tom 600 4194304 2 目标
0x00000000 458756 tom 600 393216 2 目标
--------- 信号量数组 -----------
键 semid 拥有者 权限 nsems
0x00000000 98304 zabbix 600 13
0x00000000 131073 zabbix 600 13
0x00000000 163842 zabbix 600 13
0x00000000 196611 zabbix 600 13
0x00000000 229380 zabbix 600 13
0x00000000 262149 zabbix 600 13
0x00000000 294918 zabbix 600 13
0x00000000 327687 zabbix 600 13
0x00000000 360456 zabbix 600 13
0x00000000 393225 zabbix 600 13
0x00000000 425994 zabbix 600 13
0x00000000 458763 zabbix 600 13
0x00000000 491532 zabbix 600 13
0x00000000 524301 zabbix 600 13
0x00000000 557070 zabbix 600 13
0x00000000 589839 zabbix 600 13
0x00000000 622608 zabbix 600 13
0x00000000 655377 zabbix 600 13
0x00000000 688146 zabbix 600 13
0x00000000 720915 zabbix 600 13
0x00000000 753684 zabbix 600 13
0x00000000 786453 zabbix 600 13
0x00000000 819222 zabbix 600 13
0x00000000 851991 zabbix 600 13
0x00000000 884760 zabbix 600 13
0x00000000 917529 zabbix 600 13
0x00000000 950298 zabbix 600 13
0x00000000 983067 zabbix 600 13
0x00000000 1015836 zabbix 600 13
0x00000000 1048605 zabbix 600 13
0x00000000 1081374 zabbix 600 13
0x00000000 1114143 zabbix 600 13
0x00000000 1146912 zabbix 600 13
0x00000000 1179681 zabbix 600 13
0x00000000 1212450 zabbix 600 13
0x00000000 1245219 zabbix 600 13
0x00000000 1277988 zabbix 600 13
0x00000000 1310757 zabbix 600 13
0x00000000 1343526 zabbix 600 13
0x00000000 1376295 zabbix 600 13
0x00000000 1409064 zabbix 600 13
0x00000000 1441833 zabbix 600 13
0x00000000 1474602 zabbix 600 13
0x00000000 1507371 zabbix 600 13
0x00000000 1540140 zabbix 600 13
0x00000000 1572909 zabbix 600 13
0x00000000 1605678 zabbix 600 13
0x00000000 1638447 zabbix 600 13
0x00000000 1671216 zabbix 600 13
0x00000000 1703985 zabbix 600 13
0x00000000 1736754 zabbix 600 13
0x00000000 1769523 zabbix 600 13
0x00000000 1802292 zabbix 600 13
0x00000000 1835061 zabbix 600 13
0x00000000 1867830 zabbix 600 13
0x00000000 1900599 zabbix 600 13
0x00000000 1933368 zabbix 600 13
0x00000000 1966137 zabbix 600 13
0x00000000 1998906 zabbix 600 13
0x00000000 2031675 zabbix 600 13
0x00000000 2064444 zabbix 600 13
0x00000000 2097213 zabbix 600 13
0x00000000 2129982 zabbix 600 13
0x00000000 2162751 zabbix 600 13
0x00000000 2195520 zabbix 600 13
0x00000000 2228289 zabbix 600 13
0x00000000 2261058 zabbix 600 13
0x00000000 2293827 zabbix 600 13
0x00000000 2326596 zabbix 600 13
0x00000000 2359365 zabbix 600 13
0x00000000 2392134 zabbix 600 13
0x00000000 2424903 zabbix 600 13
0x00000000 2457672 zabbix 600 13
0x00000000 2490441 zabbix 600 13
0x00000000 2523210 zabbix 600 13
0x00000000 2555979 zabbix 600 13
0x00000000 2588748 zabbix 600 13
0x00000000 2621517 zabbix 600 13
0x00000000 2654286 zabbix 600 13
0x00000000 2687055 zabbix 600 13
0x00000000 2719824 zabbix 600 13
0x00000000 2752593 zabbix 600 13
0x00000000 2785362 zabbix 600 13
0x00000000 2818131 zabbix 600 13
0x00000000 2850900 zabbix 600 13
0x00000000 2883669 zabbix 600 13
0x00000000 2916438 zabbix 600 13
0x00000000 2949207 zabbix 600 13
0x00000000 2981976 zabbix 600 13
0x00000000 3014745 zabbix 600 13
0x00000000 3047514 zabbix 600 13
0x00000000 3080283 zabbix 600 13
0x00000000 3113052 zabbix 600 13
0x00000000 3145821 zabbix 600 13
0x00000000 3178590 zabbix 600 13
0x00000000 3211359 zabbix 600 13
0x00000000 3244128 zabbix 600 13
0x00000000 3276897 zabbix 600 13
0x00000000 3309666 zabbix 600 13
0x00000000 3342435 zabbix 600 13
0x00000000 3375204 zabbix 600 13
0x00000000 3407973 zabbix 600 13
0x00000000 3440742 zabbix 600 13
0x00000000 3473511 zabbix 600 13
0x00000000 3506280 zabbix 600 13
0x00000000 3539049 zabbix 600 13
0x00000000 3571818 zabbix 600 13
0x00000000 3604587 zabbix 600 13
0x00000000 3637356 zabbix 600 13
0x00000000 3670125 zabbix 600 13
0x00000000 3702894 zabbix 600 13
0x00000000 3735663 zabbix 600 13
0x00000000 3768432 zabbix 600 13
0x00000000 3801201 zabbix 600 13
0x00000000 3833970 zabbix 600 13
0x00000000 3866739 zabbix 600 13
0x00000000 3899508 zabbix 600 13
0x00000000 3932277 zabbix 600 13
0x00000000 3965046 zabbix 600 13
0x00000000 3997815 zabbix 600 13
0x00000000 4030584 zabbix 600 13
0x00000000 4063353 zabbix 600 13
0x00000000 4096122 zabbix 600 13
0x00000000 4128891 zabbix 600 13
0x00000000 4161660 zabbix 600 13
0x00000000 4194429 zabbix 600 13
0x00000000 4227198 zabbix 600 13
0x00000000 4259967 zabbix 600 13
删除它们
[root@mariadb-node-0 zabbix]# ipcrm --help
用法：
ipcrm [options]
ipcrm <shm|msg|sem> <id> [...]
选项：
-m, --shmem-id <id> 按 id 号移除共享内存段
-M, --shmem-key <键> 按键值移除共享内存段
-q, --queue-id <id> 按 id 号移除消息队列
-Q, --queue-key <键> 按键值移除消息队列
-s, --semaphore-id <id> 按 id 号移除信号量
-S, --semaphore-key <键> 按键值移除信号量
-a, --all[=<shm|msg|sem>] 全部移除
-v, --verbose 解释正在进行的操作
-h, --help 显示此帮助并退出
-V, --version 输出版本信息并退出
更多信息请参阅 ipcrm(1)。
[root@mariadb-node-0 zabbix]# ipcrm -s 98304
[root@mariadb-node-0 zabbix]#
这样就好办了，写个shell吧。
[root@mariadb-node-0 zabbix]# cat ~/t.sh
#!/usr/bin/bash
for line in `cat ./sem-id.txt`
do
echo $line
ipcrm -s $line
done
echo "game over!"
[root@mariadb-node-0 zabbix]#
ok, 全删除了。
把原有的配置文件还原，然后 重启服务 吧。
重启之后就OK了。
还有一个方向是这样的：
设置与查看 selinux
[jlch@mariadb-node-0 zabbix]$ getenforce
Permissive
[jlch@mariadb-node-0 zabbix]$ sudo setenforce 0
[jlch@mariadb-node-0 zabbix]$ sudo getenforce
Permissive
查日志：
[jlch@mariadb-node-0 zabbix]$ sudo cat /var/log/audit/audit.log |grep failed
... ## 这里有很多很多啦
'unit=zabbix-agent comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
type=SERVICE_STOP msg=audit(1507601499.089:102137): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=zabbix-agent comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
[jlch@mariadb-node-0 zabbix]$ ^C
查 audit 返回的原因
[jlch@mariadb-node-0 zabbix]$ sudo cat /var/log/audit/audit.log |grep zabbix-agent |audit2allow
[sudo] password for jlch:
Nothing to do
[jlch@mariadb-node-0 zabbix]$ sudo cat /var/log/audit/audit.log |grep zabbix-agent |audit2why
Nothing to do
[jlch@mariadb-node-0 zabbix]$
发现没有返回原因，说明原因不在这个地方嘛。（当然不在这个地方了，我们这里是因为 内存文件系统 ipcs 嘛）
所以不用再考虑 selinux 了。
zabbix bug spell¶
安装后，启动，看状态，报错了
[root@mongodb1 jlch]# sudo systemctl status zabbix-agent.service [root@mongodb1 jlch]# journalctl -xe [root@mongodb1 jlch]# tail -n 200 /var/log/messages
从这里可以看到一条信息如下：
systemd: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue 'StardardInput' in section 'Service'
google 查到这里
https://bugzilla.redhat.com/show_bug.cgi?id=1401333
说了，要： Change “Stardard” to “Standard”.
修改吧。
[root@mongodb1 jlch]# vi /usr/lib/systemd/system/anaconda-pre.service [root@mongodb1 jlch]# sudo systemctl status anaconda-pre.service ● anaconda-pre.service - pre-anaconda logging service
Loaded: loaded (/usr/lib/systemd/system/anaconda-pre.service; static; vendor preset: disabled) Active: inactive (dead)6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ 6月 30 12:50:17 mongodb1 systemd[1]: [/usr/lib/systemd/system/anaconda-pre.service:18] Unknown lvalue ‘StardardInput’ in section ‘Service’ [root@mongodb1 jlch]# sudo systemctl start anaconda-pre.service
果然，这样 anaconda-pre 就正常了，然后再启动 zabbix-agent ，也正常了。
Zabbix discoverer processes more than 75% busy¶
https://www.kaijia.me/2014/01/zabbix-poller-processes-more-than-75-busy-alert-issue-solved/
step¶
编辑Zabbix Server的配置文件/etc/zabbix/zabbix_server.conf，找到配置StartPollers的段落：
### Option: StartPollers
# Number of pre-forked instances of pollers.
#
# Mandatory: no
# Range: 0-1000
# Default:
# StartPollers=5
取消StartPollers=一行的注释或者直接在后面增加：
StartPollers=10
因为我这里本来的就是10, 所以我直接修改成20, 这样子, 报错就消失了
StartPollers=10
第2天¶
第2天, 这个问题又来了, 所以我只好选择继续看这个文章
选择每天重启zabbix_server吧.
重启Zabbix。当然另外一种从整体上降低Zabbix服务器负载的方法就是定期重启Zabbix，这种方法可以用Cron实现，运行：
crontab -e
这个计划会每天自动重启Zabbix服务以结束僵尸进程并清理内存等。目前Kaijia这样配置Zabbix后还没有再次遇到过“Zabbix poller processes more than 75% busy”的问题。
所以我只好这样了.. 等明天接着观察吧.
[root@qlw jlch]# crontab -l
SHELL=/bin/bash
PATH=/usr/local/bin:/usr/bin
#m h dom mon dow user command
10 2 * * * systemctl restart zabbix_server.service > /dev/null 2>&1
[root@qlw jlch]#
ruby¶
Contents:
ruby install¶
Contents:
ruby install by rpm¶
ruby¶
去 源 http://mirrors.aliyun.com/centos/7/os/x86_64/Packages/ 下载 必要的包
[jlch@check ruby]$ cat t.sh
#!/bin/bash
for rpm_packages in `cat packages.txt`
do
echo $rpm_packages
wget http://mirrors.aliyun.com/centos/7/os/x86_64/Packages/$rpm_packages
done
[jlch@check ruby]$ cat packages.txt
libyaml-0.1.4-11.el7_0.x86_64.rpm
ruby-2.0.0.648-30.el7.x86_64.rpm
ruby-libs-2.0.0.648-30.el7.x86_64.rpm
rubygem-io-console-0.4.2-30.el7.x86_64.rpm
rubygem-rdoc-4.0.0-30.el7.noarch.rpm
rubygem-bigdecimal-1.2.0-30.el7.x86_64.rpm
rubygem-psych-2.0.0-30.el7.x86_64.rpm
rubygems-2.0.14.1-30.el7.noarch.rpm
ruby-irb-2.0.0.648-30.el7.noarch.rpm
rubygem-json-1.7.7-30.el7.x86_64.rpm
[jlch@check ruby]$
安装 ruby, gem
rpm -ivh libyaml-0.1.4-11.el7_0.x86_64.rpm
rpm -ivh ruby-2.0.0.648-30.el7.x86_64.rpm ruby-libs-2.0.0.648-30.el7.x86_64.rpm rubygem-io-console-0.4.2-30.el7.x86_64.rpm rubygem-rdoc-4.0.0-30.el7.noarch.rpm rubygem-bigdecimal-1.2.0-30.el7.x86_64.rpm rubygem-psych-2.0.0-30.el7.x86_64.rpm rubygems-2.0.14.1-30.el7.noarch.rpm ruby-irb-2.0.0.648-30.el7.noarch.rpm rubygem-json-1.7.7-30.el7.x86_64.rpm
which ruby
ruby help
ruby --help
ruby gems ## 这个本身就不能运行的。
gem
gem list --local
ruby -v
卸载 ruby
[root@localhost ruby-install-by-rpm]# cat rm.sh
rpm -ef rubygem-io-console-0.4.2-30.el7.x86_64 rubygem-psych-2.0.0-30.el7.x86_64 rubygems-2.0.14.1-30.el7.noarch ruby-irb-2.0.0.648-30.el7.noarch ruby-libs-2.0.0.648-30.el7.x86_64 rubygem-bigdecimal-1.2.0-30.el7.x86_64 rubygem-json-1.7.7-30.el7.x86_64 ruby-2.0.0.648-30.el7.x86_64 rubygem-rdoc-4.0.0-30.el7.noarch
ruby 2.4 install by rpm¶
ruby¶
相关 sh 脚本，可见 https://gitee.com/tomt/tom_ruby_install_by_rpm
去 源 http://mirrors.aliyun.com/centos/7/os/x86_64/Packages/ 下载 必要的包
[jlch@check ruby]$ cat t.sh
#!/bin/bash
for rpm_packages in `cat packages.txt`
do
echo $rpm_packages
wget http://mirrors.aliyun.com/centos/7/os/x86_64/Packages/$rpm_packages
done
[jlch@check ruby]$ cat packages.txt
libyaml-0.1.4-11.el7_0.x86_64.rpm
ruby-2.0.0.648-30.el7.x86_64.rpm
ruby-libs-2.0.0.648-30.el7.x86_64.rpm
rubygem-io-console-0.4.2-30.el7.x86_64.rpm
rubygem-rdoc-4.0.0-30.el7.noarch.rpm
rubygem-bigdecimal-1.2.0-30.el7.x86_64.rpm
rubygem-psych-2.0.0-30.el7.x86_64.rpm
rubygems-2.0.14.1-30.el7.noarch.rpm
ruby-irb-2.0.0.648-30.el7.noarch.rpm
rubygem-json-1.7.7-30.el7.x86_64.rpm
[jlch@check ruby]$
安装 ruby, gem
rpm -ivh libyaml-0.1.4-11.el7_0.x86_64.rpm
rpm -ivh ruby-2.0.0.648-30.el7.x86_64.rpm ruby-libs-2.0.0.648-30.el7.x86_64.rpm rubygem-io-console-0.4.2-30.el7.x86_64.rpm rubygem-rdoc-4.0.0-30.el7.noarch.rpm rubygem-bigdecimal-1.2.0-30.el7.x86_64.rpm rubygem-psych-2.0.0-30.el7.x86_64.rpm rubygems-2.0.14.1-30.el7.noarch.rpm ruby-irb-2.0.0.648-30.el7.noarch.rpm rubygem-json-1.7.7-30.el7.x86_64.rpm
加载 环境变量
[root@localhost rh-ruby24]# find / -name ruby
[root@localhost rh-ruby24]# pwd
/opt/rh/rh-ruby24
[root@localhost rh-ruby24]# ls
enable root
[root@localhost rh-ruby24]# cat /opt/rh/rh-ruby24/enable >> /etc/profile
[root@localhost rh-ruby24]# source /etc/profile
验证
which ruby # 这个时候, 可能还是没有 ruby的, 因为,没有加载 环境变量
ruby help
ruby --help
ruby gems ## 这个本身就不能运行的。
gem
gem list --local
ruby -v
[root@localhost rh-ruby24]# ruby --help
卸载 ruby
[root@localhost ruby-install-by-rpm]# cat rm.sh
rpm -ef rubygem-io-console-0.4.2-30.el7.x86_64 rubygem-psych-2.0.0-30.el7.x86_64 rubygems-2.0.14.1-30.el7.noarch ruby-irb-2.0.0.648-30.el7.noarch ruby-libs-2.0.0.648-30.el7.x86_64 rubygem-bigdecimal-1.2.0-30.el7.x86_64 rubygem-json-1.7.7-30.el7.x86_64 ruby-2.0.0.648-30.el7.x86_64 rubygem-rdoc-4.0.0-30.el7.noarch
—
draft:
中间，为了环境变量，有一次用过
export RUBYLIB=/opt/jlch/ruby-2.4.2/lib/
gem install package¶
Contents:
gem install redis¶
https://rubygems.org/gems/redis/versions/3.3.5
pwd
ls /home/jlch/
pwd
wget https://rubygems.org/downloads/redis-3.3.5.gem
gem install /home/jlch/redis-4.0.1.gem ## ruby-2.4 会成功，其它报错
gem install /home/jlch/redis-4.0.0.gem ## 报错
gem install /home/jlch/redis-3.3.5.gem ## 成功
jenkins¶
Contents:
jenkins install¶
Contents:
jenkins install centos yum¶
检查 8080 端口有没有占用。
sudo lsof -i:8080
按官网安装
sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
sudo yum install jenkins -y
sudo systemctl status jenkins.service
sudo systemctl start jenkins.service
sudo systemctl status jenkins.service
sudo /etc/rc.d/init.d/jenkins start
sudo journalctl -xe
查到原因
Oct 18 13:13:11 jlch_web_001 runuser[21226]: pam_unix(runuser:session): session opened for user jenkins by (uid=0)
Oct 18 13:13:11 jlch_web_001 jenkins[21225]: Starting Jenkins bash: /usr/bin/java: No such file or directory
Oct 18 13:13:11 jlch_web_001 runuser[21226]: pam_unix(runuser:session): session closed for user jenkins
Oct 18 13:13:11 jlch_web_001 jenkins[21225]: [FAILED]
Oct 18 13:13:11 jlch_web_001 systemd[1]: jenkins.service: control process exited, code=exited status=1
Oct 18 13:13:11 jlch_web_001 systemd[1]: Failed to start LSB: Jenkins Automation Server.
-- Subject: Unit jenkins.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit jenkins.service has failed.
--
-- The result is failed.
哈哈，没有安装 java
sudo yum list installed |grep java
sudo yum search java|grep jdk
安装吧
sudo yum install java-1.8.0-openjdk -y
再启动
sudo systemctl start jenkins.service
sudo systemctl status jenkins.service
开端口
sudo firewall-cmd --help
sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload
sudo firewall-cmd --list-ports
浏览器去咯。
jenkins install jenkins.war¶
Jenkins install¶
环境：192.168.31.181:/home/tom/
java -jar jenkins.war --httpPort=8011
jlch, jlch, uuiduuid, 273412935@qq.com
Jenkins+github+nodejs¶
https://segmentfault.com/a/1190000010200161
然后，我们配置吧。
在 github.com 中要配置 webhook, 这个时候，要填写 jenkins服务器IP, 这时候，我们就要 通过 nginx 作反向代理了。
然后，我在反向代理出错了。（主要原因是 jenkins 页面，没有写成 相对路径）
悲剧了，怎么办?
- 去阿里云，然后，重新来一次
http://blog.csdn.net/frank_good/article/details/68930286
http://www.jianshu.com/p/22b7860b4e81 (我通过这个网址，做到成功了)
tomtsang- personal access token -
91e4a74a0ec87f7fce9b6344ead0caa86e7644a0
环境：120.25.204.216
安装方式：centos裸机安装。
- 去 让一灯实现一下。
Jenkins+gitlab+nodejs¶
jenkins.io¶
Contents:
jenkins.io/doc¶
Contents:
jenkins.io/doc/tutorials/¶
Contents:
building-a-java-app-with-maven¶
Contents:
docker pull jenkinsci/blueocean
docker volume create jenkins-data
#docker run --rm -u root -p 8011:8080 -v jenkins-data:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -v "$HOME":/home jenkinsci/blueocean
docker run --rm -u root -p 8011:8080 -v jenkins-data:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -v /home/tom/jenkins/home/:/home jenkinsci/blueocean
# 输出一大串了
# ...
*************************************************************
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
f4509cc0cf6543b6a85896fcad11c7ad
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
*************************************************************
*************************************************************
*************************************************************
Oct 23, 2017 2:42:14 AM hudson.model.UpdateSite updateData
后面直接按照官方文档进行.
成功的.
building-a-node-js-and-react-app-with-npm¶
Contents:
docker pull jenkinsci/blueocean
docker volume create jenkins-data-nodejs
cd /home/tom/jenkins/test/nodejs/ # my workspace
mkdir -p ./home/GitHub/
cd ./home/GitHub/
git clone https://github.com/tomtsang/simple-node-js-react-npm-app
docker run \
--rm \
-u root \
-p 8012:8080 \
-v jenkins-data-nodejs:/var/jenkins_home \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /home/tom/jenkins/test/nodejs/home:/home \
jenkinsci/blueocean
# 输出一大串了
# ...
*************************************************************
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
f4509cc0cf6543b6a85896fcad11c7ad
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
*************************************************************
*************************************************************
*************************************************************
Oct 23, 2017 2:42:14 AM hudson.model.UpdateSite updateData
后面直接按照官方文档进行.
成功的.
jenkins-the-definitive-guide¶
Contents:
jenkins-the-definitive-guide README¶
resource¶
jenkins-the-definitive-guide
http://www.wakaleo.com/books/jenkins-the-definitive-guide
https://github.com/wakaleo/jenkins-the-definitive-guide-book
从上面这2个网站上可以下载到书的 pdf 英文版, 随书代码.
我自己买了一本纸质书.哈哈.
pdf 中文版扫描, 可以从下面这下载 http://download.csdn.net/download/htjswhg_a/9938627
game over¶
chap02¶
Contents:
maven config¶
配置你的 Maven 设置
env1¶
192.168.31.181 master centos7
192.168.31.172 slave1 ubuntu16
192.168.31.240 slave2 centos7
env2¶
120.25.204.216 master jlchaliyun
120.234.23.98 slave1 jlchsecdn
202.104.136.202 slave2 jlchsetom
slave 节点安装 maven¶
http://maven.apache.org/download.cgi
上 slave节点
下载apache-maven-3.5.2-bin.tar.gz包
- ::
- cd /home/jenkins/ mkdir software && cd software wget http://mirror.bit.edu.cn/apache/maven/maven-3/3.5.2/binaries/apache-maven-3.5.2-bin.tar.gz tar -zxvf apache-maven-3.5.2-bin.tar.gz
添加成下面这个样子 这样, maven 的文件夹就是 /home/jenkins/software/apache-maven-3.5.2/ 让它生效吧.后续会把这个路径加入到 jenkins 的maven配置中来
cd /home/jenkins
vi .bash_profile
[jenkins@test_240 game-of-life-default]$ cat ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH
# maven所在的目录
export M2_HOME=/home/jenkins/software/apache-maven-3.5.2
# maven bin所在的目录
export M2=$M2_HOME/bin
# 将maven bin加到PATH变量中
export PATH=$M2:$PATH
[jenkins@test_240 game-of-life-default]$
检验一下
source .bash_profile
[jenkins@test_240 game-of-life-default]$ mvn -version
Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T15:58:13+08:00)
Maven home: /home/jenkins/software/apache-maven-3.5.2
Java version: 1.8.0_151, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-1.b12.el7_4.x86_64/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-229.14.1.el7.x86_64", arch: "amd64", family: "unix"
[jenkins@test_240 game-of-life-default]$
说明安装 Maven 成功.
配置你的 Maven 设置¶
回 master 节点.
开浏览器.
http://blog.csdn.net/tengxing007/article/details/77626628
点击 系统管理/全局tool配置(Global Tool Configuration)/ 在 Maven 项目中,写入 slave 的 maven 地址(/home/jenkins/software/apache-maven-3.5.2/)
Name 192.168.31.240.jenkins.maven3.5.2
MAVEN_HOME /home/jenkins/software/apache-maven-3.5.2/
因为 这个 maven 地址, 不是 master的maven 地址,所以会有一个黄色的小警告,如下:
/home/jenkins/software/apache-maven-3.5.2 is not a directory on the Jenkins master (but perhaps it exists on some agents)
好了, 配置完成了.
配置Job¶
源码管理 选择 Git
Repository URL 写: git@github.com:tomtsang/game-of-life.git
构建 Invoke top-level Maven targets [Help]
Maven Version 写入: 192.168.31.240.jenkins.maven3.5.2 Goals 写入: clean package
构建后操作 Archive the artifacts
- Publish JUnit the result report
- 测试报告（XML）写入: gameoflife-core/target/surefire-reports/*.xml
立即构建¶
点击
报错了.
然后,我直接上 slave 节点, 去把 工程 clone 下来, 然后直接跑 mvn clean package, 然后输出与 jenkins 日志一样, 确实是报错了.
错误如下:
No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?
怎么办?
解决 “No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?”¶
网上有一些文章都是在说 eclipse 下的配置, 与我们的环境不符嘛.
提出使用 apt-get install openjdk-7-jdk openjdk-7-doc openjdk-7-jre-lib 但是我们的 slave 是 centos7. 接着查到了 http://blog.csdn.net/cdnight/article/details/78293767
先安装 yum install java-devel -y 试一下.来吧
sudo yum install java-devel -y
cd game-of-life/
ls
mvn clean package
.....
[INFO] gameoflife-web ..................................... SUCCESS [ 35.860 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 49.793 s
[INFO] Finished at: 2017-11-17T09:40:58+08:00
[INFO] Final Memory: 30M/337M
[INFO] ------------------------------------------------------------------------
[jenkins@test_240 game-of-life]$
我去,成功了.哈哈哈哈~~~~
javadoc¶
env2¶
120.25.204.216 master jlchaliyun
120.234.23.98 slave1 jlchsecdn
202.104.136.202 slave2 jlchsetom
配置 job¶
构建:
Invoke top-level Maven targets
- Maven Version 选择: api-cm.zhangtl.com.jenkins
- Goals 写入: javadoc:javadoc 不能写 javadoc:javadoc -o, 加了 -o 会报错.
没有 plugin,报错¶
在 chap 02 中的 更多的报告–显示javadoc 小节中,要配置 javadoc
因为继前小节, 我们的环境是由 master, 调用 slave 模式进行, 因为我们的 slave 没有安装 javadoc, 所以直接报错误了.
错误如下:
[INFO] ------------------------------------------------------------------------
[ERROR] No plugin found for prefix 'javadoc' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/home/jenkins/.m2/repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoPluginFoundForPrefixException
Build step 'Invoke top-level Maven targets' marked build as failure
Archiving artifacts
Recording test results
Finished: FAILURE
一看就明白了, No plugin found for prefix ‘javadoc’ in the current project and in the plugin groups 没有plugin.
安装 javadoc¶
这里有一个疑问: 是下面中的哪一个呢?
- slave 安装 javadoc
- master 下的 jenkins 安装javadoc 插件
我选择先在 master 中安装javadoc 插件.
通过浏览器, 安装完了 javadoc 插件后, 再构建,不行,一样的错误. 这个也是可以理解的,因为,master是委托 slave 去做具体的事务的,所以,master安装了javadoc, 肯定是没有用的嘛.
好吧,去 slave安装 javadoc.
[jenkins@test_240 game-of-life]$ sudo yum search openjdk-javadoc
[jenkins@test_240 game-of-life]$ sudo yum install java-1.8.0-openjdk-javadoc -y
成功了.
然后,直接在 slave 上 ,在工程中运行 mvn javadoc:javadoc , 在3个子文件夹,成功了. 在 gameoflife-web 下失败了. 那就 cd gameoflife-web 后,单独跑一下, 果然是失败了.
[jenkins@test_240 gameoflife-web]$ /home/jenkins/software/apache-maven-3.5.2/bin/mvn javadoc:javadoc
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.wakaleo.gameoflife:gameoflife-web:war:1.0-SNAPSHOT
[WARNING] Reporting configuration should be done in <reporting> section, not in maven-site-plugin <configuration> as reportPlugins parameter.
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building gameoflife-web 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> maven-javadoc-plugin:3.0.0-M1:javadoc (default-cli) > generate-sources @ gameoflife-web >>>
[WARNING] The POM for com.wakaleo.gameoflife:gameoflife-core:jar:1.0-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.062 s
[INFO] Finished at: 2017-11-20T16:42:10+08:00
[INFO] Final Memory: 12M/150M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project gameoflife-web: Could not resolve dependencies for project com.wakaleo.gameoflife:gameoflife-web:war:1.0-SNAPSHOT: Could not find artifact com.wakaleo.gameoflife:gameoflife-core:jar:1.0-SNAPSHOT -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[jenkins@test_240 gameoflife-web]$
这样,就是说, 是工程自己的问题了. 那我们先把这个 gameoflife-web 去掉了先吧.
gameoflife-web 去掉¶
怎么去掉呢?
把 pom.xml 中的 gameoflife-web 注释掉. (大约在212行).
[jenkins@test_240 game-of-life]$ ls
gameoflife-acceptance-tests gameoflife-build gameoflife-core gameoflife-deploy gameoflife-web infinitest.filters pom.xml README.markdown target
[jenkins@test_240 game-of-life]$ vi pom.xml
[jenkins@test_240 game-of-life]$ grep gameoflife-web ./pom.xml
<!--<module>gameoflife-web</module>-->
[jenkins@test_240 game-of-life]$
再一跑, 成功了.
配置 job¶
既然成功了. 那么, 回 浏览器配置吧..
构建后操作
Publish Javadoc
Javadoc directory: gameoflife-core/target/site/apidocs 勾选上 Retain Javadoc for each successful build
jenkins git local¶
Contents:
jenkins-gitlocal¶
https://juejin.im/entry/58f18fc2570c3500562cb433
env¶
192.168.31.181 master
192.168.31.172 slave1
192.168.31.240 slave2
install¶
https://pkg.jenkins.io/redhat-stable/
sudo systemctl enable jenkins
sudo systemctl start jenkins
防火墙把 8080 打开
配置master¶
把jenkins用户设置为sudo用户
$ grep jenkins /etc/passwd
jenkins:x:496:496:Jenkins Continuous Integration Server:/var/lib/jenkins:/bin/false
修改一下,让可以登陆
$ grep jenkins /etc/passwd
jenkins:x:496:496:Jenkins Continuous Integration Server:/var/lib/jenkins:/bin/bash
$ su jenkins
$ cd ~
$ pwd
/var/lib/jenkins
配置公钥
配置slave节点¶
建立用户¶
sudo adduser jenkins
sudo passwd jenkins
配置公钥给slave¶
ssh jenkins@slave1 'mkdir -p .ssh && cat >> .ssh/authorized_keys' < ~/.ssh/id_rsa.pub
ssh jenkins@slave2 'mkdir -p .ssh && cat >> .ssh/authorized_keys' < ~/.ssh/id_rsa.pub
把 jenkins 用户加入到 sudo 用户¶
配置Node¶
来到浏览器, 192.168.31.181:8080, 登陆
系统管理/管理节点/新建节点/
注意要填写 label 哟
这个时候, 如果说能够正常连接上就说明 slave 成功与 master 连接了.
如果出错了, 马上看日志吧.
如果日志信息说是
找不到 java , 而自己去 slave 命令行下 which java, 是有正常返回的,那说明,是少了 openjdk . 怎么办?安装呗.
[jenkins@test_240 ~]$ sudo yum search openjdk
[jenkins@test_240 ~]$ sudo yum install -y java-1.8.0-openjdk
好了,这下安装成功,然后重新再来一连接, 成功了哟!
配置pipeline¶
Repository URL 时: 写入具体的 git 地址就可以了. 如: git@secdn:trend.git, 这个地址后续会在 slave 节点中运行 git clone git@secdn:trend.git 这也是说, 这些 slave 节点, 要有这个 trend 仓库的git clone 权限哟. 如果没有设置,这里要回头去设置了.
构建¶
点一下”立即构建” 看效果吧.
game over!¶
jenkins web¶
Contents:
jenkins-nodejs-publish¶
使用Jenkins自动部署nodejs应用
env1¶
120.25.204.216 jenkins master
10.10.15.181 jenkins remote-server
env2¶
10.10.15.181 jenkins master
10.10.15.181 jenkins remote-server
安装插件¶
系统管理 -> 插件管理
主要是安装 Publish Over SSH, SSH plugin
GIT client
GIT
Git server
GitHub API
GitHub
SSH Credentials
SSH
Publish Over SSH
安装完成后，重新启动 jenkins。 如果无法自动重启，可以输入 ‘/restart’ 在地址栏，即可手动重启。
添加remote计算机¶
系统管理 -> 系统设置
SSH remote hosts¶
增加remote server 这个是增加ssh remote 插件的服务器
点击 新增
- Hostname 写入 api-ct.zhangtl.com
- Port 写入 6000 Credentials 点击 Add , 选择 jenkins (这里其实是之前配置的一个credential)
点击 Check connection, 如显示 Successfull connection, 表示连接成功
增加远程拷贝文件的remote server
Publish over SSH¶
- Jenkins SSH Key
- Passphrase 选择 密码, •••••••••••••••••••••••••••••••••••••••••••••••••••••••••• Path to key 写入 /var/lib/jenkins/.ssh/id_rsa 这个就是 master 的 私钥 Key 这里可以不填写. 如果说 上面没有写私钥的地址,那这里就直接把 私钥 复制到这里.也就是2者写1就行了 Disable exec 不动, 不打勾 SSH Servers
- SSH Server 这个就是 远程的server 的信息了
- Name 写入 api-ct.zhanglt.com.6000.jenkins, 这个是给自己看的 Hostname 写入 api-ct.zhangtl.com 这个是要解析的,也就是IP了 Username 写入 jenkins 用户名 Remote Directory 写入 /var/lib/jenkins 这里我写的是 jenkins 用户的 $HOME 目录,
点击 Test Configuration, 如显示 Success 表示成功了.
创建项目¶
我现在在github上建了一个nodejs的helloworld项目： https://github.com/tomtsang/jenkinsNodejs 是 fork 于 https://github.com/DoubleSpout/jenkinsNodejs
我们需要在jenkins中也创建一个相应的项目来，依次点击：新建 -> 取名为 nodejsPublish
进入项目，点击配置，我们就可以对这个项目进行配置，让它自动部署
源码管理¶
- 设置 git 仓库路径
- Git / Repositories
- Repository URL 写入 https://github.com/tomtsang/jenkinsNodejs/ Credentials 可以不选择, 选择也可以, 我是 Add, 选择 jenkins 这个.
因为是公共仓库，所以可以不需要验证用户名，密码
构建¶
构建完毕之后，将代码打包保存, 所以增加一个构建完毕的shell脚本执行
具体的是, 打包代码，然后移动到workspace目录下
点击 增加构建步骤, 选择 Execute shell, command 写入
#rm -rf /var/lib/jenkins/jobs/nodejsPublish/nodejsPublish.tar.gz
#tar -zcvf /tmp/nodejsPublish.tar.gz -C /var/lib/jenkins/jobs/nodejsPublish/workspace/ .
tar -zcvf /tmp/nodejsPublish.tar.gz -C /var/lib/jenkins/workspace/nodejsPublish/ .
mv /tmp/nodejsPublish.tar.gz /var/lib/jenkins/workspace/nodejsPublish/
echo {$HOME}
echo "Execute shell"
接着我们需要连接到部署服务器上，将原来的服务暂时暂停，删除上一次部署的压缩包，选择添加远程shell脚本执行， 如:将pm2服务停止，这里大家可以根据自己的要求写脚本
点击 增加构建步骤, 选择 Execute shell script on remote host using ssh,
SSH site 选择 jenkins@api-ct.zhangtl.com:6000 , 这个选择项目也就是上面配置的Publish over SSH/SSH Server的结果
command 写入
pm2 stop all
pm2 delete all
echo {$HOME}
echo "Execute shell script on remote host using ssh"
最后我们要把打包好的代码发送到服务器上，然后执行安装依赖包操作，并且启动nodejs服务，
也就是 选择通过ssh发送文件并执行命令到远程服务器, 然后 解压代码，执行依赖包安装，然后通过pm2启动nodejs服务。
点击 增加构建步骤, 选择 Send files or execute commands over ssh,
SSH Publishers / SSH Server
- Name
- 选择 api-ct.zhanglt.com.6000.jenkins
- Transfers / Transfer Set
- Source files 写入 nodejsPublish.tar.gz, Source files的默认当前路径就是workspaces，也就是 /var/lib/jenkins/workspace/, 这里写入的是它的相对路径 Remove prefix 不写 Remote directory 写入 tom/nodejsPublish/ , Remote directory的当前路径就是之前添加远程服务器设置的，我当时设置为/var/lib/jenkins/, 同样写相对路径 Exec command 写入 如下:
echo {$HOME}
ls -l
ls tom/nodejsPublish/
这个 Exec command 下方会有红色提示(但是我发现没有关系, 不受影响, 这是为什么? 求赐教), 提示如下:
Either Source files, Exec command or both must be supplied
开始构建¶
配置好上述命令之后，我们点击保存，然后点击立即构建，
出现蓝色的球，就表示构建成功， 红色的球表示构建失败， 黄色的球表示执行构建命令失败，
不论构建成功还是失败，我们可以通过查看控制台输出看到构建的结果
如果我们有多台设备，就只需要配置一次，然后以后每次有代码变更，手动点击立即构建即可自动部署到目标服务器，免去我们枯草的重复劳动。
当然其他语言，例如python，php都是可以这样来部署的。
jenkins log¶
我本次的 控制台输出 如下
Started by user jlch
Building on master in workspace /var/lib/jenkins/workspace/nodejsPublish
> /usr/bin/git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> /usr/bin/git config remote.origin.url https://github.com/tomtsang/jenkinsNodejs/ # timeout=10
Fetching upstream changes from https://github.com/tomtsang/jenkinsNodejs/
> /usr/bin/git --version # timeout=10
using GIT_SSH to set credentials
> /usr/bin/git fetch --tags --progress https://github.com/tomtsang/jenkinsNodejs/ +refs/heads/*:refs/remotes/origin/*
> /usr/bin/git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> /usr/bin/git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision f48169f7462594f445a52f64e4c0a36376c089ae (refs/remotes/origin/master)
> /usr/bin/git config core.sparsecheckout # timeout=10
> /usr/bin/git checkout -f f48169f7462594f445a52f64e4c0a36376c089ae
Commit message: "tangren"
> /usr/bin/git rev-list f48169f7462594f445a52f64e4c0a36376c089ae # timeout=10
[nodejsPublish] $ /bin/sh -xe /tmp/jenkins1008771630424029501.sh
+ tar -zcvf /tmp/nodejsPublish.tar.gz -C /var/lib/jenkins/workspace/nodejsPublish/ .
./
./.git/
./.git/config
./.git/logs/
./.git/logs/refs/
./.git/logs/refs/remotes/
./.git/logs/refs/remotes/origin/
./.git/logs/refs/remotes/origin/master
./.git/logs/HEAD
./.git/index
./.git/refs/
./.git/refs/remotes/
./.git/refs/remotes/origin/
./.git/refs/remotes/origin/master
./.git/refs/heads/
./.git/refs/tags/
./.git/hooks/
./.git/hooks/pre-rebase.sample
./.git/hooks/pre-applypatch.sample
./.git/hooks/update.sample
./.git/hooks/commit-msg.sample
./.git/hooks/prepare-commit-msg.sample
./.git/hooks/pre-push.sample
./.git/hooks/applypatch-msg.sample
./.git/hooks/post-update.sample
./.git/hooks/pre-commit.sample
./.git/objects/
./.git/objects/fa/
./.git/objects/fa/296423e71ae782c22b374b6eaa07aa187bef6e
./.git/objects/89/
./.git/objects/89/8d1d090bd8da654b1fb6f3672a683e3f5942dd
./.git/objects/00/
./.git/objects/00/693b7e12fe8eebfeb8a0a50ecbc41c34f0ffe1
./.git/objects/64/
./.git/objects/64/e43ba90329a2db9f03f5fd60b825a8c69598f9
./.git/objects/fb/
./.git/objects/fb/0ea539f22a2cf415e2e9fbf2f52f6f847abfdd
./.git/objects/bd/
./.git/objects/bd/f6ae0c7f31e52c1f6a10d9ddf58e36d904a30d
./.git/objects/f4/
./.git/objects/f4/8169f7462594f445a52f64e4c0a36376c089ae
./.git/objects/ee/
./.git/objects/ee/84c56b0cac714754952079a4fecbdf17afd0de
./.git/objects/a2/
./.git/objects/a2/949bf24f3c31110982db73ee5ae5f1890201a0
./.git/objects/35/
./.git/objects/35/a95e2f5f67bab140269b1db1b34fc3cd46fdd5
./.git/objects/2b/
./.git/objects/2b/896fc37a879c86b389a1ad1dadbe9b8fb70359
./.git/objects/cb/
./.git/objects/cb/462c01a49216ca83b066c9455165070451e1e5
./.git/objects/f0/
./.git/objects/f0/224f93301c34c512c8d479e8a5fc9fc02efa13
./.git/objects/90/
./.git/objects/90/61189f9601ff1132a8101ec5bbea9d1ce2a998
./.git/objects/29/
./.git/objects/29/561a10c6484f7ef37c02abc444988a2f1c4f7d
./.git/objects/1c/
./.git/objects/1c/3f756b6ee0fc0c34333d9e4bdcd5b926e5c722
./.git/objects/08/
./.git/objects/08/5df15cd44e08c07affbe11cf2892ce58f09da2
./.git/objects/08/cf9368a5991917d492bae266a11caa20e74513
./.git/objects/87/
./.git/objects/87/448347cfe603b42c9fd63c57bb75ecd6cf8d06
./.git/objects/bc/
./.git/objects/bc/c46eb7e365aed7183db000c844bed2ac261f9a
./.git/objects/d2/
./.git/objects/d2/b2b3aad1755ab667cb880b6a8f2f153be2ea59
./.git/objects/info/
./.git/objects/14/
./.git/objects/14/fe3b780e02cb14e5247872259366dfb36047c9
./.git/objects/b7/
./.git/objects/b7/e189f6f999c05ef212c73985620004ce7dba6a
./.git/objects/b6/
./.git/objects/b6/a0ab9ce05dfeb67cdca62652462a2384b91662
./.git/objects/pack/
./.git/objects/c3/
./.git/objects/c3/096e2191d287ed22242d8f8cafcc58e6c01c50
./.git/objects/5d/
./.git/objects/5d/3a0f97ba1ebde0face66ac632000ce373bbbe4
./.git/objects/df/
./.git/objects/df/3e49b23675c87ae38b2561e02f468b3aa4465c
./.git/objects/a7/
./.git/objects/a7/56ebd1d4efc084af35056c7e2ee4e38521e8a0
./.git/objects/c5/
./.git/objects/c5/4d43514be4c21e13a5b46970a10391e5776533
./.git/objects/ac/
./.git/objects/ac/ad1d8e4c144059129f335d7672f1bdf85ecdb0
./.git/objects/8e/
./.git/objects/8e/c113bd2cc0054b69a361cc3c0d5a8813aa7c4b
./.git/objects/47/
./.git/objects/47/a836795a1aa6ef23612c12a164256044af393c
./.git/objects/1d/
./.git/objects/1d/0960dc4eb0aced6133d0b79613f14afe272735
./.git/objects/25/
./.git/objects/25/3f1497de9565b220ed807c2e68e00509142d9d
./.git/branches/
./.git/info/
./.git/info/exclude
./.git/description
./.git/HEAD
./.git/FETCH_HEAD
./applist.json
./app.js
./package.json
./README.md
+ mv /tmp/nodejsPublish.tar.gz /var/lib/jenkins/workspace/nodejsPublish/
+ echo '{/var/lib/jenkins}'
{/var/lib/jenkins}
+ echo 'Execute shell'
Execute shell
[SSH] script:
HOME="/var/lib/jenkins"
pm2 stop all
pm2 delete all
echo {$HOME}
echo "Execute shell script on remote host using ssh"
[SSH] executing...
[PM2][WARN] No process found
[PM2][WARN] No process found
┌──────────┬────┬──────┬─────┬────────┬─────────┬────────┬────────┬──────────┐
│ App name │ id │ mode │ pid │ status │ restart │ uptime │ memory │ watching │
└──────────┴────┴──────┴─────┴────────┴─────────┴────────┴────────┴──────────┘
Use `pm2 show <id|name>` to get more details about an app
┌──────────┬────┬──────┬─────┬────────┬─────────┬────────┬────────┬──────────┐
│ App name │ id │ mode │ pid │ status │ restart │ uptime │ memory │ watching │
└──────────┴────┴──────┴─────┴────────┴─────────┴────────┴────────┴──────────┘
Use `pm2 show <id|name>` to get more details about an app
{/var/lib/jenkins}
Execute shell script on remote host using ssh
[SSH] completed
[SSH] exit-status: 0
SSH: Connecting from host [jlch_web_001]
SSH: Connecting with configuration [api-ct.zhanglt.com.6000.jenkins] ...
SSH: EXEC: STDOUT/STDERR from command [echo {/var/lib/jenkins}
ls -l
ls tom/nodejsPublish/] ...
{/var/lib/jenkins}
total 100
drwxr-xr-x 6 jenkins jenkins 4096 Nov 16 12:51 caches
-rw-r--r-- 1 jenkins jenkins 1592 Nov 17 09:59 config.xml
-rw-r--r-- 1 jenkins jenkins 2522 Nov 16 09:09 credentials.xml
drwxr-xr-x 3 jenkins jenkins 15 Nov 16 15:14 fingerprints
-rw-r--r-- 1 jenkins jenkins 159 Nov 15 14:46 hudson.model.UpdateCenter.xml
-rw-r--r-- 1 jenkins jenkins 1254 Nov 16 14:15 hudson.plugins.emailext.ExtendedEmailPublisher.xml
-rw-r--r-- 1 jenkins jenkins 370 Nov 17 09:59 hudson.plugins.git.GitTool.xml
-rw-r--r-- 1 jenkins jenkins 173 Nov 17 09:59 hudson.plugins.gradle.Gradle.xml
-rw-r--r-- 1 jenkins jenkins 145 Nov 17 09:59 hudson.tasks.Ant.xml
-rw-r--r-- 1 jenkins jenkins 374 Nov 17 09:59 hudson.tasks.Maven.xml
-rw------- 1 jenkins jenkins 1712 Nov 15 14:46 identity.key.enc
-rw-r--r-- 1 jenkins jenkins 94 Nov 15 14:46 jenkins.CLI.xml
-rw-r--r-- 1 jenkins jenkins 6 Nov 15 15:34 jenkins.install.InstallUtil.lastExecVersion
-rw-r--r-- 1 jenkins jenkins 6 Nov 15 15:34 jenkins.install.UpgradeWizard.state
-rw-r--r-- 1 jenkins jenkins 247 Nov 17 09:59 jenkins.mvn.GlobalMavenConfig.xml
drwxr-xr-x 6 jenkins jenkins 86 Nov 16 14:33 jobs
drwxr-xr-x 4 jenkins jenkins 31 Nov 15 16:44 logs
-rw-r--r-- 1 jenkins jenkins 907 Nov 15 14:46 nodeMonitors.xml
drwxr-xr-x 4 jenkins jenkins 56 Nov 15 18:06 nodes
-rw-r--r-- 1 jenkins jenkins 298 Nov 17 09:59 org.jenkinsci.plugins.docker.commons.tools.DockerTool.xml
-rw-r--r-- 1 jenkins jenkins 255 Nov 17 09:59 org.jenkinsci.plugins.gitclient.JGitApacheTool.xml
-rw-r--r-- 1 jenkins jenkins 243 Nov 17 09:59 org.jenkinsci.plugins.gitclient.JGitTool.xml
-rw-r--r-- 1 jenkins jenkins 46 Nov 16 14:11 org.jenkinsci.plugins.workflow.flow.FlowExecutionList.xml
drwxr-xr-x 73 jenkins jenkins 8192 Nov 15 14:51 plugins
-rw-r--r-- 1 jenkins jenkins 64 Nov 15 14:45 secret.key
-rw-r--r-- 1 jenkins jenkins 0 Nov 15 14:45 secret.key.not-so-secret
drwx------ 4 jenkins jenkins 4096 Nov 16 09:55 secrets
drwxrwxr-x 3 jenkins jenkins 67 Nov 16 16:15 software
drwxrwxr-x 3 jenkins jenkins 26 Nov 25 08:17 tom
drwxr-xr-x 2 jenkins jenkins 4096 Nov 24 14:46 updates
drwxr-xr-x 2 jenkins jenkins 23 Nov 15 14:46 userContent
drwxr-xr-x 3 jenkins jenkins 17 Nov 15 15:34 users
drwxr-xr-x 2 jenkins jenkins 6 Nov 15 14:50 workflow-libs
nodejsPublish.tar.gz
SSH: EXEC: completed after 201 ms
SSH: Disconnecting configuration [api-ct.zhanglt.com.6000.jenkins] ...
SSH: Transferred 1 file(s)
Build step 'Send files or execute commands over SSH' changed build result to SUCCESS
Finished: SUCCESS
game over
jenkins line¶
Contents:
jenkins line¶
http://www.jianshu.com/p/144254a055ba
http://m.blog.csdn.net/wh211212/article/details/53484028
http://hengyunabc.github.io/deploy-system-build-with-jenkins-ansible-supervisor/
http://m.it168.com/article_1673109.html
https://zhuanlan.zhihu.com/p/23255035#comment-177559091
测试
line¶
http://www.moye.me/2016/03/03/nodejs_ci_by_jenkins/ https://johnhamelink.com/2012/11/13/testing-nodejs-with-jenkins https://cnodejs.org/topic/58439ca33ebad99b336b1d50 https://codeforgeek.com/2016/04/continuous-integration-deployment-jenkins-node-js/ https://mp.weixin.qq.com/s/ETo_IGKu8ia0uWFs798arw
http://www.cnblogs.com/moye/p/nodejs_project_with_jenkins_ci.html https://www.jianyujianyu.com/using-github-and-jenkins-deploy-hexo/ https://www.ctolib.com/topics-120724.html https://segmentfault.com/a/1190000010200161
soft-test¶
Contents:
web-test¶
Contents:
web-test line¶
https://www.zhihu.com/question/41440770?sort=created
http://www.cnblogs.com/imyalost/p/5622867.html
https://www.zhihu.com/question/24248575
selenium + python
http://www.testclass.net/selenium_python/
http://www.seleniumhq.org/download/#side_plugins
vsphere¶
Contents:
vsphere-60天过期¶
https://jingyan.baidu.com/article/3c48dd3499c717e10be358ee.html
http://www.mamicode.com/info-detail-1488564.html
VMWare:vSphere6 企业版参考序列号
我在 192.168.31.196 上用的是这个：
HV4WC-01087-1ZJ48-031XP-9A843
mac¶
Contents:
macOSX-sshd-开启或禁止¶
如果说我就是想禁止我的mac被远程登陆呢.
http://blog.csdn.net/govago/article/details/8536165
====方式1: UI设置开启============== 系统偏好设置 => 共享＝>勾选允许远程登陆。
====方式2:手动设置开启============== 1. 配置文件路径： /etc/sshd_config
jlch¶
Contents:
deploy¶
Contents:
factor 后台部署说明¶
以下部署行为的操作系统为CentOS，均在终端上操作
[部署ruby与redis]¶
将文件夹 ~/ruby与~/redis复制到需要部署的机器上
1 、cd ~/ruby/ruby 目录下，部署ruby及安装相关依赖文件 2 、执行命令：./rpm-install-ruby-2.4.sh 3 、cd ~/ruby/gem-install-redis 目录下，部署redis 4 、执行命令：./install.sh
5 、cd ~/redis/by-rpm 目录下，安装相关的依赖包 6 、执行命令：./install.sh
注意：emalloc-3.6.0-1.el7.x86_64.rpm安装不成功,可以忽略7 、cd ~/redis/redis-conf-haitong 目录下，进行redis配置 8 、执行命令：./redis-deploy-ip-port-passwd.sh $1 $2 $3。
注意：$1是部署机器的ip地址，$2是部署机器的端口号（开放端口），$3是redis的密码。建议，每台部署机器，提供两个端口号进行部署 该命令会会在当前目录下生成一个文件夹，名称为：node-{$2} ，新生成的文件夹有一个配置文件redis.conf eg : ./redis-deploy-ip-port-passwd.sh 188.190.117.122 7000 wudang
生成的文件夹为 node-7000，node-7000下有redis.conf文件9 、执行命令：systemctl stop redis。 停止部署机器的redis服务 10、执行命令：redis-server ./node-{$2}/redis.conf &。启动部署机器的redis服务，“&”保持后台运行。eg: redis-server ./node-7000/redis.conf & 11、执行命令：ps -ef | grep redis 查看redis进程 12、选择ip为188.190.117.122的部署机器，cd ~/redis/redis-conf-haitong目录下，创建redis集群。
执行命令： ./redis-trib.rb create –replicas 1 188.190.117.122:7000 188.190.117.122:7001 188.190.117.123:7002 188.190.117.123:7003 188.190.117.124:7004 188.190.117.124:7005(可以跟多台机器的redis地址)
- 13、进入集群配置连接redis的密码：
- 首次连接：redis-cli -h 188.190.117.122 -p 7000 -c
- 188.190.117.122:7000> CONFIG SET requirepass “wudang” OK 188.190.117.122:7000> AUTH wudang OK 188.190.117.122:7000> exit
14、测试：redis-cli -h 188.190.117.122 -p 7000 -a wudang -c
[部署node]¶
- 1、cd ~/node目录下，部署node:
- xz -d node-v6.11.4-linux-x64.tar.xz
- tar -xvf node-v6.11.4-linux-x64.tar sudo ln -s /home/jlch/node-v6.11.4-linux-x64/bin/node /usr/bin/node node -v
- 2、部署pm2管理：
- tar -zxvf pm2.tar.gz
- mv ~/pm2/ node-v6.11.4-linux-x64/lib/node_modules/ sudo ln -s /home/jlch/node-v6.11.4-linux-x64/lib/node_modules/pm2/bin/pm2 pm2 -v
[部署API服务]¶
1、cd ~/api目录下 2、tar -xvf factor_ht_svr*.tar.gz 3、修改./conf/下的配置文件：
1）conf/app.js修改API的port 2）conf/mysql.js修改mysql的连接地址，及其用户密码 3）conf/redis.js修改redis的连接地址，及其用户密码4、pm2命令启动API服务。eg pm2 start index.js –name “factor_ht_svr”
[部署Nginx]¶
1、cd ~/node/nginx目录下，安装nginx。 2、执行命令：./install.sh 3、查看Nginx服务状态，及启动Nginx服务。相关命令：systemctl status nginx.service ；systemctl stop nginx.service；systemctl start nginx.service 4、Nginx配置：
- 若需要配置nginx性能，那么需要替换/etc/nginx/nginx.conf文件，替换步骤：
- 1）cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.20171030 2) cp ./nginx.conf /etc/nginx/
5、API代理的配置：cp 3001.conf /etc/nginx/conf.d/ 6、重启Nginx服务，建议命令：systemctl restart nginx.service
[部署Web客户端]¶
需要部署的机器都需重新部署 1、备份nginx下的文件：
mv /usr/share/nginx/html/factor/ /usr/share/nginx/html/factor.20171027/
- 2、cd ~/html目录下，解压文件：
- tar -zxvf factor.20171030.tar.gz
- 3、替换新的文件：
- mv factor/ /usr/share/nginx/html/factor/
不连网 部署 factor 海通版本¶
redis¶
tar -zxvf redis.tar.gz
nginx¶
ls
wget http://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.12.1-1.el7.ngx.x86_64.rpm
ls
rpm -ivh nginx-1.12.1-1.el7.ngx.x86_64.rpm
sudo rpm -ivh nginx-1.12.1-1.el7.ngx.x86_64.rpm
ls
which nginx
ls /etc/nginx/conf.d/
ls /usr/share/nginx/html/
nodejs¶
ls
which xz
xz -d node-v6.11.4-linux-x64.tar.xz
ls
tar -xvf node-v6.11.4-linux-x64.tar
ls
cd node-v6.11.4-linux-x64/
ls
cd bin/
sudo ln -s /home/jlch/node-v6.11.4-linux-x64/bin/node /usr/bin/node
cd
node
pm2¶
ls
tar -zxvf pm2.tar.gz
ls
cd pm2/
ls
cd bin/
ls
pwd
./pm2 list
pwd
cd ~
ls
cd node-v6.11.4-linux-x64/
ls
cd lib/node_modules/
ls
cd npm/
ls
cd ..
pwd
ls
mv ~/pm2/ .
ls
cd pm2
ls
cd bin
ls
pwd
sudo ln -s /home/jlch/node-v6.11.4-linux-x64/lib/node_modules/pm2/bin/pm2 /usr/bin/pm2
cd
pm2 l
- 如果这里出错了，可能是因为之前操作过pm2, 所以请将 ~/.pm2/ 删除，也就是
- rm -rf ~/.pm2/
factor_svr¶
tar -zxvf factor_svr.tar.gz
factor-html¶
cp factor-html/ /usr/share/nginx/html/factor/
海通 短线精灵¶
我们把 120.234.23.100 是 从 移动线路，修改成电信线路，同时IP有变动：换成了 202.104.136.202 。 120.234.23.99，保留移动线路
120.234.23.99:6379 => 10.10.12.19:6379 202.104.136.202:6379 => 10.10.13.103:6379
transmitter¶
判断外网访问是否正常¶
curl -X POST "http://qliang.com.cn/transmitter_api/v1/session" -H "accept: application/json" -H "content-type: application/json" -d "{ \"login_name\": \"htqilei\", \"login_pwd\": \"123456\"}"
curl -X POST "https://api-cm.zhangtl.com/transmitter_api/v1/session" -H "accept: application/json" -H "content-type: application/json" -d "{ \"login_name\": \"htqilei\", \"login_pwd\": \"123456\"}"
curl -X POST "https://api-ct.zhangtl.com:8443/transmitter_api/v1/session" -H "accept: application/json" -H "content-type: application/json" -d "{ \"login_name\": \"htqilei\", \"login_pwd\": \"123456\"}"
openx¶
Contents:
singals 信号¶
Contents:
tick signals debug¶
配置 config.json¶
让 redis 指向 测试环境 192.168.31.240
回放行情¶
node quote_player.js 201710311448
redis 查看效果¶
2 个 terminal
psubscribe Topics.signals.quote.1Min.S*
psubscribe Topics.market.snapshot.SZ
生成信号¶
node tick-signals.js SZ
测试程序¶
node quote_test.js
tick singals feedback¶
20171031 obj.ID: 002036 GTTS¶
第1次出现：
obj.TM: 20171031 145406 000 change5: -0.02016546018614274 price: 18.95
- 第1次消失：
- obj.TM: 20171031 145415 000 change5: -0.019658561821003572 price: 18.95
第2次出现：
obj.TM: 20171031 145418 000 change5: -0.020175892395240407 price: 18.94
第2次消失：
obj.TM: 20171031 145421 000 change5: -0.019141231246766553 price: 18.96
第3次出现：
obj.TM: 20171031 145427 000 change5: -0.02016546018614274 price: 18.95
第3次消失：
obj.TM: 20171031 145439 000 change5: -0.019141231246766553 price: 18.96
左总：针对 002036.SZ, GTTS（高台跳水），经核查，我们给出的信号是正确的。
jlch record¶
Contents:
zhongtai-a¶
中泰，资金流向需求
中泰将数据给蛟龙，蛟龙返回结果给中泰。
对接人：胡显锋 外部需求方：中泰 事由：觉得蛟龙资金流向不准确。 期限：长期项目（1年以上） 周期：交易日 时间：
- 中泰：15:00-17:00
- 蛟龙：17:00-20:00
- 数据量：
- 第一次给10天：3000*7*10
- 之后：3000*7, 且不会新增
中泰缺少应急方案。
jlch network¶
Contents:
把ip从办公31段移动到生产10段¶
env¶
192.168.31.247 =》 10.10.12.20
step¶
修改IP配置如下¶
jlch@uRedis:~$ sudo cat /etc/network/interfaces
[sudo] password for jlch:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eno1
iface eno1 inet manual
bond-master bond0
auto eno2
iface eno2 inet manual
bond-master bond0
auto bond0
iface bond0 inet static
address 10.10.12.20
netmask 255.255.255.0
gateway 10.10.12.1
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 202.96.134.133
bond-mode 0
bond-miimon 100
bond-slaves eno1 eno2
jlch@uRedis:~$
重启网络¶
jlch@ubuntu:~$ sudo /etc/init.d/networking status
jlch@ubuntu:~$ sudo /etc/init.d/networking restart
jlch@ubuntu:~$ sudo /etc/init.d/networking status
[sudo] password for jlch:
● networking.service - Raise network interfaces
Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
Drop-In: /run/systemd/generator/networking.service.d
└─50-insserv.conf-$network.conf
Active: active (exited) since Thu 2017-11-30 02:23:26 CST; 7h ago
Docs: man:interfaces(5)
Process: 50569 ExecStop=/sbin/ifdown -a --read-environment (code=exited, status=0/SUCCESS)
Process: 50624 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
Process: 50617 ExecStartPre=/bin/sh -c [ "$CONFIGURE_INTERFACES" != "no" ] && [ -n "$(ifquery --read-environment --list --exclude=lo)" ] && udevadm settle (code=exited, status=0/SUCCESS)
Main PID: 50624 (code=exited, status=0/SUCCESS)
Tasks: 0
Memory: 0B
CPU: 0
CGroup: /system.slice/networking.service
Nov 30 02:23:25 ubuntu systemd[1]: Starting Raise network interfaces...
Nov 30 02:23:25 ubuntu ifup[50624]: Waiting for bonding kernel module to be ready (will timeout after 5s)
Nov 30 02:23:25 ubuntu ifup[50624]: Waiting for bond master bond0 to be ready
Nov 30 02:23:26 ubuntu systemd[1]: Started Raise network interfaces.
在重启后，可能就会断开连接了。 这个时候需要网络工程师作调整（请通知他），则，直到网络工程师调整完成后，再次 ssh 连接。
检查¶
jlch@ubuntu:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 18:66:da:9d:ca:62 brd ff:ff:ff:ff:ff:ff
inet 192.168.31.247/24 brd 192.168.31.255 scope global eno1
valid_lft forever preferred_lft forever
3: eno2: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN group default qlen 1000
link/ether 18:66:da:9d:ca:62 brd ff:ff:ff:ff:ff:ff
4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 18:66:da:9d:ca:64 brd ff:ff:ff:ff:ff:ff
5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 18:66:da:9d:ca:65 brd ff:ff:ff:ff:ff:ff
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 18:66:da:9d:ca:62 brd ff:ff:ff:ff:ff:ff
inet 10.10.12.20/24 brd 10.10.12.255 scope global bond0
valid_lft forever preferred_lft forever
inet6 fe80::1a66:daff:fe9d:ca62/64 scope link
valid_lft forever preferred_lft forever
jlch@ubuntu:~$ ping 10.10.12.1
jlch@ubuntu:~$
ping 10.10.12.1 正常，说明网络已经OK了。
注意，这里虽然 eno1 为 192.168.31.247，且可以做到 ping 192.168.31.247, 但是，ping 192.168.31.1 已不通。这个无关紧要了。
蛟龙 防火墙 硬件¶
深信服, 提供一台linux电脑, 给了普通用户帐号.然后蛟龙网管通过web方式管理
蛟龙-windows-server加路由¶
为 windows-server 加路由, 加无线段, 方便陈近南从无线(192.168.33.***)登陆windows-server(192.168.31.242)
env¶
- IP: 192.168.31.242, 10.10.15.242
- OS: windows server
- 无线IP段: 192.168.33.0/24
- 测试IP: 192.168.33.220
step¶
打开cmd
查看一下现有route
route print
先给某一个测试 无线IP(这里为192.168.33.220)设置. 因为这里是双网卡机器, 所以这里最后应该写成 192.168.31.1 而不是本机IP 192.168.31.242
route -p add 192.168.33.220 mask 255.255.255.255 192.168.31.1
然后相互ping一下(也可以telnet 192.168.31.242 3389) 成功.
删除掉刚刚的测试
route delete 192.168.33.220
正式加路由
route -p add 192.168.33.0 mask 255.255.255.0 192.168.31.1
ref¶
jlch ops¶
Contents:
蛟龙-注册用户¶
gitlab¶
各位同事:
下午好!
因免费gitee控制严格, 为更好进行项目沟通, 现对工程进行迁移...
麻烦各同事进行如下操作:
1. 登陆 http://192.168.31.169/ ,
2. 注册用户名: 花名全拼(如: 小昭则使用 xiaozhao) ,
3. 注册邮箱: 使用 公司邮箱 ***@jiaolongchuhai.com ,
4. 回复: 注册后, 在群中回复以下内容: 花名OK( 如: "xiaozhao OK")
如有问题, 或需要任何相关协助, 请联系我!
谢谢!
bugzilla¶
各位同事:
下午好!
为更好进行项目bug管理, 今后对bug进行统一记录...
麻烦各同事进行如下操作:
1. 登陆 http://192.168.31.148/bugzilla/
2. 注册邮箱: 使用 公司邮箱 ***@jiaolongchuhai.com
3. 打开邮箱: 邮箱会收到确认邮件,确认邮件中包含一个链接(第一个),形如: http://192.168.31.148/token.cgi?t=9PrVMyPztk&a=request_new_account, 点击打开
4. 打开浏览器: 这个时候会显示"Not Found", 把鼠标移到地址栏中, 把 http://192.168.31.148/token.cgi?t=9PrVMyPztk&a=request_new_account 中的 "148/"后 加 "bugzilla/" , 变成 http://192.168.31.148/bugzilla/token.cgi?t=9PrVMyPztk&a=request_new_account , 按回车
5. 注册用户名: 花名全拼(如: 小昭则使用 xiaozhao)
6. 注册后, 在群中回复以下内容: 花名 OK( 如: "xiaozhao OK")
如有问题, 或需要任何相关协助, 请联系我!
谢谢!
virtualbox¶
Contents:
Moving A VirtualBox VM With Snapshots¶
这个页面用google浏览器打开时, 会重定向掉, 所以请使用 safari 浏览器打开(具体原因,我也不清楚)
https://ehikioya.com/moving-virtualbox-vm-snapshots/
env¶
machine A: virtualbox 192.168.31.184(ubuntu desktop) machine B: virtualbox 192.168.31.193(ubuntu desktop)
step¶
Cloning A VirtualBox VM¶
machine A
Step 1: Shut down the virtual machine you want to clone. A virtual machine in a saved or running state cannot be cloned.
Step 2: To start the cloning process, right click the virtual machine and select “Clone”.
Step 3: The wizard that opens will first ask you to enter a name for new machine. Enter your desired name and press “Next”. The name you choose must be unique. No other vm with that name should exist on the host machine.
Step 4: On the next screen, you need to select the type of clone you want. Since we want to move our vm to a new host, select “Full clone”.
Step 5: The final screen requires that you choose if you want only the current machine state or if you want to clone the snapshots too. We want the snapshots so choose the “Everything” option. In my experience, cloning with snapshots takes a significantly longer time.
Step 6: Press “Clone” and the process begins.
Add A New VM Using The Clone¶
machine B
Step 7: Now you need to copy the entire folder containing the cloned VM over to the new host. In my case, this folder is found in “C:UsersEhi KioyaVirtualBox VMs”. The folder name will be the machine name you entered in Step 3 above. Make sure this folder contains a “VirtualBox Machine Definition” file (.vbox). Copy this folder to an external or shared drive and then copy it onto the new host (the location doesn’t matter).
Step 8: Launch VirtualBox on the new host and press “Machine” on the menu. Select “Add” on the dropdown. A Windows explorer browser window will open. Browse to the folder you newly copied over and select the “VirtualBox Machine Definition” file within it. If your new host machine does not have any existing VMs, you should be fine. A new virtual machine containing all the snapshots from the original host will be created. That’s it. You’re done.
However, if your new host machine already has existing VMs, and assuming you always install “VirtualBox Guest Additions” accross all your VMs (which is very likely), then you should get an error like this:
We’ll fix this error in the next post.
fex¶
Contents:
Hello World¶
EvaEngine Quick Start¶
EvaEngine is a PHP5 & Zend Framework 2.0 based development engine which is still under developing.
Our goal is make EvaEngine to the best way to create your own custom website or webservice.
Features¶
Real Module Based¶
Every feature is an independent module in EvaEngine. What you need is just pick out features when you need them.
You could combine different modules into various websites : blog, social networking SNS, E-commerce or anything you want.
Everything is RESTFul¶
In EvaEngine, everything is designed by RESTFul style, you could build up a RESTFul webserice easily.
Plugins and Customize¶
EvaEngine is complete follow Zend Framework 2.0 code standards, it make developers easy to add features and install 3rd-part modules.
Installation¶
Get source code from github¶
mkdir evaengine cd evaengine git clone git://github.com/AlloVince/eva-engine.git git submodules update –init
Create mysql db tables¶
Create a database “eva” (or any name you want), run sql query file in this database
evaengine/data/database/eva.sql
Connect database¶
Create EvaEngine local config file by:
evaengine/config/autoload/local.config.php
Write config file as below and change the username/password to yours:
<?php return array(
- ‘db’ => array(
- ‘dsn’ => ‘mysql:dbname=eva;hostname=localhost’, ‘username’ => ‘dbusername’, ‘password’ => ‘dbpassword’,
- ),
- ‘superadmin’ => array(
- ‘username’ => ‘root’,
- ‘password’ => ‘123456’,
),
);
Bind local domain¶
Bind a local domain local.evaengine (or anything you want) to path
evaengine/public
Then visit the local domain http://local.evaengine .
Resources¶
Contact author AlloVince by his [blog](http://avnpc.com/) or email i#av2.me(replace # to @) .