【Calico系列】6 Calico 镜像初探

2020-05-26 tech kubernetes calico 14 mins 3 图 4949 字

这篇文章从 calico 的安装内容中了解 calico 的组件，基于calico v3.10.3。

标准的 kubernetes 安装 calico 的过程，参考官网手册：https://docs.projectcalico.org/getting-started/kubernetes/

参考安装的 yaml 文件，calico 包含一个配置文件，一个密钥文件，几个角色配置，一个deployment和一个daemonset：

secret: calico-etcd-secrets
ConfigMap： cni的网络配置：
ClusterRole： calico-kube-controllers/calico-node
ClusterRoleBinding： calico-kube-controllers/calico-node
ServiceAccount：calico-node/calico-kube-controllers
Deployment：calico-kube-controllers
DaemonSet：calico-node
1. init容器
  1. install-cni （This container installs the CNI binaries）
  2. flexvol-driver （ Felix 通过socket与api交互，socket落盘.）
2. calico-node
  1. felix 每个节点上的一个守护进程，负责编写路由和ACLs（访问控制列表）. 还有一些其它节点上需要设置的东西。
  2. bird bgp广播
  3. confd 监听etcd修改BGP配置 AS number, logging levels, IPAM信息等

我们主要来看两个容器：

calico-kube-controller
calico-node

Calico Kubernetes controllers

源码地址：https://github.com/projectcalico/kube-controllers

官方文档：https://docs.projectcalico.org/reference/kube-controllers/configuration

必须有 k8s 的 api 读取权限，才可以监控数据变化。

必须使用 etcd 存储 calico 数据，下列controller才会生效：

policy controller：监控policies事件同步到 Calico 数据存储，和配置policies 。
namespace controller: 监控 ns 事件同步到 Calico 数据存储和配置 Calico profiles 。
serviceaccount controller: service accounts 监控和配置 Calico profiles。
workloadendpoint controller: pod labels 监控和 Calico workload endpoints 配置。
node controller: 监控节点增减，移除calico相关联的数据，可以配置监控 host endpoint 的创建和同步。默认情况下不启用这个controller。

install-cni

用来在本机生成一些配置。

https://github.com/projectcalico/cni-plugin

1591000684051

flexvol-driver

FlexVolumes的驱动，用来保障 Pods 和守护进程的安全通信。

https://github.com/projectcalico/pod2daemon

1591000699990

Calico-node

核心容器，源码地址：https://github.com/projectcalico/node

1590720690537

calico-node 基本思路是将所有的组件打包到一个项目中进行管理，减少新手上手的难度。（不由得想起了Istio 1.5）。

calico-node 使用了 runit 进行进程管理，简单说就是entrypoint 中将需要运行的应用拷贝到 /etc/service/enable 目录下，当runsvdir检查应用时，runsvdir 会启动一个 runsv 进程来执行和监控run脚本。

runit 是个进程守护程序，有着纯净的进程状态、可靠的日志记录工具、并进行系统快速启停。同时可移植、封装友好、轻量。同类应用还有 systemd/monit/supervisor 等。

普通情况下 calico-node 中主要运行这三个应用：

felix 每个节点上的一个守护进程
bird bgp广播
confd 监听etcd修改BGP配置 AS number, logging levels, IPAM信息等

贴一个启动脚本rc.local，可以帮助理解：

# Handle old CALICO_NETWORKING environment by converting to the new config.
if [ -n "$CALICO_NETWORKING" ]; then
	echo "WARNING: $CALICO_NETWORKING will be deprecated: use $CALICO_NETWORKING_BACKEND instead"
	if [ "$CALICO_NETWORKING" = "false" ]; then
		export CALICO_NETWORKING_BACKEND=none
	else
		export CALICO_NETWORKING_BACKEND=bird
	fi
fi

# Run the startup initialisation script.  These ensure the node is correctly
# configured to run.
calico-node -startup || exit 1

# Set the nodename based on the value picked by the startup procedure.
if [ ! -f "/var/lib/calico/nodename" ]; then
	echo "/var/lib/calico/nodename does not exist, exiting"
	exit 1
fi
NODENAME=$(cat /var/lib/calico/nodename)
export NODENAME

# If possible pre-allocate any tunnel addresses. 
calico-node -allocate-tunnel-addrs || exit 1

# Create a directly to put enabled service files
mkdir /etc/service/enabled

# XXX: Here and below we do all manupulations on /etc/service avoiding rm'ing
# dirs contained in Docker image. This is due to bug in Docker with graphdriver
# overlay on CentOS 7.X kernels (https://github.com/docker/docker/issues/15314)

# Allow felix to be disabled, for example, if the user is running Felix
# outside the container.
if [ -z "$CALICO_DISABLE_FELIX" ]; then
  cp -a /etc/service/available/felix /etc/service/enabled/
fi

case "$CALICO_NETWORKING_BACKEND" in
	"none" )
	# If running in policy only mode, we don't need to run BIRD / Confd.
	echo "CALICO_NETWORKING_BACKEND is none - no BGP daemon running"
	;;
	"vxlan" )
	# If running in VXLAN-only mode, we don't need to run BIRD / Confd.
	echo "CALICO_NETWORKING_BACKEND is vxlan - no need to run a BGP daemon"
	;;
	"gobgp" )
	# Run calico-bgp-daemon instead of BIRD / Confd.
	echo "CALICO_NETWORKING_BACKEND is gobgp - run calico-bgp-daemon"
	cp -a /etc/service/available/calico-bgp-daemon /etc/service/enabled/
	sh -c 'for file in `find /etc/calico/confd/conf.d/ -not -name 'tunl-ip.toml' -type f`; do rm $file; done'
	cp -a /etc/service/available/confd /etc/service/enabled/
	;;
	* )

	# Enable the confd and bird services
	cp -a /etc/service/available/bird  /etc/service/enabled/
	cp -a /etc/service/available/bird6 /etc/service/enabled/
	cp -a /etc/service/available/confd /etc/service/enabled/
	;;
esac

if [ "$CALICO_DISABLE_FILE_LOGGING" = "true" ]; then
	rm -rf /etc/service/enabled/bird/log
	rm -rf /etc/service/enabled/bird6/log
	rm -rf /etc/service/enabled/confd/log
	rm -rf /etc/service/enabled/felix/log
	rm -rf /etc/service/enabled/calico-bgp-daemon/log
fi

echo "Calico node started successfully"