在上篇文章中介绍了Operator SDK的安装和设定过程,这篇文章继续使用官方的示例来介绍一下如何使用Operator SDK进行Operator的创建和构建等过程。
- 事前准备
- 步骤1: init操作
- 步骤2: 创建API
- 步骤3: 构建镜像
- 步骤4: 运行Operator
- 步骤5: 创建自定义资源
- 步骤6: 删除CR和相关资源
- 总结
- 参考内容
liumiaocn:~ liumiao$ which operator-sdk /usr/local/bin/operator-sdk liumiaocn:~ liumiao$ operator-sdk version operator-sdk version: "v1.2.0", commit: "215fc50b2d4acc7d92b36828f42d7d1ae212015c", kubernetes version: "v1.18.8", go version: "go1.15.3", GOOS: "darwin", GOARCH: "amd64" liumiaocn:~ liumiao$ go version go version go1.15.5 darwin/amd64 liumiaocn:~ liumiao$步骤1: init操作
执行命令:operator-sdk init --domain=example.com --repo=github.com/example-inc/memcached-operator
liumiaocn:~ liumiao$ mkdir memcached-operator liumiaocn:~ liumiao$ cd memcached-operator/ liumiaocn:memcached-operator liumiao$ operator-sdk init --domain=example.com --repo=github.com/example-inc/memcached-operator Writing scaffold for you to edit... Get controller runtime: $ go get sigs.k8s.io/controller-runtime@v0.6.3 go: downloading sigs.k8s.io/controller-runtime v0.6.3 ...省略 go: downloading google.golang.org/appengine v1.5.0 Update go.mod: $ go mod tidy go: downloading go.uber.org/zap v1.10.0 ...省略 go: downloading github.com/kr/text v0.1.0 Running make: $ make go: creating new go.mod: module tmp ...省略 /Users/liumiao/go/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." go fmt ./... go vet ./... go build -o bin/manager main.go Next: define a resource with: $ operator-sdk create api liumiaocn:memcached-operator liumiao$
执行完毕,整体目录结构如下所示
liumiaocn:memcached-operator liumiao$ ls Dockerfile PROJECT config go.sum main.go Makefile bin go.mod hack liumiaocn:memcached-operator liumiao$ tree . . ├── Dockerfile ├── Makefile ├── PROJECT ├── bin │ └── manager ├── config │ ├── certmanager │ │ ├── certificate.yaml │ │ ├── kustomization.yaml │ │ └── kustomizeconfig.yaml │ ├── default │ │ ├── kustomization.yaml │ │ ├── manager_auth_proxy_patch.yaml │ │ ├── manager_webhook_patch.yaml │ │ └── webhookcainjection_patch.yaml │ ├── manager │ │ ├── kustomization.yaml │ │ └── manager.yaml │ ├── prometheus │ │ ├── kustomization.yaml │ │ └── monitor.yaml │ ├── rbac │ │ ├── auth_proxy_client_clusterrole.yaml │ │ ├── auth_proxy_role.yaml │ │ ├── auth_proxy_role_binding.yaml │ │ ├── auth_proxy_service.yaml │ │ ├── kustomization.yaml │ │ ├── leader_election_role.yaml │ │ ├── leader_election_role_binding.yaml │ │ └── role_binding.yaml │ ├── scorecard │ │ ├── bases │ │ │ └── config.yaml │ │ ├── kustomization.yaml │ │ └── patches │ │ ├── basic.config.yaml │ │ └── olm.config.yaml │ └── webhook │ ├── kustomization.yaml │ ├── kustomizeconfig.yaml │ └── service.yaml ├── go.mod ├── go.sum ├── hack │ └── boilerplate.go.txt └── main.go 12 directories, 34 files liumiaocn:memcached-operator liumiao$步骤2: 创建API
执行命令:operator-sdk create api --group cache --version v1 --kind Memcached --resource=true --controller=true
liumiaocn:memcached-operator liumiao$ operator-sdk create api --group cache --version v1 --kind Memcached --resource=true --controller=true Writing scaffold for you to edit... api/v1/memcached_types.go controllers/memcached_controller.go Running make: $ make go: creating new go.mod: module tmp go: found sigs.k8s.io/controller-tools/cmd/controller-gen in sigs.k8s.io/controller-tools v0.3.0 /Users/liumiao/go/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." go fmt ./... go vet ./... go build -o bin/manager main.go liumiaocn:memcached-operator liumiao$步骤3: 构建镜像
执行命令:make docker-build IMG=liumiaocn/memcache:v1
liumiaocn:memcached-operator liumiao$ make docker-build IMG=liumiaocn/memcache:v1 go: creating new go.mod: module tmp go: found sigs.k8s.io/controller-tools/cmd/controller-gen in sigs.k8s.io/controller-tools v0.3.0 /Users/liumiao/go/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." go fmt ./... go vet ./... /Users/liumiao/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases mkdir -p /Users/liumiao/memcached-operator/testbin test -f /Users/liumiao/memcached-operator/testbin/setup-envtest.sh || curl -sSLo /Users/liumiao/memcached-operator/testbin/setup-envtest.sh https://raw.githubusercontent.com/kubernetes-sigs/controller-runtime/v0.6.3/hack/setup-envtest.sh source /Users/liumiao/memcached-operator/testbin/setup-envtest.sh; fetch_envtest_tools /Users/liumiao/memcached-operator/testbin; setup_envtest_env /Users/liumiao/memcached-operator/testbin; go test ./... -coverprofile cover.out fetching envtest tools@1.16.4 (into '/Users/liumiao/memcached-operator/testbin') x bin/ x bin/etcd x bin/kubectl x bin/kube-apiserver setting up env vars ? github.com/example-inc/memcached-operator [no test files] ? github.com/example-inc/memcached-operator/api/v1 [no test files] ok github.com/example-inc/memcached-operator/controllers 6.020s coverage: 0.0% of statements docker build . -t liumiaocn/memcache:v1 Sending build context to Docker daemon 323.3MB Step 1/14 : FROM golang:1.13 as builder 1.13: Pulling from library/golang d6ff36c9ec48: Pull complete c958d65b3090: Pull complete edaf0a6b092f: Pull complete 80931cf68816: Pull complete 813643441356: Pull complete 799f41bb59c9: Pull complete 16b5038bccc8: Pull complete Digest: sha256:8ebb6d5a48deef738381b56b1d4cd33d99a5d608e0d03c5fe8dfa3f68d41a1f8 Status: Downloaded newer image for golang:1.13 ---> d6f3656320fe Step 2/14 : WORKDIR /workspace ---> Running in eb0a5f21e3fe Removing intermediate container eb0a5f21e3fe ---> dbb6e3e63541 Step 3/14 : COPY go.mod go.mod ---> 4ab6a945e245 Step 4/14 : COPY go.sum go.sum ---> 293a4eec3dbb Step 5/14 : RUN go mod download ---> Running in d057c3fac1d5 go: finding cloud.google.com/go v0.38.0 go: finding github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78 ...省略 go: finding sigs.k8s.io/yaml v1.2.0 Removing intermediate container d057c3fac1d5 ---> e83e6409c8f6 Step 6/14 : COPY main.go main.go ---> 72236a1f1ba2 Step 7/14 : COPY api/ api/ ---> 2e8254e6e37e Step 8/14 : COPY controllers/ controllers/ ---> 879e2f79eab2 Step 9/14 : RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager main.go ---> Running in ca5c33fa8fb0 Removing intermediate container ca5c33fa8fb0 ---> ab3a3a66312d Step 10/14 : FROM gcr.io/distroless/static:nonroot nonroot: Pulling from distroless/static e59bd8947ac7: Already exists Digest: sha256:dfabf95aa48990eec883b8b75462b1304678ebe9f36bdbca7841f5ddbd32af96 Status: Downloaded newer image for gcr.io/distroless/static:nonroot ---> aa99000bc55d Step 11/14 : WORKDIR / ---> Running in 3913d069fe17 Removing intermediate container 3913d069fe17 ---> 3241720e1ab6 Step 12/14 : COPY --from=builder /workspace/manager . ---> 4e2497bbc978 Step 13/14 : USER nonroot:nonroot ---> Running in d55a63335b21 Removing intermediate container d55a63335b21 ---> 740854feb14e Step 14/14 : ENTRYPOINT ["/manager"] ---> Running in 5271112557de Removing intermediate container 5271112557de ---> 46e4c9af60d8 Successfully built 46e4c9af60d8 Successfully tagged liumiaocn/memcache:v1 liumiaocn:memcached-operator liumiao$
注:如果事前docker login设定完毕,可以直接执行make docker-build docker-push推送至镜像仓库中。
构建之后相关镜像信息如下所示:
liumiaocn:memcached-operator liumiao$ docker images |grep memcache liumiaocn/memcache v1 46e4c9af60d8 6 minutes ago 46.5MB liumiaocn:memcached-operator liumiao$步骤4: 运行Operator
使用如下命令运行刚刚创建的Operator
执行命令:make install && make deploy IMG=liumiaocn/memcache:v1
liumiaocn:memcached-operator liumiao$ make install && make deploy IMG=liumiaocn/memcache:v1 go: creating new go.mod: module tmp go: found sigs.k8s.io/controller-tools/cmd/controller-gen in sigs.k8s.io/controller-tools v0.3.0 /Users/liumiao/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases go: creating new go.mod: module tmp go: downloading sigs.k8s.io/kustomize/kustomize/v3 v3.5.4 ...省略 go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1 /Users/liumiao/go/bin/kustomize build config/crd | kubectl apply -f - customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com created go: creating new go.mod: module tmp go: found sigs.k8s.io/controller-tools/cmd/controller-gen in sigs.k8s.io/controller-tools v0.3.0 /Users/liumiao/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases go: creating new go.mod: module tmp cd config/manager && /Users/liumiao/go/bin/kustomize edit set image controller=liumiaocn/memcache:v1 /Users/liumiao/go/bin/kustomize build config/default | kubectl apply -f - namespace/memcached-operator-system created customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com configured role.rbac.authorization.k8s.io/memcached-operator-leader-election-role created clusterrole.rbac.authorization.k8s.io/memcached-operator-manager-role created clusterrole.rbac.authorization.k8s.io/memcached-operator-metrics-reader created clusterrole.rbac.authorization.k8s.io/memcached-operator-proxy-role created rolebinding.rbac.authorization.k8s.io/memcached-operator-leader-election-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/memcached-operator-manager-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/memcached-operator-proxy-rolebinding created service/memcached-operator-controller-manager-metrics-service created deployment.apps/memcached-operator-controller-manager created liumiaocn:memcached-operator liumiao$
- 结果确认
liumiaocn:memcached-operator liumiao$ kubectl get service -A |grep memcache memcached-operator-system memcached-operator-controller-manager-metrics-service ClusterIP 10.110.206.488443/TCP 110s liumiaocn:memcached-operator liumiao$ kubectl get pod -A |grep memcache memcached-operator-system memcached-operator-controller-manager-5df85cb9cb-fmcns 2/2 Running 0 119s liumiaocn:memcached-operator liumiao$ liumiaocn:memcached-operator liumiao$ kubectl get deployment -A |grep memcache memcached-operator-system memcached-operator-controller-manager 1/1 1 1 2m13s liumiaocn:memcached-operator liumiao$
- crd确认
liumiaocn:memcached-operator liumiao$ kubectl get crds NAME CREATED AT memcacheds.cache.example.com 2020-11-24T06:32:57Z liumiaocn:memcached-operator liumiao$
详细信息确认
liumiaocn:memcached-operator liumiao$ kubectl describe crd memcacheds.cache.example.com Name: memcacheds.cache.example.com Namespace: Labels: <none> Annotations: controller-gen.kubebuilder.io/version: v0.3.0 API Version: apiextensions.k8s.io/v1 Kind: CustomResourceDefinition Metadata: Creation Timestamp: 2020-11-24T06:32:57Z Generation: 1 Managed Fields: API Version: apiextensions.k8s.io/v1 Fields Type: FieldsV1 fieldsV1: f:status: f:acceptedNames: f:listKind: f:singular: Manager: kube-apiserver Operation: Update Time: 2020-11-24T06:32:57Z API Version: apiextensions.k8s.io/v1beta1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:controller-gen.kubebuilder.io/version: f:kubectl.kubernetes.io/last-applied-configuration: f:spec: f:conversion: .: f:strategy: f:group: f:names: f:kind: f:listKind: f:plural: f:singular: f:preserveUnknownFields: f:scope: f:subresources: .: f:status: f:validation: .: f:openAPIV3Schema: .: f:description: f:properties: .: f:apiVersion: .: f:description: f:type: f:kind: .: f:description: f:type: f:metadata: .: f:type: f:spec: .: f:description: f:properties: .: f:foo: .: f:description: f:type: f:type: f:status: .: f:description: f:type: f:type: f:version: f:versions: f:status: f:acceptedNames: f:kind: f:plural: f:conditions: f:storedVersions: Manager: kubectl Operation: Update Time: 2020-11-24T06:33:00Z Resource Version: 35707 Self Link: /apis/apiextensions.k8s.io/v1/customresourcedefinitions/memcacheds.cache.example.com UID: 9ca0a50d-797b-4361-b9e8-44ee46979cbd Spec: Conversion: Strategy: None Group: cache.example.com Names: Kind: Memcached List Kind: MemcachedList Plural: memcacheds Singular: memcached Preserve Unknown Fields: true Scope: Namespaced Versions: Name: v1 Schema: openAPIV3Schema: Description: Memcached is the Schema for the memcacheds API Properties: API Version: Description: APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources Type: string Kind: Description: Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds Type: string Metadata: Type: object Spec: Description: MemcachedSpec defines the desired state of Memcached Properties: Foo: Description: Foo is an example field of Memcached. Edit Memcached_types.go to remove/update Type: string Type: object Status: Description: MemcachedStatus defines the observed state of Memcached Type: object Type: object Served: true Storage: true Subresources: Status: Status: Accepted Names: Kind: Memcached List Kind: MemcachedList Plural: memcacheds Singular: memcached Conditions: Last Transition Time: 2020-11-24T06:32:57Z Message: no conflicts found Reason: NoConflicts Status: True Type: NamesAccepted Last Transition Time: 2020-11-24T06:32:57Z Message: the initial names have been accepted Reason: InitialNamesAccepted Status: True Type: Established Stored Versions: v1 Events: <none> liumiaocn:memcached-operator liumiao$步骤5: 创建自定义资源
执行命令:kubectl apply -f config/samples/cache_v1_memcached.yaml
liumiaocn:memcached-operator liumiao$ kubectl apply -f config/samples/cache_v1_memcached.yaml memcached.cache.example.com/memcached-sample created liumiaocn:memcached-operator liumiao$
确认Operator相关日志信息
liumiaocn:memcached-operator liumiao$ kubectl logs deployment.apps/memcached-operator-controller-manager -n memcached-operator-system -c manager 2020-11-24T06:33:10.178Z INFO controller-runtime.metrics metrics server is starting to listen {"addr": "127.0.0.1:8080"} 2020-11-24T06:33:10.179Z INFO setup starting manager I1124 06:33:10.179468 1 leaderelection.go:242] attempting to acquire leader lease memcached-operator-system/f1c5ece8.example.com... 2020-11-24T06:33:10.179Z INFO controller-runtime.manager starting metrics server {"path": "/metrics"} I1124 06:33:10.285304 1 leaderelection.go:252] successfully acquired lease memcached-operator-system/f1c5ece8.example.com 2020-11-24T06:33:10.286Z INFO controller Starting EventSource {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached", "source": "kind source: /, Kind="} 2020-11-24T06:33:10.286Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"ConfigMap","namespace":"memcached-operator-system","name":"f1c5ece8.example.com","uid":"93c817f8-5a1b-46c3-a579-974425b59d14","apiVersion":"v1","resourceVersion":"35765"}, "reason": "LeaderElection", "message": "memcached-operator-controller-manager-5df85cb9cb-fmcns_db02b1d3-c780-42ab-99f1-56da6bcb7ed9 became leader"} 2020-11-24T06:33:10.387Z INFO controller Starting Controller {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached"} 2020-11-24T06:33:10.387Z INFO controller Starting workers {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached", "worker count": 1} 2020-11-24T06:37:51.245Z DEBUG controller Successfully Reconciled {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached", "name": "memcached-sample", "namespace": "default"} liumiaocn:memcached-operator liumiao$步骤6: 删除CR和相关资源
执行命令:kubectl delete -f config/samples/cache_v1_memcached.yaml
liumiaocn:memcached-operator liumiao$ kubectl delete -f config/samples/cache_v1_memcached.yaml memcached.cache.example.com "memcached-sample" deleted liumiaocn:memcached-operator liumiao$ kubectl logs deployment.apps/memcached-operator-controller-manager -n memcached-operator-system -c manager 2020-11-24T06:33:10.178Z INFO controller-runtime.metrics metrics server is starting to listen {"addr": "127.0.0.1:8080"} 2020-11-24T06:33:10.179Z INFO setup starting manager I1124 06:33:10.179468 1 leaderelection.go:242] attempting to acquire leader lease memcached-operator-system/f1c5ece8.example.com... 2020-11-24T06:33:10.179Z INFO controller-runtime.manager starting metrics server {"path": "/metrics"} I1124 06:33:10.285304 1 leaderelection.go:252] successfully acquired lease memcached-operator-system/f1c5ece8.example.com 2020-11-24T06:33:10.286Z INFO controller Starting EventSource {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached", "source": "kind source: /, Kind="} 2020-11-24T06:33:10.286Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"ConfigMap","namespace":"memcached-operator-system","name":"f1c5ece8.example.com","uid":"93c817f8-5a1b-46c3-a579-974425b59d14","apiVersion":"v1","resourceVersion":"35765"}, "reason": "LeaderElection", "message": "memcached-operator-controller-manager-5df85cb9cb-fmcns_db02b1d3-c780-42ab-99f1-56da6bcb7ed9 became leader"} 2020-11-24T06:33:10.387Z INFO controller Starting Controller {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached"} 2020-11-24T06:33:10.387Z INFO controller Starting workers {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached", "worker count": 1} 2020-11-24T06:37:51.245Z DEBUG controller Successfully Reconciled {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached", "name": "memcached-sample", "namespace": "default"} 2020-11-24T06:41:34.574Z DEBUG controller Successfully Reconciled {"reconcilerGroup": "cache.example.com", "reconcilerKind": "Memcached", "controller": "memcached", "name": "memcached-sample", "namespace": "default"} liumiaocn:memcached-operator liumiao$
最终的目录和文件结构如下所示
liumiaocn:memcached-operator liumiao$ pwd /Users/liumiao/memcached-operator liumiaocn:memcached-operator liumiao$ tree . . ├── Dockerfile ├── Makefile ├── PROJECT ├── api │ └── v1 │ ├── groupversion_info.go │ ├── memcached_types.go │ └── zz_generated.deepcopy.go ├── bin │ └── manager ├── config │ ├── certmanager │ │ ├── certificate.yaml │ │ ├── kustomization.yaml │ │ └── kustomizeconfig.yaml │ ├── crd │ │ ├── bases │ │ │ └── cache.example.com_memcacheds.yaml │ │ ├── kustomization.yaml │ │ ├── kustomizeconfig.yaml │ │ └── patches │ │ ├── cainjection_in_memcacheds.yaml │ │ └── webhook_in_memcacheds.yaml │ ├── default │ │ ├── kustomization.yaml │ │ ├── manager_auth_proxy_patch.yaml │ │ ├── manager_webhook_patch.yaml │ │ └── webhookcainjection_patch.yaml │ ├── manager │ │ ├── kustomization.yaml │ │ └── manager.yaml │ ├── prometheus │ │ ├── kustomization.yaml │ │ └── monitor.yaml │ ├── rbac │ │ ├── auth_proxy_client_clusterrole.yaml │ │ ├── auth_proxy_role.yaml │ │ ├── auth_proxy_role_binding.yaml │ │ ├── auth_proxy_service.yaml │ │ ├── kustomization.yaml │ │ ├── leader_election_role.yaml │ │ ├── leader_election_role_binding.yaml │ │ ├── memcached_editor_role.yaml │ │ ├── memcached_viewer_role.yaml │ │ ├── role.yaml │ │ └── role_binding.yaml │ ├── samples │ │ ├── cache_v1_memcached.yaml │ │ └── kustomization.yaml │ ├── scorecard │ │ ├── bases │ │ │ └── config.yaml │ │ ├── kustomization.yaml │ │ └── patches │ │ ├── basic.config.yaml │ │ └── olm.config.yaml │ └── webhook │ ├── kustomization.yaml │ ├── kustomizeconfig.yaml │ └── service.yaml ├── controllers │ ├── memcached_controller.go │ └── suite_test.go ├── cover.out ├── go.mod ├── go.sum ├── hack │ └── boilerplate.go.txt ├── main.go └── testbin ├── bin │ ├── etcd │ ├── kube-apiserver │ └── kubectl └── setup-envtest.sh 21 directories, 54 files liumiaocn:memcached-operator liumiao$总结
看似只是按照步骤创建了一系列的操作,实际上如果有真正进行CRD封装的经验的会了解这个过程中脚手架确实能帮助省去了很多精力,比如可以直接将精力放在生成的controller的实现之上。
参考内容https://sdk.operatorframework.io/docs/building-operators/golang/quickstart/