Operator Lifecycle Manager – Core Tasks

Docs: Creating operator manifests

Wed, 25 Mar 2020 00:00:00 +0000

OLM requires you to provide metadata about your operator, so that the operator’s lifecycle can be managed safely on a cluster. This section introduces you to the process of packaging the metadata in a format that is compatible with OLM.

This is very similar to packaging software for a traditional operating system - think of the packaging step for OLM as the stage at which you make your rpm, dep, or apk bundle.

Writing your Operator Manifests

OLM uses a CRD called ClusterServiceVersion (CSV) to describe a single instance of a version of an operator. This is the main entry point for packaging an operator for OLM.

There are two important ways to think about the CSV:

Like an rpm or deb, it collects metadata about the operator that is required to install it onto the cluster.
Like a Deployment that can stamp out Pods from a template, the ClusterServiceVersion describes a template for the operator Deployment and can stamp them out.

This is all in service of ensuring that when a user installs an operator from OLM, they can understand what changes are happening to the cluster, and OLM can ensure that installing the operator is a safe operation.

Starting from an existing set of operator manifests

For this example, we’ll use the example manifests from the example memcached operator.

These manifests consist of:

CRDs that define the APIs your operator will manage.
Operator (operator.yaml), containing theDeployment that runs your operator pods.
RBAC (role.yaml, role_binding.yaml, service_account.yaml) that configures the service account permissions your operator requires.

Building a minimal ClusterServiceVersion from these manifests requires transplanting the contents of the Operator definition and the RBAC definitions into a CSV. Together, your CSV and CRDs will form the package that you give to OLM to install an operator.

Basic Metadata (Optional)

Let’s start with a CSV that only contains some descriptive metadata:

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
 annotations:
 name: memcached-operator.v0.10.0
spec:
 description: This is an operator for memcached.
 displayName: Memcached Operator
 keywords:
 - memcached
 - app
 maintainers:
 - email: corp@example.com
 name: Some Corp
 maturity: alpha
 provider:
 name: Example
 url: www.example.com
 version: 0.10.0
 minKubeVersion: 1.20.0

Most of these fields are optional, but they provide an opportunity to describe your operator to potential or current users.

Installation Metadata (Required)

The next section to add to the CSV is the Install Strategy - this tells OLM about the runtime components of your operator and their requirements.

Here is an example of the basic structure of an install strategy:

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
 annotations:
 name: memcached-operator.v0.10.0
spec:
 install:
 # strategy indicates what type of deployment artifacts are used
 strategy: deployment
 # spec for the deployment strategy is a list of deployment specs and required permissions - similar to a pod template used in a deployment
 spec:
 permissions:
 - serviceAccountName: memcached-operator
 rules:
 - apiGroups:
 - ""
 resources:
 - pods
 verbs:
 - '*'
 # the rest of the rules
 # permissions required at the cluster scope
 clusterPermissions:
 - serviceAccountName: memcached-operator
 rules:
 - apiGroups:
 - ""
 resources:
 - serviceaccounts
 verbs:
 - '*'
 # the rest of the rules
 deployments:
 - name: memcached-operator
 spec:
 replicas: 1
 # the rest of a deployment spec

deployments is an array - your operator may be composed of several seperate components that should all be deployed and versioned together.

It’s also important to tell OLM the ways in which your operator can be deployed, or its installModes. InstallModes indicate if your operator can be configured to watch, one, some, or all namespaces. Please see the document on operator scoping with operatorgroups for more information.

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
 name: memcached-operator.v0.10.0
spec:
 # ...
 installModes:
 - supported: true
 type: OwnNamespace
 - supported: true
 type: SingleNamespace
 - supported: false
 type: MultiNamespace
 - supported: true
 type: AllNamespaces

Using faq to build an install strategy from an existing deployment and rbac

faq is a wrapper around jq that can handle multiple input and output formats, like the yaml we’re working with now. The following example requires that faq be installed and references the example memcached operator.

Here is a simple faq script that can generate an install strategy from a single deployment:

faq -f yaml '{install: {strategy: "deployment", spec:{ deployments: [{name: .metadata.name, template: .spec }] }}}' operator.yaml

If you have an existing CSV csv.yaml (refer to the example from Basic Metadata) and you’d like to insert or update an install strategy from a deployment operator.yaml, a role role.yaml, and a service account service_account.yaml, that is also possible:

faq -f yaml -o yaml --slurp '.[0].spec.install = {strategy: "deployment", spec:{ deployments: [{name: .[1].metadata.name, template: .[1].spec }], permissions: [{serviceAccountName: .[3].metadata.name, rules: .[2].rules }]}} | .[0]' csv.yaml operator.yaml role.yaml service_account.yaml

Defining APIs (Required)

By definition, operators are programs that can talk to the Kubernetes API. Often, they are also programs that extend the Kubernetes API, by providing an interface via CustomResourceDefinitions or, less frequently, APIServices.

Owned APIs

Exactly which APIs are used and which APIs are watched or owned is important metadata for OLM. OLM uses this information to determine if dependencies are met and ensure that no two operators fight over the same resources in a cluster.

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
 name: memcached-operator.v0.10.0
spec:
 # ...
 customresourcedefinitions:
 owned:
 # a list of CRDs that this operator owns
 # name is the metadata.name of the CRD (which is of the form <plural>.<group>)
 - name: memcacheds.cache.example.com
 # version is the spec.versions[].name value defined in the CRD
 version: v1alpha1
 # kind is the CamelCased singular value defined in spec.names.kind of the CRD.
 kind: Memcached
 # human-friendly display name of the CRD for rendering in graphical consoles (optional)
 displayName: Memcached Cluster
 # a short description of the CRDs purpose for rendering in graphical consoles (optional)
 description: Represents a memcached cluster

Required APIs

Similarly, there is a section spec.customresourcedefinitions.required, where dependencies can be specified. The operators that provide those APIs will be discovered and installed by OLM if they have not been installed.

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
 name: other-operator.v1.0
spec:
 # ...
 customresourcedefinitions:
 required:
 # a list of CRDs that this operator requires
 # name is the metadata.name of the CRD (which is of the form <plural>.<group>)
 - name: others.example.com
 # version is the spec.versions[].name value defined in the CRD
 version: v1alpha1
 # kind is the CamelCased singular value defined in spec.names.kind of the CRD.
 kind: Other

Dependency resolution and ownership is discussed more in depth in the here.

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
 name: memcached-operator.v0.10.0
spec:
 # ...
 customresourcedefinitions:
 owned:
 # a list of CRDs that this operator owns
 # name is the metadata.name of the CRD (which is of the form <plural>.<group>)
 - name: memcacheds.cache.example.com
 # version is the spec.versions[].name value defined in the CRD
 version: v1alpha1
 # kind is the CamelCased singular value defined in spec.names.kind of the CRD.
 kind: Memcached

NativeAPIs (recommended)

There are often cases where you wish to depend on an API that is either provided natively by the platform (i.e. Pod) or sometimes by another operator that is outside the control of OLM.

In those cases, those dependencies can be described in the CSV as well, via nativeAPIs

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
 name: other-operator.v1.0
spec:
 nativeAPIs:
 - group: ""
 version: v1
 kind: Pod

The absence of any required nativeAPIs from a cluster will pause the installation of the operator, and OLM will write a status into the CSV indicating the missing APIs.

TODO: example status

nativeAPIs is an optional field, but the more information you give OLM about the context in which your operator should be running, the more informed decisions OLM can make.

Packaging Additional Objects Alongside an Operator

Operators can include additional objects alongside their CSV in the /manifests directory. These objects should be YAML files and valid kubernetes objects. The following objects are supported as of OLM 0.16.0:

Secret
ClusterRole
ClusterRoleBinding
ConfigMap
ServiceAccount
Service
Role
RoleBinding
PrometheusRule
ServiceMonitor
PodDisruptionBudget
PriorityClass
VerticalPodAutoscaler
ConsoleYAMLSample
ConsoleQuickStart
ConsoleCLIDownload
ConsoleLink

Note: some of these objects can affect an upgrade of the cluster and potentially cause problems for workloads unrelated to your operator. Be sure to understand the safe use of these objects before packaging them with your operator. See the docs linked above for more information on these objects as they relate to OLM.*

Limitations on Pod Disruption Budgets

No limitations are placed on the contents of a PDB at this time when installing on-cluster. However, the following are suggested guidelines to follow when including PDB objects in a bundle.

maxUnavailable field cannot be set to 0 or 0%.
- This can make a node impossible to drain and block important lifecycle actions like operator upgrades or even cluster upgrades.
minAvailable field cannot be set to 100%.
- This can make a node impossible to drain and block important lifecycle actions like operator upgrades or even cluster upgrades.

Limitations on Priority Classes

No limitations are placed on the contents of a PriorityClass manifest at this time when installing on-cluster. However, the following is a suggested guideline to follow when including PriorityClass objects in a bundle.

globalDefault should always be false on a PriorityClass included in a bundle.
- Setting globalDefault on a PriorityClass means that all pods in the cluster without an explicit priority class will use this default PriorityClass. This can unintentionally affect other pods running in the cluster.

Extension apiservers and APIServices

TODO: Document on extension apiservers for operators that do not rely on CRDs to provide its API.

Advanced and Optional features

TODO: Documentation for advanced operator configuration which includes additional suggestions for further integration with OLM.

Operator SDK

You can also generate the manifests for your bundle using the operator-sdk binary. Checkout the documentation for generating CSV using operator-sdk here.

Docs: Creating an Operator Bundle

Mon, 11 Jan 2021 00:00:00 +0000

Prerequisites

opm
docker version 17.03+ or podman v1.2.0+ or buildah v1.7+.

Operator Bundle

An Operator Bundle is a container image that stores Kubernetes manifests and metadata associated with an operator. A bundle is meant to represent a specific version of an operator on cluster. Once you have the ClusterServiceVersion(CSV) for your operator, you can create an operator bundle using the CSV and the CRDs for your operator.

We refer to a directory of files with one ClusterServiceVersion as a bundle that includes a CSV and the CRDs in its manifest directory, though additional kubernetes objects may be included. The directory also includes an annotations file in its metadata folder which defines some higher level aggregate data that helps to describe the format and package information about how the bundle should be added into a catalog of bundles. Finally, a Dockerfile can be built from the information in the directory to build the operator bundle image.

 # example bundle
etcd
├── manifests
│ ├── etcdcluster.crd.yaml
│ └── etcdoperator.clusterserviceversion.yaml
├── metadata
│ └── annotations.yaml
└── Dockerfile

Contents of annotations.yaml and the Dockerfile

The annotations.yaml and the Dockerfile can be generated using the opm tool’s alpha bundle generate command.

Usage:
 opm alpha bundle generate [flags]

Flags:
 -c, --channels string The list of channels that bundle image belongs to
 -e, --default string The default channel for the bundle image
 -d, --directory string The directory where bundle manifests for a specific version are located.
 -h, --help help for generate
 -u, --output-dir string Optional output directory for operator manifests
 -p, --package string The name of the package that bundle image belongs to

 Note:
 * All manifests yaml must be in the same directory.

For example, to generate the annotations.yaml and Dockerfile for the example bundle mentioned above, the command for the generate task is:

$ opm alpha bundle generate --directory ./etcd --package etcd --channels stable --default stable

After the generate command is executed, the Dockerfile is generated in the directory where command is run. By default, the annotations.yaml file is located in a folder named metadata in the same root directory as the input directory containing manifests.

If the --output-dir parameter is specified, that directory becomes the parent for a new pair of folders manifests/ and metadata/, where manifests/ is a copy of the passed in directory of manifests and metadata/ is the folder containing annotations.yaml:

$ tree etcd
etcd
├── manifests
│ ├── etcdcluster.crd.yaml
│ └── etcdoperator.clusterserviceversion.yaml
├── my-output-manifest-dir
│ ├── manifests
│ │ ├── etcdcluster.crd.yaml
│ │ └── etcdoperator.clusterserviceversion.yaml
│ └── metadata
│ └── annotations.yaml
└── Dockerfile

The annotations.yaml contains the following information as labels that are used to annotate the operator bundle container image:

The label operators.operatorframework.io.bundle.mediatype.v1 reflects the media type or format of the operator bundle. It could be helm charts, plain Kubernetes manifests etc.
The label operators.operatorframework.io.bundle.manifests.v1 reflects the path in the image to the directory that contains the operator manifests. This label is reserved for the future use and is set to manifests/ for the time being.
The label operators.operatorframework.io.bundle.metadata.v1 reflects the path in the image to the directory that contains metadata files about the bundle. This label is reserved for the future use and is set to metadata/ for the time being.
The manifests.v1 and metadata.v1 labels imply the bundle type:
- The value manifests.v1 implies that this bundle contains operator manifests.
- The value metadata.v1 implies that this bundle has operator metadata.
The label operators.operatorframework.io.bundle.package.v1 reflects the package name of the bundle.
The label operators.operatorframework.io.bundle.channels.v1 reflects the list of channels the bundle is subscribing to when added into an operator registry
The label operators.operatorframework.io.bundle.channel.default.v1 reflects the default channel an operator should be subscribed to when installed from a registry. This label is optional if the default channel has been set by previous bundles and the default channel is unchanged for this bundle.

The annotations.yaml file generated in the example above would look like:

annotations:
 operators.operatorframework.io.bundle.mediatype.v1: "registry+v1"
 operators.operatorframework.io.bundle.manifests.v1: "manifests/"
 operators.operatorframework.io.bundle.metadata.v1: "metadata/"
 operators.operatorframework.io.bundle.package.v1: "etcd"
 operators.operatorframework.io.bundle.channels.v1: "stable"
 operators.operatorframework.io.bundle.channel.default.v1: "stable"

The Dockerfile generated in the example above would look like:

FROM scratch

LABEL operators.operatorframework.io.bundle.mediatype.v1=registry+v1
LABEL operators.operatorframework.io.bundle.manifests.v1=manifests/
LABEL operators.operatorframework.io.bundle.metadata.v1=metadata/
LABEL operators.operatorframework.io.bundle.package.v1=test-operator
LABEL operators.operatorframework.io.bundle.channels.v1=beta,stable
LABEL operators.operatorframework.io.bundle.channel.default.v1=stable

ADD test/*.yaml /manifests
ADD test/metadata/annotations.yaml /metadata/annotations.yaml

Bundle images

An Operator Bundle is built as a scratch (i.e non-runnable) container image that contains information about the operator manifests and metadata inside the image(stored in a database inside the image). The image can then be pushed and pulled from an OCI-compliant container registry.

The opm tool can be used to interact directly with these images. Once you have your manifests defined and have created a directory in the format defined above, building the image is as simple as defining a Dockerfile and building that image:

$ podman build -t quay.io/my-container-registry-namespace/my-manifest-bundle:latest -f bundle.Dockerfile .

Once you have built the container, you can publish it like any other container image:

$ podman push quay.io/my-container-registry-namespace/my-manifest-bundle:latest

Of course, this build step can be done with any other OCI spec container tools like docker, buildah, libpod, etc

Validating your bundle

Once you’ve created your bundle, you will want to ensure that your bundle is valid and in the correct format. The api library contains a validation library that is used by operator-framework tools like operator-sdk and opm to validate operator bundles. For more information on validating via the operator-sdk see the operator-sdk bundle validate documentation.

The opm alpha bundle validate command will validate a bundle image from a remote registry to determine if its format and content information are accurate. The following validators will run by default on every invocation of the command.

CSV validator - validates the CSV name and replaces fields.
CRD validator - validates the CRDs OpenAPI V3 schema.
Bundle validator - validates the bundle format and annotations.yaml file as well as the optional dependencies.yaml file.

For example:

$ opm alpha bundle validate --tag quay.io/test/test-operator:latest --image-builder docker

Optional Validation

Some validators are disabled by default and can be optionally enabled via the --optional-validators or -o flag.

Operatorhub validator - performs operatorhub.io validation which will check your bundle against the common criteria to distributed with OLM. To validate a bundle using custom categories use the OPERATOR_BUNDLE_CATEGORIES environmental variable to point to a json-encoded categories file. Enable this option via --optional-validators=operatorhub. This validator allows you to validate that your manifests can work with a Kubernetes cluster of a particular version using the k8s-version optional key value. (e.g. --optional-values=k8s-version=1.22)
Bundle objects validator - performs validation on resources like PodDisruptionBudgets and PriorityClasses. Enable this option via --optional-validators=bundle-objects. Multiple optional validators can be enabled at once, for example --optional-validators=operatorhub,bundle-objects.
Community validator - performs community operator bundle validation which will check your bundle against the criteria to distribute your project on the Community Catalogs. For further information see its docs. This validator allows you to validate the required labels in the catalog image by using the index-path optional key value. (e.g. --optional-values=index-path=bundle.Dockerfile).

Custom bundle categories

The operatorhub validator can verify against custom bundle categories by setting the OPERATOR_BUNDLE_CATEGORIES environment variable. Setting the OPERATOR_BUNDLE_CATEGORIES environment variable to the path to a json file containing a list of categories will enable those categories to be used when comparing CSV categories for operatorhub validation. The json file should be in the following format:

{
 "categories":[
 "Cloud Pak",
 "Registry",
 "MyCoolThing",
 ]
}

For example:

$ OPERATOR_BUNDLE_CATEGORIES=./validate/categories.json ./bin/opm alpha bundle validate --tag <bundle-tag> --image-builder docker -o operatorhub will validate the bundle using the provided categories file.

If OPERATOR_BUNDLE_CATEGORIES is not set, and operatorhub validation is enabled, the default categories will be used when performing operatorhub validation. The default categories are the following:

AI/Machine Learning
Application Runtime
Big Data
Cloud Provider
Developer Tools
Database
Integration & Delivery
Logging & Tracing
Monitoring
Networking
OpenShift Optional
Security
Storage
Streaming & Messaging

Docs: Creating a Catalog of operators

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

opm v1.19.0+ (for file-based catalogs), OR
opm v1.23.1+ (for catalog templates)

Note: This document discusses creating a catalog of operators using plaintext files to store catalog metadata, which is the latest feature of OLM catalogs. If you are looking to build catalogs using the deprecated sqlite database format to store catalog metadata instead, please read the v0.18.z version of this doc instead.

Note: catalog templates are ALPHA functionality and may adopt breaking changes

Creating a Catalog

OLM’s CatalogSource CRD accepts a container image reference to a catalog of operators that can be made available to install in a cluster. You can make your operator bundle available to install in a cluster by adding it to a catalog, packaging the catalog in a container image, and then using that image reference in the CatalogSource. This image contains all of the metadata required for OLM to manage the lifecycle of all of the operators it contains.

OLM uses a plaintext file-based catalog format (JSON or YAML) to store these records in a Catalog, and there are two approaches we can take to creating a Catalog, adding operators to it, and validating it. Let’s walk through a simple example for both approaches.

Catalog Creation Using Catalog Templates

Catalog Templates are a purpose-built simplification of File-Based Catalogs to ease common catalog operations. For this example, we’ll be using the semver template.

Note: We strongly recommend that authors create and maintain their templates in a version-controlled environment discrete from their generated catalogs. Further, we recommend that authors focus on the template as the sole artifact connecting the operator to the catalog (even going so far as only generating the file-based catalog during CI/CD tooling so it is only provided for catalog contribution.)

Catalog Creation

First we need to create the Catalog hierarchy and Dockerfile for generating the image

$ mkdir -p cool-catalog/example-operator
$ opm generate dockerfile cool-catalog

Organizing the Bundles into Channels

Let’s assume that this isn’t the first time that we have released this operator into the catalog, but it’s our first foray into templates. We need to ensure an upgrade graph edge between the older bundle version and the new one. We also want to promote this latest version in a “stable” channel. Lastly, we already use Semantic Versioning for our release numbering, and we really only care about new major (e.g. X.#.#) releases.

Note: we presume this step and template processing are performed in the source-controlled location related to operator bundle release, or at least separate from the catalog

$ cat << EOF >> example-operator-template.yaml
Schema: olm.semver
GenerateMajorChannels: true
GenerateMinorChannels: false
Stable:
 Bundles:
 - Image: repository-uri/example-operator:v0.8.9
 - Image: repository-uri/example-operator:v0.9.0
EOF

Generating the Catalog

opm alpha render-template semver -o yaml < example-operator-template.yaml > cool-catalog/catalog.yaml

Validate the catalog to ensure that the result is functional

$ opm validate cool-catalog
$ echo $?
0

Catalog Creation with Raw File-Based Catalogs

Catalog Creation

First, we need to initialize our Catalog, so we’ll make a directory for it, generate a Dockerfile that can build a Catalog image, and then populate our catalog with our operator.

Initializing the Catalog

$ mkdir cool-catalog
$ opm generate dockerfile cool-catalog
$ opm init example-operator \
 --default-channel=preview \
 --description=./README.md \
 --icon=./example-operator.svg \
 --output yaml > cool-catalog/operator.yaml

Let’s validate our catalog to see if we’re ready to ship!

$ opm validate cool-catalog
FATA[0000] invalid index:
└── invalid package "example-operator":
 └── invalid channel "preview":
 └── channel must contain at least one bundle

Alright, so it’s not valid. It looks like we need to add a bundle, so let’s do that next…

Add a bundle to the Catalog

$ opm render quay.io/example-inc/example-operator-bundle:v0.1.0 \
 --output=yaml >> cool-catalog/operator.yaml

Let’s validate again:

$ opm validate cool-catalog
FATA[0000] package "example-operator", bundle "example-operator.v0.1.0" not found in any channel entries

Add a channel entry for the bundle

We rendered the bundle, but we still haven’t yet added it to any channels. Let’s initialize a channel:

cat << EOF >> cool-catalog/operator.yaml
---
schema: olm.channel
package: example-operator
name: preview
entries:
 - name: example-operator.v0.1.0
EOF

Is the third time the charm for opm validate?

$ opm validate cool-catalog
$ echo $?
0

Success! There were no errors and we got a 0 error code.

Raw File-Based Catalogs Summary

In the general case, adding a bundle involves three discrete steps:

Render the bundle into the catalog using opm render <bundleImage>.
Add the bundle into desired channels and update the channels’ upgrade edges to stitch the bundle into the correct place.
Validate the resulting catalog.

NOTE: catalog metadata should be stored in a version control system (e.g. git) and catalog images should be rebuilt from source whenever updates are made to ensure that all changes to the catalog are auditable. Here is an example of catalog metadata being stored in github: https://github.com/operator-framework/cool-catalog, with the catalog image being rebuilt whenever there is a change: https://github.com/operator-framework/cool-catalog/blob/main/.github/workflows/build-push.yml.

Step 1 is just a simple opm render command.

Step 2 has no defined standards other than that the result must pass validation in step 3. Some operator authors may decide to hand edit channels and upgrade edges. Others may decide to implement automation (e.g. to idempotently build semver-based channels and upgrade graphs based solely on the versions of the operators in the package). There is no right or wrong answer for implementing this step as long as opm validate is successful.

There are some guidelines to keep in mind though:

Once a bundle is present in a Catalog, you should assume that one of your users has installed it. With that in mind, you should take care to avoid stranding users that have that version installed. Put another way, make sure that all previously published bundles in a catalog have a path to the current/new channel head.
Keep the semantics of the upgrade edges you use in mind. opm validate is not able to tell you if you have a sane upgrade graph. To learn more about the upgrade graph of an operator, checkout the creating an upgrade graph doc.

Build and push the catalog image

The last step is building and pushing the catalog image. For either approach, this is the same:

$ docker build . \
 -f cool-catalog.Dockerfile \
 -t quay.io/example-inc/cool-catalog:latest
$ docker push quay.io/example-inc/cool-catalog:latest

Now the catalog image is available for clusters to use and reference with CatalogSources on their cluster.

Docs: Make a Catalog available on Cluster

Wed, 28 Apr 2021 00:00:00 +0000

Once you have a catalog of operators, you can make it available on cluster by referencing it from a CatalogSource.

For example, if you have a catalog image quay.io/my-namespace/cool-catalog:latest, you can create a CatalogSource with your image:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
 name: cool-catalog
 namespace: operator
spec:
 sourceType: grpc
 image: quay.io/my-namespace/cool-catalog:latest
 displayName: Coolest Catalog
 publisher: Me
 updateStrategy:
 registryPoll:
 interval: 10m

Explanation of spec.updateStrategy

When you create a CatalogSource, it deploys a pod that serves the content you’ve stored in the catalog via a grpc API endpoint.

$ kubectl apply -f cool-catalog.yaml
catalogsource.operators.coreos.com/cool-catalog created

$ kubectl get catsrc
NAME DISPLAY TYPE PUBLISHER AGE
cool-catalog Coolest Catalog grpc Me 38s


$ kubectl get pods
NAME READY STATUS RESTARTS AGE
cool-catalog-dtqv2 1/1 Running 0 30s

It is possible to configure the CatalogSource to poll a source, such as an image registry, to check whether the catalog source pod should be updated. A common use case would be pushing new bundles to the same catalog source tag, and seeing updated operators from those bundles being installed in the cluster.

For example, say currently you have Operator X v1.0 installed in the cluster from the catalog quay.io/my-namespace/cool-catalog:main. This is the latest version of the X operator in the catalog. When a new v2.0 of Operator X is published, the v2.0 version of the X operator can be included in the catalog using the same steps described in the creating catalog doc, then the catalog image can be rebuilt and pushed to the same main tag. With catalog polling enabled, OLM will pull down the newer version of the catalog image and make the new information available.

Each type of check for an updated catalog source is called an updateStrategy. Only one updateStrategy is supported at a time. registryPoll is a type of updateStrategy that checks an image registry for an updated version of the same tag(via image SHAs). The interval defines the amount of time between each successive poll.

Caveats

The polling sequence is not instantaneous - it can take up to 15 minutes from each poll for the new catalog source pod to be deployed into the cluster. It may take longer for larger clusters.
Because OLM pulls down the image every poll interval and starts the pod, to see if its updated, the updated catalog pod must be able to be scheduled onto the cluster. If the cluster is at absolutely maximum capacity, without autoscaling enabled, this feature may not work.
OLM checks to see whether the container ImageID has changed between the old and new catalog source image when determining if an upgrade is in order. It does not actually parse the image content itself to check for later CSVs. If there is a bad upgrade to the catalog source image, simply overwrite the tag with another version and it will be pulled down, or delete and recreate the catalog source.
The polling interval should be reasonably high to ensure the update functionality works as intended. Avoid intervals less than 15m.

Using registry images that require authentication as Catalog/bundle/operator/operand images

If certain images are hosted in an authenticated container image registry, i.e a private registry, OLM is unable to pull the images by default. To enable access, you can create a pull secret that contains the authentication credentials for the registry. By referencing one or more pull secrets in a CatalogSource, OLM can handle placing the secrets in the operator and catalog namespace to allow installation.

Note: Bundle images that require authentication to pull are handled by this method. Other images required by an Operator or its Operands might require access to private registries as well. OLM does not handle placing the secrets in target tenant namespaces for this scenario, but authentication credentials can be added to the global cluster pull secret or individual namespace service accounts to enable the required access. Alternatively, if providing access to the entire cluster is not permissible, the pull secret can be added to the default service accounts of the target tenant namespaces.

To use images from private registries as Catalog/bundle images, create a secret for each required private registry. Once you have the secrets, kubectl apply your secrets to the cluster, and the create or update an existing CatalogSource object to reference one or more secrets to add a spec.secrets section and specify all secrets.

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
 name: cool-catalog
 namespace: operator
spec:
 sourceType: grpc
 secrets:
 - "<secret_name_1>"
 - "<secret_name_2>"
 image: quay.io/my-namespace/cool-catalog:latest
 displayName: Coolest Catalog
 publisher: Me
 updateStrategy:
 registryPoll:
 interval: 10m

If the imagePullSecret is referenced in the bundle, for instance when the controller-manager image is pulled from a private registry, there is no place in the API to tell OLM to attach the imagePullSecrets. As a consequence, permissions to pull the image should be added directly to the operator Deployment’s manifest by adding the required secret name to the list deployment.spec.template.spec.imagePullSecrets.

For the operator-sdk abstraction, the operator Deployment’s manifest is found under config/manager/manager.yaml. Below is an example of a controller-manager Deployment’s manifest configured with an imagePullSecret to pull container images from a private registry.

# config/manager/manager.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
 name: controller-manager
 namespace: system
 ...
spec:
 ...
 spec:
 imagePullSecrets:
 - name: "registry-auth-secret-name"

Note: It is required for the imagePullSecret to be present in the same namespace where the controller is deployed for the controller pod to start.

Docs: List operators available to install on cluster

Thu, 05 Jan 2017 00:00:00 +0000

List your operator

The PackageManifest API exposes content from existing CatalogSources on cluster. Querying that API reveals the list of operators available to install.

Note: CatalogSources in OLM are either global or namespaced. Global CatalogSources contain operators that will be available for installing in all namespaces, while namespaced CatalogSources only contains operators that are available to be installed in a specific namespace.

Using the PackageManifest API

The PackageManifest API when queried, will return the union of globally available as well as namespaced available operators, from the namespace you’re querying in.

kubectl get packagemanifest -n <namespace>

The list of available operators will be displayed as an output of those above commands:

$ kubectl get packagemanifest
NAME CATALOG AGE
cassandra-operator Community Operators 26m
etcd Community Operators 26m
postgres-operator Community Operators 26m
prometheus Community Operators 26m
wildfly Community Operators 26m

Docs: Install your operator with OLM

Fri, 30 Apr 2021 00:00:00 +0000

Once you have a catalog of operators loaded onto the cluster via a CatalogSource, you can install your operator by creating a Subscription to a specific channel.

Prerequisites

Before installing an operator into a namespace, you will need to create an OperatorGroup that targets the namespaces your operator is planning to watch, to generate the required RBACs for your operator in those namespaces. You can read more about OperatorGroup here.

Note: The namespaces targeted by the OperatorGroup must align with the installModes specified in the ClusterServiceVersion of the operator’s package. To know the installModes of an operator, inspect the packagemanifest:

kubectl get packagemanifest <operator-name> -o jsonpath="{.status.channels[0].currentCSVDesc.installModes}"

Note: This document uses a global OperatorGroup in the examples to install operators. To learn more about installing namespaced scoped operators, check out operator scoping with OperatorGroups.

Install your operator

To install an Operator, simply create a Subscription for your operator. This represents the intent to subscribe to a stream of available versions of this Operator from a CatalogSource:

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
 name: <name-of-your-subscription>
 namespace: <namespace-you-want-your-operator-installed-in>
spec:
 channel: <channel-you-want-to-subscribe-to>
 name: <name-of-your-operator>
 source: <name-of-catalog-operator-is-part-of>
 sourceNamespace: <namespace-that-has-catalog>
 installPlanApproval: <Automatic/Manual>

You can read more about the Subscription object and what the different fields mean here.

The Subscription object creates an InstallPlan, which is either automatically approved (if sub.spec.installPlanApproval: Automatic), or needs to be approved (if sub.spec.installPlanApproval: Manual), following which the operator is installed in the namespace you want.

Example: Install the latest version of an Operator

If you want to install an operator named my-operator in the namespace foo that is cluster scoped (i.e installModes:AllNamespaces), from a catalog named my-catalog that is in the namespace olm, and you want to subscribe to the channel stable,

create a global OperatorGroup (which selects all namespaces):

$ cat og.yaml

 apiVersion: operators.coreos.com/v1
 kind: OperatorGroup
 metadata:
 name: my-group
 namespace: foo

$ kubectl apply -f og.yaml
 operatorgroup.operators.coreos.com/my-group created

Then, create a subscription for the operator:

$ cat sub.yaml

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
 name: sub-to-my-operator
 namespace: foo
spec:
 channel: stable
 name: my-operator
 source: my-catalog
 sourceNamespace: olm
 installPlanApproval: Manual

$ kubectl apply -f sub.yaml
subscription.operators.coreos.com/sub-to-my-operator created

Since installPlanApproval is set to Manual, we need to manually go in and approve the InstallPlan

$ kubectl get ip -n foo

NAME CSV APPROVAL APPROVED
install-nlwcw my-operator.v0.9.2 Automatic false

$ kubectl edit ip install-nlwcw -n foo

And then change the spec.approved from false to true.

This should spin up the ClusterServiceVersion of the operator in the foo namespace, following which the operator pod will spin up.

To ensure the operator installed successfully, check for the ClusterServiceVersion and the operator deployment in the namespace it was installed in.

$ kubectl get csv -n <namespace-operator-was-installed-in>

NAME DISPLAY VERSION REPLACES PHASE
<name-of-csv> <operator-name> <version> <csv-of-previous-version> Succeeded
...
$ kubectl get deployments -n <namespace-operator-was-installed-in>
NAME READY UP-TO-DATE AVAILABLE AGE
<name-of-your-operator> 1/1 1 1 9m48s

If the ClusterServiceVersion fails to show up or does not reach the Succeeded phase, please check the troubleshooting documentation to debug your installation.

Example: Install a specific version of an Operator

If you want to install a particular version of your Operator, specify the startingCSV property in your Subscription like so:

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
 name: sub-to-my-operator
 namespace: foo
spec:
 channel: stable
 name: my-operator
 source: my-catalog
 sourceNamespace: olm
 installPlanApproval: Manual
 startingCSV: 1.1.0

Notice that approval has been set to Manual as well in order to keep OLM from immediately updating your Operator, if 1.1.0 happens to be superseded by a newer version in my-catalog. Follow the instructions from the previous example to approve the initial InstallPlan for this Subscription, so 1.1.0 is allowed to be installed.

If your intent is to pin an installed Operator to the particular version 1.1.0 you don’t need to do anything. After approving the initial InstallPlan OLM will install version 1.1.0 of your Operator and keep it at that version. When updates are discovered in the catalog, OLM will wait not proceed unless you manually approve the update.

Docs: Uninstall your operator

Wed, 25 Mar 2020 00:00:00 +0000

When uninstalling an operator managed by OLM, a Cluster Admin must decide whether or not to remove the CustomResourceDefinitions (CRD), APIServices, and resources related to these types owned by the operator. By design, when OLM uninstalls an operator it does not remove any of the operator’s owned CRDs, APIServices, or CRs in order to prevent data loss. Instead, it is left to the Cluster Admin to remove any unwanted types and resources from the cluster. This document will discuss the steps a Cluster Admin should take when uninstalling an operator.

Step 1: Identifying Resources to Remove

The cluster admin should first understand which types (CRDs and APIServices) are owned by the operator, which is available in the operator’s ClusterServiceVersion (CSV) under the spec.customresourcedefinitions.owned and spec.apiservicedefinitions.owned arrays. It is likely that users have created resources for these types since the operator was installed. The cluster admin should should decide which of these resources to delete on a case-by-case basis. If the resource is not required, delete it. The Cluster Admin should delete all unwanted resources before moving to the next step.

Note: Although deleting the CRD or APIService removes all resource of the type from the cluster, this action may lead to unintended consequences. Operators often use finalizers to execute application specific cleanup routines before removing the CR. If the API is removed, the operator will be unable to properly remove the resource, and the cluster may appear to be “stuck” as defined in this Kubernetes issue.

Step 2: Unsubscribe from the Operator

OLM uses the Subscription resource to convey a user’s intent to subscribe to the latest version of an operator. If the operator was installed with Automatic Updates (spec.InstallPlanApproval: "Automatic"), OLM will reinstall a new version of the operator even if the operator’s CSV was deleted earlier. In effect, you must tell OLM that you do not want new versions of the operator to be installed by deleting the Subscription associated with the operator. It should be noted that opting out of future upgrades by deleting a Subscription does not delete the associated CSV as this ensures that a specific version of the operator is available on cluster and is never upgraded.

You can list existing Subscription in a specific namespace with the following kubectl command:

$ kubectl get subscription -n <namespace>
# Example output
NAME PACKAGE SOURCE CHANNEL
foo-sub foo foo-catalog alpha

Note: The name of the operator installed by the Subscription is available under the Package column.

The Subscription can be deleted by running this command:

kubectl delete subscription <subscription-name> -n <namespace>

Step 3: Delete the Operator’s ClusterServiceVersion (CSV)

The CSV contains all the information that OLM needs to manage an operator, and it effectively represents an operator that is installed on cluster. By deleting a CSV, OLM will delete the resources it created for the operator such as the deployment, RBAC, and any corresponding CSVs that OLM “Copied” into other namespaces watched by the operator.

Note that deleting a CSV only deletes objects derived from the CSV and does not delete objects from the bundle. For example, while RBAC resources derived from the CSV’s permissions and clusterPermissions fields will be deleted, any RBAC resources included directly in the bundle itself will remain on the cluster.

If you wish to look up a list of ClusterServiceVersion in a specific namespace to see which ClusterServiceVersion you need to delete, you can use the example kubectl command:

$ kubectl get clusterserviceversion -n <namespace>
# Example output
NAME DISPLAY VERSION REPLACES PHASE
foo Foo Operator 1.0.0 Succeeded

You can delete the ClusterServiceVersion in the namespace that the operator was installed into using this command:

kubectl delete clusterserviceversion <csv-name> -n <namespace>

Combine steps 2 and 3

Alternatively, you can delete both Subscription and its CSV using a sequence of commands:

CSV=$(kubectl get subscription <subscription-name> -n <namespace> -o json | jq -r '.status.installedCSV')
kubectl delete subscription <subscription-name> -n <namespace>
kubectl delete csv $CSV -n <namespace>

Step 4: Deciding whether or not to delete the CRDs and APIServices

The fourth step consists of deciding whether or not to delete the CRDs and APIServices that were introduced to the cluster by the operator. Assuming you have already deleted all unwanted resources on cluster as enumerated in Step 1, if no resources remain it is safe to remove the CRD or APISerivces. Otherwise, you should not delete the type as the wanted resources will be deleted automatically when the CRD or APISerivce is deleted.

Step 5: Deleting the Operator CR

OLM recently introduced the view-only operator CRD which communicates the list of resources associated with an operator bundle installed by OLM. When installing an operator bundle, OLM will create an operator CR named:

<packageName> if the operator is All Namespaced scoped.
<packageName>.<Namespace> if the operator is not All Namespaced scoped.

OLM will then the update the operator’s status.Components.Refs array to include all resources associated with the operator. Let’s consider OLM’s behavior after creating the operator CR named foo:

All resources associated with the foo operator CR will have the operators.coreos.com/foo label applied to it.
OLM will create or recreate the foo operator CR if any resources exist with the operators.coreos.com/foo label.

This ultimately means that in order to delete the foo operator CR, users will need to ensure that no resources are labeled with the operators.coreos.com/foo label. Typically, OLM should not attempt to recreate the foo operator after a user deletes it if they have completed steps 1 through 4 above. However, if OLM is still recreating the foo operator, a user should:

Delete each resource found in the foo operator's status.Components.Refs array. Alternatively, if you have deleted the foo operator's CSV and Subscription you may remove the operators.coreos.com/foo label from any resources you do not wish to delete.
Delete the foo operator CR. The final step consists of deciding whether or not to delete the CRDs and APIServices that were introduced to the cluster by the operator. Assuming you have already deleted all unwanted resources on cluster as enumerated in Step 1, if no resources remain it is safe to remove the CRD or APISerivces. Otherwise, you should not delete the type as the wanted resources will be deleted automatically when the CRD or APISerivce is deleted.