Installation and upgrade v1.1.2

Installing the operator on Kubernetes

Obtaining an EDB subscription token

Installing EDB Postgres Distributed for Kubernetes requires an EDB Repos 2.0 token to gain access to the EDB private software repositories.

You can obtain the token by visiting your EDB Account Profile. You will have to sign in if you aren't already logged in.

Your account profile page displays the token next to the label Repos 2.0 Token. By default, the token is obscured. Select Show (an eye icon) to reveal it.

Your token entitles you to access one of two repositories: standard or enterprise.

• standard - Includes the operators and the EDB Postgres Extended operand images.
• enterprise - Includes the operators and the operand images for EDB Postgres Advanced and EDB Postgres Extended.

Set the relevant value, determined by your subscription, as an environment variable EDB_SUBSCRIPTION_PLAN:

EDB_SUBSCRIPTION_PLAN=enterprise

Then set the Repos 2.0 token as an environment variable EDB_SUBSCRIPTION_TOKEN:

EDB_SUBSCRIPTION_TOKEN=<your-token>

You can now proceed with the installation.

Using the Helm chart

You can install the operator using the provided Helm chart.

Directly using the operator manifest

You can deploy the EDB Postgres Distributed for Kubernetes operator directly using the manifest. This manifest installs both EDB Postgres Distributed for Kubernetes operator and the latest supported EDB Postgres for Kubernetes operator in the same namespace.

Install the cert-manager

EDB Postgres Distributed for Kubernetes requires Cert Manager 1.10 or higher. You can follow the installation guide or use this command to deploy cert-manager:

kubectl apply -f \
  https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml

Install the EDB pull secret

Before installing EDB Postgres Distributed for Kubernetes, you need to create a pull secret for the EDB container registry.

The pull secret needs to be saved in the namespace where the operator will reside (pgd-operator-system by default). Create the pgd-operator-system namespace using this command:

kubectl create namespace pgd-operator-system

To create the pull secret, run the following command:

kubectl create secret -n pgd-operator-system docker-registry edb-pull-secret \
  --docker-server=docker.enterprisedb.com \
  --docker-username=k8s_${EDB_SUBSCRIPTION_PLAN}_pgd \
  --docker-password=${EDB_SUBSCRIPTION_TOKEN}

Install the operator manifest

After the pull-secret is added to the namespace, you can install the operator like any other resource in Kubernetes: through a YAML manifest applied via kubectl.

Two different manifests are available, depending on your subscription plan:

• Standard: The latest standard operator manifest.
• Enterprise: The latest enterprise operator manifest.

To install the manifest for the latest version of the operator:

kubectl apply --server-side -f \
  https://get.enterprisedb.io/pg4k-pgd/pg4k-pgd-${EDB_SUBSCRIPTION_PLAN}-1.1.2.yaml

Check the operator deployment:

kubectl get deployment -n pgd-operator-system pgd-operator-controller-manager

Details about the deployment

In Kubernetes, the operator is by default installed in the pgd-operator-system namespace as a Kubernetes Deployment. The name of this deployment depends on the installation method. When installed through the manifest, by default it's named pgd-operator-controller-manager. When installed via Helm, by default the deployment name is derived from the Helm release name, appended with the suffix -edb-postgres-distributed-for-kubernetes (that is, <name>-edb-postgres-distributed-for-kubernetes).

You can get more information using the describe command in kubectl:

$ kubectl get deployments -n pgd-operator-system
NAME                READY   UP-TO-DATE   AVAILABLE   AGE
<deployment-name>   1/1     1            1           18m

kubectl describe deploy \
  -n pgd-operator-system \
  <deployment-name>

As with any deployment, it sits on top of a ReplicaSet and supports rolling upgrades. The default configuration of the EDB Postgres Distributed for Kubernetes operator is a deployment of a single replica, which is suitable for most installations. If the node where the pod is running isn't reachable anymore, the pod will be rescheduled on another node.

If you require high availability at the operator level, it's possible to specify multiple replicas in the deployment configuration, given that the operator supports leader election. In addition, you can take advantage of taints and tolerations to make sure that the operator does not run on the same nodes where the actual PostgreSQL clusters are running. (This might even include the control plane for self-managed Kubernetes installations.)

Deploy PGD clusters

Be sure to create a cert issuer before you start deploying PGD clusters. The Helm chart prompts you to do this, but in case you miss it, you can run, for example:

kubectl apply -f \
  https://raw.githubusercontent.com/EnterpriseDB/edb-postgres-for-kubernetes-charts/main/hack/samples/issuer-selfsigned.yaml

With the operators and a self-signed cert issuer deployed, you can start creating PGD clusters. See Quick start for an example.

Default operand images

By default, each operator release binds a default version and flavor for PGD and PGD Proxy images. If the image names aren't specified in the spec.pgd.imageName and spec.pgdproxy.imageName fields of the PGDGroup YAML file, the default images are used.

You can overwrite default images using the pgd-operator-controller-manager-config operator configuration map. For more details, see EDB Postgres Distributed for Kubernetes operator configuration.

You can find the images the PGD cluster is using by checking the PGDGroup status:

kubectl get pgdgroup <pgdgroup name> -o yaml | yq ".status.image"

Specifying operand images

This example shows a PGD cluster using explicit image names:

apiVersion: pgd.k8s.enterprisedb.io/v1beta1
kind: PGDGroup
metadata:
  name: group-example-customized
spec:
  instances: 2
  proxyInstances: 2
  witnessInstances: 1
  imageName: docker.enterprisedb.com/k8s_enterprise_pgd/edb-postgres-extended-pgd:17.5-pgd581-ubi9
  pgdProxy:
    imageName: docker.enterprisedb.com/k8s_enterprise_pgd/edb-pgd-proxy:5.8.1-ubi9
  imagePullSecrets:
  - name: registry-pullsecret
  pgd:
    parentGroup:
      name: world
      create: true
  cnp:
    storage:
      size: 1Gi
  pgdProxy:
    imageName: docker.enterprisedb.com/k8s_enterprise_pgd/edb-pgd-proxy:5.7.1-1

Specifying operand images using ImageCatalog

Since the release of version 1.1.1, the PGD4K-PGD operator supports using ImageCatalog to specify operand images.

Different ImageCatalogs are available based on PGD versions for each PostgreSQL flavor. Note that the image included in the ImageCatalog are ubi-9 based.

• EDB Postgres Advanced PGD: https://get.enterprisedb.io/pgd-k8s-image-catalogs/epas-k8s-enterprise-pgd-<PGD_VERSION>-ubi9.yaml
• EDB Postgres Extended PGD: https://get.enterprisedb.io/pgd-k8s-image-catalogs/pgextended-k8s-enterprise-pgd-<PGD_VERSION>-ubi9.yaml
• Postgres Community PGD: https://get.enterprisedb.io/pgd-k8s-image-catalogs/postgresql-k8s-enterprise-pgd-<PGD_VERSION>-ubi9.yaml

You can create an Imagecatalog in the PGD cluster namespace and reference it in your YAML file.

This command creates an EDB Postgres Extended PGD 5.7 ImageCatalog:

kubectl create -f \
        https://get.enterprisedb.io/pgd-k8s-image-catalogs/epas-k8s-enterprise-pgd-5.8-ubi9.yaml

This example shows how to use the EDB Postgres Extended PGD 5.8 ImageCatalog to specify a PostgreSQL major version of 17 for the PGD operand image. The PGD Proxy image is sourced from the ImageCatalog.

apiVersion: pgd.k8s.enterprisedb.io/v1beta1
kind: PGDGroup
metadata:
  name: group-example-catalog
spec:
  instances: 2
  proxyInstances: 2
  witnessInstances: 1
  imageCatalogRef:
    apiGroup: pgd.k8s.enterprisedb.io
    kind: ImageCatalog
    major: 17
    name: epas-k8s-enterprise-pgd58-ubi9
...

Operator upgrade

The EDB Postgres Distributed for Kubernetes (PGD4K) operator relies on the EDB Postgres for Kubernetes (PG4K) operator to manage clusters. To upgrade the EDB PGD4K operator, the PG4K operator must also be upgraded to a supported version. We recommend keeping the EDB PG4K operator on the long-term support (LTS) version, as this is the tested version compatible with the PGD4K operator.

To upgrade the EDB Postgres Distributed (PGD) for Kubernetes operator:

1. Upgrade the EDB Postgres for Kubernetes operator as a dependency.
2. Upgrade the PGD4K controller and the related Kubernetes resources.

When you upgrade the EDB Postgres for Kubernetes operator, the instance manager on each PGD node is also upgraded. For more details, see EDB Postgres for Kubernetes upgrades.

Unless differently stated in the release notes, those steps are normally done by applying the manifest of the newer version for plain Kubernetes installations.

Compatibility among versions

EDB Postgres Distributed for Kubernetes (PGD4K) follows semantic versioning. Every release of the operator within the same API version is compatible with the previous one. The current API version is v1, corresponding to versions 1.x.y of the operator.

The minor version of PGD4K operator is tracking a PG4K LTS release change. For example:

• PGD4K operator 1.0.x is fully tested against PG4K LTS 1.22.
• PGD4K operator 1.1.x is fully tested against PG4K LTS 1.25.

A PGD4K operator release has the same support scope as the PG4K LTS release it's tracking.

In addition to new features, new versions of the operator contain bug fixes and stability enhancements.

The release notes contains a detailed list of the changes introduced in every released version of EDB Postgres Distributed for Kubernetes. Read them before upgrading to a newer version of the software.

Most versions are directly upgradable. In that case, applying the newer manifest for plain Kubernetes installations will complete the upgrade.

When versions aren't directly upgradable, you must remove the old version (of both PGD4K and PG4K) before installing the new one. This won't affect user data, only the operator.

Upgrading to 1.0.1 on Red Hat OpenShift

On the OpenShift platform, starting from version 1.0.1, the EDB Postgres Distributed for Kubernetes operator is required to reference the LTS releases of the PG4K operator. For example, PGD4K v1.0.1 specifically references the PG4K LTS version 1.22.x. (Any patch release of the 1.22 LTS branch is valid.)

To upgrade PGD4K operator, ensure that the PG4K operator is upgraded to a supported version first. Only then can you upgrade the PGD4K operator.

Server-side apply of manifests

To ensure compatibility with Kubernetes 1.29 and upcoming versions, EDB Postgres Distributed for Kubernetes now mandates the use of server-side apply when deploying the operator manifest.

While employing this installation method poses no challenges for new deployments, updating existing operator manifests using the --server-side option may result in errors like the following:

Apply failed with 1 conflict: conflict with "kubectl-client-side-apply" using..

If such errors arise, you can resolve them by explicitly specifying the --force-conflicts option to enforce conflict resolution:

kubectl apply --server-side --force-conflicts -f <OPERATOR_MANIFEST>

From then on, kube-apiserver is acknowledged as a recognized manager for the CRDs, eliminating the need for any further manual intervention on this matter.

Operand upgrade

Operand upgrades fall into two categories based on PostgreSQL and PGD versions:

• Minor version upgrades (for example, PostgreSQL from 17.4 to 17.5 and PGD from 5.7 to 5.8)
• Major version upgrades (for example, PostgreSQL from 16.x to 17.5, no PGD major version upgrade as only PGD 5.x is supported now)

Checking current PGD and proxy versions

Before upgrading, you can check the current PGD and PGD proxy versions:

kubectl get pgdgroup <pgdgroup name> -o yaml | yq ".status.image"

Minor version upgrade

The PGD cluster supports in-place upgrades of the operand image's minor version, though the PostgreSQL service is temporarily unavailable during the upgrade.

Upgrade procedure

• Using default or customized image name: To upgrade the operand to a new minor version, replace the imageName in the spec.imageName and spec.pgdproxy.imageName sections of the PGD group YAML file with the new imageName. The images on each node will be upgraded sequentially and restarted accordingly.

• Using image catalog: If the PGD cluster manages image versions using an ImageCatalog, upgrade the image version specified in the referenced ImageCatalog. The PGD cluster applies the new image version.

Major version upgrade

Since release v1.1.2, the PGD4K operator supports in-place upgrades for major PostgreSQL versions. During the process, each PGD node is upgraded sequentially, with the write leader transferred to an available node before the upgrade.

Prerequisites for major version in-place upgrade

• PGD4K operator v1.1.2 or higher
• PG4K operator v1.26.0 or higher
• PGD operand 5.8 or greater

Upgrade procedure

Like minor version upgrades, initiating a major version upgrade involves updating the spec.imageName in the PGDGroup to point to the new operand image.

Example scenario:

Suppose you have a PGDGroup named pgd-sample, currently running with PostgreSQL 16.9 plus PGD 5.8.1. You plan to upgrade to PostgreSQL 17.5 plus PGD 5.8.1.

Step-by-step guidance

1. Check the current operand version you're using:

kubectl -n pgd get pgdgroup pgd-sample -o yaml | yq ".status.image"

Output:

pgd: docker.enterprisedb.com/k8s_enterprise_pgd/edb-postgres-advanced-pgd:16.9-pgd581-ubi9
proxy: docker.enterprisedb.com/k8s_enterprise_pgd/edb-pgd-proxy:5.8.1-ubi9

2. Update the operand image

Edit the PGDGroup and update the spec.imageName or patch it directly to docker.enterprisedb.com/k8s_enterprise_pgd/edb-postgres-advanced-pgd:17.5-pgd581-ubi9

kubectl patch pgdgroup pgd-sample -n pgd --patch \
'{"spec": {"imageName": "docker.enterprisedb.com/k8s_enterprise_pgd/edb-postgres-advanced-pgd:17.5-pgd581-ubi9"}}' \
--type=merge

3. Monitor the node-by-node major version upgrade

The cluster begins upgrading nodes sequentially:

Observe the upgrading node, for example, pgd-sample-3, shows "Upgrading Postgres major version".

> kubectl -n pgd get cluster
NAME           AGE    INSTANCES   READY   STATUS                             PRIMARY
pgd-sample-1   121m   1           1       Cluster in healthy state           pgd-sample-1-1
pgd-sample-2   118m   1           1       Cluster in healthy state           pgd-sample-2-1
pgd-sample-3   115m   1                   Upgrading Postgres major version   pgd-sample-3-1

During the process, the PGDGroup status is:

> kubectl -n pgd get pgdgroup
NAME         DATA INSTANCES   WITNESS INSTANCES   PHASE                                                         AGE
pgd-sample   2                1                   PGDGroup - Waiting for nodes to be ready   123m

The upgrade of individual nodes is managed via dedicated jobs:

> kubectl -n pgd get job
NAME                           STATUS    COMPLETIONS   DURATION   AGE
pgd-sample-3-1-major-upgrade   Running   0/1           3m22s      3m22s

Check logs in the upgrade job pod for detail upgrade status

kubectl -n pgd logs -f pgd-sample-3-1-major-upgrade-ldtnj

Once a node's major version upgrade completes, the process moves to the next node:

NAME           AGE    INSTANCES   READY   STATUS                             PRIMARY
pgd-sample-1   128m   1           1       Cluster in healthy state           pgd-sample-1-1
pgd-sample-2   125m   1                   Upgrading Postgres major version   pgd-sample-2-1
pgd-sample-3   122m   1           1       Cluster in healthy state           pgd-sample-3-1

4. Confirm completion and health

Once all nodes are upgraded, the PGDGroup phase switches to Healthy.

NAME         DATA INSTANCES   WITNESS INSTANCES   PHASE                AGE
pgd-sample   2                1                   PGDGroup - Healthy   137m

Verify the overall image version:

kubectl -n pgd get pgdgroup  pgd-sample -o yaml | yq ".status.image"

Output:

pgd: docker.enterprisedb.com/k8s_enterprise_pgd/edb-postgres-advanced-pgd:17.5-pgd581-ubi9
proxy: docker.enterprisedb.com/k8s_enterprise_pgd/edb-pgd-proxy:5.8.1-ubi9

5. Confirm PostgreSQL version on each node:

kubectl -n pgd exec -it pgd-sample-1-1 -c postgres -- psql -c "select version()"

Output:

                                                                       version
------------------------------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 17.5 (EnterpriseDB Advanced Server 17.5.0) on aarch64-unknown-linux-gnu, compiled by gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-5), 64-bit
(1 row)