Skip to main content
Version: v0.4.0

Container image caching with Harbor

The container images composing the MoAI Inference Framework are distributed through Amazon ECR. Although this approach works well in general, fetching images from a remote registry during deployment and scaling of inference environments may incur substantial delays.

This document explains how to build a local image registry with Harbor, automatically cache container images in Moreh's upstream Amazon ECR registry, and use them when deploying production inference environments. It covers the following high-level flow.

  1. Installing Harbor and configuring HTTP/HTTPS access.
  2. Registering Moreh's Amazon ECR as a registry endpoint in Harbor.
  3. Creating a Harbor project to cache or replicate images.
  4. Configuring Kubernetes nodes to pull images from Harbor instead of ECR.

Install Harbor​

Refer to Harbor Installation and Configuration for more details.

Pull a chart from the Harbor Helm repository.

helm repo add harbor https://helm.goharbor.io
helm repo update harbor

Decide whether to configure Harbor over HTTP (insecure but simple) or HTTPS, depending on your environment. Then, deploy Harbor using the following command. If you choose HTTPS, the externalURL must start with https:// instead of http://. You also need to replace <password> and <storageClass> with your own values.

helm upgrade -i harbor harbor/harbor \
    --version 1.18.2 \
    -n harbor \
    --create-namespace \
    --set harborAdminPassword <password> \
    --set persistence.persistentVolumeClaim.registry.storageClass <storageClass> \
    --set persistence.persistentVolumeClaim.jobservice.jobLog.storageClass <storageClass> \
    --set persistence.persistentVolumeClaim.database.storageClass <storageClass> \
    --set persistence.persistentVolumeClaim.redis.storageClass <storageClass> \
    --set persistence.persistentVolumeClaim.trivy.storageClass <storageClass> \
    --set externalURL http://harbor.harbor.svc.cluster.local \
    --set expose.tls.enabled true \
    --set expose.tls.certSource secret \
    --set expose.tls.secret.secretName harbor-tls
info

If you want to modify additional configurations beyond those specified in the command above to better fit your cluster environment, you can customize and use the values.yaml file provided by Harbor.

Then, apply additional configuration depending on whether you are using HTTP or HTTPS.

HTTP/HTTPS configuration​

Harbor can be exposed over HTTP(insecure) or HTTPS. Choose one based on your environment.

warning

This configuration must be applied to all nodes.

If you want to use HTTP, your container runtime must allow an insecure registry for the Harbor endpoint; otherwise image pulls may fail.

Configure containerd

Configure the registry as insecure:

/etc/containerd/config.toml
[plugins]
...
  [plugins."io.containerd.cri.v1.images".registry]
    config_path = "/etc/containerd/certs.d"

Create a registry host file.

mkdir -p /etc/containerd/certs.d/harbor.harbor.svc.cluster.local:80
cat > /etc/containerd/certs.d/harbor.harbor.svc.cluster.local:80/hosts.toml << 'EOF'
server = "https://harbor.harbor.svc.cluster.local:80"

[host."http://harbor.harbor.svc.cluster.local:80"]
  capabilities = ["pull","resolve","push"]
  skip_verify = true
  override_path = false
EOF

Restart containerd

sudo systemctl restart containerd

Register Moreh ECR endpoint​

Administration > Registries > New Endpoint

  • Provider: AWS ECR
  • Endpoint URL: https://255250787067.dkr.ecr.ap-northeast-2.amazonaws.com
  • Please refer to moai-inference-framework for the Access ID and Access Secret.

Create New Endpoint

Create Harbor project​

This approach makes the container runtime pull images through Harbor as a proxy (pull-through cache). Images are cached on demand when they are first pulled.

Create a project

Create a new Harbor project and enable Proxy cache.

Create New Project as Proxy

Configure mirror or rewrite image path​

warning

This configuration must be applied to all nodes.

On every node that may pull images, create the directory and hosts.toml file below.

sudo mkdir -p /etc/containerd/certs.d/255250787067.dkr.ecr.ap-northeast-2.amazonaws.com
sudo tee /etc/containerd/certs.d/255250787067.dkr.ecr.ap-northeast-2.amazonaws.com/hosts.toml >/dev/null <<'EOF'
server = "https://255250787067.dkr.ecr.ap-northeast-2.amazonaws.com"

# [host."https://harbor.harbor.svc.cluster.local:443/v2/mif"]
[host."http://harbor.harbor.svc.cluster.local:80/v2/mif"]
  capabilities = ["pull","resolve"]
  skip_verify = true
  override_path = false
EOF

Restart containerd

sudo systemctl restart containerd

For verifying, Pull an image once from any node configured and then confirm the image cached under the proxy cache project(mif) in Harbor web UI.

sudo crictl pull \
  255250787067.dkr.ecr.ap-northeast-2.amazonaws.com/quickstart/<image>:<tag>

Disable ECR token refresher​

When pulling images through Harbor instead of directly from Amazon ECR, the ecrTokenRefresher CronJob is no longer needed. Harbor handles ECR authentication on its own using the registry endpoint credentials configured earlier. Disable it in your values file:

moai-inference-framework-values.yaml
ecrTokenRefresher:
  enabled: false