# CCR Token Rotation in Kubernetes

# Rotating Registry Tokens in Kubernetes

By default, CCR tokens are static and short-lived. We recommend rotating tokens frequently. The Crusoe Token Rotator Helm chart runs a periodic job in your cluster to refresh the tokens and update your Kubernetes Secret. The source code and chart can be found in the [crusoe-registry-token-rotator-helm-charts](https://github.com/crusoecloud/crusoe-registry-token-rotator-helm-charts) repository.

## Overview

The Token Rotator runs as a Kubernetes `CronJob`. On a defined schedule, it:

1. Uses your CMK cluster's Crusoe API credentials to generate a new short-lived registry token.
2. Updates the `crusoe-image-pull-secrets` in your cluster with the new token.
3. Cleans up any old tokens managed by the rotator.

This ensures that your Kubernetes nodes always have valid credentials to pull images without manual intervention.

## Prerequisites

Before installing the rotator, you will need:

- An existing **CCR Repository**.
- A Kubernetes cluster with `helm >= 3.x` and `kubectl >= 1.21.x` installed.

### Create Namespace and Secret (Non-CMK Clusters Only)

If you are not using Crusoe Managed Kubernetes (CMK), you will need Crusoe API keys. You can create them by following the instructions in ["Manage your API Keys"](../identity-and-security/managing-api-keys.mdx).

You must then create both the `crusoe-system` namespace and the `crusoe-secrets` secret:

```sh
kubectl create namespace crusoe-system

kubectl create secret generic crusoe-secrets \
  --from-literal=CRUSOE_ACCESS_KEY=<your-access-key> \
  --from-literal=CRUSOE_SECRET_KEY=<your-secret-key> \
  -n crusoe-system
```

## Installation

### Step 1: Download the Rotator

First, ensure your `kubectl` context is set to the correct cluster:

```sh
kubectl config current-context
```

Now clone the repository with the Token Rotator Helm chart:

```sh
git clone https://github.com/crusoecloud/crusoe-registry-token-rotator-helm-charts.git
cd crusoe-registry-token-rotator-helm-charts
```

### Step 2: Configure the Rotator

Update at least the following fields in `charts/crusoe-registry-token-rotator/values.yaml`:

- `targetSecret.registryUrl` (required): Set to your CCR repository URL
- `targetSecret.registryUsername` (required): Set to your CCR username, which is the email address you use in Crusoe
- `targetSecret.namespaces` (optional): Update if you want the Secret created in namespaces other than `default`

For example:

```yaml
image:
  repository: ghcr.io/crusoecloud/crusoe-registry-token-rotator
  tag: "latest"
  pullPolicy: IfNotPresent
targetSecret:
  name: crusoe-image-pull-secrets
  namespaces:
    - default
    # - <namespace1>
    # - <namespace2>
  registryUrl: "<crusoe-registry-url>"
  registryUsername: "<crusoe-registry-username>"
crusoeCredentialsSecretName: crusoe-secrets
schedule: "0 */6 * * *"
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 50m
    memory: 64Mi
```

### Step 3: Install the Token Rotator Helm Chart

```sh
helm install crusoe-registry-token-rotator ./charts/crusoe-registry-token-rotator \
  --namespace crusoe-system
```

Or to upgrade:

```sh
helm upgrade --install crusoe-registry-token-rotator ./charts/crusoe-registry-token-rotator \
  --namespace crusoe-system
```

### Step 4: Verify the installation

After installing the chart, verify that the release and its resources were created successfully:

```sh
helm list --namespace crusoe-system
```

You should see output similar to the following:

```
NAME                         NAMESPACE      REVISION UPDATED                              STATUS   CHART                               APP VERSION
crusoe-registry-token-rotator crusoe-system 1        2026-01-06 13:55:57.593474 -0700 PDT  deployed crusoe-registry-token-rotator-1.0.0
```

To verify the CronJob was created, run:

```sh
kubectl get cronjobs -n crusoe-system
```

You should see output similar to:

```
NAME                            SCHEDULE      TIMEZONE   SUSPEND   ACTIVE   LAST SCHEDULE   AGE
crusoe-registry-token-rotator   0 */6 * * *   <none>     False     0        5h29m           9d
```

### Step 5: (Optional) Trigger a Test Run of the CronJob

By default, the CronJob runs on a schedule. To manually trigger a run for testing:

```sh
kubectl create job --from=cronjob/crusoe-registry-token-rotator crusoe-registry-token-rotator-manual-test -n crusoe-system
```

You can monitor the job with:

```sh
kubectl get jobs -n crusoe-system
kubectl logs job/crusoe-registry-token-rotator-manual-test -n crusoe-system
```

## Troubleshooting

- **Crusoe API Permissions:** Ensure the API key used in `crusoe-secrets` is from a user with at least `reader` permissions in the Crusoe project.
- **Note on Rotator Permissions:** To manage Secrets in each of the namespaces you specify, the rotator is granted `GET`, `CREATE`, `UPDATE`, and `PATCH` on Secrets in Kubernetes RBAC.