Configuring Migrate Job for Cloudentity Platform

Every time Cloudentity starts up, it checks SQL data for new migrations. This operation takes an extra time on every startup (i.e autoscaling) before Cloudentity serves user requests. Additionally, during upgrade procedure or in the multi-region scenario, multiple Cloudentity instances can start migrations at the same time which can lead to restarts due tp the SQL table lock. To mitigate this issues, migration job was introduced.

The migrate job creates a Helm hook to create K8s job which utilizes Cloudentity the sql migrate command. At the same time, migrations check is disabled on Cloudentity startup.

See below for an example of how to migrate job:

migrateJob:
  enabled: true            

Migration job uses sql.url as a SQL endpoint. You can provide additional parameters via migrateJob.config

See below for an example of how to increase SQL migration timeout:

migrateJob:
  config:
    sql:
      migrations:
        timeout: 10m            

Cloudentity could be integrated with timescale to store audit events. This functionality is disabled by default. If you use timescale integration, you should enable migrate job for it. If you store your timescale url parameter in secret, it will be picked up automatically.

See below for an example of how to enable migration job for timescale:

migrateJob:
  config:
    timescale:
      enabled: true
      url: "postgres://postgres@timescaledb.svc.cluster.local:5432/acpdb?sslcert=%2Ftls%2Ftls.crt&sslkey=%2Ftls%2Ftls.key&sslmode=verify-full&sslrootcert=%2Ftls%2Fca.crt"            

Additionally migration job can utilize configuration provided in secret config to authenticate for migrations.

migrateJob:
  enabled: true
  configPath: /data/config.yaml,/secret/secret.yaml            

Sample secret.yaml can look like:

redis:
  password: "password"
timescale:
  url: "postgres://postgres:password@timescaledb.timescaledb-local.svc.cluster.local:5432/acpdb?sslcert=%2Ftls%2Ftls.crt&sslkey=%2Ftls%2Ftls.key&sslmode=verify-full&sslrootcert=%2Ftls%2Fca.crt"
  replicas:
    - "postgres://postgres:password@timescaledb-replica.timescaledb-local.svc.cluster.local:5432/acpdb?sslcert=%2Ftls%2Ftls.crt&sslkey=%2Ftls%2Ftls.key&sslmode=verify-full&sslrootcert=%2Ftls%2Fca.crt"         

See below for an example of how to enable all migrations with default:

migrateJob:
     enabled: true
     configPath: /data/config.yaml,/secret/secret.yaml
     config:
       redis:
         enabled: true
         addrs:
           - "redis-local-redis-cluster-headless.redis:6379"
           - "redis-local-redis-cluster-headless.redis:6379"
         redis_search: true
         redis_search_index_name: acp-local
         tls:
           enabled: true
           ca: "/tls/ca.crt"
           key: "/tls/tls.key"
           cert: "/tls/tls.crt"
           insecure_skip_verify: true
       sql:
         migrations:
           timeout: 30m
       timescale:
         enabled: true