Skip to main content

Hardening guidelines for SecureAuth on K8s

Learn how to harden your SecureAuth deployment on K8s to reduce the security risk by eliminating potential attack vectors.

About Hardening

SecureAuth hardening is a collection of guidelines, best practices, and steps that aim at improving your SecureAuth deployment's security. Their goal is to eliminate potential attack vectors and minimize the system's attack surface. As the National Institure of Standards and Technology (NIST) states, hardening is a process intended to eliminate a means of attack by patching vulnerabilities and turning off nonessential services.

In general, hardening should be performed throughout the life cycle of your SecureAuth deployment. It should start from the installation, last throughout the deployment, configuration, and finish only after the end-of-life date of your SecureAuth deployment. You should remember that hardening should take place anytime a new system or application is introduced to your environment.

Deployment-agnostic Hardening

Configure IDPs

By default, the admin, developer, and default workspaces are configured to use static identity providers that allow you to jump-start your experience with SecureAuth. It is recommended to disable or remove those default static IDPs.

To learn more about setting up identity providers, see the Connecting identity providers to SecureAuth documentation.

Delete Default Workspaces

SecureAuth recommends to delete the default authorization workspace and the default developer portal. You can delete both from your Workspace Directory by selecting the meatballs menu and the Delete workspace/portal button.

Hardening SecureAuth on Kubernetes

Configure Secrets

SecureAuth, by default, uses pre-configured system tenant's secrets and encryption secrets to allow you to use SecureAuth as soon as possible. One of the first steps you should take is to change the default values for both secrets.

  1. Create override.yaml file that is responsible for providing configuration for your deployment, set the create flag to false under the secretConfig key.

    For example:

    secretConfig:
   create: false
   name: secret

  2. To avoid storing confidential data in your application code, you need to create an opaque kubernetes secret that will be responsible for storing your system tenant's secret and SQL encryption secrets.

    For example:

    apiVersion: v1
kind: Secret
type: opaque
metadata:
    name: secret
data:
    system:
       secret: YourSystemTenantSecret
       secrets:
          id: 1
          key: YourEncryptionKey

    Security recommendation

    To increase security of secrets stored in your repository, it is recommended to encrypt your kubernetes secrets. You can use tools like Mozilla SOPS or Bitnami Sealed Secrets to encrypt your secrets.

    When the secrets are applied to your kubernetes deployment, the secrets are encrypted and visible as plain text. Anyone who is authorized to create a Pod in a namespace can read any secret in that namespace; this includes indirect access such as the ability to create a Deployment. To mitigate the risks, kubernetes recommends to:

    • Enable encryption at Rest for secrets.

    • Enable or configure RBAC rules that restrict reading data in secrets.

    • Where appropriate, use mechanisms such as RBAC to limit which principals are allowed to create or replace secrets.

    To learn more, visit kubernetes secrets documentation

  3. Apply the secret to your kubernetes deployment using the kubernetes apply command and upgrade helm with new values.

    For example:

    kubectl apply -f acp-secrets.yaml
helm upgrade RELEASE_NAME -f override.yaml acp/kube-acp-stack

Configure Certificate

By default helm chart will install mock certificate that is available here. This certificate is publicly available for everyone and should not be used. The purpose of this certificate is to provide easy way to setup local environment. Production certificate should be provided under certificate key or by enabling integration with cert-manager.

Example:

certificate:
  cert: |
    -----BEGIN CERTIFICATE-----
    <certificate body>
    -----END CERTIFICATE-----    

  key: |
    -----BEGIN RSA PRIVATE KEY-----
    <certificate body>
    -----END RSA PRIVATE KEY-----

Check Security Context

Security context play an important role in K8s as it defines privilege and access control for a pod or container. By default, SecureAuth follows best practices for those values as shown below. In cases where your cluster administrator enforces some extra security policies, you should update those values.

Example:

podSecurityContext:
  fsGroup: 1000
  runAsGroup: 1000
  runAsNonRoot: true
  runAsUser: 1000

containerSecurityContext:
  readOnlyRootFilesystem: true
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL

Check Deployment Configuration

In SecureAuth, some of the configuration flags are used to speed up the development, configuration, and administration of your SecureAuth deployment. Before moving to a production environment, in your values.yaml configuration file, check if the following configuration options are disabled for your deployment:

  • --dev flag responsible for enabling the development mode that allows to update templates and CSS styles without the need to restart the environment.

  • --swagger flag responsible for creating default clients for the Swagger UI that allows you to test REST APIs.

  • --demo flag responsible for creating a demo application which can be used to test authentication flows.

  • --create-default-tenant that creates a pre-configured default tenant.

  • --create-default-workspaces that creates pre-configured default workspaces that you can use.

Create Network Policies

Network policies should be used to control traffic flow within the K8s cluster. By default, all pod to pod communication is unrestricted. In case of SecureAuth, it should only be allowed to communicate with databases and ingress controller. If you use fission capabilities of SecureAuth, helm chart will deploy network policy that only allows external egress traffic for fission pods.

Backup and restore

Cockroachdb

Cockroach database allows you to create backups of clusters used to deploy your SecureAuth instance. In case of any system failure, you can restore your backup to quickly recover.

Our recommendations:

  • create a schedule for your backups so they are made on regular time intervals.

  • Send backups into two independent storage locations (i.e two S3 buckets on different regions)

  • Monitor if your backups are created successfully

  • Encrypt your backups

  • Create Incremental backups (requires enterprise license)

  • Create backups with Revision History (requires enterprise license)

  • Test and document your backup restore procedures

If running on kubernetes, it might be useful to deploy cockroachdb-client-secure to connect to database.

To learn more, visit CockroachDB Backup Up and Restore Data documentation.

Redis

While an available Redis cluster is critical for ACP to function, the data residing in Redis is not mission-critical and does not need to be backed up. The data that Redis processes include, but is not limited to, tokens, session types, and codes. The loss of these data will not interfere with the functionality of SecureAuth once SecureAuth is back online and a new Redis cluster has been deployed.

To recover from a "disaster" scenario:

  1. Remove the existing Redis cluster

  2. Re-deploy Redis cluster based on the method you used during the initial deployment of Redis. Two possible methods you may have utilized to deploy redis-cluster are listed below.

  3. Once the Redis cluster is deployed successfully, SecureAuth will start communicating with the Redis cluster.

Timescaledb

Timescaledb database allows you to create backups of clusters used to deploy your SecureAuth instance. In case of any system failure, you can bootstrap from backup.

Our recommendations:

  • Control a backup schedule for your backups so they are made on regular time intervals.

  • Send backups into two independent storage locations (i.e two S3 buckets on different regions)

  • Monitor if your backups are created successfully

  • Encrypt your backups

  • Test and document your backup restore procedures

High Availability

High Availability (HA) is a critical aspect of any production-level application. It refers to a system's ability to remain accessible and perform well under most circumstances, including the failure of some of its components. Kubernetes, as a container orchestration platform, provides various features to ensure HA for applications deployed on it. This section explores the some concept we achieve HA for ACP stack running on Kubernetes cluster.

Recommendations

Below is a check-list-style enumeration for all concepts we use on a daily basis to ensure High Availability of SecureAuth in the SaaS environment. We recommend using as many of them as possible in your on-premises environment.

  1. Replicate the application

    The primary method to achieve HA on Kubernetes is to use multiple replicas of an application's pods. With multiple replicas, if one pod crashes or becomes unresponsive, traffic can be directed to the other healthy replicas. You can achieve the replication using autoscaling.minReplicas and autoscaling.maxReplicas described in the autoscaling article.

  2. Use Horizontal Pod Autoscaling

    Horizontal Pod Autoscaling (HPA) automatically scales the number of pod replicas based on defined metrics. This not only helps to ensure HA by adjusting to the load but also optimizes resource usage. You can tweak the SecureAuth HPA behaviors with autoscaling.behavior parameter described in the autoscaling article.

  3. Configure data store

    Redis cluster requires at least 3 master nodes for deployment. With 3 master nodes, the cluster has multiple write points.

    The masters should be setup with replicas so that high-availability can be achieved. 3 master nodes and 3 replica nodes (total of 6 nodes) will provide availabilty and sufficient processing capability for most deployments of Redis cluster for SecureAuth.

    You may need to adjust the master and replica node count depending on your situation. For more detailed information on configuring the number of nodes, please refer to cluster topology.

  4. Implement multi-zone and multi-region clusters

    Deploying SecureAuth across multiple zones and regions ensures that it remains available even if an entire zone or region goes down. Kubernetes supports multi-zone and multi-region clusters, which allow you to spread SecureAuth's pods across different geographical locations. For more information on running Kubernetes in multiple zones refer to the official documentation.

  5. Configure a highly available control plane

    For a highly available Kubernetes cluster, it's important to have multiple control plane nodes. If you lose the control plane, you lose the ability to interact with the cluster, schedule workloads, and respond to changes in workload requirements. More details can bo found on the official Kubernetes documentation.

  6. Ensure Robust Monitoring and Logging

    A robust monitoring and logging solution is critical to quickly identify and rectify issues. Tools such as Prometheus for monitoring and Logstash for logging can be integrated into your Kubernetes environment to provide real-time insight into application performance and health.

  7. Regularly Backup SecureAuth Data

    Regular backups are crucial to ensure that SecureAuth can be restored in the event of data loss. It's crucial to establish a backup procedure for Redis and Cockroachdb. Refer to the Backup and restore section for detailed instruction on it.

  8. Plan for Disaster Recovery

    Even with High Availability, it's important to plan for disaster recovery. This involves creating strategies for data backup and restore, failover to a secondary site, and regular testing of your recovery procedures.

Conclusion

Achieving High Availability for SecureAuth running on Kubernetes is a multi-faceted process, involving strategic planning, appropriate configuration of Kubernetes resources, and continuous monitoring. While Kubernetes provides robust features to support HA, the ultimate effectiveness depends on a thorough understanding of SecureAuth and proactive management of your Kubernetes environment. Remember, the goal of high availability is not just to maximize uptime but also to ensure a seamless, uninterrupted user experience.

In this section: