Ocean Explained: Dive Deep with the Ocean Controller – The Spot by NetApp Blog

As manager data planeservice for containerized applications, Spot Ocean provides a seamless experience for running containers in the cloud. Ocean integrates with the control plane of your choice and handles key areas of infrastructure management, from compute provisioning and autoscaling to price optimization and right sizing.

A central component of Ocean’s architecture is the Ocean Controller, which is how Ocean and your Kubernetes cluster integrate and interact. The Ocean Controller exports pod metadata to the Spot backend and enables Ocean to take automated actions to manage, control, and optimize your infrastructure.

How does the Ocean Controller work?

A controller is usually deployed per cluster and consists of 6 Kubernetes resources:

• ConfigMap
• secret (optional)
• service account
• Cluster role
• ClusterRoleBindingClusterRoleBinding
• Deployment

The controller binds the Kubernetes cluster to the appropriate Ocean resource based on the configured Spot account ID, Spot token, and a unique cluster ID specified per cluster. Ocean is responsible for managing the cluster’s infrastructure, and since the controller resides in a pod inside your Kubernetes cluster, it’s crucial that the controller is always available so Ocean can handle newly created workloads. added. To ensure reliability, the controller is configured with a priority class (system-node-critical priorityClassName) so that the Kubernetes scheduler prioritizes the Ocean controller pod in case it gets kicked out or becomes pending (due to shrink event, continuous update, node health failure, etc.).

In the rare event that the Ocean controller becomes unavailable, the current cluster capacity will be maintained by the Spot platform. Workloads running on Spot Instances will be replaced when in danger of disruption, with on-demand fallback still enabled, and workloads can continue to use committed capacity. In this scenario, however, Ocean will not be able to react to pending (scale-up) or scale-down pods for optimization reasons. Additionally, the secure drain of terminated nodes will not be affected.

Ocean Controller Architecture

Deployed within the Kubernetes cluster, the Ocean Controller communicates with the Kubernetes API and Metric Server to retrieve and ship data to Ocean SaaS, which is responsible for making scaling decisions. In the event of a decision to scale down, the Ocean controller will ensure a graceful draining process of nodes removed from the cluster.

The controller communicates using the HTTPS protocol and only requires outbound communication to the Ocean SaaS.

[Link]

Controller Permissions

The controller policy consists of several sections described below:

• Read Only: Permissions to retrieve data. Required for functional operation of Ocean.
• Manipulate Nodes/Pods: Permissions to update nodes and evict pods. This section is required for purposes of flushing, updating nodes as unscheduled, and evicting pods.
• CleanUp function: Required for Ocean AKS-Engine integration.
• CSR Approval: Required for the CSR Approval feature.
• Auto Update: This section grants the controller the necessary permissions to update its deployment and role. This is only required for the auto-update feature.
• Full CRUD for Resources: Currently, resources include pods, deployments, and daemon sets. This is required for the Run Workloads feature.
• Ocean for Apache Spark: Required by the Ocean for Spark feature.

The policy can be adjusted depending on the desired capabilities. You can safely remove sections if you want to disable certain features.

You can extend the ocean Cost analysisfeature with CustomResourceDefinitionsCustomResourceDefinitionsadding another ClusterRole & ClusterRoleBinding, as CRDs extend the Kubernetes API, the controller needs to know the specific extension name in order to query and collect the relevant data.

Please note –if automatic updating is disabled, these changes can be made directly to the Ocean Controller ClusterRole object.

Controller auto-update – disable auto-update

By default, the controller auto-update feature is enabled to allow users to get the latest versions of the Ocean controller. Controller updates improve stability, performance, new metrics collection, and support for new Kuberenetes versions and features. With auto-update enabled, users don’t have to monitor for new controller versions or upgrade manually. Users can find version history here.

To disable automatic updates, add the disable-auto-update: “true” parameter in the controller’s configMap object and restart the controller (delete the current pod).

Update process

When rolling out a new release, updates are handled by Spot Developers. During this process, the Controller Deployment and ClusterRole objects are updated.

Please note –If the controller is unavailable during the release cycle (for any reason, including use of the shutdown/running hours feature), it will not be updated until the next release. controller day.

Please note –the automatic update process will override any role changes applied by the user

Proxy usage, proxy-url

For customers using a proxy for outbound communications, the Ocean Controller supports the (optional) proxy-url parameter, which allows traffic sent from the Ocean Controller to Ocean SaaS to pass through the configured proxy server. This feature is supported from controller version 1.0.45.

Static endpoint, base URL

Customers using strict firewall rules for outbound communication may need to configure static IP addresses in their firewall rules. By default, the Ocean Controller communicates with the Ocean SaaS using the following endpoint: api.spotinst.io.

The Ocean Controller can be configured to communicate through a different endpoint that has a static IP address. If needed, contact your Spot representative/Spot support team for details.

Certificate Signing Request (CSR) Approval, enable-csr-approval

Kubernetes clusters can be configured with TLS seedingkeep communication private, free from interference, and ensure that when a node joins a cluster, it creates a certificate signing request that must be approved before the node is registered and can operate as part of the cluster.

Ocean can approve Node Registration Certificate Signing Requests (CSRs), which can easily be used by adding the following configuration to the controller configMap: enable-csr-approval=true

When enabled, the Ocean Controller will only approve CSRs for Ocean-initiated nodes, removing the risk of unknown nodes registering with the cluster.

To note – Ocean’s support for GKE Shielded Nodes performs a similar practice, but does not require controller configMap changes.

Container image location

The Ocean Controller image is located in two main repositories:

1. Hub Docker – spotinst/kubernetes-cluster-controller: newest
2. GCR – gcr.io/spotinst-artifacts/kubernetes-cluster-controller:latest

Due to Docker hub rate limit restrictions, our install scripts default to the GCR image location.

The controller does not signal?

Follow the following troubleshooting guideif this does not resolve the issue, contact our support team through the console for further assistance.

Summary

The controller enables Ocean’s powerful automated features and capabilities so you can quickly and efficiently scale fully optimized Kubernetes clusters.

To learn more about Ocean, read the other articles in our Ocean Explained series on auto margin and container-based autoscaling.