The challenge
Our client aimed to increase the efficiency of their Kubernetes scaling services, especially during unexpected traffic spikes. Typically, this would require allocating additional resources like memory, storage, and CPU to the Kubernetes cluster, which comes with a risk of overprovisioning. The client was looking for an alternative solution that would also cut an instance creation time to 30 seconds or less.
VirtusLab designed and implemented a Kubernetes operator that steered clear of overprovisioning. The goal was to improve the efficiency and cost-effectiveness of Kubernetes scaling, especially when it involves hundreds of nodes.
The solution
VirtusLab adopted two key Kubernetes performance optimization techniques, namely hibernation and internal scaling, to make sure the operator works efficiently and saves money.
Additionally, VirtusLab has integrated distributed tracing powered by OpenTelemetry’s set of standards and tools. This combination will help the client with future maintenance and system monitoring.
To enhance the efficiency of scaling up in Kubernetes, VirtusLab implemented an intelligent, AI-driven system that stores and rapidly accesses recently used container images, maintaining their cache. By doing so, the system quickly accesses these resources without having to retrieve them from a more distant source every time.
The results
The client successfully deployed the completed Kubernetes operator for multiple customers. This decreased the time required to create a single instance from 60 to 22 seconds, without exceeding the limitations of Kubernetes and the AWS API. Additionally, the client observed:
- Reduced costs for each cluster
- Immediate response of cluster scaling to traffic surges
- Enhanced AWS Spot instance protection
The tech-stack
Programming language: Go
Platform: Kubernetes
Database: MongoDB
Cloud: Amazon Web Services (AWS)
Tooling: Various autoscaling products such as EKS Cluster AutoScaler, Karpenter and Spot.io, ContainerD and Docker, OpenTelemetry, DataDog, Prometheus, MongoDB
