Scaling Database Deployment on Kubernetes with Kelsey Hightower
Exploring the Best Practices, Misconceptions, and Challenges for Running Databases on Kubernetes.
Introduction
I had the honor of being the host of a Twitter Space on March 16, 2023, where I asked Kelsey Hightower to speak about databases on Kubernetes. Kelsey is a well-known authority on Kubernetes and cloud-native infrastructure, and his knowledge was crucial to the debate.
In the Twitter Space, we talked about the different kinds of databases that work well with Kubernetes and those that don't, as well as the typical misunderstandings and difficulties that organizations and developers encounter when operating databases on Kubernetes for the first time.
Link to Twitter Space
- Here is the link to the Twitter space that I had with Kelsey.
Databases That Are Ideal for Kubernetes Deployment
According to Kelsey, stateless and horizontally scalable databases, such as NoSQL databases, are a good match when it comes to deployment on Kubernetes. These databases are extremely available and fault-tolerant due to their simplicity in scaling up and down in response to demand.
In addition to NoSQL databases, Kelsey also emphasized that CockroachDB and other databases created to operate in a containerized context are suitable for deployment on Kubernetes. A distributed SQL database known as CockroachDB was created with Kubernetes and other container orchestration systems in mind. It offers high consistency guarantees and is horizontally scalable.
Types of Databases Not Suitable for Kubernetes Deployment
Despite the fact that Kubernetes serves a variety of databases. It's possible that some databases can't be deployed on Kubernetes groups or in communities, while others might need extra security measures.
To give each developer a sandbox setting, you can divide a cluster into virtual clusters, also referred to as namespaces, in a development environment. Using a straightforward YAML file, a Postgres container, and network-attached storage, for instance, developers may deploy Postgres to be used for development. However, in production settings, this configuration might not be adequate.
It's possible that Postgres cannot expand horizontally, so you may need to modify its configuration file and figure out how to integrate a daemon set with numerous Postgres instances.
Services, backups, and dynamic IPs are additional issues you must handle. If you don't take precautions, the database could wind up sharing resources on a node with other workloads, and Kubernetes could kick the Postgres container out if its workload isn't given priority.
As a result, you must include pod disruption budget rules to safeguard the node and give network and file systems precedence.
Common difficulties for organizations when running databases on Kubernetes
We then talked about some typical misunderstandings and difficulties that businesses and developers encounter when deploying databases on Kubernetes for the first time.
As Kelsey noted that even though legacy software is frequently disregarded and relegated to the background as a problem no one wants to deal with, it is still crucial if that is how the enterprise works. Many businesses have never developed strong database management practices, much less in Kubernetes.
Many organizations are lacking experience and skill in managing databases and most enterprises have never gotten good at even managing databases on static infrastructure.
However, Kubernetes is the least of a company's problems if it lacks expertise in managing its own existing stack and demonstrating agility there. Prior to contemplating Kubernetes, it is crucial to get better at managing the current stack.
Conclusion
The conversation on databases on Kubernetes was insightful and entertaining, and it took place during the Twitter Space with Kelsey Hightower. Regarding the kinds of databases that are ideal for implementation on Kubernetes, Kelsey offered insightful information.
That's it for now. Did you like this blog? Please let me know.
You can Buy Me a Coffee if you want to and please don't forget to follow me on YouTube, Twitter, and LinkedIn also.
Happy Learning!