Osaka, Umeda - Participation Report for Kubernetes Meetup Tokyo 19 Osaka Satellite - May 31, 2019

I participated in the Kubernetes Meetup Tokyo from Osaka. I learned about the history of Kubernetes from Joe Beda, one of the three creators of Kubernetes. His talk was very easy to understand and insightful, so I would like to share it.

https://k8sjp-osaka.connpass.com/event/131981/

※ The following content includes my interpretation and may be incorrect. Please understand. The content of the presentation is all on Youtube, which is correct. Please refer to it.

https://www.youtube.com/watch?v=ETHGx8_Q-1k

Who is Joe Beda ?

Joe Beda is a co-founder of Kubernetes (one of the first three developers), CTO of Heptio which was acquired by VMware last year, co-author of O'Reilly's "Kubernetes: Up & Running" (Japanese title "Introduction to Kubernetes"), and is still one of the leaders of Kubernetes. This time, he will talk about the past and future of Kubernetes.

※ https://k8sjp.connpass.com/event/126207/

He is one of the first committers of Kubernetes and a very famous person. When he was working at Google, he was creating Kubernetes and Compute Engine.

According to Joe, the interesting thing about developing on a platform is the balance between the following two points:

Making it easy for users to use
Flexibility in case of unexpected usage

In my interpretation, for example, if you use GKE in the GCP platform, you can create a cluster just by clicking buttons. It's easy and makes you want to try it.

However, if it's only easy, it can't meet detailed requirements, so it's improved to allow option settings and easier customization. Is this what flexibility is? If this flexibility is overdone, it becomes complicated and there is a risk that users will stop using it (although maniac users may remain). I thought that balance is important.

For a detailed explanation by Joe, see here.

The origins and future of Kubernetes (en/English)

Joe spoke in English. Mr. Tanaka of CPCAmerica(?) was interpreting, and it was very easy to understand. Thank you! Also, his memory is amazing...

https://twitter.com/mumoshu/status/1134438272518635521?s=20

※ The following is based on a summary Tweet by @‏apstndb. God!!!

https://twitter.com/silverbirder/status/1134406467744804864?s=20

History of Kubernetes

Birth of Borg

At Google, they developed MapReduce to process BigData. To handle MapReduce, they developed something called GlobalWorkQueue(GWQ), which was primarily created for batch processing. From there, Borg was born to handle not only batch processing but also services that want to be executed in real-time (such as search). In a large-scale search like Google, even a few percent efficiency improvement can lead to significant cost savings. This became the basis for Kubernetes.

Birth of Kubernetes

While developing Borg at Google, it seemed that many users in the world were dealing with virtual machines. Borg was proprietary software, and from the desire to let people know about the world of Borg and to attract developers, Kubernetes was born as OSS. Also, Kubernetes was first about increasing developer productivity through an API-driven approach, with efficiency and security coming later.

The appeal of Kubernetes

Kubernetes is known to many as a "container orchestrator", which is a major point of its popularity. From another perspective, there was talk that the design of "managing a cluster with just one database" is attractive. (This may be my own interpretation. Sorry)

In Kubernetes, a distributed KVS called etcd is used to manage the state of the cluster (other state management is said to be cached). Since you cannot access etcd without going through the APIServer, you can maintain consistent data. Around that etcd, controllers that implement business logic (Scheduler, Controller Manager) provide value. For example, assigning Pods to Nodes, providing endpoints, replicating, and so on...

With the control plane of Kubernetes, the APIServer, Scheduler, and Controller Manager, it works on both single-node and multi-node. When I ran Kubernetes on DockerForMac, it was a single node. I had the image of it being multi-node.

Kubernetes is often referred to as container orchestration, but it's not an orchestration where everything is planned in advance, but rather a philosophy that's closer to planning and assembling on the fly, like jazz. I'm not very familiar with music, but I understood the meaning. (laughs) I'm not good at improvisation by nature.

CRD and Operators

There are various resources such as Pods, Replications, Deployments, etc. But what should you do to implement something that Kubernetes doesn't have? That's where Custom Resource Definitions (CRD) come in. What is that...?

https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/

https://qiita.com/cvusk/items/773e222e0971a5391a51

In short, you can create your own resources like Pods or Deployments. Oh, what's that! To create your own functions, you need a Custom Resource and a Custom Controller, and together they form Operators were born.

For example, there are things like the following. https://github.com/oracle/mysql-operator

https://github.com/kubeflow/tf-operator

At Yahoo, it seems that they introduced Kubernetes using an OSS called gimbal. https://github.com/heptio/gimbal

https://techblog.yahoo.co.jp/advent-calendar-2018/oss-gimbal/

I don't know the details, but I learned that thanks to such extensible features, it is becoming more and more popular.

Q&A

Q4. How do you see the movement to utilize things like Virtual Kubelet and k3s at the edge, which is felt in the community?

https://twitter.com/apstndb/status/1134413224839745536?s=20

https://twitter.com/apstndb/status/1134413431316987904?s=20

Other

All the questions from the participants were sharp. I tweeted that I wanted to ask a moderate question... but it didn't reach... https://twitter.com/silverbirder/status/1134412867988480000?s=20

Osaka Venue

The venue was provided by Aiming Inc.

https://aiming-inc.com/ja/

The venue was located on the 18th floor of Grand Front Osaka Tower B. (High!) The place we used this time was probably a conference room. It was a space that could accommodate about 30 to 40 people, and it was clean.

The relay with Tokyo sometimes had interruptions in the sound, but it was clearly visible. However, as for the content, it is uploaded on YouTube, so I thought it might not be necessary to attend Osaka specifically.

However, I think there may be good aspects to attending Osaka as well.

Able to communicate with others
Able to listen to presentations and discuss together

Well, I'm not very good at communication, so it was almost non-existent...

As an improvement point, I'm hoping that it will be possible to ask questions from relay points.

Finally

I learned about the background of Kubernetes and also about CRD. Also, although it's different from Kubernetes, I strongly felt that "the power of OSS" is important in the world of engineers. Engineers almost certainly use OSS in their daily development. For engineers, OSS is an indispensable existence, and they use it all the time.

Like Google did, from the desire to "want to be widely used, want to involve engineers", I thought that one of the factors that Kubernetes spread as OSS. What would have happened if this was paid? Would it have spread this much?

There are many companies in Japan that contribute to OSS. Individuals can also contribute to OSS, and there is also OSS Gate for beginners. The hurdle for Kubernetes contributors is a bit high, but I will continue to contribute to OSS as an engineer.