Minikube is ideal tool to setup Kubernetes (k8s from now on) locally to test and experiment with your deployments.
Minikube Setup: Docker for Mac / Sierra. GitHub Gist: instantly share code, notes, and snippets. More information on metalink downloads is available from www.metalinker.org. If you have newer archives or archives for platforms not already present in this table, we'd like to add them to this table with a pointer to your location. Minikube is an executable: Re-install minikube is an upgrade. Optionally you can remove /.minikube which will make minikube completely re-initialize and also update your kubernetes version (if a new version has been made available for minikube).
In this guide I will try to help you get it up and running on your local machine, drop some tips on where and how particular stuff should be done and also make it helm capable (I assume when you use k8s that at some point you will want to learn about and use Helm, etcd, istio etc).
This is your local k8s environment scaffolding guide.
Minikube installation
Minikube works with virtual machine, and for this it can use various options depending on your preference and operating system. My preference in this case is Oracle’s VirtualBox.
You can use brew to install everything:
In this case you could get some kind of inconclusive installation error related to virtualbox installation, especially on Mojave and probably everything afterwards.
Whatever it says, it is most probably a new security feature in MacOS X that is in your way.
Go to System Preferences > Security & Privacy and on General screen you will see one (or few) messages about some software needing approval to install. You should carefully review the list if there is more than one and allow installation of software you need — in this case software by Oracle.
After that is done you can re-run the command above and when it is done you should be ready for next steps.
Running and accessing the cluster
Starting it would be as easy as
In order to optimally utilize your local machine’s resources I would suggest stopping it when you do not need it any more… With VirtualBox in center of it, it will go through you laptop’s battery pretty quickly.
Starting it again later will get you back where you left off:
Kubernetes dashboard is also available to you (while minikube is running):
I will assume you have kubectl installed locally and that you are already using it for some remote clusters so you got multiple contexts. In this case, you need to list contexts and switch to minikube one (in following commands assuming default name that is, ofc, “minikube”)
Now you are in the context of your local k8s cluster that runs on minikube and you can do all the k8s things in it.
Ingress controller
To run your deployments that have ingress (and I assume most of them will), you will need ingress add-on:
Make sure that you setup ingress based on your local hosts. It basically means that whatever you set as host in your ingress rules needs to be set up in your /etc/hosts file:
Where “[minikube ip]” should be replaced with actual minikube ip. It also works with multiple, space separated local hosts after minikube ip.
Here is shortcut to do it in bash:
Container registry — i.e. Docker registry
Reality of the real container registry usage in local environment is a rough one, so I will provide easy, quick and dirty option that makes it quite easy to deploy your local work to your local k8s, but deprives you of really important experience of using proper container registry.
Local container registry
Get your local docker context to point to minikube context:
To revert: $ eval $(docker-machine env -u)
When in minikube context, to start local docker registry:
So, now you have local registry to push stuff to (as long as your docker is in context of minikube).
You can now do:
At this point you can use localhost:5000/: as image in your deployment and that is it.
Using remote container repo
To use remote container repo locally you need to provide way to authenticate, which is through k8s secrets.
For local secrets management for ECR, GCR and Docker registry I recommend using minikube addon called registry-creds. I do not consider it safe enough to be used anywhere but in local env.
Note on ECR setup: Make sure that, if you are setting it for AWS ECR, and you do not have role arn you want to use (you usually wont have and it is optional), you set it as something random as “changeme” or smt… It requires value, if you just press enter (since it is optional) deployment of creds pod will fail and make your life miserable.
In case of AWS ECR, that will let you pull from your repo directly setting url as container image and adding pull secret named awsecr-cred:
I have to note here that running this locally worked quite chaotically for me and every session was new experience and new hack to make it work… Not a happy path.
Helm
Helm is package manager for k8s, and is often used for configuration management across deployments. With high popularity of the tool and raising adoption, I want to end this guide with the note about adding helm to your local k8s env.
It is quite easy at this point, just have minikube up and:
This should be deprecated information pretty soon, but in current case helm uses backend called Tiller and that is what gets installed/deployed during helm init execution.
You should check tiller deployment with: $ kubectl describe deploy tiller-deploy — namespace=kube-system
Valuable read:https://docs.helm.sh/using_helm/
Now you have full k8s local environment able to accept all of your test deployments before you decide to put them in the cloud (or “raw iron” server somewhere).
Minikube is ideal tool to setup Kubernetes (k8s from now on) locally to test and experiment with your deployments.
Minikube Windows 10
In this guide I will try to help you get it up and running on your local machine, drop some tips on where and how particular stuff should be done and also make it helm capable (I assume when you use k8s that at some point you will want to learn about and use Helm, etcd, istio etc).
This is your local k8s environment scaffolding guide.
Minikube installation
Minikube works with virtual machine, and for this it can use various options depending on your preference and operating system. My preference in this case is Oracle’s VirtualBox.
You can use brew to install everything:
In this case you could get some kind of inconclusive installation error related to virtualbox installation, especially on Mojave and probably everything afterwards.
Whatever it says, it is most probably a new security feature in MacOS X that is in your way.
Go to System Preferences > Security & Privacy and on General screen you will see one (or few) messages about some software needing approval to install. You should carefully review the list if there is more than one and allow installation of software you need — in this case software by Oracle.
After that is done you can re-run the command above and when it is done you should be ready for next steps.
Running and accessing the cluster
Minikube Update
Starting it would be as easy as
In order to optimally utilize your local machine’s resources I would suggest stopping it when you do not need it any more… With VirtualBox in center of it, it will go through you laptop’s battery pretty quickly.
Starting it again later will get you back where you left off:
Kubernetes dashboard is also available to you (while minikube is running):
I will assume you have kubectl installed locally and that you are already using it for some remote clusters so you got multiple contexts. In this case, you need to list contexts and switch to minikube one (in following commands assuming default name that is, ofc, “minikube”)
Now you are in the context of your local k8s cluster that runs on minikube and you can do all the k8s things in it.
Ingress controller
To run your deployments that have ingress (and I assume most of them will), you will need ingress add-on:
Make sure that you setup ingress based on your local hosts. It basically means that whatever you set as host in your ingress rules needs to be set up in your /etc/hosts file:
Where “[minikube ip]” should be replaced with actual minikube ip. It also works with multiple, space separated local hosts after minikube ip.
Here is shortcut to do it in bash:
Container registry — i.e. Docker registry
Reality of the real container registry usage in local environment is a rough one, so I will provide easy, quick and dirty option that makes it quite easy to deploy your local work to your local k8s, but deprives you of really important experience of using proper container registry.
Local container registry
Get your local docker context to point to minikube context:
To revert: $ eval $(docker-machine env -u)
When in minikube context, to start local docker registry:
So, now you have local registry to push stuff to (as long as your docker is in context of minikube).
You can now do:
At this point you can use localhost:5000/: as image in your deployment and that is it.
Using remote container repo
To use remote container repo locally you need to provide way to authenticate, which is through k8s secrets.
For local secrets management for ECR, GCR and Docker registry I recommend using minikube addon called registry-creds. I do not consider it safe enough to be used anywhere but in local env.
Note on ECR setup: Make sure that, if you are setting it for AWS ECR, and you do not have role arn you want to use (you usually wont have and it is optional), you set it as something random as “changeme” or smt… It requires value, if you just press enter (since it is optional) deployment of creds pod will fail and make your life miserable.
In case of AWS ECR, that will let you pull from your repo directly setting url as container image and adding pull secret named awsecr-cred:
I have to note here that running this locally worked quite chaotically for me and every session was new experience and new hack to make it work… Not a happy path.
Helm
Helm is package manager for k8s, and is often used for configuration management across deployments. With high popularity of the tool and raising adoption, I want to end this guide with the note about adding helm to your local k8s env.
It is quite easy at this point, just have minikube up and:
This should be deprecated information pretty soon, but in current case helm uses backend called Tiller and that is what gets installed/deployed during helm init execution.
You should check tiller deployment with: $ kubectl describe deploy tiller-deploy — namespace=kube-system
Valuable read:https://docs.helm.sh/using_helm/
Now you have full k8s local environment able to accept all of your test deployments before you decide to put them in the cloud (or “raw iron” server somewhere).