EXSUM: Ray on Minikube

I was asked about how to setup Kubernetes and deploy Ray on it. The last time I did this was before this was officially supported – I just manually built a container with Ray and deployed it as master on K8s master and workers on K8s workers. However, over the last 3 years or so a lot has happened and we can simply use off-the-shelf Helm charts now. Here is a minimal example. Since I like the idea of having online IDEs I have also included a command to run Eclipse Che. I should write a dedicated post about why online IDEs that natively support container development are compelling.

Minikube

Setup kubectl, see docs:

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Install Helm, see docs:

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

Setup minikube, see docs:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
sudo dpkg -i minikube_latest_amd64.deb

minikube start --addons=ingress --memory=8192
kubectl version

Note: It is very likely that just minikube start would not work.

Ray

See docs.

Deploy

cd ${RAY_DIR}/deploy/charts
helm -n ray install example-cluster --create-namespace ./ray

Run Test Job

$ kubectl -n ray get pods
NAME                                    READY   STATUS      RESTARTS   AGE
example-cluster-ray-head-type-lsbwn     1/1     Running     0          28m
example-cluster-ray-worker-type-bltlj   1/1     Running     0          22m
example-cluster-ray-worker-type-srwtl   1/1     Running     0          28m
ray-test-job-5gwx8                      0/1     Completed   0          22m

Now run test job directly on head:

$ kubectl -n ray cp ray/doc/kubernetes/example_scripts/run_on_head.py example-cluster-ray-head-type-lsbwn:/home/ray
$ kubectl -n ray exec example-cluster-ray-head-type-lsbwn -- python /home/ray/run_on_head.py

Istio, KNative, Che and Camel K

Minikube with GPU Acceleration

Setup minikube, see docs:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
sudo dpkg -i minikube_latest_amd64.deb

minikube start --addons=ingress --memory=8192

kubectl version
kubectl get pod

kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/master/nvidia-device-plugin.yml

Note: It is very likely that just minikube start would not work.

Eclipse Che

Install Che into Minikube:

bash <(curl -sL  https://www.eclipse.org/che/chectl/)
chectl --version

chectl server:deploy --platform minikube

To update chectl use chectl update.

Set certificate

Documentation.

cp /tmp/cheCA.crt ~/Downloads/

Chrome -> chrome://settings/security -> Advanced -> Privacy and security -> Manage Certificates -> Authorities Import cheCA.crt, “Trust this certificate for identifying websites”, OK, reload.

Go to https://che-eclipse-che.192.168.49.2.nip.io login with admin/admin, change password.

Istio

sudo mkdir -p /opt/istio && cd $_
sudo chown -R ${USER} .
curl -L https://istio.io/downloadIstio | sh -
cd istio-1.9.3
export PATH=$PWD/bin:$PATH
istioctl install

KNative

KNative requires Istio.

See docs. Pick release.

kubectl apply -f https://github.com/knative/operator/releases/download/v0.22.1/operator.yaml
kubectl logs -f deploy/knative-operator
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
 name: knative-serving
---
apiVersion: operator.knative.dev/v1alpha1
kind: KnativeServing
metadata:
  name: knative-serving
  namespace: knative-serving
EOF
watch -n 5 kubectl get deployment -n knative-serving
kubectl get KnativeServing knative-serving -n knative-serving

Camel K

ReleasesExamples

cd ~/Downloads/
wget https://github.com/apache/camel-k/releases/download/v1.3.2/camel-k-client-1.3.2-linux-64bit.tar.gz
mkdir ~/Downloads/tmp_camel_k && tar -xf camel-k-client-1.3.2-linux-64bit.tar.gz -C ~/Downloads/tmp_camel_k
sudo cp ~/Downloads/tmp_camel_k/kamel /usr/local/bin/
kamel version
rm -rf ~/Downloads/tmp_camel_k/
cd -
kubectl create namespace camel-basic
kubectl config set-context --current --namespace=camel-basic
minikube addons enable registry
kamel install

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s