Skip to content

Add helm chart #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,15 @@ Make sure you have the runtime containers images pulled.
- TODO use a more generic image tag here, e.g. latest

```
docker pull lihebi/codepod-kernel-python:v0.4.3-alpha.3
docker pull lihebi/codepod-runtime:v0.4.3-alpha.3
docker pull lihebi/codepod-kernel-python:v0.4.3-alpha.4
docker pull lihebi/codepod-runtime:v0.4.3-alpha.4
```

Or you can build the images:

```
docker build -t lihebi/codepod-kernel-python:v0.4.3-alpha.3 ./runtime/kernel
docker build -t lihebi/codepod-runtime:v0.4.3-alpha.3 ./runtime
docker build -t lihebi/codepod-kernel-python:v0.4.3-alpha.4 ./runtime/kernel
docker build -t lihebi/codepod-runtime:v0.4.3-alpha.4 ./runtime
```

## Usage
Expand Down
26 changes: 14 additions & 12 deletions api/src/spawner-k8s.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import * as k8s from "@kubernetes/client-node";
import * as fs from "fs";

import { ApolloClient, InMemoryCache, gql } from "@apollo/client/core";

Expand Down Expand Up @@ -117,15 +118,6 @@ function getServiceSpec(name) {
};
}

async function k8s_test() {
try {
let res: any = await k8sApi.listNamespacedPod("codepod-dev");
console.log(res.body);
} catch (e: any) {
console.log("ERROR", e.body.message);
}
}

export async function spawnRuntime(_, { sessionId }) {
let url = `/${sessionId}`;
sessionId = sessionId.replaceAll("_", "-").toLowerCase();
Expand All @@ -136,9 +128,14 @@ export async function spawnRuntime(_, { sessionId }) {
// check if exist?
// 1. create a jupyter kernel pod
// 2. create a ws pod
// await k8s_test();
console.log("Creating namespaced pod ..");
let ns = "codepod-runtime";

let ns =
process.env.RUNTIME_NS ||
fs
.readFileSync("/var/run/secrets/kubernetes.io/serviceaccount/namespace")
.toString();
console.log("Using k8s ns:", ns);
try {
// TODO if exists, skip
// await k8sApi.createNamespacedPod(ns, getPodSpec(k8s_name));
Expand Down Expand Up @@ -254,7 +251,12 @@ export async function killRuntime(_, { sessionId }) {
// sessionId = "k8s-user-UGY6YAk7TM";
let k8s_name = `k8s-${sessionId}`;
console.log("killRuntime", url, k8s_name);
let ns = "codepod-runtime";
let ns =
process.env.RUNTIME_NS ||
fs
.readFileSync("/var/run/secrets/kubernetes.io/serviceaccount/namespace")
.toString();
console.log("Using k8s ns:", ns);
console.log("Killing pod ..");
// await k8sApi.deleteNamespacedPod(getPodSpec(k8s_name).metadata.name, ns);
await k8sAppsApi.deleteNamespacedDeployment(
Expand Down
5 changes: 2 additions & 3 deletions compose/dev/compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ services:
JWT_SECRET: ${JWT_SECRET}
KERNEL_NETWORK: "codepod"
PROXY_API_URL: "http://proxy:4011/graphql"
ZMQ_KERNEL_IMAGE: "lihebi/codepod-kernel-python:v0.4.3-alpha.3"
WS_RUNTIME_IMAGE: "lihebi/codepod-runtime:v0.4.3-alpha.3"
ZMQ_KERNEL_IMAGE: "lihebi/codepod-kernel-python:v0.4.3-alpha.4"
WS_RUNTIME_IMAGE: "lihebi/codepod-runtime:v0.4.3-alpha.4"

ui:
image: node:18
Expand Down Expand Up @@ -69,7 +69,6 @@ volumes:
proxy-node-modules:
prisma-node-modules:


networks:
default:
name: codepod
Expand Down
4 changes: 2 additions & 2 deletions compose/prod/compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ services:
JWT_SECRET: ${JWT_SECRET}
KERNEL_NETWORK: "codepod"
PROXY_API_URL: "http://proxy:4011/graphql"
ZMQ_KERNEL_IMAGE: "lihebi/codepod-kernel-python:v0.4.3-alpha.3"
WS_RUNTIME_IMAGE: "lihebi/codepod-runtime:v0.4.3-alpha.3"
ZMQ_KERNEL_IMAGE: "lihebi/codepod-kernel-python:v0.4.3-alpha.4"
WS_RUNTIME_IMAGE: "lihebi/codepod-runtime:v0.4.3-alpha.4"

ui:
build: ../../ui
Expand Down
24 changes: 12 additions & 12 deletions deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,21 @@
Build the docker images:

```
docker build -t lihebi/codepod-ui:v0.4.3-alpha.3 ./ui
docker build -t lihebi/codepod-api:v0.4.3-alpha.3 ./api
docker build -t lihebi/codepod-proxy:v0.4.3-alpha.3 ./proxy
docker build -t lihebi/codepod-kernel-python:v0.4.3-alpha.3 ./runtime/kernel
docker build -t lihebi/codepod-runtime:v0.4.3-alpha.3 ./runtime
docker build -t lihebi/codepod-ui:v0.4.3-alpha.4 ./ui
docker build -t lihebi/codepod-api:v0.4.3-alpha.4 ./api
docker build -t lihebi/codepod-proxy:v0.4.3-alpha.4 ./proxy
docker build -t lihebi/codepod-kernel-python:v0.4.3-alpha.4 ./runtime/kernel
docker build -t lihebi/codepod-runtime:v0.4.3-alpha.4 ./runtime
```

Push to registry:

```
docker push lihebi/codepod-ui:v0.4.3-alpha.3
docker push lihebi/codepod-api:v0.4.3-alpha.3
docker push lihebi/codepod-proxy:v0.4.3-alpha.3
docker push lihebi/codepod-kernel-python:v0.4.3-alpha.3
docker push lihebi/codepod-runtime:v0.4.3-alpha.3
docker push lihebi/codepod-ui:v0.4.3-alpha.4
docker push lihebi/codepod-api:v0.4.3-alpha.4
docker push lihebi/codepod-proxy:v0.4.3-alpha.4
docker push lihebi/codepod-kernel-python:v0.4.3-alpha.4
docker push lihebi/codepod-runtime:v0.4.3-alpha.4
```

Create a cloud VM with docker support. Add DNS from domain name to the cloud
Expand Down Expand Up @@ -61,8 +61,8 @@ npx prisma migrate dev --name init
Pull the kernel image:

```
docker pull lihebi/codepod-kernel-python:v0.4.3-alpha.3
docker tag lihebi/codepod-kernel-python:v0.4.3-alpha.3 codepod-kernel-python
docker pull lihebi/codepod-kernel-python:v0.4.3-alpha.4
docker tag lihebi/codepod-kernel-python:v0.4.3-alpha.4 codepod-kernel-python
```

Now go to
Expand Down
10 changes: 0 additions & 10 deletions k8s/deployments/codepod-01-secret.yml

This file was deleted.

48 changes: 0 additions & 48 deletions k8s/deployments/codepod-04-prisma.yml

This file was deleted.

81 changes: 0 additions & 81 deletions k8s/deployments/codepod-04.5-rbac.yml

This file was deleted.

23 changes: 23 additions & 0 deletions k8s/helm-chart/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
24 changes: 24 additions & 0 deletions k8s/helm-chart/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: v2
name: codepod
description: A Helm chart for Kubernetes

# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "v0.4.3-alpha.4"
56 changes: 56 additions & 0 deletions k8s/helm-chart/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Helm charts for CodePod

## Prerequist

Also need the secrets:

```
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: mysecret
data:
POSTGRES_USER:
POSTGRES_PASSWORD:
POSTGRES_DB:
JWT_SECRET:
```

Note: the secrets must be base64 encoded with:

echo -n your-password | base64

Need the -n there, otherwise there will be a space when this secret is used in
env variables. Ref: https://github.com/kubernetes/kubernetes/issues/28086

Need to manually apply these secrets:

```
kubectl apply -f secrets.yaml
```

## Install

We need to create one more namespace manually:

kubectl create ns codepod-staging-runtime

Install:

helm install codepod-staging . --namespace codepod-staging --create-namespace --values=./values.staging.yaml

Upgrade:

helm upgrade codepod-staging . --namespace codepod-staging --values=./values.staging.yaml

Optionally initialize the DB (run in one api pod):

npx prisma migrate dev --name init

- TODO: automate this
- TODO: restore from backup

Uninstall:

helm uninstall codepod-staging
Loading