diff --git a/README.md b/README.md index fa32a5b..53dd48b 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ To quickly run a small [Montage workflow](https://github.com/hyperflow-wms/monta - Install [kind](https://kind.sigs.k8s.io/) and [Helm](https://helm.sh/) - Create a small kind cluster using provided configuration: ``` -kind create cluster --config local/kind-config-3n.yaml +kind create cluster --config local/kind-config-3n.yaml # CHECK SECTION: RUN WITH CUSTOM HYPERFLOW !!! ``` - Jump to section [Install resources](#install-resources) @@ -109,6 +109,19 @@ If you don't want to use labels, you can use values from `minikube` directory th ### Install resources Assuming you are in repository main directory, install Kubernetes resources as follows: ``` +# otel +helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts +helm upgrade -i my-opentelemetry-collector open-telemetry/opentelemetry-collector -f ./values/cluster/collector-values.yaml + +# jaeger +helm repo add jaegertracing https://jaegertracing.github.io/helm-charts +kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.6.3/cert-manager.yaml +# kubectl create namespace observability +kubectl create -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.41.0/jaeger-operator.yaml # -n observability +kubectl apply -f ./values/cluster/jaeger-all-in-one.yaml + + +# hyperflow - main helm upgrade -i nfs-server-provisioner charts/nfs-ganesha-server-and-external-provisioner/charts/nfs-server-provisioner --values values/cluster/nfs-server-provisioner.yml helm upgrade -i nfs-pv charts/nfs-volume --values values/cluster/nfs-volume.yaml helm upgrade -i redis charts/redis --values values/cluster/redis.yml @@ -125,6 +138,26 @@ Once all pods are up and running or completed, you can manually run the workflow * `cd /work_dir` * `hflow run .` +### RUN WITH CUSTOM HYPERFLOW +1. Create docker registry and run kind cluster +``` +sh ./local/run_docker_registry.sh +kind create cluster --config local/kind-config-3n-local-registry.yaml +kubectl apply -f local/kind-local-registry.yaml +sh ./local/connect_docker_registry.sh +``` +2. Adding images to local repo +``` +docker tag localhost:5001/myhfw +docker push localhost:5001/myhfw +docker pull localhost:5001/myhfw +``` +3. Using images in file +Edit `values/cluster/hyperflow-engine.yaml` file +``` + localhost:5000/myhfw:latest +``` + ## Using Google Kubernetes Engine Here are additional steps you need to do in order to run HyperFlow on the Google Kubernetes Engine. diff --git a/local/kind-config-3n-local-registry.yaml b/local/kind-config-3n-local-registry.yaml new file mode 100644 index 0000000..f976405 --- /dev/null +++ b/local/kind-config-3n-local-registry.yaml @@ -0,0 +1,18 @@ +# three node (two workers) cluster config +kind: Cluster +apiVersion: kind.x-k8s.io/v1alpha4 +containerdConfigPatches: +- |- + [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"] + endpoint = ["http://kind-registry:5000"] +nodes: +- role: control-plane +- role: worker + labels: + hyperflow-wms/nodepool: hfmaster +- role: worker + labels: + hyperflow-wms/nodepool: hfworker +- role: worker + labels: + hyperflow-wms/nodepool: hfworker diff --git a/values/cluster/collector-values.yaml b/values/cluster/collector-values.yaml new file mode 100644 index 0000000..0a096da --- /dev/null +++ b/values/cluster/collector-values.yaml @@ -0,0 +1,36 @@ +mode: deployment +config: + receivers: + otlp: + protocols: + http: + endpoint: 0.0.0.0:4318 + grpc: + endpoint: 0.0.0.0:4317 + exporters: + logging: + loglevel: info + jaeger: + endpoint: simplest-collector:14250 + tls: + insecure: true + extensions: + pprof: + endpoint: :1888 + zpages: + endpoint: :55679 + processors: + batch: + timeout: 1s + resource: + attributes: + - key: test.key + value: "test-value" + action: insert + service: + extensions: [pprof, zpages] + pipelines: + traces: + receivers: [otlp] + processors: [batch, resource] + exporters: [logging, jaeger] \ No newline at end of file diff --git a/values/cluster/jaeger-all-in-one.yaml b/values/cluster/jaeger-all-in-one.yaml new file mode 100644 index 0000000..38eff50 --- /dev/null +++ b/values/cluster/jaeger-all-in-one.yaml @@ -0,0 +1,4 @@ +apiVersion: jaegertracing.io/v1 +kind: Jaeger +metadata: + name: simplest