Skip to content

[FLINK-17857][e2e] Fix Kubernetes and docker e2e tests on Mac OS #12451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

[FLINK-17857][e2e] Fix Kubernetes and docker e2e tests on Mac OS #12451

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
072992f
[FLINK-17857][e2e] Fix Kubernetes e2e tests on Mac OS
wangyang0918 May 29, 2020
4c93b68
[FLINK-17857][e2e] Fix docker e2e tests on Mac OS
wangyang0918 Jun 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion flink-end-to-end-tests/test-scripts/common_docker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ function containers_health_check() {

function build_image() {
local image_name=${1:-flink-job}
local default_file_server_address="localhost"
[[ "${OS_TYPE}" != "linux" ]] && default_file_server_address="host.docker.internal"
local file_server_address=${2:-${default_file_server_address}}

echo "Starting fileserver for Flink distribution"
pushd ${FLINK_DIR}/..
Expand All @@ -47,7 +50,7 @@ function build_image() {
echo "Preparing Dockeriles"
git clone https://github.com/apache/flink-docker.git --branch dev-master --single-branch
cd flink-docker
./add-custom.sh -u localhost:9999/flink.tgz -n ${image_name}
./add-custom.sh -u ${file_server_address}:9999/flink.tgz -n ${image_name}

echo "Building images"
docker build --no-cache --network="host" -t ${image_name} dev/${image_name}-debian
Expand Down
12 changes: 12 additions & 0 deletions flink-end-to-end-tests/test-scripts/common_kubernetes.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,9 @@ function start_kubernetes {
echo "$NON_LINUX_ENV_NOTE"
exit 1
fi
# Mount Flink dist into minikube virtual machine because we need to mount hostPath as usrlib
minikube mount $FLINK_DIR:$FLINK_DIR &
export minikube_mount_pid=$!
else
setup_kubernetes_for_linux
if ! retry_times ${MINIKUBE_START_RETRIES} ${MINIKUBE_START_BACKOFF} start_kubernetes_if_not_running; then
Expand All @@ -107,6 +110,7 @@ function start_kubernetes {
function stop_kubernetes {
if [[ "${OS_TYPE}" != "linux" ]]; then
echo "$NON_LINUX_ENV_NOTE"
kill $minikube_mount_pid 2> /dev/null
else
echo "Stopping minikube ..."
stop_command="sudo minikube stop"
Expand Down Expand Up @@ -175,4 +179,12 @@ appender.console.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%t] %-60c %x
END
}

function get_host_machine_address {
if [[ "${OS_TYPE}" != "linux" ]]; then
echo $(minikube ssh "route -n | grep ^0.0.0.0 | awk '{ print \$2 }' | tr -d '[:space:]'")
else
echo "localhost"
fi
}

on_exit cleanup
2 changes: 1 addition & 1 deletion flink-end-to-end-tests/test-scripts/test_kubernetes_application.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ setConsoleLogging

start_kubernetes

build_image ${FLINK_IMAGE_NAME}
build_image ${FLINK_IMAGE_NAME} $(get_host_machine_address)

kubectl create clusterrolebinding ${CLUSTER_ROLE_BINDING} --clusterrole=edit --serviceaccount=default:default --namespace=default

Expand Down
2 changes: 1 addition & 1 deletion flink-end-to-end-tests/test-scripts/test_kubernetes_embedded_job.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ start_kubernetes

mkdir -p $OUTPUT_VOLUME

build_image ${FLINK_IMAGE_NAME}
build_image ${FLINK_IMAGE_NAME} $(get_host_machine_address)

export USER_LIB=${FLINK_DIR}/examples/batch
kubectl create -f ${CONTAINER_SCRIPTS}/job-cluster-service.yaml
Expand Down
2 changes: 1 addition & 1 deletion flink-end-to-end-tests/test-scripts/test_kubernetes_session.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ setConsoleLogging

start_kubernetes

build_image ${FLINK_IMAGE_NAME}
build_image ${FLINK_IMAGE_NAME} $(get_host_machine_address)

kubectl create clusterrolebinding ${CLUSTER_ROLE_BINDING} --clusterrole=edit --serviceaccount=default:default --namespace=default

Expand Down