Skip to content

HDFS-14907. [Dynamometer] DataNode can't find junit jar when using Hadoop-3 binary #1680

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 29, 2019
Merged

HDFS-14907. [Dynamometer] DataNode can't find junit jar when using Hadoop-3 binary #1680

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2fb22b1
HDFS-14907. [Dynamometer] DataNode can't find junit jar when using Ha…
tasanuma Oct 28, 2019
21a295b
fix a shellcheck issue
tasanuma Oct 28, 2019
e625662
fix a shellcheck issue again
tasanuma Oct 28, 2019
995a879
add more comments
tasanuma Oct 29, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion ...p-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/resources/start-component.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,13 @@ function hdfs_original {
extraClasspathDir="$(pwd)/additionalClasspath/"
mkdir -p "${extraClasspathDir}"

# DataNodes need junit jar to run SimulatedDataNodes
junitClassPath="$(find "${hadoopHome}" -name "junit*.jar" | head -n 1)"
Copy link
Contributor

@xkrogen xkrogen Oct 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment here explaining why this is necessary?

if [[ -z "$junitClassPath" ]]; then
echo "Can't find junit jar file in ${hadoopHome}."
exit 1
fi

# Change environment variables for the Hadoop process
export HADOOP_HOME="$hadoopHome"
export HADOOP_PREFIX="$hadoopHome"
Expand All @@ -107,7 +114,7 @@ export HADOOP_CONF_DIR="${confDir}"
export YARN_CONF_DIR="${confDir}"
export HADOOP_LOG_DIR="${logDir}"
export HADOOP_PID_DIR="${pidDir}"
HADOOP_CLASSPATH="$(pwd)/dependencies/*:$extraClasspathDir"
HADOOP_CLASSPATH="$(pwd)/dependencies/*:$extraClasspathDir:$junitClassPath"
export HADOOP_CLASSPATH
echo "Environment variables are set as:"
echo "(note that this doesn't include changes made by hadoop-env.sh)"
Expand Down