Skip to content

HADOOP-17465. Upgrade to Focal #2613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jan 26, 2021
+46 −59
Merged

HADOOP-17465. Upgrade to Focal #2613

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
b9a44de
HADOOP-17465. Upgrade to Focal
GauthamBanasandra Jan 12, 2021
23f9922
Install python-pip separately
GauthamBanasandra Jan 12, 2021
8085cb5
Use Boost from Focal toolchain
GauthamBanasandra Jan 13, 2021
fda981a
Use Java protobuf from Focal toolchain
GauthamBanasandra Jan 13, 2021
ad8f707
Use nodejs from Focal toolchain
GauthamBanasandra Jan 13, 2021
b659d6f
Use node-yarn from Focal toolchain
GauthamBanasandra Jan 13, 2021
a59bf66
Use hugo from Focal toolchain
GauthamBanasandra Jan 13, 2021
013ead2
Use npm from Focal toolchain
GauthamBanasandra Jan 13, 2021
37ae7b1
Add comment for pip
GauthamBanasandra Jan 13, 2021
e7b5b23
Update docker aarch64 to use Focal
GauthamBanasandra Jan 13, 2021
d16aea4
Touch files to run tests
GauthamBanasandra Jan 13, 2021
b91b923
Downgrade Boost version to 1.71.0
GauthamBanasandra Jan 14, 2021
3076f21
Touch file to build
GauthamBanasandra Jan 14, 2021
b1d2f16
Install Boost separately
GauthamBanasandra Jan 14, 2021
72b4ac2
Revert "Use Java protobuf from Focal toolchain"
GauthamBanasandra Jan 14, 2021
76f3616
Revert "Touch files to run tests"
GauthamBanasandra Jan 22, 2021
6027c1f
Install Java Protobuf for aarch64
GauthamBanasandra Jan 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 23 additions & 30 deletions dev-support/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
Expand All @@ -18,7 +17,7 @@
# Dockerfile for installing the necessary dependencies for building Hadoop.
# See BUILDING.txt.

FROM ubuntu:bionic
FROM ubuntu:focal

WORKDIR /root

Expand Down Expand Up @@ -51,6 +50,7 @@ RUN apt-get -q update \
gcc \
git \
gnupg-agent \
hugo \
libbcprov-java \
libbz2-dev \
libcurl4-openssl-dev \
Expand All @@ -65,16 +65,17 @@ RUN apt-get -q update \
locales \
make \
maven \
nodejs \
node-yarn \
npm \
openjdk-11-jdk \
openjdk-8-jdk \
pinentry-curses \
pkg-config \
python \
python2.7 \
python-pip \
python-pkg-resources \
python-setuptools \
python-wheel \
rsync \
shellcheck \
software-properties-common \
Expand All @@ -93,7 +94,7 @@ ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV FINDBUGS_HOME /usr

#######
# Install Boost 1.72 (1.65 ships with Bionic)
# Install Boost 1.72 (1.71 ships with Focal)
#######
# hadolint ignore=DL3003
RUN mkdir -p /opt/boost-library \
Expand All @@ -107,8 +108,20 @@ RUN mkdir -p /opt/boost-library \
&& cd /root \
&& rm -rf /opt/boost-library

####
# Install pip (deprecated from Focal toolchain)
####
# hadolint ignore=DL3003
RUN mkdir -p /opt/pip \
&& curl -L https://bootstrap.pypa.io/get-pip.py > get-pip.py \
&& mv get-pip.py /opt/pip \
&& cd /opt/pip \
&& python2.7 get-pip.py \
&& cd /root \
&& rm -rf /opt/pip

######
# Install Google Protobuf 3.7.1 (3.0.0 ships with Bionic)
# Install Google Protobuf 3.7.1 (3.6.1 ships with Focal)
######
# hadolint ignore=DL3003
RUN mkdir -p /opt/protobuf-src \
Expand Down Expand Up @@ -139,25 +152,11 @@ RUN pip2 install \
####
RUN pip2 install python-dateutil==2.7.3

###
# Install node.js 10.x for web UI framework (4.2.6 ships with Xenial)
###
# hadolint ignore=DL3008
RUN curl -L -s -S https://deb.nodesource.com/setup_10.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& npm install -g [email protected]

###
## Install Yarn 1.22.5 for web UI framework
####
RUN curl -s -S https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& echo 'deb https://dl.yarnpkg.com/debian/ stable main' > /etc/apt/sources.list.d/yarn.list \
&& apt-get -q update \
&& apt-get install -y --no-install-recommends yarn=1.22.5-1 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Install bower
####
# hadolint ignore=DL3008
RUN npm install -g [email protected]

###
# Install hadolint
Expand Down Expand Up @@ -203,12 +202,6 @@ ENV HADOOP_SKIP_YETUS_VERIFICATION true
# YETUS CUT HERE
###

# Hugo static website generator for new hadoop site
RUN curl -L -o hugo.deb https://github.com/gohugoio/hugo/releases/download/v0.58.3/hugo_0.58.3_Linux-64bit.deb \
&& dpkg --install hugo.deb \
&& rm hugo.deb


# Add a welcome message and environment checks.
COPY hadoop_env_checks.sh /root/hadoop_env_checks.sh
RUN chmod 755 /root/hadoop_env_checks.sh
Expand Down
52 changes: 23 additions & 29 deletions dev-support/docker/Dockerfile_aarch64
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
# Dockerfile for installing the necessary dependencies for building Hadoop.
# See BUILDING.txt.

FROM ubuntu:bionic
FROM ubuntu:focal

WORKDIR /root

Expand Down Expand Up @@ -54,6 +54,7 @@ RUN apt-get -q update \
gcc \
git \
gnupg-agent \
hugo \
libbcprov-java \
libbz2-dev \
libcurl4-openssl-dev \
Expand All @@ -68,16 +69,17 @@ RUN apt-get -q update \
locales \
make \
maven \
nodejs \
node-yarn \
npm \
openjdk-11-jdk \
openjdk-8-jdk \
pinentry-curses \
pkg-config \
python \
python2.7 \
python-pip \
python-pkg-resources \
python-setuptools \
python-wheel \
rsync \
shellcheck \
software-properties-common \
Expand All @@ -96,7 +98,7 @@ ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-arm64
ENV FINDBUGS_HOME /usr

#######
# Install Boost 1.72 (1.65 ships with Bionic)
# Install Boost 1.72 (1.71 ships with Focal)
#######
# hadolint ignore=DL3003
RUN mkdir -p /opt/boost-library \
Expand All @@ -110,8 +112,20 @@ RUN mkdir -p /opt/boost-library \
&& cd /root \
&& rm -rf /opt/boost-library

####
# Install pip (deprecated from Focal toolchain)
####
# hadolint ignore=DL3003
RUN mkdir -p /opt/pip \
&& curl -L https://bootstrap.pypa.io/get-pip.py > get-pip.py \
&& mv get-pip.py /opt/pip \
&& cd /opt/pip \
&& python2.7 get-pip.py \
&& cd /root \
&& rm -rf /opt/pip

######
# Install Google Protobuf 3.7.1 (3.0.0 ships with Bionic)
# Install Google Protobuf 3.7.1 (3.6.1 ships with Focal)
######
# hadolint ignore=DL3003
RUN mkdir -p /opt/protobuf-src \
Expand Down Expand Up @@ -142,25 +156,11 @@ RUN pip2 install \
####
RUN pip2 install python-dateutil==2.7.3

###
# Install node.js 10.x for web UI framework (4.2.6 ships with Xenial)
###
# hadolint ignore=DL3008
RUN curl -L -s -S https://deb.nodesource.com/setup_10.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& npm install -g [email protected]

###
## Install Yarn 1.22.5 for web UI framework
####
RUN curl -s -S https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& echo 'deb https://dl.yarnpkg.com/debian/ stable main' > /etc/apt/sources.list.d/yarn.list \
&& apt-get -q update \
&& apt-get install -y --no-install-recommends yarn=1.22.5-1 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Install bower
####
# hadolint ignore=DL3008
RUN npm install -g [email protected]

###
# Install phantomjs built for aarch64
Expand All @@ -187,12 +187,6 @@ ENV HADOOP_SKIP_YETUS_VERIFICATION true
# YETUS CUT HERE
###

# Hugo static website generator (for new hadoop site docs)
RUN curl -L -o hugo.deb https://github.com/gohugoio/hugo/releases/download/v0.58.3/hugo_0.58.3_Linux-ARM64.deb \
&& dpkg --install hugo.deb \
&& rm hugo.deb


# Add a welcome message and environment checks.
COPY hadoop_env_checks.sh /root/hadoop_env_checks.sh
RUN chmod 755 /root/hadoop_env_checks.sh
Expand Down