Skip to content

Commit 7991159

Browse files
committed
SUBMARINE-82. Fix english grammar mistakes in documentation. Contributed by Szilard Nemeth.
1 parent 5d5081e commit 7991159

File tree

7 files changed

+95
-77
lines changed

7 files changed

+95
-77
lines changed

hadoop-submarine/hadoop-submarine-core/src/site/markdown/Examples.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414

1515
# Examples
1616

17-
Here're some examples about Submarine usage.
17+
Here are some examples about how to use Submarine:
1818

1919
[Running Distributed CIFAR 10 Tensorflow Job](RunningDistributedCifar10TFJobs.html)
2020

hadoop-submarine/hadoop-submarine-core/src/site/markdown/HowToInstall.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -14,23 +14,23 @@
1414

1515
# How to Install Dependencies
1616

17-
Submarine project uses YARN Service, Docker container, and GPU (when GPU hardware available and properly configured).
17+
Submarine project uses YARN Service, Docker container and GPU.
18+
GPU could only be used if a GPU hardware is available and properly configured.
1819

19-
That means as an admin, you have to properly setup YARN Service related dependencies, including:
20+
As an administrator, you have to properly setup YARN Service related dependencies, including:
2021
- YARN Registry DNS
22+
- Docker related dependencies, including:
23+
- Docker binary with expected versions
24+
- Docker network that allows Docker containers to talk to each other across different nodes
2125

22-
Docker related dependencies, including:
23-
- Docker binary with expected versions.
24-
- Docker network which allows Docker container can talk to each other across different nodes.
26+
If you would like to use GPU, you need to set up:
27+
- GPU Driver
28+
- Nvidia-docker
2529

26-
And when GPU wanna to be used:
27-
- GPU Driver.
28-
- Nvidia-docker.
29-
30-
For your convenience, we provided installation documents to help you to setup your environment. You can always choose to have them installed in your own way.
30+
For your convenience, we provided some installation documents to help you setup your environment. You can always choose to have them installed in your own way.
3131

3232
Use Submarine installer to install dependencies: [EN](https://github.com/hadoopsubmarine/hadoop-submarine-ecosystem/tree/master/submarine-installer) [CN](https://github.com/hadoopsubmarine/hadoop-submarine-ecosystem/blob/master/submarine-installer/README-CN.md)
3333

34-
Alternatively, you can follow manual install dependencies: [EN](InstallationGuide.html) [CN](InstallationGuideChineseVersion.html)
34+
Alternatively, you can follow this guide to manually install dependencies: [EN](InstallationGuide.html) [CN](InstallationGuideChineseVersion.html)
3535

36-
Once you have installed dependencies, please follow following guide to [TestAndTroubleshooting](TestAndTroubleshooting.html).
36+
Once you have installed all the dependencies, please follow this guide: [TestAndTroubleshooting](TestAndTroubleshooting.html).

hadoop-submarine/hadoop-submarine-core/src/site/markdown/Index.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,20 +21,20 @@ Goals of Submarine:
2121

2222
- Can launch services to serve Tensorflow/MXNet models.
2323

24-
- Support run distributed Tensorflow jobs with simple configs.
24+
- Supports running distributed Tensorflow jobs with simple configs.
2525

26-
- Support run standalone PyTorch jobs with simple configs.
26+
- Supports running standalone PyTorch jobs with simple configs.
2727

28-
- Support run user-specified Docker images.
28+
- Supports running user-specified Docker images.
2929

30-
- Support specify GPU and other resources.
30+
- Supports specifying GPU and other resources.
3131

32-
- Support launch tensorboard for training jobs if user specified.
32+
- Supports launching Tensorboard for training jobs (optional, if specified).
3333

34-
- Support customized DNS name for roles (like tensorboard.$user.$domain:6006)
34+
- Supports customized DNS name for roles (like tensorboard.$user.$domain:6006)
3535

3636

37-
Click below contents if you want to understand more.
37+
If you want to deep-dive, please check these resources:
3838

3939
- [QuickStart Guide](QuickStart.html)
4040

hadoop-submarine/hadoop-submarine-core/src/site/markdown/InstallationGuide.md

Lines changed: 47 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -16,20 +16,25 @@
1616

1717
## Prerequisites
1818

19-
(Please note that all following prerequisites are just an example for you to install. You can always choose to install your own version of kernel, different users, different drivers, etc.).
19+
Please note that the following prerequisites are just an example for you to install Submarine.
20+
21+
You can always choose to install your own version of kernel, different users, different drivers, etc.
2022

2123
### Operating System
2224

23-
The operating system and kernel versions we have tested are as shown in the following table, which is the recommneded minimum required versions.
25+
The operating system and kernel versions we have tested against are shown in the following table.
26+
The versions in the table are the recommended minimum required versions.
2427

25-
| Enviroment | Verion |
28+
| Environment | Version |
2629
| ------ | ------ |
2730
| Operating System | centos-release-7-5.1804.el7.centos.x86_64 |
28-
| Kernal | 3.10.0-862.el7.x86_64 |
31+
| Kernel | 3.10.0-862.el7.x86_64 |
2932

3033
### User & Group
3134

32-
As there are some specific users and groups recommended to be created to install hadoop/docker. Please create them if they are missing.
35+
There are specific users and groups recommended to be created to install Hadoop with Docker.
36+
37+
Please create these users if they do not exist.
3338

3439
```
3540
adduser hdfs
@@ -80,7 +85,9 @@ lspci | grep -i nvidia
8085

8186
### Nvidia Driver Installation (Only for Nvidia GPU equipped nodes)
8287

83-
To make a clean installation, if you have requirements to upgrade GPU drivers. If nvidia driver/cuda has been installed before, They should be uninstalled firstly.
88+
To make a clean installation, if you have requirements to upgrade GPU drivers.
89+
90+
If nvidia driver / CUDA has been installed before, they should be uninstalled as a first step.
8491

8592
```
8693
# uninstall cuda:
@@ -90,7 +97,7 @@ sudo /usr/local/cuda-10.0/bin/uninstall_cuda_10.0.pl
9097
sudo /usr/bin/nvidia-uninstall
9198
```
9299

93-
To check GPU version, install nvidia-detect
100+
To check GPU version, install nvidia-detect:
94101

95102
```
96103
yum install nvidia-detect
@@ -107,7 +114,9 @@ Pay attention to `This device requires the current xyz.nm NVIDIA driver kmod-nvi
107114
Download the installer like [NVIDIA-Linux-x86_64-390.87.run](https://www.nvidia.com/object/linux-amd64-display-archive.html).
108115

109116

110-
Some preparatory work for nvidia driver installation. (This is follow normal Nvidia GPU driver installation, just put here for your convenience)
117+
Some preparatory work for Nvidia driver installation.
118+
119+
The steps below are for Nvidia GPU driver installation, just pasted here for your convenience.
111120

112121
```
113122
# It may take a while to update
@@ -152,7 +161,7 @@ Would you like to run the nvidia-xconfig utility to automatically update your X
152161
```
153162

154163

155-
Check nvidia driver installation
164+
Check Nvidia driver installation
156165

157166
```
158167
nvidia-smi
@@ -165,7 +174,7 @@ https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
165174

166175
### Docker Installation
167176

168-
The following steps show how to install docker 18.06.1.ce. You can choose other approaches to install Docker.
177+
The following steps show you how to install docker 18.06.1.ce. You can choose other approaches to install Docker.
169178

170179
```
171180
# Remove old version docker
@@ -205,7 +214,9 @@ Reference:https://docs.docker.com/install/linux/docker-ce/centos/
205214

206215
### Docker Configuration
207216

208-
Add a file, named daemon.json, under the path of /etc/docker/. Please replace the variables of image_registry_ip, etcd_host_ip, localhost_ip, yarn_dns_registry_host_ip, dns_host_ip with specific ips according to your environments.
217+
Add a file, named daemon.json, under the path of /etc/docker/.
218+
219+
Please replace the variables of image_registry_ip, etcd_host_ip, localhost_ip, yarn_dns_registry_host_ip, dns_host_ip with specific IPs according to your environment.
209220

210221
```
211222
{
@@ -294,7 +305,7 @@ import tensorflow as tf
294305
tf.test.is_gpu_available()
295306
```
296307

297-
The way to uninstall nvidia-docker V2
308+
If you want to uninstall nvidia-docker V2:
298309
```
299310
sudo yum remove -y nvidia-docker2-2.0.3-1.docker18.06.1.ce
300311
```
@@ -304,12 +315,14 @@ https://github.com/NVIDIA/nvidia-docker
304315

305316
### Tensorflow Image
306317

307-
There is no need to install CUDNN and CUDA on the servers, because CUDNN and CUDA can be added in the docker images. We can get basic docker images by referring to [Write Dockerfile](WriteDockerfileTF.html).
318+
There is no need to install CUDNN and CUDA on the servers, because CUDNN and CUDA can be added in the docker images.
319+
320+
We can get or build basic docker images by referring to [Write Dockerfile](WriteDockerfileTF.html).
308321

309322
### Test tensorflow in a docker container
310323

311324
After docker image is built, we can check
312-
Tensorflow environments before submitting a yarn job.
325+
Tensorflow environments before submitting a Submarine job.
313326

314327
```shell
315328
$ docker run -it ${docker_image_name} /bin/bash
@@ -336,8 +349,8 @@ If there are some errors, we could check the following configuration.
336349

337350
### Etcd Installation
338351

339-
etcd is a distributed reliable key-value store for the most critical data of a distributed system, Registration and discovery of services used in containers.
340-
You can also choose alternatives like zookeeper, Consul.
352+
etcd is a distributed, reliable key-value store for the most critical data of a distributed system, Registration and discovery of services used in containers.
353+
You can also choose alternatives like ZooKeeper, Consul or others.
341354

342355
To install Etcd on specified servers, we can run Submarine-installer/install.sh
343356

@@ -366,8 +379,10 @@ b3d05464c356441a: name=etcdnode1 peerURLs=http://${etcd_host_ip3}:2380 clientURL
366379

367380
### Calico Installation
368381

369-
Calico creates and manages a flat three-tier network, and each container is assigned a routable ip. We just add the steps here for your convenience.
370-
You can also choose alternatives like Flannel, OVS.
382+
Calico creates and manages a flat three-tier network, and each container is assigned a routable IP address.
383+
384+
We are listing the steps here for your convenience.
385+
You can also choose alternatives like Flannel, OVS or others.
371386

372387
To install Calico on specified servers, we can run Submarine-installer/install.sh
373388

@@ -379,7 +394,7 @@ systemctl status calico-node.service
379394
#### Check Calico Network
380395

381396
```shell
382-
# Run the following command to show the all host status in the cluster except localhost.
397+
# Run the following command to show all host status in the cluster except localhost.
383398
$ calicoctl node status
384399
Calico process is running.
385400

@@ -412,7 +427,7 @@ docker exec workload-A ping workload-B
412427
You can either get Hadoop release binary or compile from source code. Please follow the https://hadoop.apache.org/ guides.
413428

414429

415-
### Start yarn service
430+
### Start YARN service
416431

417432
```
418433
YARN_LOGFILE=resourcemanager.log ./sbin/yarn-daemon.sh start resourcemanager
@@ -421,7 +436,7 @@ YARN_LOGFILE=timeline.log ./sbin/yarn-daemon.sh start timelineserver
421436
YARN_LOGFILE=mr-historyserver.log ./sbin/mr-jobhistory-daemon.sh start historyserver
422437
```
423438

424-
### Start yarn registery dns service
439+
### Start YARN registry DNS service
425440

426441
```
427442
sudo YARN_LOGFILE=registrydns.log ./yarn-daemon.sh start registrydns
@@ -441,13 +456,13 @@ sudo YARN_LOGFILE=registrydns.log ./yarn-daemon.sh start registrydns
441456

442457
#### Clean up apps with the same name
443458

444-
Suppose we want to submit a tensorflow job named standalone-tf, destroy any application with the same name and clean up historical job directories.
459+
Suppose we want to submit a TensorFlow job named standalone-tf, destroy any application with the same name and clean up historical job directories.
445460

446461
```bash
447462
./bin/yarn app -destroy standalone-tf
448463
./bin/hdfs dfs -rmr hdfs://${dfs_name_service}/tmp/cifar-10-jobdir
449464
```
450-
where ${dfs_name_service} is the hdfs name service you use
465+
where ${dfs_name_service} is the HDFS name service you use
451466

452467
#### Run a standalone tensorflow job
453468

@@ -471,7 +486,7 @@ where ${dfs_name_service} is the hdfs name service you use
471486
./bin/hdfs dfs -rmr hdfs://${dfs_name_service}/tmp/cifar-10-jobdir
472487
```
473488

474-
#### Run a distributed tensorflow job
489+
#### Run a distributed TensorFlow job
475490

476491
```bash
477492
./bin/yarn jar /home/hadoop/hadoop-current/share/hadoop/yarn/hadoop-yarn-submarine-3.2.0-SNAPSHOT.jar job run \
@@ -490,11 +505,11 @@ where ${dfs_name_service} is the hdfs name service you use
490505
```
491506

492507

493-
## Tensorflow Job with GPU
508+
## TensorFlow Job with GPU
494509

495-
### GPU configurations for both resourcemanager and nodemanager
510+
### GPU configurations for both ResourceManager and NodeManager
496511

497-
Add the yarn resource configuration file, named resource-types.xml
512+
Add the YARN resource configuration file, named resource-types.xml
498513

499514
```
500515
<configuration>
@@ -505,9 +520,9 @@ Add the yarn resource configuration file, named resource-types.xml
505520
</configuration>
506521
```
507522

508-
#### GPU configurations for resourcemanager
523+
#### GPU configurations for ResourceManager
509524

510-
The scheduler used by resourcemanager must be capacity scheduler, and yarn.scheduler.capacity.resource-calculator in capacity-scheduler.xml should be DominantResourceCalculator
525+
The scheduler used by ResourceManager must be the capacity scheduler, and yarn.scheduler.capacity.resource-calculator in capacity-scheduler.xml should be DominantResourceCalculator
511526

512527
```
513528
<configuration>
@@ -518,7 +533,7 @@ The scheduler used by resourcemanager must be capacity scheduler, and yarn.sche
518533
</configuration>
519534
```
520535

521-
#### GPU configurations for nodemanager
536+
#### GPU configurations for NodeManager
522537

523538
Add configurations in yarn-site.xml
524539

@@ -536,7 +551,7 @@ Add configurations in yarn-site.xml
536551
</configuration>
537552
```
538553

539-
Add configurations in container-executor.cfg
554+
Add configurations to container-executor.cfg
540555

541556
```
542557
[docker]
@@ -560,7 +575,7 @@ Add configurations in container-executor.cfg
560575
yarn-hierarchy=/hadoop-yarn
561576
```
562577

563-
### Run a distributed tensorflow gpu job
578+
### Run a distributed TensorFlow GPU job
564579

565580
```bash
566581
./yarn jar /home/hadoop/hadoop-current/share/hadoop/yarn/hadoop-yarn-submarine-3.2.0-SNAPSHOT.jar job run \

0 commit comments

Comments
 (0)