Use rogue instead of pygments.rb which is not Python 3 compatible

README.md

@@ -6,8 +6,7 @@ Building the site requires [Jekyll](http://jekyllrb.com/docs) 3.3.0 or newer.
 The easiest way to install jekyll is via a Ruby Gem. This will create a directory called `site` 
 containing `index.html` as well as the rest of the compiled directories and files.
 
-To install Jekyll and its required dependencies, execute `sudo gem install jekyll pygments.rb` 
-and `sudo pip install Pygments`.
+To install Jekyll and its required dependencies, execute `sudo gem install jekyll rogue`.
 See also https://github.com/apache/spark/blob/master/docs/README.md
 
 You can generate the html website by running `jekyll build` in this directory. Use the `--watch` 
@@ -17,7 +16,7 @@ In addition to generating the site as HTML from the markdown files, jekyll can s
 a web server. To build the site and run a web server use the command `jekyll serve` which runs 
 the web server on port 4000, then visit the site at http://localhost:4000.
 
-Please make sure you always run `jekyll build` after testing your changes with `jekyll server`,
+Please make sure you always run `jekyll build` after testing your changes with `jekyll serve`,
 otherwise you end up with broken links in a few places.
 
 ## Docs sub-dir

_config.yml

@@ -1,4 +1,4 @@
-highlighter: pygments
+highlighter: rouge
 markdown: kramdown
 kramdown:
   entity_output: symbol

css/pygments-default.css

@@ -9,6 +9,10 @@ Also, I was thrown off for a while at first when I was using markdown
 code block inside my {% highlight scala %} ... {% endhighlight %} tags
 (I was using 4 spaces for this), when it turns out that pygments will
 insert the code (or pre?) tags for you.
+
+Note that due to Python 3 compatibility in the project, now we use
+Rouge which claims Pygments compatibility, instead of pygments.rb which
+does not support Python 3. See SPARK-28752.
 */
 
 .hll { background-color: #ffffcc }
@@ -71,4 +75,4 @@ insert the code (or pre?) tags for you.
 .vc { color: #bb60d5 } /* Name.Variable.Class */
 .vg { color: #bb60d5 } /* Name.Variable.Global */
 .vi { color: #bb60d5 } /* Name.Variable.Instance */
-.il { color: #40a070 } /* Literal.Number.Integer.Long */
+.il { color: #40a070 } /* Literal.Number.Integer.Long */

site/committers.html

@@ -553,7 +553,7 @@ <h3>Review Process</h3>
 <a href="/contributing.html">Contributing to Spark</a>. 
 In particular, if you are working on an area of the codebase you are unfamiliar with, look at the 
 Git history for that code to see who reviewed patches before. You can do this using 
-<code>git log --format=full &lt;filename&gt;</code>, by examining the &#8220;Commit&#8221; field to see who committed each patch.</p>
+<code class="highlighter-rouge">git log --format=full &lt;filename&gt;</code>, by examining the &#8220;Commit&#8221; field to see who committed each patch.</p>
 
 <h3>How to Merge a Pull Request</h3>
 
@@ -562,34 +562,34 @@ <h3>How to Merge a Pull Request</h3>
 
 <h4>Setting up Remotes</h4>
 
-<p>To use the <code>merge_spark_pr.py</code> script described below, you 
-will need to add a git remote called <code>apache</code> at <code>https://github.com/apache/spark</code>, 
-as well as one called <code>apache-github</code> at <code>git://github.com/apache/spark</code>.</p>
+<p>To use the <code class="highlighter-rouge">merge_spark_pr.py</code> script described below, you 
+will need to add a git remote called <code class="highlighter-rouge">apache</code> at <code class="highlighter-rouge">https://github.com/apache/spark</code>, 
+as well as one called <code class="highlighter-rouge">apache-github</code> at <code class="highlighter-rouge">git://github.com/apache/spark</code>.</p>
 
-<p>You will likely also have a remote <code>origin</code> pointing to your fork of Spark, and
-<code>upstream</code> pointing to the <code>apache/spark</code> GitHub repo.</p>
+<p>You will likely also have a remote <code class="highlighter-rouge">origin</code> pointing to your fork of Spark, and
+<code class="highlighter-rouge">upstream</code> pointing to the <code class="highlighter-rouge">apache/spark</code> GitHub repo.</p>
 
-<p>If correct, your <code>git remote -v</code> should look like:</p>
+<p>If correct, your <code class="highlighter-rouge">git remote -v</code> should look like:</p>
 
-<pre><code>apache	https://github.com/apache/spark.git (fetch)
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apache	https://github.com/apache/spark.git (fetch)
 apache	https://github.com/apache/spark.git (push)
 apache-github	git://github.com/apache/spark (fetch)
 apache-github	git://github.com/apache/spark (push)
 origin	https://github.com/[your username]/spark.git (fetch)
 origin	https://github.com/[your username]/spark.git (push)
 upstream	https://github.com/apache/spark.git (fetch)
 upstream	https://github.com/apache/spark.git (push)
-</code></pre>
+</code></pre></div></div>
 
-<p>For the <code>apache</code> repo, you will need to set up command-line authentication to GitHub. This may
+<p>For the <code class="highlighter-rouge">apache</code> repo, you will need to set up command-line authentication to GitHub. This may
 include setting up an SSH key and/or personal access token. See:</p>
 
 <ul>
   <li>https://help.github.com/articles/connecting-to-github-with-ssh/</li>
   <li>https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/</li>
 </ul>
 
-<p>Ask <code>[email protected]</code> if you have trouble with these steps, or want help doing your first merge.</p>
+<p>Ask <code class="highlighter-rouge">[email protected]</code> if you have trouble with these steps, or want help doing your first merge.</p>
 
 <h4>Merge Script</h4>
 
@@ -601,9 +601,9 @@ <h4>Merge Script</h4>
 
 <p>If you want to amend a commit before merging – which should be used for trivial touch-ups – 
 then simply let the script wait at the point where it asks you if you want to push to Apache. 
-Then, in a separate window, modify the code and push a commit. Run <code>git rebase -i HEAD~2</code> and 
+Then, in a separate window, modify the code and push a commit. Run <code class="highlighter-rouge">git rebase -i HEAD~2</code> and 
 &#8220;squash&#8221; your new commit. Edit the commit message just after to remove your commit message. 
-You can verify the result is one change with <code>git log</code>. Then resume the script in the other window.</p>
+You can verify the result is one change with <code class="highlighter-rouge">git log</code>. Then resume the script in the other window.</p>
 
 <p>Also, please remember to set Assignee on JIRAs where applicable when they are resolved. The script 
 can do this automatically in most cases. However where the contributor is not yet a part of the
@@ -615,7 +615,7 @@ <h4>Merge Script</h4>
 
 <h3>Policy on Backporting Bug Fixes</h3>
 
-<p>From <a href="https://www.mail-archive.com/[email protected]/msg10284.html"><code>pwendell</code></a>:</p>
+<p>From <a href="https://www.mail-archive.com/[email protected]/msg10284.html"><code class="highlighter-rouge">pwendell</code></a>:</p>
 
 <p>The trade off when backporting is you get to deliver the fix to people running older versions 
 (great!), but you risk introducing new or even worse bugs in maintenance releases (bad!). 

site/community.html

@@ -208,7 +208,7 @@ <h3>Have Questions?</h3>
 <h4>StackOverflow</h4>
 
 <p>For usage questions and help (e.g. how to use this Spark API), it is recommended you use the 
-StackOverflow tag <a href="https://stackoverflow.com/questions/tagged/apache-spark"><code>apache-spark</code></a> 
+StackOverflow tag <a href="https://stackoverflow.com/questions/tagged/apache-spark"><code class="highlighter-rouge">apache-spark</code></a> 
 as it is an active forum for Spark users&#8217; questions and answers.</p>
 
 <p>Some quick tips when using StackOverflow:</p>
@@ -217,17 +217,17 @@ <h4>StackOverflow</h4>
   <li>Prior to asking submitting questions, please:
     <ul>
       <li>Search StackOverflow&#8217;s 
-<a href="https://stackoverflow.com/questions/tagged/apache-spark"><code>apache-spark</code></a> tag to see if 
+<a href="https://stackoverflow.com/questions/tagged/apache-spark"><code class="highlighter-rouge">apache-spark</code></a> tag to see if 
 your question has already been answered</li>
       <li>Search the nabble archive for
 <a href="http://apache-spark-user-list.1001560.n3.nabble.com/">[email protected]</a></li>
     </ul>
   </li>
   <li>Please follow the StackOverflow <a href="https://stackoverflow.com/help/how-to-ask">code of conduct</a></li>
-  <li>Always use the <code>apache-spark</code> tag when asking questions</li>
+  <li>Always use the <code class="highlighter-rouge">apache-spark</code> tag when asking questions</li>
   <li>Please also use a secondary tag to specify components so subject matter experts can more easily find them.
- Examples include: <code>pyspark</code>, <code>spark-dataframe</code>, <code>spark-streaming</code>, <code>spark-r</code>, <code>spark-mllib</code>, 
-<code>spark-ml</code>, <code>spark-graphx</code>, <code>spark-graphframes</code>, <code>spark-tensorframes</code>, etc.</li>
+ Examples include: <code class="highlighter-rouge">pyspark</code>, <code class="highlighter-rouge">spark-dataframe</code>, <code class="highlighter-rouge">spark-streaming</code>, <code class="highlighter-rouge">spark-r</code>, <code class="highlighter-rouge">spark-mllib</code>, 
+<code class="highlighter-rouge">spark-ml</code>, <code class="highlighter-rouge">spark-graphx</code>, <code class="highlighter-rouge">spark-graphframes</code>, <code class="highlighter-rouge">spark-tensorframes</code>, etc.</li>
   <li>Please do not cross-post between StackOverflow and the mailing lists</li>
   <li>No jobs, sales, or solicitation is permitted on StackOverflow</li>
 </ul>
@@ -258,14 +258,14 @@ <h4>Mailing Lists</h4>
 <ul>
   <li>Prior to asking submitting questions, please:
     <ul>
-      <li>Search StackOverflow at <a href="https://stackoverflow.com/questions/tagged/apache-spark"><code>apache-spark</code></a> 
+      <li>Search StackOverflow at <a href="https://stackoverflow.com/questions/tagged/apache-spark"><code class="highlighter-rouge">apache-spark</code></a> 
 to see if your question has already been answered</li>
       <li>Search the nabble archive for
 <a href="http://apache-spark-user-list.1001560.n3.nabble.com/">[email protected]</a></li>
     </ul>
   </li>
   <li>Tagging the subject line of your email will help you get a faster response, e.g. 
-<code>[Spark SQL]: Does Spark SQL support LEFT SEMI JOIN?</code></li>
+<code class="highlighter-rouge">[Spark SQL]: Does Spark SQL support LEFT SEMI JOIN?</code></li>
   <li>Tags may help identify a topic by:
     <ul>
       <li>Component: Spark Core, Spark SQL, ML, MLlib, GraphFrames, GraphX, TensorFrames, etc</li>