Skip to content

Commit a75dc80

Browse files
huaxingaosrowen
authored andcommitted
[SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
### What changes were proposed in this pull request? Remove the unneeded embedded inline HTML markup by using the basic markdown syntax. Please see #28414 ### Why are the changes needed? Make the doc cleaner and easily editable by MD editors. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Manually build and check Closes #28451 from huaxingao/html_cleanup. Authored-by: Huaxin Gao <[email protected]> Signed-off-by: Sean Owen <[email protected]>
1 parent ce63bef commit a75dc80

File tree

80 files changed

+1979
-2506
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

80 files changed

+1979
-2506
lines changed

docs/_data/menu-sql.yaml

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -156,22 +156,22 @@
156156
url: sql-ref-syntax-qry-select-distribute-by.html
157157
- text: LIMIT Clause
158158
url: sql-ref-syntax-qry-select-limit.html
159+
- text: Common Table Expression
160+
url: sql-ref-syntax-qry-select-cte.html
161+
- text: Inline Table
162+
url: sql-ref-syntax-qry-select-inline-table.html
159163
- text: JOIN
160164
url: sql-ref-syntax-qry-select-join.html
161165
- text: Join Hints
162166
url: sql-ref-syntax-qry-select-hints.html
167+
- text: LIKE Predicate
168+
url: sql-ref-syntax-qry-select-like.html
163169
- text: Set Operators
164170
url: sql-ref-syntax-qry-select-setops.html
165171
- text: TABLESAMPLE
166172
url: sql-ref-syntax-qry-sampling.html
167173
- text: Table-valued Function
168174
url: sql-ref-syntax-qry-select-tvf.html
169-
- text: Inline Table
170-
url: sql-ref-syntax-qry-select-inline-table.html
171-
- text: Common Table Expression
172-
url: sql-ref-syntax-qry-select-cte.html
173-
- text: LIKE Predicate
174-
url: sql-ref-syntax-qry-select-like.html
175175
- text: Window Function
176176
url: sql-ref-syntax-qry-window.html
177177
- text: EXPLAIN
@@ -213,20 +213,20 @@
213213
subitems:
214214
- text: SHOW COLUMNS
215215
url: sql-ref-syntax-aux-show-columns.html
216+
- text: SHOW CREATE TABLE
217+
url: sql-ref-syntax-aux-show-create-table.html
216218
- text: SHOW DATABASES
217219
url: sql-ref-syntax-aux-show-databases.html
218220
- text: SHOW FUNCTIONS
219221
url: sql-ref-syntax-aux-show-functions.html
222+
- text: SHOW PARTITIONS
223+
url: sql-ref-syntax-aux-show-partitions.html
220224
- text: SHOW TABLE
221225
url: sql-ref-syntax-aux-show-table.html
222226
- text: SHOW TABLES
223227
url: sql-ref-syntax-aux-show-tables.html
224228
- text: SHOW TBLPROPERTIES
225229
url: sql-ref-syntax-aux-show-tblproperties.html
226-
- text: SHOW PARTITIONS
227-
url: sql-ref-syntax-aux-show-partitions.html
228-
- text: SHOW CREATE TABLE
229-
url: sql-ref-syntax-aux-show-create-table.html
230230
- text: SHOW VIEWS
231231
url: sql-ref-syntax-aux-show-views.html
232232
- text: CONFIGURATION MANAGEMENT

docs/sql-ref-ansi-compliance.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,10 @@ This means that in case an operation causes overflows, the result is the same wi
4141
On the other hand, Spark SQL returns null for decimal overflows.
4242
When `spark.sql.ansi.enabled` is set to `true` and an overflow occurs in numeric and interval arithmetic operations, it throws an arithmetic exception at runtime.
4343

44-
{% highlight sql %}
44+
```sql
4545
-- `spark.sql.ansi.enabled=true`
4646
SELECT 2147483647 + 1;
47-
java.lang.ArithmeticException: integer overflow
47+
java.lang.ArithmeticException: integer overflow
4848

4949
-- `spark.sql.ansi.enabled=false`
5050
SELECT 2147483647 + 1;
@@ -53,7 +53,7 @@ SELECT 2147483647 + 1;
5353
+----------------+
5454
| -2147483648|
5555
+----------------+
56-
{% endhighlight %}
56+
```
5757

5858
### Type Conversion
5959

@@ -64,15 +64,15 @@ On the other hand, `INSERT INTO` syntax throws an analysis exception when the AN
6464
Currently, the ANSI mode affects explicit casting and assignment casting only.
6565
In future releases, the behaviour of type coercion might change along with the other two type conversion rules.
6666

67-
{% highlight sql %}
67+
```sql
6868
-- Examples of explicit casting
6969

7070
-- `spark.sql.ansi.enabled=true`
7171
SELECT CAST('a' AS INT);
72-
java.lang.NumberFormatException: invalid input syntax for type numeric: a
72+
java.lang.NumberFormatException: invalid input syntax for type numeric: a
7373

7474
SELECT CAST(2147483648L AS INT);
75-
java.lang.ArithmeticException: Casting 2147483648 to int causes overflow
75+
java.lang.ArithmeticException: Casting 2147483648 to int causes overflow
7676

7777
-- `spark.sql.ansi.enabled=false` (This is a default behaviour)
7878
SELECT CAST('a' AS INT);
@@ -94,8 +94,8 @@ CREATE TABLE t (v INT);
9494

9595
-- `spark.sql.storeAssignmentPolicy=ANSI`
9696
INSERT INTO t VALUES ('1');
97-
org.apache.spark.sql.AnalysisException: Cannot write incompatible data to table '`default`.`t`':
98-
- Cannot safely cast 'v': StringType to IntegerType;
97+
org.apache.spark.sql.AnalysisException: Cannot write incompatible data to table '`default`.`t`':
98+
- Cannot safely cast 'v': StringType to IntegerType;
9999

100100
-- `spark.sql.storeAssignmentPolicy=LEGACY` (This is a legacy behaviour until Spark 2.x)
101101
INSERT INTO t VALUES ('1');
@@ -105,7 +105,7 @@ SELECT * FROM t;
105105
+---+
106106
| 1|
107107
+---+
108-
{% endhighlight %}
108+
```
109109

110110
### SQL Functions
111111

docs/sql-ref-datatypes.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,7 @@ Specifically:
240240

241241
#### Examples
242242

243-
{% highlight sql %}
243+
```sql
244244
SELECT double('infinity') AS col;
245245
+--------+
246246
| col|
@@ -313,4 +313,4 @@ SELECT COUNT(*), c2 FROM test GROUP BY c2;
313313
| 2|-Infinity|
314314
| 3| Infinity|
315315
+---------+---------+
316-
{% endhighlight %}
316+
```

docs/sql-ref-functions-builtin.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ license: |
7070
### JSON Functions
7171
{% include_relative generated-json-funcs-table.html %}
7272
#### Examples
73-
{% include_relative generated-agg-funcs-examples.html %}
73+
{% include_relative generated-json-funcs-examples.html %}
7474
{% break %}
7575
{% endif %}
7676
{% endfor %}

docs/sql-ref-functions-udf-aggregate.md

Lines changed: 45 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -27,46 +27,35 @@ User-Defined Aggregate Functions (UDAFs) are user-programmable routines that act
2727

2828
A base class for user-defined aggregations, which can be used in Dataset operations to take all of the elements of a group and reduce them to a single value.
2929

30-
* IN - The input type for the aggregation.
31-
* BUF - The type of the intermediate value of the reduction.
32-
* OUT - The type of the final output result.
30+
***IN*** - The input type for the aggregation.
31+
32+
***BUF*** - The type of the intermediate value of the reduction.
33+
34+
***OUT*** - The type of the final output result.
35+
36+
* **bufferEncoder: Encoder[BUF]**
3337

34-
<dl>
35-
<dt><code><em>bufferEncoder: Encoder[BUF]</em></code></dt>
36-
<dd>
3738
Specifies the Encoder for the intermediate value type.
38-
</dd>
39-
</dl>
40-
<dl>
41-
<dt><code><em>finish(reduction: BUF): OUT</em></code></dt>
42-
<dd>
39+
40+
* **finish(reduction: BUF): OUT**
41+
4342
Transform the output of the reduction.
44-
</dd>
45-
</dl>
46-
<dl>
47-
<dt><code><em>merge(b1: BUF, b2: BUF): BUF</em></code></dt>
48-
<dd>
43+
44+
* **merge(b1: BUF, b2: BUF): BUF**
45+
4946
Merge two intermediate values.
50-
</dd>
51-
</dl>
52-
<dl>
53-
<dt><code><em>outputEncoder: Encoder[OUT]</em></code></dt>
54-
<dd>
47+
48+
* **outputEncoder: Encoder[OUT]**
49+
5550
Specifies the Encoder for the final output value type.
56-
</dd>
57-
</dl>
58-
<dl>
59-
<dt><code><em>reduce(b: BUF, a: IN): BUF</em></code></dt>
60-
<dd>
61-
Aggregate input value <code>a</code> into current intermediate value. For performance, the function may modify <code>b</code> and return it instead of constructing new object for <code>b</code>.
62-
</dd>
63-
</dl>
64-
<dl>
65-
<dt><code><em>zero: BUF</em></code></dt>
66-
<dd>
51+
52+
* **reduce(b: BUF, a: IN): BUF**
53+
54+
Aggregate input value `a` into current intermediate value. For performance, the function may modify `b` and return it instead of constructing new object for `b`.
55+
56+
* **zero: BUF**
57+
6758
The initial value of the intermediate result for this aggregation.
68-
</dd>
69-
</dl>
7059

7160
### Examples
7261

@@ -95,16 +84,16 @@ For example, a user-defined average for untyped DataFrames can look like:
9584
{% include_example untyped_custom_aggregation java/org/apache/spark/examples/sql/JavaUserDefinedUntypedAggregation.java%}
9685
</div>
9786
<div data-lang="SQL" markdown="1">
98-
{% highlight sql %}
87+
```sql
9988
-- Compile and place UDAF MyAverage in a JAR file called `MyAverage.jar` in /tmp.
10089
CREATE FUNCTION myAverage AS 'MyAverage' USING JAR '/tmp/MyAverage.jar';
10190

10291
SHOW USER FUNCTIONS;
103-
-- +------------------+
104-
-- | function|
105-
-- +------------------+
106-
-- | default.myAverage|
107-
-- +------------------+
92+
+------------------+
93+
| function|
94+
+------------------+
95+
| default.myAverage|
96+
+------------------+
10897

10998
CREATE TEMPORARY VIEW employees
11099
USING org.apache.spark.sql.json
@@ -113,26 +102,26 @@ OPTIONS (
113102
);
114103

115104
SELECT * FROM employees;
116-
-- +-------+------+
117-
-- | name|salary|
118-
-- +-------+------+
119-
-- |Michael| 3000|
120-
-- | Andy| 4500|
121-
-- | Justin| 3500|
122-
-- | Berta| 4000|
123-
-- +-------+------+
105+
+-------+------+
106+
| name|salary|
107+
+-------+------+
108+
|Michael| 3000|
109+
| Andy| 4500|
110+
| Justin| 3500|
111+
| Berta| 4000|
112+
+-------+------+
124113

125114
SELECT myAverage(salary) as average_salary FROM employees;
126-
-- +--------------+
127-
-- |average_salary|
128-
-- +--------------+
129-
-- | 3750.0|
130-
-- +--------------+
131-
{% endhighlight %}
115+
+--------------+
116+
|average_salary|
117+
+--------------+
118+
| 3750.0|
119+
+--------------+
120+
```
132121
</div>
133122
</div>
134123

135124
### Related Statements
136125

137-
* [Scalar User Defined Functions (UDFs)](sql-ref-functions-udf-scalar.html)
138-
* [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html)
126+
* [Scalar User Defined Functions (UDFs)](sql-ref-functions-udf-scalar.html)
127+
* [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html)

docs/sql-ref-functions-udf-hive.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Spark SQL supports integration of Hive UDFs, UDAFs and UDTFs. Similar to Spark U
2828
Hive has two UDF interfaces: [UDF](https://github.com/apache/hive/blob/master/udf/src/java/org/apache/hadoop/hive/ql/exec/UDF.java) and [GenericUDF](https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java).
2929
An example below uses [GenericUDFAbs](https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java) derived from `GenericUDF`.
3030

31-
{% highlight sql %}
31+
```sql
3232
-- Register `GenericUDFAbs` and use it in Spark SQL.
3333
-- Note that, if you use your own programmed one, you need to add a JAR containing it
3434
-- into a classpath,
@@ -52,12 +52,12 @@ SELECT testUDF(value) FROM t;
5252
| 2.0|
5353
| 3.0|
5454
+--------------+
55-
{% endhighlight %}
55+
```
5656

5757

5858
An example below uses [GenericUDTFExplode](https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFExplode.java) derived from [GenericUDTF](https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java).
5959

60-
{% highlight sql %}
60+
```sql
6161
-- Register `GenericUDTFExplode` and use it in Spark SQL
6262
CREATE TEMPORARY FUNCTION hiveUDTF
6363
AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDTFExplode';
@@ -79,12 +79,12 @@ SELECT hiveUDTF(value) FROM t;
7979
| 3|
8080
| 4|
8181
+---+
82-
{% endhighlight %}
82+
```
8383

8484
Hive has two UDAF interfaces: [UDAF](https://github.com/apache/hive/blob/master/udf/src/java/org/apache/hadoop/hive/ql/exec/UDAF.java) and [GenericUDAFResolver](https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFResolver.java).
8585
An example below uses [GenericUDAFSum](https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java) derived from `GenericUDAFResolver`.
8686

87-
{% highlight sql %}
87+
```sql
8888
-- Register `GenericUDAFSum` and use it in Spark SQL
8989
CREATE TEMPORARY FUNCTION hiveUDAF
9090
AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum';
@@ -105,4 +105,4 @@ SELECT key, hiveUDAF(value) FROM t GROUP BY key;
105105
| b| 3|
106106
| a| 3|
107107
+---+---------------+
108-
{% endhighlight %}
108+
```

docs/sql-ref-functions-udf-scalar.md

Lines changed: 11 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -26,24 +26,18 @@ User-Defined Functions (UDFs) are user-programmable routines that act on one row
2626
### UserDefinedFunction
2727

2828
To define the properties of a user-defined function, the user can use some of the methods defined in this class.
29-
<dl>
30-
<dt><code><em>asNonNullable(): UserDefinedFunction</em></code></dt>
31-
<dd>
29+
30+
* **asNonNullable(): UserDefinedFunction**
31+
3232
Updates UserDefinedFunction to non-nullable.
33-
</dd>
34-
</dl>
35-
<dl>
36-
<dt><code><em>asNondeterministic(): UserDefinedFunction</em></code></dt>
37-
<dd>
33+
34+
* **asNondeterministic(): UserDefinedFunction**
35+
3836
Updates UserDefinedFunction to nondeterministic.
39-
</dd>
40-
</dl>
41-
<dl>
42-
<dt><code><em>withName(name: String): UserDefinedFunction</em></code></dt>
43-
<dd>
37+
38+
* **withName(name: String): UserDefinedFunction**
39+
4440
Updates UserDefinedFunction with a given name.
45-
</dd>
46-
</dl>
4741

4842
### Examples
4943

@@ -57,5 +51,5 @@ To define the properties of a user-defined function, the user can use some of th
5751
</div>
5852

5953
### Related Statements
60-
* [User Defined Aggregate Functions (UDAFs)](sql-ref-functions-udf-aggregate.html)
61-
* [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html)
54+
* [User Defined Aggregate Functions (UDAFs)](sql-ref-functions-udf-aggregate.html)
55+
* [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html)

0 commit comments

Comments
 (0)