Skip to content

Commit ace4079

Browse files
rxinhvanhovell
authored andcommitted
[SPARK-18714][SQL] Add a simple time function to SparkSession
## What changes were proposed in this pull request? Many Spark developers often want to test the runtime of some function in interactive debugging and testing. This patch adds a simple time function to SparkSession: ``` scala> spark.time { spark.range(1000).count() } Time taken: 77 ms res1: Long = 1000 ``` ## How was this patch tested? I tested this interactively in spark-shell. Author: Reynold Xin <[email protected]> Closes #16140 from rxin/SPARK-18714. (cherry picked from commit cb1f10b) Signed-off-by: Herman van Hovell <[email protected]>
1 parent e362d99 commit ace4079

File tree

1 file changed

+16
-0
lines changed

1 file changed

+16
-0
lines changed

sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -618,6 +618,22 @@ class SparkSession private(
618618
@InterfaceStability.Evolving
619619
def readStream: DataStreamReader = new DataStreamReader(self)
620620

621+
/**
622+
* Executes some code block and prints to stdout the time taken to execute the block. This is
623+
* available in Scala only and is used primarily for interactive testing and debugging.
624+
*
625+
* @since 2.1.0
626+
*/
627+
@InterfaceStability.Stable
628+
def time[T](f: => T): T = {
629+
val start = System.nanoTime()
630+
val ret = f
631+
val end = System.nanoTime()
632+
// scalastyle:off println
633+
println(s"Time taken: ${(end - start) / 1000 / 1000} ms")
634+
// scalastyle:on println
635+
ret
636+
}
621637

622638
// scalastyle:off
623639
// Disable style checker so "implicits" object can start with lowercase i

0 commit comments

Comments
 (0)