Skip to content

SPARK-1349: spark-shell gets its own command history #267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

SPARK-1349: spark-shell gets its own command history #267

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion repl/src/main/scala/org/apache/spark/repl/SparkJLineReader.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@

package org.apache.spark.repl

import scala.reflect.io.{Path, File}
import scala.tools.nsc._
import scala.tools.nsc.interpreter._
import scala.tools.nsc.interpreter.session.JLineHistory.JLineFileHistory

import scala.tools.jline.console.ConsoleReader
import scala.tools.jline.console.completer._
Expand All @@ -25,7 +27,7 @@ class SparkJLineReader(_completion: => Completion) extends InteractiveReader {
val consoleReader = new JLineConsoleReader()

lazy val completion = _completion
lazy val history: JLineHistory = JLineHistory()
lazy val history: JLineHistory = new SparkJLineHistory

private def term = consoleReader.getTerminal()
def reset() = term.reset()
Expand Down Expand Up @@ -78,3 +80,11 @@ class SparkJLineReader(_completion: => Completion) extends InteractiveReader {
def readOneLine(prompt: String) = consoleReader readLine prompt
def readOneKey(prompt: String) = consoleReader readOneKey prompt
}

/** Changes the default history file to not collide with the scala repl's. */
class SparkJLineHistory extends JLineFileHistory {
import Properties.userHome

def defaultFileName = ".spark_history"
override protected lazy val historyFile = File(Path(userHome) / defaultFileName)
Copy link
Contributor

@pwendell pwendell Mar 30, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering - does this handle systems with different path separators correctly?

Does historyFile need to be scala.reflect.io.File? Or can it be a java.io.File? It seems a little heavyweight to pull in the reflection library to create a file path, but it makes sense if this is the needed type of historyFile.

Copy link
Contributor Author

@aarondav aarondav Apr 5, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I did this was to fully mimic the code from the Scala interpreter. I think this will make it easier to compare in case the Scala version is updated some time in the future, but you're right we could use a lighter weight construct, we just risk having different behavior than the standard Scala shell history.

Not a big deal to me, but I'm inclined to stick with cloning the Scala shell behavior. Let me know if you still prefer the java.io.File route, I'd be happy to change to that.

Copy link
Contributor

@pwendell pwendell Apr 5, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay sounds good - just wondering why it was there.

}