[Spark-18080][ML][PYTHON] Python API & Examples for Locality Sensitive Hashing #16715

Yunni · 2017-01-26T20:28:22Z

What changes were proposed in this pull request?

This pull request includes python API and examples for LSH. The API changes was based on @yanboliang 's PR #15768 and resolved conflicts and API changes on the Scala API. The examples are consistent with Scala examples of MinHashLSH and BucketedRandomProjectionLSH.

How was this patch tested?

API and examples are tested using spark-submit:
bin/spark-submit examples/src/main/python/ml/min_hash_lsh.py
bin/spark-submit examples/src/main/python/ml/bucketed_random_projection_lsh.py

User guide changes are generated and manually inspected:
SKIP_API=1 jekyll build

…o spark-18080

…8080

…rk-18080

Yunni · 2017-01-26T20:31:30Z

@yanboliang @jkbradley Please take a look. Thanks!

SparkQA · 2017-01-26T20:33:47Z

Test build #72042 has finished for PR 16715 at commit 65dab3e.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-01-26T21:09:45Z

Test build #72044 has finished for PR 16715 at commit 3d3bcf0.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-01-26T21:50:45Z

Test build #72046 has finished for PR 16715 at commit 69dccde.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-01-26T22:31:40Z

Test build #72050 has finished for PR 16715 at commit e7542d0.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-01-26T23:38:59Z

Test build #72055 has finished for PR 16715 at commit 5cfc9c5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

jkbradley · 2017-01-26T23:46:17Z

@yanboliang Would you have time to take a look? Thanks!

yanboliang · 2017-01-28T15:43:01Z

@jkbradley @Yunni I'll have a look at next week. Thanks.

Yunni · 2017-01-28T18:25:41Z

Thanks very much, @yanboliang ~~

Yunni · 2017-02-06T18:20:01Z

@yanboliang, just a friendly reminder please don't forget to review the PR when you have time. Thanks!

yanboliang · 2017-02-07T01:29:27Z

@Yunni I'm on travel at Spark Summit East these days, and will review after the summit. Thanks for your patience.

sethah

Leaving comments on the code, will check docs and examples too.

sethah · 2017-02-07T16:52:06Z

python/pyspark/ml/feature.py


+class LSHParams(Params):
+    """
+    Mixin for Locality Sensitive Hashing(LSH) algorithm parameters.


space after "Hashing"

sethah · 2017-02-07T16:52:31Z

python/pyspark/ml/feature.py

+
+class LSHModel():
+    """
+    Mixin for Locality Sensitive Hashing(LSH) models.


space here too

sethah · 2017-02-07T16:54:51Z

python/pyspark/ml/feature.py

+    """
+
+    @since("2.2.0")
+    def approxNearestNeighbors(self, dataset, key, numNearestNeighbors, singleProbing=True,


Can we just leave singe probing out since it has no effect and we aren't including it in the doc?

sethah · 2017-02-07T16:56:51Z

python/pyspark/ml/feature.py

+    @since("2.2.0")
+    def approxSimilarityJoin(self, datasetA, datasetB, threshold, distCol="distCol"):
+        """
+        Join two dataset to approximately find all pairs of rows whose distance are smaller than


"two datasets"

sethah · 2017-02-07T16:58:03Z

python/pyspark/ml/feature.py

+    .. note:: Experimental
+
+    LSH class for Jaccard distance.
+    The input can be dense or sparse vectors, but it is more efficient if it is sparse.


This notation is not correct. For Python it should be Vectors.sparse(10, [(2, 1.0), (3, 1.0), (5, 1.0)]).

sethah · 2017-02-07T19:52:14Z

python/pyspark/ml/feature.py

+    ...         (Vectors.dense([1.0, -1.0 ]),),
+    ...         (Vectors.dense([1.0, 1.0]),)]
+    >>> df = spark.createDataFrame(data, ["keys"])
+    >>> rp = BucketedRandomProjectionLSH(inputCol="keys", outputCol="values",


can we call them "features" and "hashes" ? I'm open to other names but "keys" and "values" is unclear to me

EDIT: I see this is the Scala example convention. I still think "features" and "hashes" is better, but either way is acceptable.

I agree with you. I have changed the terms to "features" and "hashes"

sethah · 2017-02-07T20:41:32Z

python/pyspark/ml/feature.py

+    >>> df2 = spark.createDataFrame(data2, ["keys"])
+    >>> model.approxNearestNeighbors(df2, Vectors.dense([1.0, 2.0]), 1).collect()
+    [Row(keys=DenseVector([2.0, 2.0]), values=[DenseVector([1.0])], distCol=1.0)]
+    >>> model.approxSimilarityJoin(df, df2, 3.0).select("distCol").head()[0]


Since doctests are also partly used to demonstrate the usage of the algorithm, I don't think this line is particularly useful. It is quite hard to interpret. I think it might be nicer to add an "id" column to the dataframes and then do a "show" here to see the joined dataframes, as in the Scala example. Then again, you end up with:

+--------------------+--------------------+----------------+ | datasetA| datasetB| distCol| +--------------------+--------------------+----------------+ |[[1.0,1.0],Wrappe...|[[3.0,2.0],Wrappe...|2.23606797749979| +--------------------+--------------------+----------------+

Which is also confusing! Thoughts on which option is better?

I think showing the ids would be more interpretable, as users are able to see the feature vectors of the ids from the examples.

sethah · 2017-02-07T21:05:20Z

python/pyspark/ml/feature.py

+    """
+
+    @property
+    @since("2.2.0")


This should not be exposed, since it's private in Scala. Also, Array[(Int, Int)] does not serialize to Python.

sethah · 2017-02-07T21:09:47Z

python/pyspark/ml/feature.py

+        transform the data; if the :py:attr:`outputCol` exists, it will use that. This allows
+        caching of the transformed data when necessary.
+
+        :param dataset: The dataset to search for nearest neighbors of the key.


add the note that's in the scala doc?

sethah · 2017-02-07T21:11:11Z

python/pyspark/ml/feature.py

        return self.getOrDefault(self.threshold)


+class LSHParams(Params):


The classes in this file are alphabetized for the most part. Let's keep the convention here.

It's not alphabetized here because the declaration order matters for PySpark shell.

sethah · 2017-02-07T21:56:46Z

examples/src/main/python/ml/bucketed_random_projection_lsh.py

+    model = brp.fit(dfA)
+
+    # Feature Transformation
+    model.transform(dfA).show()


Other examples typically will output some print statements along with the output, explaining what you're seeing. As it is, this example just spits out a bunch of dataframes with no explanations. I'd like us to add that here, and for the Scala examples really.

Done for Scala/Java/Python Examples.

sethah · 2017-02-07T21:57:38Z

examples/src/main/python/ml/bucketed_random_projection_lsh.py

+"""
+An example demonstrating BucketedRandomProjectionLSH.
+Run with:
+  bin/spark-submit examples/src/main/python/ml/bucketed_random_projection_lsh.py


the file names usually end with "_example". Have we not done that here because of how long the name is already? I slightly prefer to stick with the convention.

That was a mistake. Sorry about it!

sethah · 2017-02-07T22:15:11Z

examples/src/main/python/ml/min_hash_lsh.py

+    model.approxSimilarityJoin(dfA, dfA, 0.6).filter("datasetA.id < datasetB.id").show()
+
+    # Approximate nearest neighbor search
+    model.approxNearestNeighbors(dfA, key, 2).show()


These two output empty dataframes.

Increased the number of HashTables.

sethah · 2017-02-07T22:15:28Z

examples/src/main/python/ml/min_hash_lsh.py

+    model = mh.fit(dfA)
+
+    # Feature Transformation
+    model.transform(dfA).show()


same comment about print statements here

sethah · 2017-02-07T22:37:15Z

First pass, thanks @Yunni and @yanboliang !

…8080

SparkQA · 2017-02-10T01:22:15Z

Test build #72664 has finished for PR 16715 at commit b1da01e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-10T02:29:11Z

Test build #72668 has finished for PR 16715 at commit 8f1d708.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

jkbradley · 2017-02-11T04:38:09Z

Could you please add tag "[PYTHON]" to the PR title?
Also, please remove "Please review http://spark.apache.org/contributing.html before opening a pull request." from the PR description since that will become part of the commit message.
Thanks!

sethah

This is nearly ready. With the suggested changes, all of the following are good:

Doc tests pass
Docs build and examples can be copy/pasted to REPL for python and scala
examples run and have coherent output
python API docs look good

sethah · 2017-02-13T18:53:32Z

python/pyspark/ml/feature.py

+        :param numNearestNeighbors: The maximum number of nearest neighbors.
+        :param distCol: Output column for storing the distance between each result row and the key.
+                        Use "distCol" as default value if it's not specified.
+        :return: A dataset containing at most k items closest to the key. A distCol is added


"A distCol" -> "A column 'distCol'"

sethah · 2017-02-13T18:54:30Z

python/pyspark/ml/feature.py

+        :param distCol: Output column for storing the distance between each result row and the key.
+                        Use "distCol" as default value if it's not specified.
+        :return: A joined dataset containing pairs of rows. The original rows are in columns
+                "datasetA" and "datasetB", and a distCol is added to show the distance of


indentation

"a distCol" -> "a column distCol"

sethah · 2017-02-13T18:54:36Z

mllib/src/main/scala/org/apache/spark/ml/feature/BucketedRandomProjectionLSH.scala

 * The input is dense or sparse vectors, each of which represents a point in the Euclidean
- * distance space. The output will be vectors of configurable dimension. Hash values in the
- * same dimension are calculated by the same hash function.
+ * distance space. The output will be vectors of configurable dimension. Hash values in the same


Can we revert this?

sethah · 2017-02-13T18:57:39Z

python/pyspark/ml/feature.py

+        return self._call_java("approxNearestNeighbors", dataset, key, numNearestNeighbors,
+                               distCol)
+
+    @since("2.2.0")


I think we've decided not to put since tags in parent classes, since they'll be wrong for future derived classes.

Removed in 4 places.

sethah · 2017-02-13T19:25:46Z

examples/src/main/scala/org/apache/spark/examples/ml/MinHashLSHExample.scala

+    // neighbor search.
+    // We could avoid computing hashes by passing in the already-transformed dataset, e.g.
+    // `model.approxNearestNeighbors(transformedA, key, 2)`
+    // It may return less than 2 rows because of lack of elements in the hash buckets.


change it to "It may return less than 2 rows when not enough approximate near-neighbor candidates are found." ?

sethah · 2017-02-13T19:32:25Z

examples/src/main/java/org/apache/spark/examples/ml/JavaBucketedRandomProjectionLSHExample.java

 import org.apache.spark.sql.types.StructField;
 import org.apache.spark.sql.types.StructType;
+
+import static org.apache.spark.sql.functions.*;


just import col here and minhash

sethah · 2017-02-13T19:35:54Z

docs/ml-features.md

+Refer to the [BucketedRandomProjectionLSH Python docs](api/python/pyspark.ml.html#pyspark.ml.feature.BucketedRandomProjectionLSH)
+for more details on the API.
+
+{% include_example python/ml/bucketed_random_projection_lsh.py %}


These are not correct and the docs don't build because of it. In the future, can you check that the docs build when you make changes?

cd docs; SKIP_API=1 jekyll serve --watch

More detailed instructions here. Also you can build the python docs by cd python/docs; make html

Yup, should be bucketed_random_projection_lsh_example.py (and similarly for minhash include_example below)

Sorry I forgot to retest after renaming the python examples. Thanks for the in formation.

sethah · 2017-02-13T19:42:51Z

examples/src/main/python/ml/bucketed_random_projection_lsh_example.py

+"""
+An example demonstrating BucketedRandomProjectionLSH.
+Run with:
+  bin/spark-submit examples/src/main/python/ml/bucketed_random_projection_lsh_example.py


Can we add the appropriate note for this to the Scala and Java examples as well?

Added in 4 places.

MLnick

Left a few minor comments.

MLnick · 2017-02-13T23:39:09Z

python/pyspark/ml/feature.py

+        :param datasetA: One of the datasets to join.
+        :param datasetB: Another dataset to join.
+        :param threshold: The threshold for the distance of row pairs.
+        :param distCol: Output column for storing the distance between each result row and the key.


shouldn't this be "distance between each pair of rows", rather than "between each result row and the key"?

MLnick · 2017-02-13T23:39:48Z

python/pyspark/ml/feature.py

+        :param distCol: Output column for storing the distance between each result row and the key.
+                        Use "distCol" as default value if it's not specified.
+        :return: A joined dataset containing pairs of rows. The original rows are in columns
+                "datasetA" and "datasetB", and a distCol is added to show the distance of


nit - "distance between each pair" rather than "distance of"

MLnick · 2017-02-13T23:56:59Z

examples/src/main/scala/org/apache/spark/examples/ml/BucketedRandomProjectionLSHExample.scala

+    model.approxSimilarityJoin(dfA, dfB, 1.5)
+      .select(col("datasetA.id").alias("idA"),
+        col("datasetB.id").alias("idB"),
+        col("distCol").alias("EuclideanDistance")).show()


We can just pass distCol = EuclideanDistance here, and for approxNearestNeighbors.

We can do this throughout the examples (and obviously for min hash change it to jaccard accordingly).

Done in 6 places.

MLnick · 2017-02-14T00:02:47Z

docs/ml-features.md

+Refer to the [BucketedRandomProjectionLSH Python docs](api/python/pyspark.ml.html#pyspark.ml.feature.BucketedRandomProjectionLSH)
+for more details on the API.
+
+{% include_example python/ml/bucketed_random_projection_lsh.py %}


Yup, should be bucketed_random_projection_lsh_example.py (and similarly for minhash include_example below)

…8080

sethah

Just a few minor things. Thanks for all the work on this!

sethah · 2017-02-14T07:07:11Z

examples/src/main/scala/org/apache/spark/examples/ml/MinHashLSHExample.scala

+    model.approxSimilarityJoin(dfA, dfB, 0.6)
+      .select(col("datasetA.id").alias("idA"),
+        col("datasetB.id").alias("idB"),
+        col("distCol").alias("JaccardDistance")).show()


pass distCol as method parameter instead of alias

sethah · 2017-02-14T07:08:02Z

mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala

+   * @param distCol Output column for storing the distance between each pair of rows.
   * @return A joined dataset containing pairs of rows. The original rows are in columns
-   *         "datasetA" and "datasetB", and a distCol is added to show the distance of each pair.
+   *         "datasetA" and "datasetB", and a distCol is added to show the distance between each


a column "distCol"

sethah · 2017-02-14T07:11:13Z

examples/src/main/scala/org/apache/spark/examples/ml/MinHashLSHExample.scala

 // $example on$
 import org.apache.spark.ml.feature.MinHashLSH
 import org.apache.spark.ml.linalg.Vectors
+import org.apache.spark.sql.functions._


just import col here and above

sethah · 2017-02-14T07:16:43Z

python/pyspark/ml/feature.py

+    >>> mh2.getOutputCol() == mh.getOutputCol()
+    True
+    >>> modelPath = temp_path + "/mh-model"
+    >>> model.save(modelPath)


let's add an equality check here and for BRP. For example for IDFModel we have:

loadedModel.transform(df).head().idf == model.transform(df).head().idf

SparkQA · 2017-02-14T07:24:39Z

Test build #72856 has finished for PR 16715 at commit c64d50b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-14T18:22:58Z

Test build #72881 has finished for PR 16715 at commit 5d55752.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

sethah

One minor thing, then LGTM. Thanks @Yunni!

sethah · 2017-02-15T04:27:55Z

examples/src/main/java/org/apache/spark/examples/ml/JavaMinHashLSHExample.java


 package org.apache.spark.examples.ml;

+import org.apache.spark.ml.linalg.Vector;


move it below under example on

SparkQA · 2017-02-15T07:22:41Z

Test build #72923 has started for PR 16715 at commit d849c3a.

Yunni · 2017-02-15T07:25:53Z

@sethah Really appreciate your detailed code review and comments. :)
@MLnick @yanboliang Thank you for the help as well. Please let me know if you have any other comments.

…8080

SparkQA · 2017-02-15T22:51:54Z

Test build #72956 has finished for PR 16715 at commit 36fd9bc.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class LinearSVCWrapperWriter(instance: LinearSVCWrapper) extends MLWriter
class LinearSVCWrapperReader extends MLReader[LinearSVCWrapper]
class NoSuchDatabaseException(val db: String) extends AnalysisException(s\"Database '$db' not found\")
class ResolveBroadcastHints(conf: CatalystConf) extends Rule[LogicalPlan]
case class JsonToStruct(
case class StructToJson(
case class Hint(name: String, parameters: Seq[String], child: LogicalPlan) extends UnaryNode
case class InnerOuterEstimation(conf: CatalystConf, join: Join) extends Logging
case class LeftSemiAntiEstimation(conf: CatalystConf, join: Join)
case class NumericRange(min: JDecimal, max: JDecimal) extends Range
class FileStreamOptions(parameters: CaseInsensitiveMap[String]) extends Logging

yanboliang

LGTM, merged into master. Thanks for all.

yanboliang · 2017-02-16T00:23:06Z

examples/src/main/java/org/apache/spark/examples/ml/JavaBucketedRandomProjectionLSHExample.java

+/**
+ * An example demonstrating BucketedRandomProjectionLSH.
+ * Run with:
+ *   bin/run-example org.apache.spark.examples.ml.JavaBucketedRandomProjectionLSHExample


Actually we can simplify it as bin/run-example ml.JavaBucketedRandomProjectionLSHExample, but it's ok to leave as it is.

sethah · 2017-02-16T00:39:13Z

BTW, in the future I'd prefer to separate the examples and the Python API. I'm not sure if we ever fully decided on a normal protocol for this, but it certainly would make the review easier :)

yanboliang · 2017-02-16T00:42:21Z

+1 @sethah

…e Hashing ## What changes were proposed in this pull request? This pull request includes python API and examples for LSH. The API changes was based on yanboliang 's PR apache#15768 and resolved conflicts and API changes on the Scala API. The examples are consistent with Scala examples of MinHashLSH and BucketedRandomProjectionLSH. ## How was this patch tested? API and examples are tested using spark-submit: `bin/spark-submit examples/src/main/python/ml/min_hash_lsh.py` `bin/spark-submit examples/src/main/python/ml/bucketed_random_projection_lsh.py` User guide changes are generated and manually inspected: `SKIP_API=1 jekyll build` Author: Yun Ni <[email protected]> Author: Yanbo Liang <[email protected]> Author: Yunni <[email protected]> Closes apache#16715 from Yunni/spark-18080.

Yunni · 2017-02-16T19:31:31Z

Sure. Will do.

e-m-m · 2017-02-21T21:24:15Z

Hey, super excited about this feature! Was actually thinking of writing this, myself, until I saw this. What version of Spark is this slated to hit the Python API? Thanks :)

Yunni · 2017-02-21T21:35:49Z

Hi @e-m-m, I think the Python API will be included in Spark 2.2.

e-m-m · 2017-02-21T21:47:16Z

Wow, thanks for the quick answer, @Yunni! Sounds great. I'll definitely be using it.

jkbradley

Thanks everyone for the PR & reviews! @Yunni Would you mind sending a "[MINOR]" follow-up PR to fix my late comment + the one from @yanboliang above?

jkbradley · 2017-02-28T01:32:37Z

python/pyspark/ml/feature.py

+    def __init__(self, inputCol=None, outputCol=None, seed=None, numHashTables=1,
+                 bucketLength=None):
+        """
+        __init__(self, inputCol=None, outputCol=None, seed=None, numHashTables=1,


Missing "\" at end of line

Sure. Will do.

Actually I found this issue when reviewing this PR, but I found the generated Python API doc is correct, so I ignored it. @jkbradley Could you let me know the effect of \ at the end of line? Thanks.

Oh, I thought it was necessary for proper doc generation, but maybe it's not.

yanboliang and others added 8 commits November 4, 2016 07:22

Locality Sensitive Hashing (LSH) Python API.

85d22c3

Fix typos.

cdeca1c

Merge branch 'spark-18080' of https://github.com/yanboliang/spark int…

66d308b

…o spark-18080

Merge branch 'master' of https://github.com/apache/spark into spark-1…

d62a2d0

…8080

Changes to fix LSH Python API

dafc4d1

Merge branch 'spark-18080' of https://github.com/Yunni/spark into spa…

ac1f4f7

…rk-18080

Fix examples and class definition

3a21f26

Add python examples and updated the user guide

65dab3e

Fix lint issues

3d3bcf0

Fix python doc issues

69dccde

Fix 'Definition list ends without a blank line'

e7542d0

Fix python unit tests

5cfc9c5

yanboliang mentioned this pull request Jan 28, 2017

[SPARK-18080][ML][PySpark] Locality Sensitive Hashing (LSH) Python API. #15768

Closed

sethah reviewed Feb 7, 2017

View reviewed changes

Yun Ni added 3 commits February 7, 2017 14:49

Merge branch 'master' of https://github.com/apache/spark into spark-1…

ccabbf4

…8080

Code Review Comments

2508a2f

Merge branch 'master' of https://github.com/apache/spark into spark-1…

2dd6aad

…8080

Add alias for similarity join examples

8f1d708

Yunni changed the title ~~[Spark-18080][ML] Python API & Examples for Locality Sensitive Hashing~~ [Spark-18080][ML][PYTHON] Python API & Examples for Locality Sensitive Hashing Feb 11, 2017

sethah reviewed Feb 13, 2017

View reviewed changes

MLnick suggested changes Feb 14, 2017

View reviewed changes

Yun Ni added 2 commits February 13, 2017 20:33

Merge branch 'master' of https://github.com/apache/spark into spark-1…

49edc93

…8080

Code Review Comments

c64d50b

sethah reviewed Feb 14, 2017

View reviewed changes

Code Review Comments: Some minor fixes

5d55752

sethah reviewed Feb 15, 2017

View reviewed changes

Code Review Comment

d849c3a

Merge branch 'master' of https://github.com/apache/spark into spark-1…

36fd9bc

…8080

yanboliang approved these changes Feb 16, 2017

View reviewed changes

asfgit closed this in 08c1972 Feb 16, 2017

jkbradley reviewed Feb 28, 2017

View reviewed changes

		return self.getOrDefault(self.threshold)


		class LSHParams(Params):


		package org.apache.spark.examples.ml;

		import org.apache.spark.ml.linalg.Vector;

[Spark-18080][ML][PYTHON] Python API & Examples for Locality Sensitive Hashing #16715

[Spark-18080][ML][PYTHON] Python API & Examples for Locality Sensitive Hashing #16715

Uh oh!

Conversation

Yunni commented Jan 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

Yunni commented Jan 26, 2017

Uh oh!

SparkQA commented Jan 26, 2017

Uh oh!

SparkQA commented Jan 26, 2017

Uh oh!

SparkQA commented Jan 26, 2017

Uh oh!

SparkQA commented Jan 26, 2017

Uh oh!

SparkQA commented Jan 26, 2017

Uh oh!

jkbradley commented Jan 26, 2017

Uh oh!

yanboliang commented Jan 28, 2017

Uh oh!

Yunni commented Jan 28, 2017

Uh oh!

Yunni commented Feb 6, 2017

Uh oh!

yanboliang commented Feb 7, 2017

Uh oh!

sethah left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yunni commented Jan 26, 2017 •

edited

Loading