Fix schema/type inference issue

[HyukjinKwon]

#244
This is re-opened due to the build failure in travis.

[tanwanirahul]

@HyukjinKwon @falaki Do you want me to raise new PR with a single commit? That will make sure we have a clean commit history?

[codecov-io]

Current coverage is 85.74%

Merging #261 into master will decrease coverage by -0.33% as of d0a91c5

@@            master    #261   diff @@
======================================
  Files           12      12       
  Stmts          517     519     +2
  Branches       149     149       
  Methods          0       0       
======================================
  Hit            445     445       
  Partial          0       0       
- Missed          72      74     +2

Review entire Coverage Diff as of d0a91c5

Powered by Codecov. Updated on successful CI builds.

[HyukjinKwon]

@tanwanirahul dev/merge_pr.py will make this as a single commit log as PRs are not merged by simply pushing the merge button and these commit logs will only stay in comments in a single commit log at the end with a single author.

This is why I said here #244 (comment), the author might have to be you.

[HyukjinKwon]

BTW, does this not produce Nulltype after merging Nulltypes?
It looks the test Merging Nulltypes should yeild Nulltype is not covering this.

[tanwanirahul]

@HyukjinKwon Please elaborate. Referring to doc comments in InferSchema.scala

/**

• Similar to the JSON schema inference.
• [[org.apache.spark.sql.execution.datasources.json.InferSchema]]

• 1. Infer type of each row
  

• 2. Merge row types to find common type
  

• 3. Replace any null types with string type
  

  */

All null types should be replaced with String types at the end. This is what is happening right now?
If I understand you correctly, you feel there is a need for some more tests to be added. Could you please mention what that test is suppose to test?

As far as testing merging two nulltypes returning null type, is already covered.

[HyukjinKwon]

Ah, sorry. Please ignore my comment.

[HyukjinKwon]

@tanwanirahul Well, but If you need to change some codes here, then I think you better create another PR maybe. (And please close the previous PR)

[tanwanirahul]

I will create a JIRA on Spark and submit a patch once this is merged.

[falaki]

Thanks. This looks good and passes all tests. Merging it now.