Datafram schema comparision fails with assertSmallDataFrameEquality method, eventhough schema is same about spark-daria

[MrPowers]

Thanks for opening this issue @prkorp.

The spark-fast-tests library defines the assertSmallDataFrameEquality method that checks if the schema and data in two DataFrames is equal. In your case, the schemas might be the same, but the data might be different.

This project, spark-daria, contains a validateSchema method that's defined here to make sure two schemas are the same.

If you only want to confirm schema equality, then validateSchema will probably be more useful. You can always print out the schemas of both DataFrames with the printSchema method and manually compare the differences. Hopefully this helps!