SPARK-4910 [CORE] build failed (use of FileStatus.isFile in Hadoop 1.x)

[srowen]

Fix small Hadoop 1 compile error from SPARK-2261. In Hadoop 1.x, all we have is FileStatus.isDir, so these "is file" assertions are changed to "is not a dir". This is how similar checks are done so far in the code base.

[SparkQA]

Test build #24682 has started for PR 3754 at commit 52c5e4e.

• This patch merges cleanly.

[SparkQA]

Test build #24682 has finished for PR 3754 at commit 52c5e4e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24682/
Test PASSed.

[JoshRosen]

This seems fine to me, especially since this only affects test code. I guess that newer versions of Hadoop have isSymlink, so it may not always be the case that !isDir implies isFile. That said, it looks like this may be the best that we can do in Hadoop 1.x (see Javadoc), so I'm fine with merging this.

One minor comment, though: it looks like your PR description / commit message describes the opposite of this change:

In Hadoop 1.x, all we have is FileStatus.isFile, so these "is directory" assertions are changed to "is not a file"

but I think you meant to say

In Hadoop 1.x, all we have is FileStatus.isDir, so these "is file" assertions are changed to "is not a directory"

[srowen]

@JoshRosen Yes I agree, isDir is not quite the same as !isFile but it's the closest thing available in 1.x. I'll fix the message above, oops.

[JoshRosen]

According to HADOOP-6585, the old behavior / usage was that !isDir implies isFile. Symlinks broke this, but we don't use them in these tests.

I'm going to merge this into master to fix the Hadoop 1 builds. Thanks!