Hadoop indexing - use NumberedShardSpec instead of NoneShardSpec

[gianm]

Currently if Hadoop indexing has only one shard to generate for a particular time range, it will use NoneShardSpec. Using NumberedShardSpec(0, 1) instead of NoneShardSpec would help allocateSegment flows (like the Kafka indexing service) coexist with Hadoop indexing. allocateSegment (see #1896) depends on existing segment sets being extensible, which NoneShardSpec is not.

This could be an across the board change or it could be optional. I don't see any real usefulness for NoneShardSpec, so I lean towards making this change across the board.

[sixtus]

The same is true for IndexTask btw.

I guess https://github.com/druid-io/druid/blob/master/indexing-service/src/main/java/io/druid/indexing/common/task/IndexTask.java#L303-L309 can be replaced by the code in the else block, right?

[gianm]

Going to take a look at this. I'm thinking about adding a "forceExtendableShardSpecs" option.

[nishantmonu51]

IIRC, the reason for NoneShardSpec is to have the coordinator compact them, as the coordinator only compacts segments with NoneShardSpec.
forceExtendableShardSpecs with default to false is a good workaround until the coordinator logic is modified to handle numbered/linear shard specs.