DiskScheduler sometimes hangs on shutdown

[kayousterhout]

(Doesn't need to be fixed until post-deadline)

This is the stack trace:

"Stop DiskAccessor threads" prio=10 tid=0x00007f2184003800 nid=0x62c9 in Object.wait() [0x00007f21fdd19000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1281)
- locked <0x00007f23aa371a30> (a java.lang.Thread)
at java.lang.Thread.join(Thread.java:1355)
at org.apache.spark.monotasks.disk.DiskScheduler$$anonfun$stop$1.apply(DiskScheduler.scala:190)
at org.apache.spark.monotasks.disk.DiskScheduler$$anonfun$stop$1.apply(DiskScheduler.scala:186)
at scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:107)
at scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:107)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
at scala.collection.mutable.HashMap$$anon$2.foreach(HashMap.scala:107)
at org.apache.spark.monotasks.disk.DiskScheduler.stop(DiskScheduler.scala:186)
at org.apache.spark.monotasks.disk.DiskScheduler$$anon$1$$anonfun$run$1.apply$mcV$sp(DiskScheduler.scala:173)
at org.apache.spark.monotasks.disk.DiskScheduler$$anon$1$$anonfun$run$1.apply(DiskScheduler.scala:173)
at org.apache.spark.monotasks.disk.DiskScheduler$$anon$1$$anonfun$run$1.apply(DiskScheduler.scala:173)
at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1621)
at org.apache.spark.monotasks.disk.DiskScheduler$$anon$1.run(DiskScheduler.scala:173)