The is a follow-up from https://github.com/elastic/elasticsearch/pull/35161#discussion_r230295264 We should add a check (and test(s)) for relocating and non-started shards in AllocationRoutedStep