This issue has come up in the context of ide-backend.
The problem is that currently I need to run stack build before stack test. Otherwise, the ide-backend-server executable can be out of date. It seems that stack test doesn't build things that aren't directly required by packages which contain test-suites.
For now, stack build ide-backend-server && stack test is ok, but it requires an expensive reconfigure + rebuild.