Cache not invalidated when previously missing imports get stubs

[FuegoFro]

It looks like if you type check a file that has missing imports and use the --ignore-missing-imports flag, then add stubs for those imports (such that they're no longer missing), then running type checking again will not correctly invalidate the cache. In the repro I have here this just causes it to miss a type error, but in my work it caused internal errors in mypy (though I wasn't able to get a minimal repro for those errors). Once you get into this state, either removing the --ignore-missing-imports flag or deleting the cache will cause it to properly type check the file.

Here is a script to reproduce the issue:

#!/bin/bash
# Set up a dummy program and a dummy stubs dir
PROGRAM="$(mktemp)"
echo "import my_made_up_package; my_made_up_package.my_method()" > "${PROGRAM}"

STUBS="$(mktemp -d)"
mkdir "${STUBS}/my_made_up_package"
echo "def my_method(i: int) -> None: ..." > "${STUBS}/my_made_up_package/__init__.pyi"

# Make a dummy cache, to not mess with the real one
CACHE="$(mktemp -d)"

# Run the checker to generate a cache.
echo "RESULTS WITHOUT STUBS"
python3 -m mypy --cache-dir "${CACHE}" --ignore-missing-imports "${PROGRAM}"

# Add stubs to MYPYPATH and and run the checker again. Note it doesn't report an error!
echo "RESULTS WITH STUBS"
MYPYPATH="${STUBS}" python3 -m mypy --cache-dir "${CACHE}" --ignore-missing-imports "${PROGRAM}"

# Clear out the cache and run the checker again, see it now reports an error.
# Alternatively, you can leave the cache and remove the --ignore-missing-imports arg to correctly
# report an error.
rm -rf "${CACHE}"
echo "RESULTS WITH STUBS AFTER CLEARING CACHE"
MYPYPATH="${STUBS}" python3 -m mypy --cache-dir "${CACHE}" --ignore-missing-imports "${PROGRAM}"

# Clean up the program, stubs dir, and cache
rm "${PROGRAM}"
rm -rf "${STUBS}"
rm -rf "${CACHE}"

This is with mypy version mypy 0.600+dev-95ae4ee24dfc1c84dcc892406cd1a27f80bca3de

[ilevkivskyi]

I am raising priority to high because it makes adding stubs to large frameworks quite annoying (I need to kill the cache manually).