Add a robots.txt to stop Google from indexing Deprecated Versions of the Docs

[aribornstein]

📚 Documentation

Add a robots.txt to stop Google from indexing Deprecated Versions of the Docs
Google is currently indexing deprecated versions of the docs so that when you search for features you are getting depricated information. For example if I google "pytorch lightning save best model" The first result is associated with lighting version 0.4.9 with no indication that isn't the latest version of the docs.

This shouldn't be that hard to fix and will have major impact on developer experience.
https://docs.readthedocs.io/en/latest/faq.html#how-can-i-avoid-search-results-having-a-deprecated-version-of-my-docs

[akihironitta]

Same as #6160