Skip to content

Conversation

@jxtngx
Copy link
Contributor

@jxtngx jxtngx commented Jul 5, 2022

What does this PR do?

Fix mypy errors attributed to pytorch_lightning.loggers.logger.py for issue #13445

Fixes #<issue_number>

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

@jxtngx jxtngx changed the title Fix mypy errors attributed to pytorch_lightning.loggers.csv_logs.py for issue #13445 Fix mypy errors attributed to pytorch_lightning.loggers.logger.py for issue #13445 Jul 5, 2022
@akihironitta akihironitta added logger Related to the Loggers code quality community This PR is from the community labels Jul 5, 2022
@jxtngx
Copy link
Contributor Author

jxtngx commented Jul 5, 2022

@awaelchli @carmocca et al

regarding logger.merge_dicts:

I've looked at the references and the hierarchy of logger.merge_dicts; it seems to only exist to fulfill a test, and for backwards compatibility in loggers.base.merge_dicts. Neither base.merge_dicts or logger.merge_dicts is used by the base Logger, or any of the third party loggers. The way logger.merge_dicts is implemented violated it's own type hints, causing the following errors:

  1. line 409: error: Argument 2 to "merge_dicts" has incompatible type "Optional[Callable[[Sequence[float]], float]]"; expected "Optional[Mapping[str, Callable[[Sequence[float]], float]]]
  2. line 411: error: Incompatible types in assignment (expression has type "float", target has type "Dict[Any, Any]")

I updated the agg_key_funcs to be more generic with Optional[Mapping]. Using a defaultdict as d_out during recursion fixed the above errors; the defaultdict is returned as a dict with dict(d_out), satisfying the intended return type.

@jxtngx jxtngx marked this pull request as ready for review July 5, 2022 21:47

@wraps(fn)
def experiment(self):
def experiment(self: Callable) -> Union[ExperimentWriter, DummyExperiment]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't annotate self. self has always the pseudo type "Self"

Copy link
Contributor Author

@jxtngx jxtngx Jul 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I remove that annotation, mypy complains:

pytorch_lightning/loggers/logger.py:42: error: Function is missing a type annotation for one or more arguments.

Rather than annotate Self, I commented with # type: ignore[no-untyped-def] to exclude the error in the latest commit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This self for the local function definiton is not necessarily the "self" from the actual instance.

@jxtngx jxtngx mentioned this pull request Jul 7, 2022
52 tasks
@akihironitta akihironitta changed the title Fix mypy errors attributed to pytorch_lightning.loggers.logger.py for issue #13445 Fix mypy errors attributed to pytorch_lightning.loggers.logger.py Jul 8, 2022
@akihironitta akihironitta added this to the pl:1.7 milestone Jul 8, 2022
Copy link
Contributor

@akihironitta akihironitta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JustinGoheen Other than my comments below, it looks good to me!

Also, the following PRs are waiting on checks, is there anything more I need to do before the remaining checks can run?

Let's wait for reviews from others!

@mergify mergify bot added the ready PRs ready to be merged label Jul 8, 2022
@otaj
Copy link
Contributor

otaj commented Jul 8, 2022

Oh, things started failing as ExperimentWriter is undefined. I'll take a closer look at it on Monday

@jxtngx
Copy link
Contributor Author

jxtngx commented Jul 9, 2022

@otaj I had set a conditional import of ExperimentWriter; I removed that condition in favor of a hardcoded import.

@otaj
Copy link
Contributor

otaj commented Jul 11, 2022

@JustinGoheen Great :)

@otaj otaj enabled auto-merge (squash) July 11, 2022 09:22
@otaj otaj merged commit 933848d into Lightning-AI:master Jul 11, 2022
jerome-habana pushed a commit to jerome-habana/lightning that referenced this pull request Jul 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code quality community This PR is from the community logger Related to the Loggers ready PRs ready to be merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants