-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Description
🐛 Bug
Trainer's fit method fails when the only logger is the custom logger from the official documentation
To Reproduce
- Define a custom logger as per https://pytorch-lightning.readthedocs.io/en/latest/logging.html#make-a-custom-logger
logger = MyLogger() - Define a lightning module as per https://pytorch-lightning.readthedocs.io/en/latest/new-project.html#step-1-define-lightningmodule
- Load MNIST and call training as per https://pytorch-lightning.readthedocs.io/en/latest/new-project.html#step-2-fit-with-lightning-trainer with ONE MODIFICATION - specify Trainer(gpus=1, logger = logger)
Expected behavior
Successful execution
Environment
- CUDA:
- GPU:
- GeForce RTX 2070
- available: True
- version: 10.2 - Packages:
- numpy: 1.19.2
- pyTorch_debug: False
- pyTorch_version: 1.6.0
- pytorch-lightning: 1.0.3
- tqdm: 4.50.2 - System:
- OS: Linux
- architecture:
- 64bit
- ELF
- processor:
- python: 3.8.5
- version: Proposal for help #1 SMP PREEMPT Sat Oct 17 11:06:19 UTC 2020
Additional context
Stack trace:
`
PU available: True, used: True
TPU available: False, using: 0 TPU cores
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
TypeError Traceback (most recent call last)
in
5 # trainer = pl.Trainer(gpus=8) (if you have GPUs)
6 trainer = pl.Trainer(gpus=1, logger = logger)
----> 7 trainer.fit(autoencoder, train_loader)
~/.conda/envs/dlschool0/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py in fit(self, model, train_dataloader, val_dataloaders, datamodule)
437 self.call_hook('on_fit_start')
438
--> 439 results = self.accelerator_backend.train()
440 self.accelerator_backend.teardown()
441
~/.conda/envs/dlschool0/lib/python3.8/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py in train(self)
49
50 # set up training routine
---> 51 self.trainer.train_loop.setup_training(model)
52
53 # train or test
~/.conda/envs/dlschool0/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py in setup_training(self, model)
144 # --------------------------
145 # on pretrain routine start
--> 146 self.trainer.on_pretrain_routine_start(ref_model)
147 if self.trainer.is_function_implemented('on_pretrain_routine_start'):
148 ref_model.on_pretrain_routine_start()
~/.conda/envs/dlschool0/lib/python3.8/site-packages/pytorch_lightning/trainer/callback_hook.py in on_pretrain_routine_start(self, model)
120 """Called when the train begins."""
121 for callback in self.callbacks:
--> 122 callback.on_pretrain_routine_start(self, model)
123
124 def on_pretrain_routine_end(self, model):
~/.conda/envs/dlschool0/lib/python3.8/site-packages/pytorch_lightning/callbacks/model_checkpoint.py in on_pretrain_routine_start(self, trainer, pl_module)
159 When pretrain routine starts we build the ckpt dir on the fly
160 """
--> 161 self.__resolve_ckpt_dir(trainer, pl_module)
162
163 def on_validation_end(self, trainer, pl_module):
~/.conda/envs/dlschool0/lib/python3.8/site-packages/pytorch_lightning/callbacks/model_checkpoint.py in __resolve_ckpt_dir(self, trainer, pl_module)
405 version, name = trainer.accelerator_backend.broadcast((version, trainer.logger.name))
406
--> 407 ckpt_path = os.path.join(
408 save_dir, name, version, "checkpoints"
409 )
~/.conda/envs/dlschool0/lib/python3.8/posixpath.py in join(a, *p)
88 path += sep + b
89 except (TypeError, AttributeError, BytesWarning):
---> 90 genericpath._check_arg_types('join', a, *p)
91 raise
92 return path
~/.conda/envs/dlschool0/lib/python3.8/genericpath.py in _check_arg_types(funcname, *args)
150 hasbytes = True
151 else:
--> 152 raise TypeError(f'{funcname}() argument must be str, bytes, or '
153 f'os.PathLike object, not {s.class.name!r}') from None
154 if hasstr and hasbytes:
TypeError: join() argument must be str, bytes, or os.PathLike object, not 'method'
`