“Actor object is missing” in GRPO example run

[zch42]

🐛 Describe the bug

Following the installation tutorial (conda env setup), I ran:

python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml

with the default TORCHSTORE_RDMA_ENABLED=1, I first hit an RDMA-related failure that seems related to this issue.

After disabling RDMA (TORCHSTORE_RDMA_ENABLED=0), the run still failed with:

ActorError: A remote actor call has failed.
AssertionError: Actor object is missing when executing init_backends
Actor object is missing … local_fetcher_actor error

It appears that by the time register_fetcher runs, the policy mesh or its local_fetcher_actor has failed to start, so Monarch reports “actor object is missing” when metric loggers attempt to access it.

Versions

• torch: 2.9.0+cu128
• torchmonarch: 0.1.2
• torchtitan: 0.2.0
• vLLM: 0.10.1.dev0 (commit 6d8d0a24c, built 2025-11-14)

[felipemello1]

hey @zch42 , sorry to hear that this is happening. I just tried running it on main and things ran fine. I have the exact same versions. I am on python=3.12 though. I also installed my env ~3 days ago.

Did you run this to setup your env?

conda create -n forge python=3.12
conda activate forge
./scripts/install.sh

[felipemello1]

Can you please share the complete trace?

[zch42]

hey @zch42 , sorry to hear that this is happening. I just tried running it on main and things ran fine. I have the exact same versions. I am on python=3.12 though. I also installed my env ~3 days ago.

Did you run this to setup your env?

conda create -n forge python=3.12
conda activate forge
./scripts/install.sh

Yes I used the exact same setup to install env yesterday

[felipemello1]

i will retry tomorrow with a fresh env

[felipemello1]

For now, to unblock you, you could set env var FORGE_DISABLE_METRICS=False, but you would have no metric logging

[zch42]

Can you please share the complete trace?

$ export TORCHSTORE_RDMA_ENABLED=0
$ python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml

Warning: setting HYPERACTOR_CODEC_MAX_FRAME_LENGTH since this needs to be set to enable large RPC calls via Monarch
INFO Automatically detected platform cuda.
Launcher not provided, remote allocations will not work.

wandb: WARNING Disabling SSL verification. Connections to this server are not verified and may be insecure!
wandb: Tracking run with wandb version 0.23.0
wandb: Run data is saved locally in ./wandb/run-20251117_024352-xxxxxxx
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run <run-name>
wandb: Detected [openai] in use.

Spawning actor DatasetActor
Spawning service Generator
Spawning actor TitanTrainer
Spawning actor ReplayBuffer
Spawning actor ComputeAdvantages
Spawning service ReferenceModel
Spawning service RewardActor

Unhandled exception in actor endpoint
Traceback (most recent call last):
  File ".../site-packages/monarch/_src/actor/actor_mesh.py", line 935, in handle
    result = await the_method(*args, **kwargs)
  File ".../torchforge/src/forge/observability/metric_actors.py", line 352, in register_fetcher
    await fetcher.init_backends.call(
  File ".../monarch/_src/actor/future.py", line 138, in mark_complete
    func, value = fut.set_result, await coro
  File ".../monarch/_src/actor/endpoint.py", line 147, in process
    rank, value = await r._recv()
  File ".../monarch/_src/actor/actor_mesh.py", line 794, in _recv
    return self._process(result)
  File ".../monarch/_src/actor/actor_mesh.py", line 824, in _process
    return rank, super()._process(msg)
  File ".../monarch/_src/actor/actor_mesh.py", line 803, in _process
    raise cast(Exception, payload)
monarch._src.actor.actor_mesh.ActorError: A remote actor call has failed.

Traceback of where the remote call failed (most recent call last):
  File ".../monarch/_src/actor/actor_mesh.py", line 917, in handle
    raise AssertionError(error_message)
AssertionError: Actor object is missing when executing method init_backends on actor unix:@<actor-address>,trainer_0_<uid>,local_fetcher_actor_<uid>[0].

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "torchforge/apps/grpo/main.py", line 507, in <module>
    _main()
  File "torchforge/src/forge/util/config.py", line 313, in wrapper
    sys.exit(recipe_main(conf))
  File "torchforge/apps/grpo/main.py", line 505, in _main
    asyncio.run(main(cfg))
  File ".../asyncio/runners.py", line 195, in run
    return runner.run(main)
  File ".../asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
  File ".../asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
  File "torchforge/apps/grpo/main.py", line 322, in main
    ) = await asyncio.gather(
  File "torchforge/src/forge/controller/actor.py", line 235, in as_actor
    actor = await cls.launch(*args, **actor_kwargs)
  File "torchforge/src/forge/controller/actor.py", line 217, in launch
    proc_mesh = await get_proc_mesh(process_config=cfg)
  File "torchforge/src/forge/controller/provisioner.py", line 537, in get_proc_mesh
    return await provisioner.get_proc_mesh(
  File "torchforge/src/forge/controller/provisioner.py", line 406, in get_proc_mesh
    _ = await get_or_create_metric_logger(procs, process_name=mesh_name)
  File "torchforge/src/forge/observability/metric_actors.py", line 127, in get_or_create_metric_logger
    await global_logger.register_fetcher.call_one(local_fetcher_actor, proc._uid)
  File ".../monarch/_src/actor/future.py", line 138, in mark_complete
    func, value = fut.set_result, await coro
  File ".../monarch/_src/actor/actor_mesh.py", line 794, in _recv
    return self._process(result)
  File ".../monarch/_src/actor/actor_mesh.py", line 803, in _process
    raise cast(Exception, payload)
monarch._src.actor.actor_mesh.ActorError: A remote actor call has failed.

Traceback of where the remote call failed (most recent call last):
  File ".../monarch/_src/actor/actor_mesh.py", line 917, in handle
    raise AssertionError(error_message)
AssertionError: Actor object is missing when executing method init_backends on actor unix:@<actor-address>,trainer_0_<uid>,local_fetcher_actor_<uid>[0].


wandb:
wandb: Find logs at: wandb/run-20251117_024352-xxxxxxx/logs

Task was destroyed but it is pending!
Exception ignored in: <coroutine object _Actor.handle>
Traceback (most recent call last):
  File ".../monarch/_src/actor/actor_mesh.py", line 960, in handle
  File ".../monarch/_src/actor/actor_mesh.py", line 980, in _post_mortem_debug
ImportError: sys.meta_path is None, Python is likely shutting down

Task was destroyed but it is pending!
Exception ignored in: <coroutine object _Actor.handle>

[zch42]

i will retry tomorrow with a fresh env

Thanks @felipemello1, I’ve opened a PR with a quick fix that initializes the metric logger after all proc meshes are spawned. With this change, I can successfully run python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml with wandb metric logging.

[felipemello1]

Hey @zch42 , the PR makes sense, but it shouldnt be happening in the first place and i couldnt reproduce and no one in the team reported it either. I am a bit busy with a task atm, let me get back to that later this week. If more people complain about it, i will merge it

[zch42]

Hey @zch42 , the PR makes sense, but it shouldnt be happening in the first place and i couldnt reproduce and no one in the team reported it either. I am a bit busy with a task atm, let me get back to that later this week. If more people complain about it, i will merge it

Thanks @felipemello1, let me know if you need additional info to reproduce the bug. Also do you see any potential issues with the changes I made in the PR?

[felipemello1]

no issues at all, but it masks a bug, which is you should be able to instantiate new actors whenever, and metric logging should still work. E.g. imagine if an actors dies and needs to be rebooted, or if there is some condition that triggers a new actors later in the process

[zch42]

@felipemello1 were you able to reproduce the bug? I ran the example on a B200 cluster