High CPU overhead when using ParallelExecutor in a minimal throttled example

[maun]

Bevy version

0.8-dev main aced6af

Operating system & version

macOS 12.3.1

What you did

Used minimal example:

fn main() {
    App::new()
        .add_plugins(MinimalPlugins)
        .add_system(sleep_system)
        .run();
}

fn sleep_system() {
    std::thread::sleep(std::time::Duration::from_millis(15));
}

and measured CPU usage with top / Activity Monitor.
The app was started using cargo run --release.

What you expected to happen

Running with close to 0% CPU usage.

What actually happened

CPU usage of 10-15%

Additional information

Using SingleThreadedExecutor by hardcoding it in bevy_ecs

pub fn parallel() -> Self {
    // Self::new(Box::new(ParallelExecutor::default()))
    Self::new(Box::new(SingleThreadedExecutor::default()))
}

results in a CPU usage of about 0.3%.

Similar behaviour exists for an empty window without using the renderer feature and also in the breakout example. For the empty window the CPU usage was quite low, whereas in the breakout example the CPU usage dropped about >10% with the change, but was still unexpectedly high.

I recorded flame graphs, to get the interactive version download the svg and open them in a browser.

ParallelExecutor

The call to std::thread::sleep is at the highest "tower" on the left, you can zoom in when using a local interactive svg.

SingleThreadedExecutor

The same test without the sleep system results in 150% CPU usage with the ParallelExecutor and close to 100% with the SingleThreadedExecutor .

Issue #1462 is probably related but was closed and is less specific.

[alice-i-cecile]

This is an excellent issue report. I'd like to try and benchmark this; we have a long way to go here.

[james7132]

@maun can you please try again with the changes from #4740? This rewrite a large portion of the internals of our TaskPool, which seems to have a lot of its internals showing up in your flamegraphs. In particular, the way that scope is handled (with the local executor's try_tick usage) should be reworked to use proper suspend and Waker infrastructure.

[maun]

With the changes from #4740 I get a CPU usage of about 2.3%. This is the flamegraph for the PR:

[james7132]

With the changes from #4740 I get a CPU usage of about 2.3%. This is the flamegraph for the PR:

This might be an issue. We're currently investigating some overhead with that PR, which may be very prominently shown in your flamegraph as parking the thread, whereas main is currently spin-waiting on the results. Is a reduction from 15% idle to 2.3% acceptable here for us to close the issue or push this further? At some point this becomes a tradeoff question of latency vs resource usage.

[DigitalJokerMan]

+1 on this issue. I'd like to use Bevy for a command-line game server and running a barebones App with just MinimalPlugins results in using as many CPU resources as the machine has left.

[ypoonse]

+1 on this issue. I'd like to use Bevy for a command-line game server and running a barebones App with just MinimalPlugins results in using as many CPU resources as the machine has left.

Same here, very similar same scenario: a headless server that runs game logic. In my case I only have a system that needs to run once every 0.1s, so my setup looks like this:

    App::new()
        .add_plugins(MinimalPlugins)
        .add_system_set(
            SystemSet::new()
                .with_run_criteria(FixedTimestep::step(GAME_TIMESTEP))
                .with_system(logic_system1),
        )

But in this case, it's also still consuming a full cpu.

One possible workaround is to take control of the scheduling myself.

let mut world = World::default();

let mut update_stage = SystemStage::parallel();
update_stage.add_system(my_system);

let entity1 = world
    .spawn()
    .insert(...)
    .id();

// Adding resources
world.insert_resource(....);

// Run systems every 100 ms, 
// Note that the example is not yet correct as it doesn't take into account how long the 'run' step takes. In principle there should be a timer that calls 'updated_stage.run' every 100 ms.
loop {
    update_stage.run(&mut world);
    std::thread::sleep(std::time::Duration::from_millis(100));
}

[hymm]

@ypoonse for your usage, I would consider using the bevy_framepace crate or write a custom runner. https://github.com/bevyengine/bevy/blob/v0.8.0/examples/app/custom_loop.rs

[martinfiers]

Thank you very much @hymm ! CPU just went down with a lot by using a custom loop. This workaround works perfectly for me.