vz: lima managed vm hangs with high CPU usage intermittently.

[vsiravar]

Problem

Virtualization Framework intermittently starts consuming 100%-220%(from Activity Monitor) CPU and is unresponsive. This leads to all limactl commands being unresponsive or failing.
This intermittently happens when the lima vm is started and left alone for a while.

Behaviour observed

• limactl commands hang such as limactl shell <vm name>.
• sometimes the command fails with RC 255

Once the vm gets to this state
All limactl commands fail.

Workaround

The way around it is to recreate vm.

Related issue

docker/for-mac#6655

Expected behaviour

That the vm should not hang when the computer wakes up from sleep.

Host info

macOS version: 13.4
cpu brand: Apple M1 Pro
lima version: 0.16.0

[balajiv113]

@vsiravar
is this consistently reproducible ?

Before sleep was there any high intensive task running in vm ?

[vsiravar]

@vsiravar is this consistently reproducible ?

No, it's quite intermittent.

Before sleep was there any high intensive task running in vm ?

Not really, I just have a hello-world container running in the vm. I have not experienced this behaviour with qemu.

[balajiv113]

@vsiravar
With current master we now have support for video display. If possible could you enable display and try to replicate the same ??

When it hangs you can check from ui and see if vm is accessible. This will give an idea if the issue is with network/with vm itself

[vsiravar]

With current master we now have support for video display. If possible could you enable display and try to replicate the same ??

Sure, will try this out. Thanks!

[ningziwen]

I think this doesn't only happen when computer wakes up from sleep...

I successfully initialized the VM and ran some commands normally. But after I reply several messages in Slack and come back (around 10 mins), it starts to hang and return FATA[0928] exit status 255.

VM Service has 300% + CPU usage.

[balajiv113]

@ningziwen could you also try enabling display as mentioned above and see ??

Also do share you template which you used.

[ningziwen]

@balajiv113 Sorry I didn't get what it means. Would you like to do screen recording and upload the video? Or using any GUI? Could you point me the instruction if it is GUI?

[balajiv113]

@ningziwen
Steps to enable display

• Build and Install lima from master
• Edit your template with

   video:
     display: "vz"

• Start your template
• Do the normal work and try to reproduce the freeze.
• When hangs check if the display is usable / does contain any logs

This will give us a idea if there are some issues with network/whole vm itself.

[balajiv113]

I tired the above steps myself. Haven't got high cpu usage but the freeze happens.

On checking the GUI during the freeze even that was not responsive so i think the freeze happens on virtualization.framework level not on network.

I have also raised a support ticket with Apple with the same info.

Note: This happens to me on M1 only. My intel runs smooth for weeks with sleep and wake cases

[ningziwen]

@balajiv113 Hey. Did you get any reply from Apple? Is the support ticket link sharable?

[vsiravar]

Updated ticket description and title based on new behaviour observed.

[ryancurrah]

Maybe once #1659 is resolved you can look at the serial.log to see if there is any related log messages.