Commit Accounting related changes #100380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

cshung merged 7 commits into dotnet:main from cshung:public/fix-refresh-memory-limit

May 22, 2024

Contributor

cshung commented Mar 28, 2024 •

edited

Loading

When heap_segment are moved across heaps during change_heap_count, this change records them so that committed_by_oh_per_heap is still valid.
Keep track of the various committed_by values regardless of heap_hard_limit, which will allow us refresh_memory_limit for segments.
Various refactoring to avoid duplicating code for verification.
Moved verify_regions into _DEBUG.
When release_segment, heap_segment are released by virtual_free, we need to also decrement the committed counters when that happen.
When get_segment, we accidentally used max_generation as oh for UOH segments, this change will make it use the right oh.
If we failed to grow_brick_card_table, heap_segment are released by virtual_free, we need to also decrement the committed counters when that happen.
Compute the amount of memory spent on bookkeeping data for segments, and give up accounting for mark array

... to be continued.

cshung added the area-GC-coreclr label

cshung self-assigned this

cshung marked this pull request as draft

March 28, 2024 00:57

Contributor

dotnet-policy-service bot commented Mar 28, 2024

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

cshung force-pushed the public/fix-refresh-memory-limit branch 2 times, most recently from 9beeb42 to 34c53c5 Compare

April 12, 2024 18:45

build-analysis bot mentioned this pull request

System.Security.Cryptography.Tests.FactorySha1Tests.HashAlgorithm_TransformBlockAndDispose_ConcurrentUseDoesNotCrashProcess fails #100983

Closed

cshung mentioned this pull request

Add GC heap hard limit for 32 bit #101024

Open

cshung changed the title ~~Adding missed synchronization~~ Commit Accounting related changes

cshung force-pushed the public/fix-refresh-memory-limit branch 2 times, most recently from cc4c736 to fcbcdc2 Compare

April 28, 2024 06:24

build-analysis bot mentioned this pull request

Test failure in System.Numerics.Tensors.Tests.SingleGenericTensorPrimitives.SpanDestinationFunctions_SpecialValues #101731

Closed

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Outdated Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Outdated Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Outdated Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Outdated Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Outdated Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Show resolved Hide resolved

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Outdated Show resolved Hide resolved

cshung added 3 commits

May 8, 2024 12:19


          Changes to commit accounting

258d431


          Fix a few commit accounting issues on segments

dda9e01


          Make sure committed bytes got reduced in case we fail to grow the bri…

775fd41

…ck card table

cshung added 2 commits

May 8, 2024 12:19


          Accounting for bookkeeping data structures except mark array

5f169c0


          Avoid redundant system calls

461023e

cshung force-pushed the public/fix-refresh-memory-limit branch from 4d24a44 to 78d6849 Compare

May 8, 2024 19:36


          Code Review Feedback

1a3ef3c

cshung force-pushed the public/fix-refresh-memory-limit branch from 78d6849 to 1a3ef3c Compare

May 8, 2024 19:41

build-analysis bot mentioned this pull request

arm32 fails in CI with "/lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.34' not found" #102030

Closed

cshung marked this pull request as ready for review

May 9, 2024 00:30

Maoni0 reviewed

View reviewed changes

src/coreclr/gc/gc.cpp Outdated Show resolved Hide resolved

Maoni0 approved these changes

View reviewed changes

Member

Maoni0 left a comment

other than the one small comment, LGTM!


          NITs

cshung merged commit 3608eee into dotnet:main

cshung deleted the public/fix-refresh-memory-limit branch

May 22, 2024 20:58

Contributor Author

cshung commented May 22, 2024

This change is tested with all these combinations:

x86/x64
chk
wks/svr/datas (x64 only)
(no defines/committed_bytes_shadow/committed_bytes_shadow + stress_dynamic_heap_count)
(plug test case/gcperfsim)

Bug #102585 is found, but it is unrelated to commit accounting.

steveharter pushed a commit to steveharter/runtime that referenced this pull request


          Commit Accounting related changes (dotnet#100380)

c04aad2

Ruihan-Yin pushed a commit to Ruihan-Yin/runtime that referenced this pull request


          Commit Accounting related changes (dotnet#100380)

f3744cf

github-actions bot locked and limited conversation to collaborators

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-GC-coreclr