Skip to content

Conversation

@benwtrent
Copy link
Member

From the management side, we allow users to update configured memory limits after the analytics configuration has started.

To help users with their workflow, we should allow analytics processes to attempt to run even if the configured memory limit is lower than our memory estimate.

related to: elastic/elasticsearch#61505

@benwtrent benwtrent changed the title [ML] do not fail to start analytics process of memory estimate is low [ML] do not fail to start analytics process if memory estimate is lower than predicted memory usage Aug 31, 2020
@benwtrent
Copy link
Member Author

This change seemed simple to me. Hopefully I didn't miss any "gotchas"

Our instrumentation will automatically detect when memory usage is higher than the configured limit and throw a fatal.

See:

if (memory > memoryLimit) {

Copy link

@droberts195 droberts195 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

BOOST_REQUIRE_EQUAL(1, static_cast<int>(errors.size()));
BOOST_TEST_REQUIRE(re.matches(errors[0]));
// no error should be registered
BOOST_REQUIRE_EQUAL(0, static_cast<int>(errors.size()));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Casting to int is a leftover from CppUnit, which required the LHS and RHS types to be exactly the same. Boost.Test doesn't, so the cast is just noise now. But no need to make another commit just to change this, as it had the cast before too.

@benwtrent benwtrent merged commit 339edd0 into elastic:master Sep 1, 2020
@benwtrent benwtrent deleted the feature/ml-dont-fail-to-start-due-to-memory-limits branch September 1, 2020 13:03
benwtrent added a commit that referenced this pull request Sep 1, 2020
…er than predicted memory usage (#1465) (#1467)

From the management side, we allow users to update configured memory limits after the analytics configuration has started.

To help users with their workflow, we should allow analytics processes to attempt to run even if the configured memory limit is lower than our memory estimate. 


related to: elastic/elasticsearch#61505
benwtrent added a commit to elastic/elasticsearch that referenced this pull request Sep 2, 2020
benwtrent added a commit to benwtrent/elasticsearch that referenced this pull request Sep 2, 2020
benwtrent added a commit to elastic/elasticsearch that referenced this pull request Sep 2, 2020
)

* [ML] unmute testTooLowConfiguredMemoryStillStarts (#61846)

Native PR addresses this test failure: elastic/ml-cpp#1465


closes #61704

closes #61561
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants