[ML] Feature/forecast scale #89
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
April 17, 2018 18:28
This implements the C++ side of forecast persistence. An additional parameter allows the forecast runner to persist models on disk for temporary purposes. Models are loaded back into memory one by one. For models smaller than the current limit of 20MB nothing changes.
droberts195
approved these changes
May 4, 2018
|
|
||
| // Note: This value measures the size in memory, not the size of the persistence, | ||
| // which is likely higher and would be hard to calculate upfront | ||
| //! max memory allowed to use for forecast models persisting to disk |
There was a problem hiding this comment.
Only the last line of this begins with //!. All the lines should be consistent.
| // if you change this value also change the limit on X-pack side. | ||
| // The purpose of this value is to guard the rest of the system regarding | ||
| // an out of disk space | ||
| //! minimum disk space required for disk persistence |
There was a problem hiding this comment.
Only the last line of this begins with //!. All the lines should be consistent.
include/model/CModelFactory.h
Outdated
| //! component. | ||
| std::size_t componentSize() const; | ||
|
|
||
| // Get the minimum seasonal variance scale, specific to the model |
lib/api/CForecastRunner.cc
Outdated
| series.s_ToForecast.emplace_back( | ||
| feature, std::move(model), byFieldValue); | ||
| } else // restorer exhausted, no need for further restoring | ||
| { |
There was a problem hiding this comment.
nit: could move { after else but before comment
hendrikmuhs
pushed a commit
that referenced
this pull request
May 14, 2018
This implements the C++ side of forecast persistence. An additional parameter allows the forecast runner to persist models on disk for temporary purposes. Models are loaded back into memory one by one.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This implements forecast model cloning overflowing to disk to allow forecasting of large jobs. The tmp storage location has to be given as parameter (X-Pack change: elastic/elasticsearch#30399). If the parameter is not given, this feature is disabled. For models smaller than the existing limit (20MB) nothing changes.
Todo:
To be merged together once corresponding X-Pack PR and this PR are ready.
Release note: Forecasting of large machine learning jobs is now supported by temporarily storing
model state on disk