Added additional env vars to control the frequency of detector activating#33
Added additional env vars to control the frequency of detector activating#33symstu-tempesta wants to merge 2 commits into
Conversation
aca57a6 to
f73cced
Compare
krizhanovsky
left a comment
krizhanovsky
left a comment
There was a problem hiding this comment.
I have a couple of comments, where I think we should do different things
00cd8b4 to
2e17cd3
Compare
krizhanovsky
left a comment
krizhanovsky
left a comment
There was a problem hiding this comment.
There are still questions on the implementation
|
|
||
| :param current_time: used as the current time in functional tests | ||
| :param interval: used as the current time in functional tests | ||
| :param previous_window_interval: control group of the access log data |
There was a problem hiding this comment.
Is "control" means "training" or "history"?
There was a problem hiding this comment.
This function is shared across all data fetching, so it depends on the context. However, the "control group" always consists of much older data, and we assume that this data should be correct and free of traffic anomalies
| BLOCKING_NEW_WINDOW_OFFSET_SEC=10 | ||
|
|
||
| # Duration (in seconds) of the **new** (current) window. | ||
| BLOCKING_NEW_WINDOW_DURATION_SEC=10 |
There was a problem hiding this comment.
Why do we need these 4 new parameters? #25 requires only 2 new ones: BLOCKING_CHECK_TIMEOUT_SEC and TRAINING_MODE_HISTORY_OFFSET_MIN.
Also please write more detailed commit messages. E.g. 3890839 provides only a brief "additional variable to control data collecting intervals" and doesn't answer why do we need these 4 new parameters. There are also no comments in the code or config example, which also describes the parameters as "new" and "previous" but doesn't explain when and why the should be configured.
There was a problem hiding this comment.
I was planning to squash all commits later and add a more detailed description there, as this is a continuation of the changes and it's much easier to track them this way. The last changes I've made according to the manual on our web-site.
The example we discussed included gaps between windows — for example, the same day one week ago. In that case, we need to control the gap duration, the window duration, and its start time. While working with these parameters, I realized it would also be useful to give maximum control over the windows and make it more straightforward.
Now it is possible to configure each window independently, including its start time and duration — for example, a window that started 2 hour ago and lasts for 1 hour, and another window that started a week ago and lasts for 1 day.
8a8fe59 to
9f85301
Compare
It is now possible to set the frequency of fetching data from ClickHouse and configure the fetching queries more accurately. For instance, it's good to fetch data for 1 day from a day before and one week before. #25
Google has migrated the list of crawlers to the new url addresses
9f85301 to
1c3efe1
Compare
3 participants
No description provided.