Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,6 +489,52 @@ This filter provides the capability to enable a feature based on a time window.
}
```

This filter also provides the capability to enable a feature during some recurring time windows. The recurring time windows act as the additional filters of evaluation, supplementing the time window defined by the `Start` and `End`. If any recurring time window filter is specified, the feature will only be enabled when the current time falls between the `Start` and `End`, and also falls within a recurring time window listed in the `Filters`.
We can see some Crontab expressions are listed under the `Filters`, which specify the recurring time windows.
``` JavaScript
"EnhancedPipeline": {
"EnabledFor": [
{
"Name": "Microsoft.TimeWindow",
"Parameters": {
"Start": "2023-09-06T00:00:00+08:00",
"Filters": [
// 18:00-20:00 on weekdays
"* 18-19 * * Mon-Fri",
// 18:00-22:00 on weekends
"* 18-21 * * Sat,Sun"
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really have to support an array of crontab expressions, or is one sufficient for most scenarios? Even if someone really needs more than one, they can always add more Time Window filters with different configurations.

I'm concerned this complicates the usage of the recurring Time Window filter. I can already see this complicates the UI design.

BTW, having a 'Filters' parameter in a feature filter is also confusing.

Copy link
Member Author

@zhiyuanliang-ms zhiyuanliang-ms Sep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scenario when the user want to activate a feature flag during 18:00-20:00 on every weekend and during 18:00-22:00 on holidays requires a list to specify recurring time windows with different durations.
Using multiple timewindow filters may not be a good choice, since the feature flag may also use other filters(e.g. targeting), in this case, the "RequirementType" is "All".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the parameter name 'Filters' is kind of confusing. But the cron expression does serve as "filters" since we also have the parameter "Start" and "End" which should be considered as the timestamp when the filter starts to work("FilterStart") and when the filter ends working("FilterEnd").
The reason why we use "Filters" instead of "Recurrence" is that we will treat an empty list of "Filters" as not specified.
When "Filters" is not specifed, the TimeWindowFilter will revert to the original fixed time window filter.
The name "Filters" makes more sense.

Let's take a look at two choices:
Choice A:
"Start" : "2023-9-19-00:00",
"End": "2024-9-19-00:00",
"Filters": [ ]

Choice B:
"Start" : "2023-9-19-00:00",
"End": "2024-9-19-00:00",
"Recurrence": [ ]
If "Filters" or "Recurrence" is not specified, there is no confusion.
If "Filters" or "Recurrence" is not empty, the filter will only enable the feature during the recurring time window listed in the list "Filters" or "Recurrence". However, for the “Recurrence" choice, the parameter "Start" and "End" serve as the "RecurrenceStart" and "RecurrentEnd" instead of "FilterStart" and "FilterEnd". This will make the scenario that the "Recurrence" is specified as an empty list confusing. The function of "Start" and "End" will be inconsistent if we use name "Recurrence".
The user who uses FeatureManagement-Dotnet is expected to read the README before using it. The user who uses FeatureManagement through portal will never know the backend data schema for TimeWindowFilter unless they use "Advanced Edit"(in this case, the user should be considered as a pro who has read our doc). On the Portal, the scenarios of setting a fixed time window and setting recurring time window will have different UI and will not mention the "Filters" parameter.

For me, I will vote for the name "Filters" because it makes the role of "Start" and "End" parametes consistent. I also believe it will not cause confusion for the user who uses FeatureManagement-Dotnet lib directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scenario when the user want to activate a feature flag during 18:00-20:00 on every weekend and during 18:00-22:00 on holidays requires a list to specify recurring time windows with different durations.

I'm sure we can come up with even more complex scenarios. The question is how realistic they are. Outlook doesn't support infinite recurrence patterns, just one pattern at a time. Without the array, the parameters can be simplified to "Start", "End" and "RecurrencePattern". The UI design doesn't need an add button for more than one recurrence pattern anymore.

Using multiple timewindow filters may not be a good choice, since the feature flag may also use other filters(e.g. targeting), in this case, the "RequirementType" is "All".

We know the "RequirementType" doesn't support all possible logical operations. It was a conscious decision. It's a balance between complexity and the most common scenarios. The choice of that design shouldn't be the reason for us to complicate the design of each feature filter. We have a reasonable workaround. We know there are scenarios that won't be supported out-of-box no matter how. That's where customers should consider custom filters.

Copy link
Member Author

@zhiyuanliang-ms zhiyuanliang-ms Sep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do not want to use a single parameter "RecurrencePattern", then we cannot use cron expression any more. The reason is that we have to use at least three cron expressions to describe the time span 18:30`20:30. e.g. "30-59 18 * * ", " 19 * * *“ and "0-29 20 * * *".
I have proposed a lot of schema designs to describe the recurrence. Some of them are deprecated early and are not included in the design doc.
The best way to describe recurrence is to specify the duration and recurrence pattern (i.e. when we want to make the recuurence happen).
At first, I want to use two parameters "Duration" and "RecurWhen".

// 18:30-20:30 every Friday
"Duration": { 
  "Type": "Hour", 
  "Length": 2 
},  
"RecurWhen":  "30 18 * * 5"

This represents 18:00-22:00 every Friday.
This design will use cron expression in classic way where the cron expression represents the schedule (time points instead of time spans).
But Jimmy thought we can include the duration information in the cron expression, in this way the json schema will be very simple. The above configuration can be simplified to a single cron "* 18-22 * * 5".

Jimmy thought the current design achieved the balance between simplicity, offering to customers and implementation/maintenance. I completly agree with Jimmy, that's the reason why we finally decided to use the current cron design for the recurrence configuration.

Besides, the current design also provides a benefit that it will make the process to check whether the current time is within any recurring time window very simple. We only need to check whether five fields are compliant with current timestamp.
If we use the "Duration" and "RecurWhen" design, we need to compute the previous occurrence of the cron schedule which is the start of the recurring time window and then check whether the current time is within that recurring time windows.

These are some basic scenarios we want to support(you can find it in the design doc):

  1. Basic Scenarios
    a) Recur during a certain time window every day
  • Turn on a feature flag during 16:00~20:00 every day
    b) Recur during a certain time window on certain days of a week
  • Turn on a feature flag during 16:00~20:00 on Monday and Friday every week
  • Turn on a feature flag from Saturday evening to Monday morning every week
    c) Recur during a certain time window on certain days of a month
  • Turn on a feature flag during 18:00~20:00 every Sunday in August
  • Turn on a feature flag on the first/last three days of each month
  • Turn on a feature flag during 18:00~23:00 on Feb 14, Dec 24 and 25 every year
  • Turn on a feature flag on Dec 31 and Jan 1 every year.
    I think all of them are reasonable scenarios and we should support all of them.
    The current design is the simplest one which can support all of these scenarios.

In fact, the cron expression is the most compact format to describe the recurrence and it is a well-known and standardized format. We can invent some formats to describe the recurrence, but it would be better to use something standardized. (This discussion can be found in the comments of the design doc.)

If you go to see the AKS doc, you will find AKS has a feature called maintenance schedule: https://learn.microsoft.com/en-us/azure/aks/planned-maintenance.
This is a very similar scenario where user can configure some recurring time windows to let AKS cluster to upgrade.
The AKS team designs a quite complicated schema
Screenshot 2023-09-19 230001
The AKS team invents a very complex schema to descibe the recurring maintenance time window.
image
It will look like this.
AKS even does not support configuring this on the Portal. It can only be configured through CLI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I follow what you said here. I did NOT suggest we abandon the usage of cron expression. I am only suggesting we support one cron expression per Time Window filter instead of an array of them. Here is an example.

"EnhancedPipeline": {
    "EnabledFor": [
        {
            "Name": "Microsoft.TimeWindow",
            "Parameters": {
                "Start": "2023-09-06T00:00:00+08:00",
                "RecurrencePattern": "* 18-19 * * Mon-Fri" // 18:00-20:00 on weekdays
            }
        }
    ]
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if I want the recurrence happen during 18:30`20:30 every day?
You cannot represent 18:30-20:30 in a single cron.

Copy link
Member

@juniwang juniwang Sep 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scenario when the user want to activate a feature flag during 18:00-20:00 on every weekend and during 18:00-22:00 on holidays requires a list to specify recurring time windows with different durations.

I'm sure we can come up with even more complex scenarios. The question is how realistic they are. Outlook doesn't support infinite recurrence patterns, just one pattern at a time. Without the array, the parameters can be simplified to "Start", "End" and "RecurrencePattern". The UI design doesn't need an add button for more than one recurrence pattern anymore.

Using multiple timewindow filters may not be a good choice, since the feature flag may also use other filters(e.g. targeting), in this case, the "RequirementType" is "All".

We know the "RequirementType" doesn't support all possible logical operations. It was a conscious decision. It's a balance between complexity and the most common scenarios. The choice of that design shouldn't be the reason for us to complicate the design of each feature filter. We have a reasonable workaround. We know there are scenarios that won't be supported out-of-box no matter how. That's where customers should consider custom filters.

Hi @zhenlan, I thinks scenario like 18:30-20:30 everyday is a very common scenario, not something that rarely happens. And current design does not allow us to use one single cron to describe it. Since they are closely tied together, adding two cron expressions makes more sense to me than adding two TimeWindow filters. If we don't support cron array, probablly customers will have to add more Feature filters, which means inceasing size/complexity in another level.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juniwang Yes, we chatted offline, and I understand this now. This limitation imposes challenges on our portal to properly break down 18:30-20:30 into 3 crontabs and reconstruct it back from crontabs to a human-readable time duration. We need to rethink our design choices.

}
}
]
}
```
In UNIX, the usage of Crontab expression is to represent the time to execute a command. The Crontab serves as a schedule. The Crontab expression is designed to represent discrete time point rather than continuous time span. For example, the Cron job “* * * * * echo hi >> log” will execute “echo hi >> log” at the beginning of each minute.

However, when the time granularity is at minute-level, we can consider it as a continuous time intervals. For example, the Crontab expression “* 16 * * *” means “at every minute past the 16th hour of the day”. We can intuitively consider it as the time span 16:00-17:00. As a result, the recurring time window can be represented by a Crontab expression.

The Crontab format can be refered to the [linux man page](https://man7.org/linux/man-pages/man5/crontab.5.html). We allow the usage of ranges, lists and step values. Randomization is not supported.

The crontab format has five time fields separated by at least one blank:

```
{minute} {hour} {day-of-month} {month} {day-of-week}
```

| Crontab Expression Example | Recurring Time Window |
|--------|--------|
|* 16 * * 1-5 | 16:00-17:00 every weekday |
| * 6-9,16-19 * * * | 6:00-10:00 and 16:00-20:00 every day |
| * * 1,15 * 1 | the first and fifteenth day of each month, as well as on every Monday|
| * * * 8 5 | every Friday in August |
| * 18-21 25 12 * | 18:00-22:00 on Dec 25th |

The Crontab does not contain the information of UTC time offset.
- If the UTC offset is specified in `Start`, it will also be applied to Crontabs listed in `Filters`.
- If the UTC offset of `Start` is not specified and the UTC offset of `End` is specified, the UTC offset of `End` will be applied to Crontabs listed in `Filters`.
- Otherwise, we will use the default UTC offset: UTC+0:00.

In the above example, the `Start` is "2023-09-06T00:00:00+08:00", as a result, the UTC+8:00 will be also applied to the recurring time window filters: 18:00-20:00 on weekdays and 18:00-22:00 on weekends.
#### Microsoft.Targeting

This filter provides the capability to enable a feature for a target audience. An in-depth explanation of targeting is explained in the [targeting](./README.md#Targeting) section below. The filter parameters include an audience object which describes users, groups, excluded users/groups, and a default percentage of the user base that should have access to the feature. Each group object that is listed in the target audience must also specify what percentage of the group's members should have access. If a user is specified in the exclusion section, either directly or if the user is in an excluded group, the feature will be disabled. Otherwise, if a user is specified in the users section directly, or if the user is in the included percentage of any of the group rollouts, or if the user falls into the default rollout percentage then that user will have the feature enabled.
Expand Down
121 changes: 121 additions & 0 deletions src/Microsoft.FeatureManagement/FeatureFilters/Cron/CronExpression.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
//
using System;

namespace Microsoft.FeatureManagement.FeatureFilters.Cron
{
/// <summary>
/// The Cron expression with Minute, Hour, Day of month, Month, Day of week fields.
/// </summary>
internal class CronExpression
{
private static readonly int NumberOfFields = 5;

private readonly CronField _minute;
private readonly CronField _hour;
private readonly CronField _dayOfMonth;
private readonly CronField _month;
private readonly CronField _dayOfWeek;

private CronExpression(CronField minute, CronField hour, CronField dayOfMonth, CronField month, CronField dayOfWeek)
{
_minute = minute;
_hour = hour;
_dayOfMonth = dayOfMonth;
_month = month;
_dayOfWeek = dayOfWeek;
}

/// <summary>
/// Check whether the given expression can be parsed by a CronExpression.
/// If the expression is invalid, an ArgumentException will be thrown.
/// </summary>
/// <param name="expression">The expression to parse.</param>
/// <returns>A parsed CronExpression.</returns>
public static CronExpression Parse(string expression)
{
if (expression == null)
{
throw new ArgumentNullException(nameof(expression));
}

string InvalidCronExpressionErrorMessage = $"The provided Cron expression: '{expression}' is invalid.";

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like if we can avoid defining white space characters and instead use IsWhiteSpace

I suggest

var fields = new string[NumberOfFields];

int i = 0, pos = 0;

while (i < expression.Length)
{
    if (char.IsWhiteSpace(expression[i]))
    {
        continue;
    }

    if (pos >= fields.Length)
    {
        //
        // Invalid length
    }

    //
    // Grab fields here

    pos++;
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good!

var fields = new string[NumberOfFields];

int i = 0, pos = 0;

while (i < expression.Length)
{
if (char.IsWhiteSpace(expression[i]))
{
i++;

continue;
}

if (pos >= fields.Length)
{
throw new ArgumentException(InvalidCronExpressionErrorMessage, nameof(expression));
}

int start = i; // Start of a field

while (i < expression.Length && !char.IsWhiteSpace(expression[i]))
{
i++;
}

fields[pos] = expression.Substring(start, i - start);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even better if we can use ReadOnlySpan<char> to avoid allocating new strings.


pos++;
}

if (CronField.TryParse(CronFieldKind.Minute, fields[0], out CronField minute) &&
CronField.TryParse(CronFieldKind.Hour, fields[1], out CronField hour) &&
CronField.TryParse(CronFieldKind.DayOfMonth, fields[2], out CronField dayOfMonth) &&
CronField.TryParse(CronFieldKind.Month, fields[3], out CronField month) &&
CronField.TryParse(CronFieldKind.DayOfWeek, fields[4], out CronField dayOfWeek))
{
return new CronExpression(minute, hour, dayOfMonth, month, dayOfWeek);
}
else
{
throw new ArgumentException(InvalidCronExpressionErrorMessage, nameof(expression));
}
}

/// <summary>
/// Checks whether the Cron expression is satisfied by the given timestamp.
/// </summary>
/// <param name="time">The timestamp to check.</param>
/// <returns>True if the Cron expression is satisfied by the give timestamp, otherwise false.</returns>
public bool IsSatisfiedBy(DateTimeOffset time)
{
/*
The current time is said to be satisfied by the Cron expression when the 'minute', 'hour', and 'month of the year' fields match the current time,
and at least one of the two 'day' fields ('day of month', or 'day of week') match the current time.
If both 'day' fields are restricted (i.e., do not contain the "*" character), the current time will be considered as satisfied when it match either 'day' field and other fields.
If exactly one of 'day' fields are restricted, the current time will be considered as satisfied when it match both 'day' fields and other fields.
*/
bool isDayMatched;

if (!_dayOfMonth.MatchesAll && !_dayOfWeek.MatchesAll)
{
isDayMatched = (_dayOfMonth.Match((int)time.Day) || _dayOfWeek.Match((int)time.DayOfWeek)) &&
_month.Match((int)time.Month);
}
else
{
isDayMatched = _dayOfMonth.Match((int)time.Day) &&
_dayOfWeek.Match((int)time.DayOfWeek) &&
_month.Match((int)time.Month);
}

return isDayMatched &&
_hour.Match((int)time.Hour) &&
_minute.Match((int)time.Minute);
}
}
}
Loading