Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve filtering dates for specific operators #3362

Open
wants to merge 22 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

* Added `regexOption` and `caseSensitive` props to the `LogDisplayModel`. (Case-sensitive search
requires `hoist-core >= v16.2.0`).
* Improved filtering of `date` `fieldType` to end of day when operator is '>' or '<='. Developers
will need to indicate `localDate` as the `fieldSpec.fieldType` to trigger this behavior.

### 🎁 New Features

Expand Down
17 changes: 2 additions & 15 deletions admin/tabs/activity/tracking/ActivityTrackingModel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {FormModel} from '@xh/hoist/cmp/form';
import {GridModel, TreeStyle} from '@xh/hoist/cmp/grid';
import {HoistModel, LoadSpec, managed, XH} from '@xh/hoist/core';
import {Cube, CubeFieldSpec, FieldSpec} from '@xh/hoist/data';
import {fmtDate, fmtNumber} from '@xh/hoist/format';
import {fmtNumber} from '@xh/hoist/format';
import {action, computed, makeObservable} from '@xh/hoist/mobx';
import {LocalDate} from '@xh/hoist/utils/datetime';
import * as Col from '@xh/hoist/admin/columns';
Expand Down Expand Up @@ -129,20 +129,7 @@ export class ActivityTrackingModel extends HoistModel {
},
{
field: 'dateCreated',
example: 'YYYY-MM-DD',
valueParser: (v, op) => {
let ret = moment(v, ['YYYY-MM-DD', 'YYYYMMDD'], true);
if (!ret.isValid()) return null;

// Note special handling for '>' & '<=' queries.
if (['>', '<='].includes(op)) {
ret = moment(ret).endOf('day');
}

return ret.toDate();
},
valueRenderer: v => fmtDate(v),
ops: ['>', '>=', '<', '<=']
fieldType: 'localDate'
}
],
persistWith: this.persistWith
Expand Down
16 changes: 15 additions & 1 deletion admin/tabs/activity/tracking/detail/ActivityDetailModel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,21 @@ export class ActivityDetailModel extends HoistModel {
sortBy: 'dateCreated|desc',
colChooserModel: true,
enableExport: true,
filterModel: true,
filterModel: {
fieldSpecs: [
'username',
'category',
'msg',
'device',
'browser',
'userAgent',
'elapsed',
{
field: 'dateCreated',
fieldType: 'localDate'
}
]
},
exportOptions: {
columns: 'ALL',
filename: `${XH.appCode}-activity-detail`
Expand Down
5 changes: 5 additions & 0 deletions cmp/filter/FilterChooserFieldSpec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,11 @@ export class FilterChooserFieldSpec extends BaseFilterFieldSpec {
if (!valueParser && (this.fieldType === 'int' || this.fieldType === 'number')) {
return input => parseNumber(input);
}

// Default date parser if fieldSpec's fieldType does not match source field's fieldType.
if (!valueParser && this.isMismatchedDateFieldType) {
return (v, op) => this.parseDate(v, op);
}
return valueParser;
}

Expand Down
2 changes: 1 addition & 1 deletion data/Field.ts
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ export function parseFieldValue(
case 'date':
return isDate(val) ? val : new Date(val);
case 'localDate':
return isLocalDate(val) ? val : LocalDate.get(val);
return isLocalDate(val) ? val : LocalDate.from(val);
}

throw XH.exception(`Unknown field type '${type}'`);
Expand Down
37 changes: 31 additions & 6 deletions data/filter/BaseFilterFieldSpec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import {HoistBase} from '@xh/hoist/core';
import {Field, Store, FieldFilter, FieldType, genDisplayName, View} from '@xh/hoist/data';
import {isEmpty} from 'lodash';
import moment from 'moment';
import {FieldFilterOperator} from './Types';

export interface BaseFilterFieldSpecConfig {
Expand Down Expand Up @@ -74,7 +75,9 @@ export abstract class BaseFilterFieldSpec extends HoistBase {
this.forceSelection = forceSelection ?? false;
this.values = values ?? (this.isBoolFieldType ? [true, false] : null);
this.hasExplicitValues = !isEmpty(this.values);
this.enableValues = this.hasExplicitValues || (enableValues ?? this.isEnumerableByDefault);
this.enableValues =
!this.isMismatchedDateFieldType &&
(this.hasExplicitValues || (enableValues ?? this.isEnumerableByDefault));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we said we don't want to enable the Values tab in this situation, right? It could be the dev's responsibility to set enableValues to false on the fieldSpec, but I think the default behavior of not showing it for dates makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to enableValues if we could find a clean way to restrict the values to the unique local dates in the dataset, rather than the full date timestamp. If this proves too complex however perhaps its not worth it.

}

/** Full Field derived from source. */
Expand Down Expand Up @@ -116,11 +119,6 @@ export abstract class BaseFilterFieldSpec extends HoistBase {
return this.filterType === 'collection';
}

get isDateBasedFieldType(): boolean {
const {fieldType} = this;
return fieldType === 'date' || fieldType === 'localDate';
}

get isNumericFieldType(): boolean {
const {fieldType} = this;
return fieldType === 'int' || fieldType === 'number';
Expand All @@ -130,6 +128,20 @@ export abstract class BaseFilterFieldSpec extends HoistBase {
return this.fieldType === 'bool';
}

get isDateBasedFieldType(): boolean {
const {fieldType} = this;
return fieldType === 'date' || fieldType === 'localDate';
}

/**
* Detects difference in date-based fieldType defined on fieldSpec and source field, indicating
* custom value parsing for > and <= operators.
*/
get isMismatchedDateFieldType(): boolean {
const sourceField = this.source.fields.find(f => f.name === this.field);
return this.fieldType === 'localDate' && sourceField.type === 'date';
}

loadValues() {
if (!this.hasExplicitValues && this.enableValues) {
this.loadValuesFromSource();
Expand All @@ -149,6 +161,18 @@ export abstract class BaseFilterFieldSpec extends HoistBase {
);
}

parseDate(v: any, op: FieldFilterOperator): Date {
let ret = moment(v, ['YYYY-MM-DD', 'YYYYMMDD'], true);
if (!ret.isValid()) return null;

// Note special handling for '>' & '<=' queries.
if (['>', '<='].includes(op)) {
ret = ret.endOf('day');
}

return ret.toDate();
}

//------------------------
// Abstract
//------------------------
Expand All @@ -165,6 +189,7 @@ export abstract class BaseFilterFieldSpec extends HoistBase {
private getDefaultOperators(): FieldFilterOperator[] {
if (this.isBoolFieldType) return ['='];
if (this.isCollectionType) return ['includes', 'excludes'];
if (this.isMismatchedDateFieldType) return ['>', '>=', '<', '<='];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ensures the FilterChooser and GridFilter will offer the same operators for this field, which is not currently the case in the ActivityTracking example - only the GridFilter offers equals/does not equal.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should be using the ActivityTracking tab as gospel for the operators we want to support - it would be better if we could support the full range of operators normally available for localDates.

That said, to support e.g. date = 2023-05-31, we need to produce a compound "between" filter like date >= 2023-05-31 00:00:00 AND date < 2023-06-01 00:00:00. Honestly not sure its worth the complexity :|

return this.isValueType
? ['=', '!=', 'like', 'not like', 'begins', 'ends']
: ['>', '>=', '<', '<=', '=', '!='];
Expand Down
4 changes: 3 additions & 1 deletion desktop/cmp/grid/impl/filter/custom/CustomRow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,9 @@ const inputField = hoistCmp.factory<CustomRowModel>(({model}) => {
} else if (fieldSpec.isDateBasedFieldType) {
return dateInput({
...props,
valueType: fieldSpec.fieldType as 'localDate' | 'date'
valueType: fieldSpec.isMismatchedDateFieldType
? 'date'
: (fieldSpec.fieldType as 'localDate' | 'date')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that if you picked a date and then closed and reopened the popover, the dateInput wouldn't render the value, due to the difference in fieldTypes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense - we need to input to be able to read dates since that is what the filter is actually using 👍

});
} else if (fieldSpec.supportsSuggestions(op as FieldFilterOperator)) {
return select({
Expand Down
4 changes: 3 additions & 1 deletion desktop/cmp/grid/impl/filter/custom/CustomRowModel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export class CustomRowModel extends HoistModel {
/** FieldFilter config output of this row. */
@computed.struct
get value(): FieldFilterSpec {
const {field} = this.fieldSpec;
const {field, isMismatchedDateFieldType} = this.fieldSpec;

let op = this.op,
value = this.inputVal;
Expand All @@ -39,6 +39,8 @@ export class CustomRowModel extends HoistModel {
} else if (op === 'not blank') {
op = '!=';
value = null;
} else if (isMismatchedDateFieldType) {
value = this.fieldSpec.parseDate(value, op);
} else if (isNil(value)) {
return null;
}
Expand Down