Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cmd parser with commands #2733

Draft
wants to merge 74 commits into
base: v5
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
605d5c1
implement parser conversion with firs GET command implemented
sjpotter Mar 13, 2024
14b6ad7
missed handling for multi in cluster
sjpotter Mar 13, 2024
761542b
fix all multis for preserve
sjpotter Mar 13, 2024
c4328b4
change how prefix is handled for script/functions
sjpotter Mar 21, 2024
2fd1585
simplify parser, only have a single parser.
sjpotter Mar 25, 2024
df228c9
nit
sjpotter Mar 30, 2024
d96aacd
move some usage of RedisArgument arrays to ReadonlyArray
sjpotter Mar 25, 2024
657167c
address review comments
sjpotter Apr 3, 2024
93f897d
Add all commands to new parser syntax for basic/graph/json
sjpotter Apr 4, 2024
3892f92
HSCAN VALUES support (v5)
sjpotter May 15, 2024
54d2c0d
enable module commands to be marked as unstable and require clients t…
sjpotter May 26, 2024
5806f68
update all search commands to v5 / unstable interface
sjpotter May 28, 2024
0f2e8d5
update architecture to how we throw an error.
sjpotter May 30, 2024
f8a300f
revert changes needed for resp3 erroring with multi, as now handled b…
sjpotter May 30, 2024
1c77ab0
bring all modules / all commands into v5
sjpotter Jun 2, 2024
f13f47a
leibele's type changing and cleaning up disabling of type mapping
sjpotter Jun 3, 2024
5daf11b
move tuples to map to requiring simple string for map key
sjpotter Jun 3, 2024
a91dded
missed on removal of unstable flag
sjpotter Jun 3, 2024
9baec20
reove ignoreTypeMapping per offline discussion
sjpotter Jun 5, 2024
33adc6f
revert 2 buffer/simple string changes from experimentation
sjpotter Jun 5, 2024
8e7f11c
make resp2/resp3 have same return value
sjpotter Jun 5, 2024
465911e
cleanup
leibale Jun 6, 2024
72e2673
Merge remote-tracking branch 'upstream/v5' into cmd-parser-with-commands
sjpotter Jun 19, 2024
e53da7e
keep in sync with v5
sjpotter Jun 19, 2024
f08e94f
nit
leibale Jun 25, 2024
6e60233
address leibele's comments + more time series stuff I found in process
sjpotter Jun 26, 2024
6c34d6b
more work on review comments
sjpotter Jun 27, 2024
d896b09
attempt to implement leibele's suggestion to clean up code
sjpotter Jul 8, 2024
7327f5e
wip - changes of personal review
sjpotter Jul 10, 2024
9a85b10
more cleanup
sjpotter Jul 17, 2024
e374a08
add field expiration commands to parser set
sjpotter Jul 29, 2024
f555c4d
add command/command info and xread/xreadgroup to v5 support
sjpotter Aug 11, 2024
17e4dd5
update tests to actually work right
sjpotter Aug 11, 2024
629e790
hscan values isn't part of 7.4
sjpotter Aug 11, 2024
59b77a0
wip - removing transformArguments
sjpotter Aug 11, 2024
418f79e
add hash field expiration commands ported over from v4
sjpotter Aug 11, 2024
2405923
move as much of module ocmmands as I can to parser - depends on v5 un…
sjpotter Aug 12, 2024
9dde283
Merge remote-tracking branch 'upstream/v5' into v5-search-broken
sjpotter Aug 14, 2024
4dea5cb
revert cluster broadcasting todo
sjpotter Aug 14, 2024
4f74a86
update test.yaml to have 7.4
sjpotter Aug 15, 2024
f0af1eb
redo time series commands to work with resp3 correctly
sjpotter Aug 15, 2024
4bee30f
unwrap json.type in resp3 from extra array
sjpotter Aug 15, 2024
77592d2
add comment to json.type for what we are doing
sjpotter Aug 19, 2024
60499fc
update time series for resp3 to work consistently
sjpotter Aug 19, 2024
2ac0964
move bloom* info commands to not error out if map type mapping is used
sjpotter Aug 19, 2024
aef901e
tweak unstable search module flag name + add proper error message
sjpotter Aug 19, 2024
60fd22c
have topk info retur a DoubleReply in resp3
sjpotter Aug 19, 2024
4066097
fix xread/xreadgroup for map->array type mapping
sjpotter Aug 19, 2024
2b2bbb5
handle xread/xreadgroup with typing.
sjpotter Aug 19, 2024
9ac2ee8
redo resp2 "map" types to return Objects to be resp3 default compatible
sjpotter Aug 22, 2024
0337343
adding "type mapping" to transformReply
sjpotter Aug 25, 2024
f3013f1
move all 'DoubleReply' in resp2 to new type mapping system
sjpotter Aug 25, 2024
11bd5d2
fix bug + tests
sjpotter Aug 25, 2024
142dfb4
change Buffer('a') to Syumbol() to force TypeError
sjpotter Aug 25, 2024
d42fd70
remove some (unnecessary) jsdoc
leibale Sep 2, 2024
78f1de6
revert and comment out "connect, ready and end events" test
leibale Sep 3, 2024
e57ca82
move info commands to discussed format
sjpotter Sep 25, 2024
bf8a93c
Merge remote-tracking branch 'origin/v5-search-broken' into v5-search…
sjpotter Sep 25, 2024
52bd5c0
fix bloom info command update
sjpotter Sep 25, 2024
3d0485b
fix cms.info test
sjpotter Sep 25, 2024
0b7bac6
wip
sjpotter Sep 25, 2024
03acaa3
Merge branch 'v5-search-broken' into cmd-parser-with-commands
sjpotter Sep 25, 2024
c9f08e0
big refactor
sjpotter Sep 26, 2024
b375570
fixes for scripting, tests
sjpotter Sep 26, 2024
64dd0b3
fix info tests and fix setting preserve attribute
sjpotter Sep 26, 2024
93538f4
tweak to make functions internal
sjpotter Sep 27, 2024
30abc9f
nit
sjpotter Sep 27, 2024
e3eabcc
nits
sjpotter Sep 27, 2024
9adac9d
rename unstableResp3SearchModule cofnig + xread/group support
sjpotter Sep 29, 2024
9d891c7
Merge branch 'v5-search-broken' into cmd-parser-with-commands
sjpotter Sep 29, 2024
22823b1
nit - cleanup xadd helper function usage
sjpotter Oct 1, 2024
1673d06
nit
sjpotter Oct 2, 2024
ea51f39
revert to be v4 compat for xread/xreadgroup
sjpotter Oct 6, 2024
54d7748
Merge remote-tracking branch 'origin/v5-search-broken' into cmd-parse…
sjpotter Oct 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
fail-fast: false
matrix:
node-version: ['18', '20']
redis-version: ['5', '6.0', '6.2', '7.0', '7.2']
redis-version: ['5', '6.0', '6.2', '7.0', '7.2', '7.4']
steps:
- uses: actions/checkout@v4
with:
Expand Down
34 changes: 30 additions & 4 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion packages/bloom/lib/commands/bloom/ADD.spec.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
import { strict as assert } from 'node:assert';
import testUtils, { GLOBAL } from '../../test-utils';
import ADD from './ADD';
import { parseArgs } from '@redis/client/dist/lib/commands/generic-transformers';

describe('BF.ADD', () => {
it('transformArguments', () => {
assert.deepEqual(
ADD.transformArguments('key', 'item'),
parseArgs(ADD, 'key', 'item'),
['BF.ADD', 'key', 'item']
);
});
Expand Down
8 changes: 5 additions & 3 deletions packages/bloom/lib/commands/bloom/ADD.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
import { CommandParser } from '@redis/client/dist/lib/client/parser';
import { RedisArgument, Command } from '@redis/client/dist/lib/RESP/types';
import { transformBooleanReply } from '@redis/client/dist/lib/commands/generic-transformers';

export default {
FIRST_KEY_INDEX: 1,
IS_READ_ONLY: false,
transformArguments(key: RedisArgument, item: RedisArgument) {
return ['BF.ADD', key, item];
parseCommand(parser: CommandParser, key: RedisArgument, item: RedisArgument) {
parser.push('BF.ADD');
parser.pushKey(key);
parser.push(item);
},
transformReply: transformBooleanReply
} as const satisfies Command;
3 changes: 2 additions & 1 deletion packages/bloom/lib/commands/bloom/CARD.spec.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
import { strict as assert } from 'node:assert';
import testUtils, { GLOBAL } from '../../test-utils';
import CARD from './CARD';
import { parseArgs } from '@redis/client/dist/lib/commands/generic-transformers';

describe('BF.CARD', () => {
it('transformArguments', () => {
assert.deepEqual(
CARD.transformArguments('bloom'),
parseArgs(CARD, 'bloom'),
['BF.CARD', 'bloom']
);
});
Expand Down
7 changes: 4 additions & 3 deletions packages/bloom/lib/commands/bloom/CARD.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import { CommandParser } from '@redis/client/dist/lib/client/parser';
import { RedisArgument, NumberReply, Command } from '@redis/client/dist/lib/RESP/types';

export default {
FIRST_KEY_INDEX: 1,
IS_READ_ONLY: true,
transformArguments(key: RedisArgument) {
return ['BF.CARD', key];
parseCommand(parser: CommandParser, key: RedisArgument) {
parser.push('BF.CARD');
parser.pushKey(key);
},
transformReply: undefined as unknown as () => NumberReply
} as const satisfies Command;
3 changes: 2 additions & 1 deletion packages/bloom/lib/commands/bloom/EXISTS.spec.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
import { strict as assert } from 'node:assert';
import testUtils, { GLOBAL } from '../../test-utils';
import EXISTS from './EXISTS';
import { parseArgs } from '@redis/client/dist/lib/commands/generic-transformers';

describe('BF.EXISTS', () => {
it('transformArguments', () => {
assert.deepEqual(
EXISTS.transformArguments('key', 'item'),
parseArgs(EXISTS, 'key', 'item'),
['BF.EXISTS', 'key', 'item']
);
});
Expand Down
8 changes: 5 additions & 3 deletions packages/bloom/lib/commands/bloom/EXISTS.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
import { CommandParser } from '@redis/client/dist/lib/client/parser';
import { RedisArgument, Command } from '@redis/client/dist/lib/RESP/types';
import { transformBooleanReply } from '@redis/client/dist/lib/commands/generic-transformers';

export default {
FIRST_KEY_INDEX: 1,
IS_READ_ONLY: true,
transformArguments(key: RedisArgument, item: RedisArgument) {
return ['BF.EXISTS', key, item];
parseCommand(parser: CommandParser, key: RedisArgument, item: RedisArgument) {
parser.push('BF.EXISTS');
parser.pushKey(key);
parser.push(item);
},
transformReply: transformBooleanReply
} as const satisfies Command;
13 changes: 7 additions & 6 deletions packages/bloom/lib/commands/bloom/INFO.spec.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
import { strict as assert } from 'node:assert';
import testUtils, { GLOBAL } from '../../test-utils';
import INFO from './INFO';
import { parseArgs } from '@redis/client/dist/lib/commands/generic-transformers';

describe('BF.INFO', () => {
it('transformArguments', () => {
assert.deepEqual(
INFO.transformArguments('bloom'),
parseArgs(INFO, 'bloom'),
['BF.INFO', 'bloom']
);
});
Expand All @@ -17,10 +18,10 @@ describe('BF.INFO', () => {
]);

assert.equal(typeof reply, 'object');
assert.equal(reply.capacity, 100);
assert.equal(typeof reply.size, 'number');
assert.equal(typeof reply.numberOfFilters, 'number');
assert.equal(typeof reply.numberOfInsertedItems, 'number');
assert.equal(typeof reply.expansionRate, 'number');
assert.equal(reply['Capacity'], 100);
assert.equal(typeof reply['Size'], 'number');
assert.equal(typeof reply['Number of filters'], 'number');
assert.equal(typeof reply['Number of items inserted'], 'number');
assert.equal(typeof reply['Expansion rate'], 'number');
}, GLOBAL.SERVERS.OPEN);
});
26 changes: 20 additions & 6 deletions packages/bloom/lib/commands/bloom/INFO.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,25 @@
import { RedisArgument, Command } from '@redis/client/dist/lib/RESP/types';
import { CommandParser } from '@redis/client/dist/lib/client/parser';
import { RedisArgument, Command, UnwrapReply, NullReply, NumberReply, TuplesToMapReply, Resp2Reply, SimpleStringReply, TypeMapping } from '@redis/client/dist/lib/RESP/types';
import { transformInfoV2Reply } from '.';

export type BfInfoReplyMap = TuplesToMapReply<[
[SimpleStringReply<'Capacity'>, NumberReply],
[SimpleStringReply<'Size'>, NumberReply],
[SimpleStringReply<'Number of filters'>, NumberReply],
[SimpleStringReply<'Number of items inserted'>, NumberReply],
[SimpleStringReply<'Expansion rate'>, NullReply | NumberReply]
]>;

export default {
FIRST_KEY_INDEX: 1,
IS_READ_ONLY: true,
transformArguments(key: RedisArgument) {
return ['BF.INFO', key];
parseCommand(parser: CommandParser, key: RedisArgument) {
parser.push('BF.INFO');
parser.pushKey(key);
},
// TODO
transformReply: undefined as unknown as () => any
transformReply: {
2: (reply: UnwrapReply<Resp2Reply<BfInfoReplyMap>>, _, typeMapping?: TypeMapping): BfInfoReplyMap => {
return transformInfoV2Reply<BfInfoReplyMap>(reply, typeMapping);
},
3: undefined as unknown as () => BfInfoReplyMap
}
} as const satisfies Command;
15 changes: 8 additions & 7 deletions packages/bloom/lib/commands/bloom/INSERT.spec.ts
Original file line number Diff line number Diff line change
@@ -1,54 +1,55 @@
import { strict as assert } from 'node:assert';
import testUtils, { GLOBAL } from '../../test-utils';
import INSERT from './INSERT';
import { parseArgs } from '@redis/client/dist/lib/commands/generic-transformers';

describe('BF.INSERT', () => {
describe('transformArguments', () => {
it('simple', () => {
assert.deepEqual(
INSERT.transformArguments('key', 'item'),
parseArgs(INSERT, 'key', 'item'),
['BF.INSERT', 'key', 'ITEMS', 'item']
);
});

it('with CAPACITY', () => {
assert.deepEqual(
INSERT.transformArguments('key', 'item', { CAPACITY: 100 }),
parseArgs(INSERT, 'key', 'item', { CAPACITY: 100 }),
['BF.INSERT', 'key', 'CAPACITY', '100', 'ITEMS', 'item']
);
});

it('with ERROR', () => {
assert.deepEqual(
INSERT.transformArguments('key', 'item', { ERROR: 0.01 }),
parseArgs(INSERT, 'key', 'item', { ERROR: 0.01 }),
['BF.INSERT', 'key', 'ERROR', '0.01', 'ITEMS', 'item']
);
});

it('with EXPANSION', () => {
assert.deepEqual(
INSERT.transformArguments('key', 'item', { EXPANSION: 1 }),
parseArgs(INSERT, 'key', 'item', { EXPANSION: 1 }),
['BF.INSERT', 'key', 'EXPANSION', '1', 'ITEMS', 'item']
);
});

it('with NOCREATE', () => {
assert.deepEqual(
INSERT.transformArguments('key', 'item', { NOCREATE: true }),
parseArgs(INSERT, 'key', 'item', { NOCREATE: true }),
['BF.INSERT', 'key', 'NOCREATE', 'ITEMS', 'item']
);
});

it('with NONSCALING', () => {
assert.deepEqual(
INSERT.transformArguments('key', 'item', { NONSCALING: true }),
parseArgs(INSERT, 'key', 'item', { NONSCALING: true }),
['BF.INSERT', 'key', 'NONSCALING', 'ITEMS', 'item']
);
});

it('with CAPACITY, ERROR, EXPANSION, NOCREATE and NONSCALING', () => {
assert.deepEqual(
INSERT.transformArguments('key', 'item', {
parseArgs(INSERT, 'key', 'item', {
CAPACITY: 100,
ERROR: 0.01,
EXPANSION: 1,
Expand Down
Loading