[RFC] JSON result comparison with regular expression support #224
Comments
For this case, I prefer using json schema.
I think mixing json and regex is unreadable but your idea with
The next version is ready, I publish it when a sufficient number of person ask me to publish it. I can wait your PR and publish the 3.0 version after. |
Yeah but you cannot check the values, only the keys.
Yeah it‘s just another tool you can use. You should try to choose wisely and find the best option :) I‘ll work on a PR then :) |
Of course you can: http://json-schema.org/example2.html#the-diskuuid-storage-type |
|
I think you don't get what I'm trying to test. The UUID regex test here is not the main advantage. I could've implemented a keyword like "uuid": "---placeholder---",to accept whatever value in there. The schema validation would allow allow to test for a valid UUID, yes, but I cannot test all the rest of the response to check for the values. {
"@context": "\/contexts\/Organisation",
"@id": "\/organisations\/39C147DE-C84B-48BF-9D38-4F66202D1080",
"@type": "Organisation",
"title": "Acme",
"street": null
}I want to do the following assertions:
To do that, as of today, I need to do this: Then the response should be in JSON
And the JSON should be valid according to this schema:
"""
{
"type": "object",
"$schema": "http://json-schema.org/draft-03/schema",
"required":true,
"properties": {
"@context": {
"type": "string",
"required":true
},
"@id": {
"type": "string",
"pattern": "^\\/organisations\\/[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$",
"required":true
},
"@type": {
"type": "string",
"required":true
},
"title": {
"type": "string",
"required":true
},
"street": {
{"type": ["string", "null"]},
"required":true
}
}
}
"""
And the JSON nodes should be equal to:
| @context | \/contexts\/Organisation |
| @type | Organisation |
| title | Acme |
And the JSON node "street" should be nullAll of this is needed to do all my desired assertions. And the JSON should be equal to considering regular expressions:
"""
{
"@context": "\/contexts\/Organisation",
"@id": "!regex:@\\/organisations\\/[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}@",
"@type": "Organisation",
"title": "Acme",
"street": null
}
""" |
|
@Toflar For your desired assertions, you actually can do this (note And the JSON should be valid according to this schema:
"""
{
"type": "object",
"$schema": "http://json-schema.org/draft-03/schema",
"required":true,
"properties": {
"@context": {
"enum": ["\/contexts\/Organisation"],
"required":true
},
"@id": {
"type": "string",
"pattern": "^\\/organisations\\/[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$",
"required":true
},
"@type": {
"enum": ["Organisation"],
"required":true
},
"title": {
"enum": ["Acme"],
"required":true
},
"street": {
"enum": [null],
"required":true
}
},
"additionalProperties": false
}
"""which you could also write like this: And the JSON should be valid according to this schema:
"""
{
"$schema": "http://json-schema.org/draft-03/schema",
"required": true,
"type": "object",
"additionalProperties": false,
"properties": {
"@context": {"required":true, "enum":["\/contexts\/Organisation"]},
"@id": {"required":true, "type":"string", "pattern":"^\\/organisations\\/[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$"},
"@type": {"required":true, "enum":["Organisation"]},
"title": {"required":true, "enum":["Acme"]},
"street": {"required":true, "enum":[null]}
}
}
"""which is even closer to your " |
|
Yeah, my |
|
Do you mean that you eventually implemented it? (I couldn't find any PR from you) |
|
Yeah I've added a custom step in my own context to do this and it works but it's actually a chunk of ugly code that would require polishing, testing etc. There you go: /**
* @Then the JSON should be equal to considering regular expressions:
*/
public function theJsonShouldBeEqualToConsideringRegularExpressions(PyStringNode $content)
{
$expected = json_decode((string) $content, true);
$result = json_decode((string) $this->httpCallResultPool->getResult()->getValue(), true);
if (!$this->checkIfTwoArraysHaveSameStructure($result, $expected)) {
throw new \Exception(
sprintf('Cannot compare regular expressions as the JSON does not even have the same structure (must already fail!). Result was "%s", expected was "%s".',
json_encode($result),
json_encode($expected)
));
}
$expected = $this->recursiveHandleRegexMatching($expected, $result);
$this->assert($result === $expected, sprintf(
'The JSON result "%s" does not match the expected "%s".',
json_encode($result),
json_encode($expected)
));
}
private function checkIfTwoArraysHaveSameStructure(array $array1, array $array2): bool
{
if (0 !== \count(array_diff_key($array1, $array2)) || 0 !== \count(array_diff_key($array2, $array1))) {
return false;
}
foreach ($array1 as $k => $v) {
if (\is_array($v)) {
if (!\is_array($array2[$k])) {
return false;
}
if (!$this->checkIfTwoArraysHaveSameStructure($v, $array2[$k])) {
return false;
}
}
}
return true;
}
/**
* @throws Exception
*/
private function recursiveHandleRegexMatching(array $expected, array $result): array
{
if (0 === \count($expected)) {
return $expected;
}
foreach ($expected as $k => $v) {
if (\is_string($k) && false !== strpos($k, '!regex:')) {
throw new Exception('Cannot assert regex in JSON keys.');
}
if (\is_array($v)) {
$expected[$k] = $this->recursiveHandleRegexMatching($v, $result[$k]);
continue;
}
if (null === $v || !\is_string($v)) {
continue;
}
preg_match_all('/!regex:((?:[^"\\\\]|\\\\.)*)/', $v, $matches, PREG_OFFSET_CAPTURE);
if (0 === \count($matches[0])) {
continue;
}
$rgxp = $matches[1][0][0];
$offset = $matches[0][0][1];
// Assert the match fulfills the regex
if (1 !== preg_match($rgxp, $result[$k])) {
throw new \Exception(sprintf(
'The regex "%s" does not match value "%s".',
$rgxp,
$result[$k]
));
}
// Replace inner match with original content to then globally compare
// if the rest of the content matches
$start = substr($v, 0, $offset);
$end = substr($v, $offset + \strlen($rgxp) + 7); // 7 = !regex:
$v = $start.$result[$k].$end;
$expected[$k] = $v;
}
return $expected;
}
/**
* @throws Exception
*/
private function assert(bool $test, string $message)
{
if (false === $test) {
throw new \Exception($message);
}
} |
|
Thanks for sharing anyway 🙂 |
Hey guys,
I thought I'll open an issue for the general concept before I work on a PR for this.
I very often find myself using something like
because I cannot use "And the JSON should be equal to" as I get a new UUID every time the test runs again. I find it very complicated to write these tables and copy all the values just because I cannot compare the whole document. Also, because the table uses strings to compare, I have to use a different expression to compare
nullvalues (And the JSON node "street" should be null), as if I'd addnullto the table it would compare it to the string value"null"instead.That's why I came up with the idea of having a special regular expression comparison within the document.
Here's what I'm thinking of:
So there's a special
!regex:control string that allows you to validate a value for a custom regular expression within". Internally it would evaluate the regular expression and fail if that regular expression does not match. If it does match, the value gets replaced with the value of the original JSON so after testing/replacing all!regex:instructions, we can normally compare the two JSON documents. That would simplify a lot of my tests and make them better because with the current version, what happens if I add a new property? The tests would all be green. In the new variant it would fail because I'm not expecting to get a new property 😄Do you like that concept? If so, any preference regarding
!regex:or would that be fine? And if I'd work on a PR for that, how fast could that be released in a new version? (asking because I'd need it for a project).The text was updated successfully, but these errors were encountered: