test parser against various policy sources

[vgramer]

Description

There is already a lot of tests on the parser but several bugs in the grammar have been found (#74 #67 #64 #59 ). AFAIK there is no way to be sure at 100% that parser is compliant but we can test it against various policy sources to try to find bug

Policy sources:

• open-policy-agent/contrib
• open-policy-agent/conftest
• open-policy-agent/gatekeeper-library
• anderseknert/opal

If you find more relevant sources, please comment on this issue.

TODO

1. brut force approach, just test the parser against policy to try to find bugs
   • fix Grammar: imports could be defined between rule  #76
   • fix Grammar: infix operation are not allowed as value of dict  #77
   • fix Grammar: infix operation should be allowed in else statement #78
   • fix Grammar: array index could be complex expression #79
2. improve test suite by adding "cases" that are currently valid for the parser but include in tests

[anderseknert]

Some quirky functions and more here https://github.com/anderseknert/opal :)

[vgramer]

I have tested the parser against the different projects.

Test Results

145 files have been tested. 5 are not compliant with the parser but should be.

open-policy-agent/contrib

Parsing fails on the following files:

• api_authz/docker/policy/api_authz.rego: parser does not allow to defined import between rules. Issue: Grammar: imports could be defined between rule  #76
• cloud-foundry/polices/pas/cert/certificate.rego and cloud-foundry/polices/pas/diego/diego.rego: infix operation are not allowed as value of dict. Issue: Grammar: infix operation are not allowed as value of dict  #77

open-policy-agent/conftest

Tests Ok

open-policy-agent/gatekeeper-library

Tests Ok

anderseknert/opal

Parsing fails on the following files:

• lib/math.rego infix operation should be allowed in else statement. Issue Grammar: infix operation should be allowed in else statement #78
• lib/lists.rego array index could be complex expression Issue Grammar: array index could be complex expression #79