Skip to content
This repository was archived by the owner on Apr 25, 2024. It is now read-only.

Regression new updated expected output #1044

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
project = 'pyk'
author = 'Runtime Verification, Inc'
copyright = '2024, Runtime Verification, Inc'
version = '0.1.755'
release = '0.1.755'
version = '0.1.756'
release = '0.1.756'

# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
Expand Down
2 changes: 1 addition & 1 deletion package/version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.1.755
0.1.756
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "pyk"
version = "0.1.755"
version = "0.1.756"
description = ""
authors = [
"Runtime Verification, Inc. <[email protected]>",
Expand Down
20 changes: 12 additions & 8 deletions regression-new/bad-flags/no-flags.out
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
usage: pyk kompile [-h] [--verbose] [--debug] [--warnings WARNINGS] [-w2e] [-I INCLUDES]
[--main-module MAIN_MODULE] [--syntax-module SYNTAX_MODULE]
[--md-selector MD_SELECTOR] [--output-definition DEFINITION_DIR] [--backend BACKEND]
[--type-inference-mode TYPE_INFERENCE_MODE] [--emit-json] [-ccopt CCOPTS]
[--no-llvm-kompile] [--with-llvm-library] [--enable-llvm-debug]
[--llvm-kompile-type LLVM_KOMPILE_TYPE] [--llvm-kompile-output LLVM_KOMPILE_OUTPUT]
[--read-only-kompiled-directory] [-O0] [-O1] [-O2] [-O3] [--enable-search]
[--coverage] [--gen-bison-parser] [--gen-glr-bison-parser] [--bison-lists]
usage: pyk kompile [-h] [--verbose] [--debug] [--warnings WARNINGS] [-w2e]
[-I INCLUDES] [--main-module MAIN_MODULE]
[--syntax-module SYNTAX_MODULE] [--md-selector MD_SELECTOR]
[--output-definition DEFINITION_DIR] [--backend BACKEND]
[--type-inference-mode TYPE_INFERENCE_MODE] [--emit-json]
[-ccopt CCOPTS] [--no-llvm-kompile] [--with-llvm-library]
[--enable-llvm-debug]
[--llvm-kompile-type LLVM_KOMPILE_TYPE]
[--llvm-kompile-output LLVM_KOMPILE_OUTPUT]
[--read-only-kompiled-directory] [-O0] [-O1] [-O2] [-O3]
[--enable-search] [--coverage] [--gen-bison-parser]
[--gen-glr-bison-parser] [--bison-lists]
[--llvm-proof-hint-instrumentation] [--no-exc-wrap]
main_file
pyk kompile: error: the following arguments are required: main_file
73 changes: 38 additions & 35 deletions regression-new/cell-sort-haskell/1.test.out
Original file line number Diff line number Diff line change
@@ -1,35 +1,38 @@
<T>
<k>
"done" ~> .K
</k>
<nextFunction>
3
</nextFunction>
<functions>
<function>
<fId>
0
</fId>
</function> <function>
<fId>
1
</fId>
</function> <function>
<fId>
2
</fId>
</function>
</functions>
<contracts>
<contract>
<cName>
foo ~> .K
</cName>
<cFunctions>
f1 |-> 0
f2 |-> 1
f3 |-> 2
</cFunctions>
</contract>
</contracts>
</T>
<generatedTop>
<T>
<k>
"done" ~> .K
</k>
<nextFunction>
3
</nextFunction>
<functions>
<function>
<fId>
0
</fId>
</function> <function>
<fId>
2
</fId>
</function> <function>
<fId>
1
</fId>
</function>
</functions>
<contracts>
<contract>
<cName>
foo ~> .K
</cName>
<cFunctions>
f1 |-> 0 f3 |-> 2 f2 |-> 1
</cFunctions>
</contract>
</contracts>
</T>
<generatedCounter>
0
</generatedCounter>
</generatedTop>
29 changes: 28 additions & 1 deletion regression-new/checkClaimError/rule-spec.k.out
Original file line number Diff line number Diff line change
@@ -1,6 +1,33 @@
[Error] Compiler: Only claims and simplification rules are allowed in proof modules.
[Error] Compiler: Only claims and simplification rules are allowed in proof
modules.
Source(rule-spec.k)
Location(6,10,6,43)
6 | rule <k> doIt(foo) => doIt(0) ... </k>
. ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[Error] Compiler: Had 1 structural errors.
Traceback (most recent call last):
Comment on lines -1 to +8
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess for these errors we can:

  • Add option --no-wrap-exc to pyk prove ..., and pass it through to kprove --dry-run ...
  • Implement custom error handling for pyk prove ... similar to what we do for kompile

The only challenge is that here we are using a temporary file as the JSON dump, and that will show up in the error message. So I guess we also need to:

  • Allow specifying --tmp-dir ... for pyk prove ..., and use that directory for dumping the temporary files.
  • Use a more deterministic filename for the temporary JSON file we dump.

OR

  • Dump the JSON in a file immediately next to the spec file as a convention.

OR

  • Strip that line of output in the testing harness.

File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 105, in _kprove
return run_process(run_args, logger=_LOGGER, env=env, check=check)
File "/home/dev/src/pyk/src/pyk/utils.py", line 451, in run_process
res.check_returncode()
File "/usr/lib/python3.10/subprocess.py", line 457, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '('kprove', 'rule-spec.k', '--definition', 'errorClaim-kompiled', '--output', 'json', '--type-inference-mode', 'checked', '--dry-run', '--emit-json-spec', '/tmp/tmppacrxgx1')' returned non-zero exit status 113.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/dev/src/pyk/src/pyk/__main__.py", line 90, in main
execute(options)
File "/home/dev/src/pyk/src/pyk/__main__.py", line 252, in exec_prove
proofs = kprove.prove_rpc(options=options)
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 380, in prove_rpc
all_claims = self.get_claims(
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 425, in get_claims
flat_module_list = self.get_claim_modules(
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 400, in get_claim_modules
_kprove(
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 107, in _kprove
raise RuntimeError(
RuntimeError: ('Command kprove exited with code 113 for: rule-spec.k', '', None)
32 changes: 30 additions & 2 deletions regression-new/checkClaimError/syntax-spec.k.out
Original file line number Diff line number Diff line change
@@ -1,11 +1,39 @@
[Error] Compiler: Found syntax declaration in proof module. Only tokens for existing sorts are allowed.
[Error] Compiler: Found syntax declaration in proof module. Only tokens for
existing sorts are allowed.
Source(syntax-spec.k)
Location(6,18,6,29)
6 | syntax X ::= "errorHere"
. ^~~~~~~~~~~
[Error] Compiler: Found syntax declaration in proof module. Only tokens for existing sorts are allowed.
[Error] Compiler: Found syntax declaration in proof module. Only tokens for
existing sorts are allowed.
Source(syntax-spec.k)
Location(7,5,7,13)
7 | syntax Y
. ^~~~~~~~
[Error] Compiler: Had 2 structural errors.
Traceback (most recent call last):
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 105, in _kprove
return run_process(run_args, logger=_LOGGER, env=env, check=check)
File "/home/dev/src/pyk/src/pyk/utils.py", line 451, in run_process
res.check_returncode()
File "/usr/lib/python3.10/subprocess.py", line 457, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '('kprove', 'syntax-spec.k', '--definition', 'errorClaim-kompiled', '--output', 'json', '--type-inference-mode', 'checked', '--dry-run', '--emit-json-spec', '/tmp/tmp2z_mabgd')' returned non-zero exit status 113.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/dev/src/pyk/src/pyk/__main__.py", line 90, in main
execute(options)
File "/home/dev/src/pyk/src/pyk/__main__.py", line 252, in exec_prove
proofs = kprove.prove_rpc(options=options)
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 380, in prove_rpc
all_claims = self.get_claims(
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 425, in get_claims
flat_module_list = self.get_claim_modules(
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 400, in get_claim_modules
_kprove(
File "/home/dev/src/pyk/src/pyk/ktool/kprove.py", line 107, in _kprove
raise RuntimeError(
RuntimeError: ('Command kprove exited with code 113 for: syntax-spec.k', '', None)
2 changes: 1 addition & 1 deletion regression-new/checks/checkCircularList.k.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[Error] Compiler: Had 1 parsing errors.
[Error] Compiler: Illegal circular relation: Exp < ExpList < Exp
[Error] Compiler: Illegal circular relation: ExpList < Exp < ExpList
[ERROR] Running process failed with returncode 113:
kompile checkCircularList.k --md-selector k --emit-json --warnings none --no-exc-wrap --backend llvm --output-definition checkCircularList-kompiled --type-inference-mode checked
4 changes: 2 additions & 2 deletions regression-new/checks/existentialCheck.k
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ module EXISTENTIALCHECK
rule isDone(stateB) => true
rule isDone(_) => false [owise]

claim <k> ?X:Int => . </k>
claim <k> ?X:Int => .K </k>

rule <k> ?X:Int => . </k>
rule <k> ?X:Int => .K </k>

rule <k> stateA => ?STATE </k>
requires isDone(?STATE)
Expand Down
14 changes: 7 additions & 7 deletions regression-new/checks/existentialCheck.k.out
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
[Error] Compiler: Claims are not allowed in the definition.
Source(existentialCheck.k)
Location(15,9,15,29)
15 | claim <k> ?X:Int => . </k>
. ^~~~~~~~~~~~~~~~~~~~
Location(15,9,15,30)
15 | claim <k> ?X:Int => .K </k>
. ^~~~~~~~~~~~~~~~~~~~~
[Error] Compiler: Existential variable ?X found in LHS. Existential variables are only allowed in the RHS.
Source(existentialCheck.k)
Location(15,13,15,15)
15 | claim <k> ?X:Int => . </k>
15 | claim <k> ?X:Int => .K </k>
. ^~
[Error] Compiler: Found existential variable not supported by concrete backend.
Source(existentialCheck.k)
Location(15,13,15,15)
15 | claim <k> ?X:Int => . </k>
15 | claim <k> ?X:Int => .K </k>
. ^~
[Error] Compiler: Existential variable ?X found in LHS. Existential variables are only allowed in the RHS.
Source(existentialCheck.k)
Location(17,12,17,14)
17 | rule <k> ?X:Int => . </k>
17 | rule <k> ?X:Int => .K </k>
. ^~
[Error] Compiler: Found existential variable not supported by concrete backend.
Source(existentialCheck.k)
Location(17,12,17,14)
17 | rule <k> ?X:Int => . </k>
17 | rule <k> ?X:Int => .K </k>
. ^~
[Error] Compiler: Found existential variable not supported by concrete backend.
Source(existentialCheck.k)
Expand Down
2 changes: 1 addition & 1 deletion regression-new/checks/macroWithFunction.k
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,5 @@ module MACROWITHFUNCTION
syntax Pgm ::= "go" FInt | Pgm ";" Pgm [left]
// ---------------------------------------------
rule <k> P ; Q => P ~> Q ... </k>
rule <k> go wad(0) => . ... </k>
rule <k> go wad(0) => .K ... </k>
endmodule
2 changes: 1 addition & 1 deletion regression-new/checks/macroWithFunction.k.out
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@
[Error] Compiler: Had 1 structural errors after macro expansion.
while executing phase "expand macros" on sentence at
Source(macroWithFunction.k)
Location(23,10,23,37)
Location(23,10,23,38)
[ERROR] Running process failed with returncode 113:
kompile macroWithFunction.k --md-selector k --emit-json --warnings none --no-exc-wrap --backend llvm --output-definition macroWithFunction-kompiled --type-inference-mode checked
6 changes: 3 additions & 3 deletions regression-new/checks/priorityError.k
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ module PRIORITYERROR
syntax Exp ::= Exp "+" Exp [klabel(plus), symbol]
> "l" Exp [klabel(lguard), symbol]
| Exp "r" [klabel(rguard), symbol]
rule . => 1 + l 2 // unable to disambiguate - error
rule . => 1 r + 2 // unable to disambiguate - error
rule . => l 1 r // ambiguous - error
rule .K => 1 + l 2 // unable to disambiguate - error
rule .K => 1 r + 2 // unable to disambiguate - error
rule .K => l 1 r // ambiguous - error
endmodule
18 changes: 9 additions & 9 deletions regression-new/checks/priorityError.k.out
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
[Error] Inner Parser: Priority filter exception. Cannot use lguard as an immediate child of plus. Consider using parentheses around lguard
Source(priorityError.k)
Location(21,19,21,22)
21 | rule . => 1 + l 2 // unable to disambiguate - error
. ^~~
Location(21,20,21,23)
21 | rule .K => 1 + l 2 // unable to disambiguate - error
. ^~~
[Error] Inner Parser: Priority filter exception. Cannot use rguard as an immediate child of plus. Consider using parentheses around rguard
Source(priorityError.k)
Location(22,15,22,18)
22 | rule . => 1 r + 2 // unable to disambiguate - error
. ^~~
Location(22,16,22,19)
22 | rule .K => 1 r + 2 // unable to disambiguate - error
. ^~~
[Error] Inner Parser: Parsing ambiguity.
1: syntax Exp ::= "l" Exp [klabel(lguard), symbol]
lguard(rguard(#token("1","Int")))
2: syntax Exp ::= Exp "r" [klabel(rguard), symbol]
rguard(lguard(#token("1","Int")))
Source(priorityError.k)
Location(23,15,23,20)
23 | rule . => l 1 r // ambiguous - error
. ^~~~~
Location(23,16,23,21)
23 | rule .K => l 1 r // ambiguous - error
. ^~~~~
[Error] Compiler: Had 3 parsing errors.
[ERROR] Running process failed with returncode 113:
kompile priorityError.k --md-selector k --emit-json --warnings none --no-exc-wrap --backend llvm --output-definition priorityError-kompiled --type-inference-mode checked
51 changes: 27 additions & 24 deletions regression-new/concrete-function-cache/a2-spec.k.out
Original file line number Diff line number Diff line change
@@ -1,24 +1,27 @@
kore-exec: [] Warning (WarnStuckClaimState):
The configuration's term unifies with the destination's term, but the implication check between the conditions has failed. Location: a2-spec.k:7:1-8:18
Context:
(InfoReachability) while checking the implication
#Ceil ( bar ( X ) )
#And
#Not ( #Ceil ( baz ( X ) )
#And
{
bar ( X )
#Equals
baz ( X )
} )
#And
<k>
end ( bar ( X ) ) ~> .K
</k>
#And
{
true
#Equals
X <Int 0
}
[Error] Prover: backend terminated because the configuration cannot be rewritten further. See output for more details.
APRProof: A2-SPEC.s2
status: ProofStatus.FAILED
admitted: False
nodes: 5
pending: 0
failing: 1
vacuous: 0
stuck: 1
terminal: 0
refuted: 0
bounded: 0
execution time: 0s
Subproofs: 0
1 Failure nodes. (0 pending and 1 failing)

Failing nodes:

Node id: 5
Failure reason:
Matching failed.
The following cells failed matching individually (antecedent #Implies consequent):
K_CELL: end ( bar ( X:Int ) ) #Implies end ( baz ( X:Int ) )
Path condition:
#Ceil ( bar ( X:Int ) )
Failed to generate a model.

Join the Runtime Verification Discord server for support: https://discord.gg/CurfmXNtbN
Comment on lines -1 to +27
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This failure information message contains largely the same information as the original message, basically the part that failed and path condition to it. Are we happy with this new way of presenting information?

Loading