Skip to content

Conversation

@lsylvestre
Copy link
Contributor

@lsylvestre lsylvestre commented Jul 11, 2021

This PR introduces a ppx facility to test expressions.

  1. It embeds compiler-libs/pprintast.cmi into the grading environment. This allows to compose reports quoting well-formed pieces of syntaxe produced by ppx-metaquot (e.g. an expression of type Parsetree.expression builded by [%expr e]).

  2. It implements the ppx extension [%printable e] specified in #379 as a shortcut for Test_lib.printable_fun e (Pprintast.string_of_expression [%expr e]).

  3. It implements the ppx extension [%code e] which build the tuple (Code.(e), Solution.(e), [%expr e]). This may be useful for testing an expression e in the scope of the student code, against the same expression e in the scope of the solution while having access to the syntax tree of e (typically for quoting e in the report).


Here is some examples.

  1. Pprintast + ppx-metaquot

The following grader:

let () =
  Test_lib.set_result @@
  let s = Pprintast.string_of_expression [%expr M.(x + 1)] in
  Report.[ Message ([Text "this expression is well-formed:"; Output s], Informative) ]

produces the following report:

this expression is well-formed:
  let open M in x + 1
  1. [%printable ...]

The following grader:

(* solution.ml *)
let apply f x = f x

(* test.ml *)
open Test_lib

let p_incr = [%printable fun n -> n + 1]
let p_fact = [%printable 
                let rec fact n = 
                  if n = 0 then 1 else n * fact (n - 1) 
                in fact]

let () =
  set_result @@ ast_sanity_check code_ast @@ fun () ->
    test_function_2_against_solution
      [%ty: (int -> int) -> int -> int] "apply" ~gen:0
      [(p_incr, 42); (p_fact, 6)]

produces the following report:

Exercice terminé                          2 pts
-----------------------------------------------
Found apply with compatible type.
Computing apply (fun n -> n + 1) 42
Correct value 43                           1 pt
---
Computing apply (let rec fact n = if n = 0 then 1 else n * (fact (n - 1)) in fact) 6
Correct value 720                          1 pt
  1. [%code ...]

The following grader:

(* solution.ml *)
class counter =
  object
    val mutable c = 0
    method get = c
    method reset = c <- 0
    method inc = c <- c + 1
  end

(* test.ml *)
open Test_lib
open Report

module type S = sig 
  class counter :
    object
      method get : int
      method reset : unit
      method inc : unit
    end
end

let test_expression (e_code, e_solution, e_ast) =
  let descr = [ Text "The expression:"; 
                Output (Pprintast.string_of_expression e_ast) ] in
  let res_code     = exec (fun () -> e_code) in
  let res_solution = exec (fun () -> e_solution) in
  if res_code = res_solution
  then Message(descr @ [Text "seems correct."],Success 1)
  else Message(descr @ [Text "produces an unexpected result."], Failure)
;;

let () =
  set_result @@ ast_sanity_check code_ast @@ fun () ->
  test_student_code [%ty:(module S)] @@ fun code ->
    let module Code = (val code : S) in
    
    (* now, we can safely use Code, since it is compatible with S *)

    [ test_expression [%code let c = new counter in c#get];
      test_expression [%code let c = new counter in 
                             for i = 1 to 10 do c#inc done; 
                             c#get];
      test_expression [%code let c = new counter in 
                             c#inc; c#reset; c#get] ]

produces the following report:

Exercice incomplet                        2 pts
-----------------------------------------------
The expression:                            1 pt
  let c = new counter  in c#get
seems correct.
---
The expression:                            1 pt
  let c = new counter  in for i = 1 to 10 do c#inc done; c#get
seems correct.
---
The expression:                            0 pt 
  let c = new counter  in c#inc; c#reset; c#get
produces an unexpected result.

@lsylvestre lsylvestre changed the title Add a ppx facility for testing expressions Add a ppx facility to test expressions Jul 12, 2021
@erikmd erikmd added the kind: feature New user-facing feature. label Aug 25, 2021
@yurug yurug self-assigned this Sep 15, 2021
@erikmd erikmd added this to the learn-ocaml 0.13 milestone Sep 15, 2021
@lsylvestre
Copy link
Contributor Author

Hi @yurug,

I have just adapted this PR :

  • for compatibility with OCaml 4.12
  • to avoid a conflict with the obsolete file src/grader/build.ocp

Thank you, have a good day.

@yurug
Copy link
Collaborator

yurug commented Sep 17, 2021

Thank you @lsylvestre !

@yurug yurug merged commit 526bc07 into ocaml-sf:master Sep 17, 2021
@lsylvestre
Copy link
Contributor Author

Many thanks @yurug !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind: feature New user-facing feature.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants