Conversation
CHANGES: Fork of Crowbar, renamed to Alcobar. - Alcotest-compatible API: `test_case`, `run`, and suite exports replace the old `add_test`/`run_test` interface. - `--gen-corpus DIR` flag to generate seed corpus files from passing test runs, removing the need for a separate `gen_corpus.ml`. - Per-test timeout (`ALCOBAR_TIMEOUT` env var, `--timeout` flag). - Per-test time budget (`--budget` flag). - GitHub Actions CI.
|
/cc @stedolan -- I'm experimenting with an Alcotest API on top of your nice crowbar. I'm publishing in the main repo as I intend to release a few libraries that uses it but I'm also fine to just keep it a local fork and disabling the test somehow when I'll release my packages. |
| bug-reports: "https://github.com/samoht/alcobar/issues" | ||
| depends: [ | ||
| "dune" {>= "3.21"} | ||
| "ocaml" {>= "4.08"} |
There was a problem hiding this comment.
| "ocaml" {>= "4.08"} | |
| "ocaml" {>= "4.10.0"} |
as List.concat_map was added in that release, thereby addressing:
https://opam.ci.ocaml.org/github/ocaml/opam-repository/commit/446e4a3df0d605e197d04cd013cc3f97a00250c8/variant/compilers,4.09,alcobar.0.3
#=== ERROR while compiling alcobar.0.3 ========================================#
# context 2.5.0 | linux/x86_64 | ocaml-base-compiler.4.09.1 | pinned(https://github.com/samoht/alcobar/releases/download/v0.3/alcobar-0.3.tbz)
# path ~/.opam/4.09/.opam-switch/build/alcobar.0.3
# command ~/.opam/opam-init/hooks/sandbox.sh build dune build -p alcobar -j 71 @install
# exit-code 1
# env-file ~/.opam/log/alcobar-7-2fe7c0.env
# output-file ~/.opam/log/alcobar-7-2fe7c0.out
### output ###
# (cd _build/default && /home/opam/.opam/4.09/bin/ocamlopt.opt -g -I src/.alcobar.objs/byte -I src/.alcobar.objs/native -I /home/opam/.opam/4.09/lib/afl-persistent -I /home/opam/.opam/4.09/lib/alcotest -I /home/opam/.opam/4.09/lib/alcotest/engine -I /home/opam/.opam/4.09/lib/alcotest/stdlib_ext -I /home/opam/.opam/4.09/lib/astring -I /home/opam/.opam/4.09/lib/cmdliner -I /home/opam/.opam/4.09/lib/fmt -I /home/opam/.opam/4.09/lib/fmt/cli -I /home/opam/.opam/4.09/lib/fmt/tty -I /home/opam/.opam/4.09/lib/re -I /home/opam/.opam/4.09/lib/seq -I /home/opam/.opam/4.09/lib/stdlib-shims -I /home/opam/.opam/4.09/lib/uutf -intf-suffix .ml -no-alias-deps -o src/.alcobar.objs/native/alcobar.cmx -c -impl src/alcobar.ml)
# File "src/alcobar.ml", line 649, characters 4-19:
# 649 | List.concat_map
# ^^^^^^^^^^^^^^^
# Error: Unbound value List.concat_map
# (cd _build/default && /home/opam/.opam/4.09/bin/ocamlc.opt -g -bin-annot -I src/.alcobar.objs/byte -I /home/opam/.opam/4.09/lib/afl-persistent -I /home/opam/.opam/4.09/lib/alcotest -I /home/opam/.opam/4.09/lib/alcotest/engine -I /home/opam/.opam/4.09/lib/alcotest/stdlib_ext -I /home/opam/.opam/4.09/lib/astring -I /home/opam/.opam/4.09/lib/cmdliner -I /home/opam/.opam/4.09/lib/fmt -I /home/opam/.opam/4.09/lib/fmt/cli -I /home/opam/.opam/4.09/lib/fmt/tty -I /home/opam/.opam/4.09/lib/re -I /home/opam/.opam/4.09/lib/seq -I /home/opam/.opam/4.09/lib/stdlib-shims -I /home/opam/.opam/4.09/lib/uutf -intf-suffix .ml -no-alias-deps -o src/.alcobar.objs/byte/alcobar.cmo -c -impl src/alcobar.ml)
# File "src/alcobar.ml", line 649, characters 4-19:
# 649 | List.concat_map
# ^^^^^^^^^^^^^^^
# Error: Unbound value List.concat_map
jmid
left a comment
There was a problem hiding this comment.
Thanks!
I see a range of runtest failures - but only on 4.14, e.g.,
https://opam.ci.ocaml.org/github/ocaml/opam-repository/commit/446e4a3df0d605e197d04cd013cc3f97a00250c8/variant/compilers,4.14,alcobar.0.3,tests
- ┌──────────────────────────────────────────────────────────────────────────────┐
- │ [FAIL] calendar 0 add_sub. │
- └──────────────────────────────────────────────────────────────────────────────┘
- ASSERT When given the input:
-
- [1711-06-30 00:19:24; 1713-03-29 00:38:50]
-
- the test failed:
-
- 1713-04-01 00:38:50 != 1713-03-29 00:38:50
-
- FAIL When given the input:
-
- [1711-06-30 00:19:24; 1713-03-29 00:38:50]
-
- the test failed:
-
- 1713-04-01 00:38:50 != 1713-03-29 00:38:50
-
- Raised at Alcotest_engine__Test.check_err in file "src/alcotest-engine/test.ml", line 173, characters 20-48
- Called from Alcotest_engine__Core.Make.protect_test.(fun) in file "src/alcotest-engine/core.ml", line 186, characters 17-23
- Called from Alcotest_engine__Monad.Identity.catch in file "src/alcotest-engine/monad.ml", line 24, characters 31-35
(impressive middle-age calendar bug-finding ability - but annoying red CI lights here on the opam repo...)
Write tests, let a fuzzer find failing cases
CHANGES:
Fork of Crowbar, renamed to Alcobar.
test_case,run, and suite exportsreplace the old
add_test/run_testinterface.--gen-corpus DIRflag to generate seed corpus files from passingtest runs, removing the need for a separate
gen_corpus.ml.ALCOBAR_TIMEOUTenv var,--timeoutflag).--budgetflag).