developer doc: ninja build fails with CUnit header problems

[bhaller]

OK, in my ongoing journey I have now made it to trying to run ninja as instructed at https://tskit.dev/tskit/docs/latest/development.html#building. Unfortunately, this fails:

bhaller@glass c % ninja -C build
ninja: Entering directory `build'
[12/56] Compiling C object test_core.p/tests_test_core.c.o
FAILED: [code=1] test_core.p/tests_test_core.c.o 
cc -Itest_core.p -I. -I.. -Isubprojects/kastore -I../subprojects/kastore -fdiagnostics-color=always -Wall -Winvalid-pch -std=c99 -O0 -g -Wall -Wextra -Werror -Wpedantic -W -Wmissing-prototypes -Wstrict-prototypes -Wconversion -Wshadow -Wpointer-arith -Wcast-align -Wcast-qual -Wwrite-strings -Wnested-externs -fshort-enums -fno-common -DTSK_TRACE_ERRORS '-DMESON_PROJECT_VERSION="1.3.1"' -MD -MQ test_core.p/tests_test_core.c.o -MF test_core.p/tests_test_core.c.o.d -o test_core.p/tests_test_core.c.o -c ../tests/test_core.c
In file included from ../tests/test_core.c:25:
../tests/testlib.h:33:10: fatal error: 'CUnit/Basic.h' file not found
   33 | #include <CUnit/Basic.h>
      |          ^~~~~~~~~~~~~~~
1 error generated.
[13/56] Compiling C object test_genotypes.p/tests_test_genotypes.c.o
FAILED: [code=1] test_genotypes.p/tests_test_genotypes.c.o 
cc -Itest_genotypes.p -I. -I.. -Isubprojects/kastore -I../subprojects/kastore -fdiagnostics-color=always -Wall -Winvalid-pch -std=c99 -O0 -g -Wall -Wextra -Werror -Wpedantic -W -Wmissing-prototypes -Wstrict-prototypes -Wconversion -Wshadow -Wpointer-arith -Wcast-align -Wcast-qual -Wwrite-strings -Wnested-externs -fshort-enums -fno-common -DTSK_TRACE_ERRORS -MD -MQ test_genotypes.p/tests_test_genotypes.c.o -MF test_genotypes.p/tests_test_genotypes.c.o.d -o test_genotypes.p/tests_test_genotypes.c.o -c ../tests/test_genotypes.c
In file included from ../tests/test_genotypes.c:25:
../tests/testlib.h:33:10: fatal error: 'CUnit/Basic.h' file not found
   33 | #include <CUnit/Basic.h>
      |          ^~~~~~~~~~~~~~~
1 error generated.
[14/56] Compiling C object test_convert.p/tests_test_convert.c.o
FAILED: [code=1] test_convert.p/tests_test_convert.c.o 
cc -Itest_convert.p -I. -I.. -Isubprojects/kastore -I../subprojects/kastore -fdiagnostics-color=always -Wall -Winvalid-pch -std=c99 -O0 -g -Wall -Wextra -Werror -Wpedantic -W -Wmissing-prototypes -Wstrict-prototypes -Wconversion -Wshadow -Wpointer-arith -Wcast-align -Wcast-qual -Wwrite-strings -Wnested-externs -fshort-enums -fno-common -DTSK_TRACE_ERRORS -MD -MQ test_convert.p/tests_test_convert.c.o -MF test_convert.p/tests_test_convert.c.o.d -o test_convert.p/tests_test_convert.c.o -c ../tests/test_convert.c
In file included from ../tests/test_convert.c:25:
../tests/testlib.h:33:10: fatal error: 'CUnit/Basic.h' file not found
   33 | #include <CUnit/Basic.h>
      |          ^~~~~~~~~~~~~~~
1 error generated.
[15/56] Compiling C object test_trees.p/tests_test_trees.c.o
FAILED: [code=1] test_trees.p/tests_test_trees.c.o 
cc -Itest_trees.p -I. -I.. -Isubprojects/kastore -I../subprojects/kastore -fdiagnostics-color=always -Wall -Winvalid-pch -std=c99 -O0 -g -Wall -Wextra -Werror -Wpedantic -W -Wmissing-prototypes -Wstrict-prototypes -Wconversion -Wshadow -Wpointer-arith -Wcast-align -Wcast-qual -Wwrite-strings -Wnested-externs -fshort-enums -fno-common -DTSK_TRACE_ERRORS -MD -MQ test_trees.p/tests_test_trees.c.o -MF test_trees.p/tests_test_trees.c.o.d -o test_trees.p/tests_test_trees.c.o -c ../tests/test_trees.c
In file included from ../tests/test_trees.c:25:
../tests/testlib.h:33:10: fatal error: 'CUnit/Basic.h' file not found
   33 | #include <CUnit/Basic.h>
      |          ^~~~~~~~~~~~~~~
1 error generated.
[17/56] Compiling C object test_stats.p/tests_test_stats.c.o
FAILED: [code=1] test_stats.p/tests_test_stats.c.o 
cc -Itest_stats.p -I. -I.. -Isubprojects/kastore -I../subprojects/kastore -fdiagnostics-color=always -Wall -Winvalid-pch -std=c99 -O0 -g -Wall -Wextra -Werror -Wpedantic -W -Wmissing-prototypes -Wstrict-prototypes -Wconversion -Wshadow -Wpointer-arith -Wcast-align -Wcast-qual -Wwrite-strings -Wnested-externs -fshort-enums -fno-common -DTSK_TRACE_ERRORS -MD -MQ test_stats.p/tests_test_stats.c.o -MF test_stats.p/tests_test_stats.c.o.d -o test_stats.p/tests_test_stats.c.o -c ../tests/test_stats.c
In file included from ../tests/test_stats.c:25:
../tests/testlib.h:33:10: fatal error: 'CUnit/Basic.h' file not found
   33 | #include <CUnit/Basic.h>
      |          ^~~~~~~~~~~~~~~
1 error generated.
[18/56] Compiling C object test_tables.p/tests_test_tables.c.o
FAILED: [code=1] test_tables.p/tests_test_tables.c.o 
cc -Itest_tables.p -I. -I.. -Isubprojects/kastore -I../subprojects/kastore -fdiagnostics-color=always -Wall -Winvalid-pch -std=c99 -O0 -g -Wall -Wextra -Werror -Wpedantic -W -Wmissing-prototypes -Wstrict-prototypes -Wconversion -Wshadow -Wpointer-arith -Wcast-align -Wcast-qual -Wwrite-strings -Wnested-externs -fshort-enums -fno-common -DTSK_TRACE_ERRORS -MD -MQ test_tables.p/tests_test_tables.c.o -MF test_tables.p/tests_test_tables.c.o.d -o test_tables.p/tests_test_tables.c.o -c ../tests/test_tables.c
In file included from ../tests/test_tables.c:25:
../tests/testlib.h:33:10: fatal error: 'CUnit/Basic.h' file not found
   33 | #include <CUnit/Basic.h>
      |          ^~~~~~~~~~~~~~~
1 error generated.
[19/56] Compiling C object test_haplotype_matching.p/tests_test_haplotype_matching.c.o
FAILED: [code=1] test_haplotype_matching.p/tests_test_haplotype_matching.c.o 
cc -Itest_haplotype_matching.p -I. -I.. -Isubprojects/kastore -I../subprojects/kastore -fdiagnostics-color=always -Wall -Winvalid-pch -std=c99 -O0 -g -Wall -Wextra -Werror -Wpedantic -W -Wmissing-prototypes -Wstrict-prototypes -Wconversion -Wshadow -Wpointer-arith -Wcast-align -Wcast-qual -Wwrite-strings -Wnested-externs -fshort-enums -fno-common -DTSK_TRACE_ERRORS -MD -MQ test_haplotype_matching.p/tests_test_haplotype_matching.c.o -MF test_haplotype_matching.p/tests_test_haplotype_matching.c.o.d -o test_haplotype_matching.p/tests_test_haplotype_matching.c.o -c ../tests/test_haplotype_matching.c
In file included from ../tests/test_haplotype_matching.c:25:
../tests/testlib.h:33:10: fatal error: 'CUnit/Basic.h' file not found
   33 | #include <CUnit/Basic.h>
      |          ^~~~~~~~~~~~~~~
1 error generated.
[25/56] Linking static target libtestlib.a
ninja: build stopped: subcommand failed.

I have no idea why this might be. I installed cunit with sudo port install cunit, and that worked (this is on macOS, using MacPorts), and then I ran meson setup build and that worked, saying that it found the cunit install. So I'd expect things to be configured for the ninja build to work – but apparently not. I have no idea what to do next, since I'm not familiar with any of these tools. Given that meson found cunit and said that it succeeded in configuring the build, my initial guess is that something about the meson configuration is incorrect – maybe it is not set up to emit a header search path for cunit correctly. I don't have time/energy to try to learn enough about meson to fix this; this seems like something that ought to work out of the box. So, I think this is the end of the road for me until somebody helps me over this hurdle. :-O

[lkirk]

@bhaller from what I understand, it seems that meson is finding cunit, but for some reason it seems the include path is not correct. Perhaps you could run grep -i cunit build/build.ninja to see the commands that will be used to build the tests. At a high level, the build.ninja file is readable enough to get a sense for how things are being compiled.

The build works for me on my end, here's a sample of what I found. I'm including the first line of my grep output

$ grep -i cunit build/build.ninja  --color=auto
 ARGS = -Ilibtestlib.a.p -I. -I.. -Isubprojects/kastore -I../subprojects/kastore -I/nix/store/wavs7s5is612kd135b5l9y44c6jigdmk-CUnit-2.1-3/include -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c99 -O0 -g -fPIC

If I look in the listed include path for cunit, I see this:

$ ls /nix/store/wavs7s5is612kd135b5l9y44c6jigdmk-CUnit-2.1-3/include/CUnit/
Automated.h  Basic.h  Console.h  CUError.h  CUnit.h  CUnit_intl.h  MyMem.h  TestDB.h  TestRun.h  Util.h

Just a small translation from nixos->normal unix file heirarchy, my /nix/store path is effectively /usr/lib or /usr/local/lib etc.

Hope that helps you figure out what's going on here.

[bhaller]

Thanks @lkirk! I'll investigate this and report back. :->