Skip to content

Register and run lit tests #498

Open
castigli wants to merge 26 commits intohw-native-sys:mainfrom
castigli:filecheck
Open

Register and run lit tests #498
castigli wants to merge 26 commits intohw-native-sys:mainfrom
castigli:filecheck

Conversation

@castigli
Copy link
Copy Markdown

This PR adds a target to register and run the lit (FileCheck) tests.

Changes

  • move lit tests to a subfolder for a clean cmake setup
  • adds a target check-pto to run the tests
  • fixes the trivially broken tests
  • add the tests to the CI

IMPORTANT

  • at the moment, the test pto/issue481_addptr_gm_slot_buffer.pto fails. This is a genuine bug.
  • the CI setup might be wrong, I am not super familiar with github actions

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates LLVM's lit testing framework into the project, adding the necessary CMake configurations, site templates, and updating existing test files to use lit-style CHECK lines. The changes include the addition of test/lit/CMakeLists.txt, lit.cfg.py, and lit.site.cfg.py.in, alongside significant updates to various .pto test files to reflect changes in generated code and error reporting. Several issues were identified in the new build scripts, including a potential copy-paste error where dependencies were appended to the wrong variable (MLIR_TEST_DEPENDS instead of PTOIR_TEST_DEPENDS), redundant configuration of %PATH% and test_exec_root in the Python config, and the inclusion of a potentially non-existent dependency pto-opt.

Comment thread test/lit/CMakeLists.txt
Comment thread test/lit/CMakeLists.txt Outdated
Comment thread test/lit/CMakeLists.txt Outdated
Comment thread test/lit/lit.cfg.py
Comment thread test/lit/lit.cfg.py Outdated
@reedhecre
Copy link
Copy Markdown

reedhecre commented Apr 16, 2026

Codex Review

该评论由 review 机器人自动更新。

  • PR: Register and run lit tests  #498 Register and run lit tests
  • Author: castigli
  • Base/Head: main / filecheck
  • Head SHA: 702777dd20f5
  • Trigger: PR 有新提交
  • Generated At: 2026-04-17T15:05:05Z
  • Previous Head SHA: 5751a4bd6402
  • Status: completed

Summary

lit registration misses one existing A5 regression test, and the new lit environment is not runnable on macOS because it does not preserve DYLD_LIBRARY_PATH.

Findings

  1. P2 One existing A5 lit regression test is left outside the registered suite test/lit/lit.cfg.py:27

The new suite only discovers tests under test/lit (config.test_source_root = os.path.dirname(__file__)), but test/basic/test_tpush_tpop_roundtrip_nosplit_a5.pto still contains a // RUN: ptoas --pto-arch=a5 %s directive and exercises the A5 no-split cube/vector round-trip path. After this PR, check-pto no longer runs that regression case, so CI still misses one of the existing lit tests this change is supposed to register.

  1. P2 `check-pto` will not work on macOS because the lit environment drops `DYLD_LIBRARY_PATH` test/lit/lit.cfg.py:36

The lit config forwards only HOME, INCLUDE, LIB, TMP, and TEMP. On macOS, ptoas is run with DYLD_LIBRARY_PATH set in the repo's own CLI smoke test setup (docker/test_ptoas_cli.sh), which indicates the binary does not run standalone without the dylib search path. Because check-pto now becomes the supported test entrypoint but does not preserve DYLD_LIBRARY_PATH, the whole lit suite will fail to launch ptoas on macOS.

@castigli castigli marked this pull request as ready for review April 17, 2026 08:37
@castigli
Copy link
Copy Markdown
Author

castigli commented Apr 17, 2026

/run sim

@MirkoDeVita98
Copy link
Copy Markdown

/run all

@reedhecre
Copy link
Copy Markdown

A3 板测完成(有跳过)

  • 触发方式:manual
  • 源码提交:789930ca022d
  • 结果汇总:OK 202 / FAIL 0 / SKIP 1
  • 日志:/home/zhongxuan/ptoas-board-monitor/runtime/logs/20260417_192005_manual_pr498.log
  • 结果 TSV:/home/zhongxuan/ptoas-board-monitor/runtime/logs/20260417_192005_manual_pr498.tsv
  • 手动指令:/run all
  • 触发人:MirkoDeVita98
  • 触发评论:Register and run lit tests  #498 (comment)

@castigli
Copy link
Copy Markdown
Author

/run sim

@castigli
Copy link
Copy Markdown
Author

/run a5

@reedhecre
Copy link
Copy Markdown

A5 手动板测当前要求显式指定 case。请使用 /run a5 <case>/run a5 case1,case2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants