Running tpdiff should give a "jit-analyze"-like result of the throughput results, i.e. specific context examples of regressions/improvements.
I think there is potential to clean up superpmi.exe as part of this work. Today we produce several .csv files to communicate the information back: base metrics, diff metrics and the diffs info. The problem is that the "diffs info" only writes a row for each context with ASM diffs, so it is not useful for this purpose.
We could subsume these three mechanisms if we introduce a new "--output-csv" argument and always output a row into this file for every context. superpmi.py would then be responsible for aggregating the metrics from this file and and could furthermore use it to produce the tpdiff examples.
Running
tpdiffshould give a "jit-analyze"-like result of the throughput results, i.e. specific context examples of regressions/improvements.I think there is potential to clean up superpmi.exe as part of this work. Today we produce several .csv files to communicate the information back: base metrics, diff metrics and the diffs info. The problem is that the "diffs info" only writes a row for each context with ASM diffs, so it is not useful for this purpose.
We could subsume these three mechanisms if we introduce a new "--output-csv" argument and always output a row into this file for every context. superpmi.py would then be responsible for aggregating the metrics from this file and and could furthermore use it to produce the tpdiff examples.