Hi Paolo,
The performance compare and bisect tools will both deal with stats and there is a library that provides some few fundamental stats functions:
https://gitee.com/wu_fengguang/lkp-tests/blob/master/lib/stats.rb
You may check and select the parts that can be referenced or directly reused when doing performance regression detection and bisection.
Xueliang wrote a simple bisect tool that works for build errors:
https://gitee.com/wu_fengguang/compass-ci/blob/master/sbin/git-bisect
Which may be referenced when doing performance bisect tool.
The BJTU students in CC list are interested in bisect. Hopefully you can cooperate on the work.
Thanks, Fengguang ---
Il giorno 9 feb 2021, alle ore 12:50, wufengguang wufengguang@huawei.com ha scritto:
Hi Anmar,
Sorry for top posting -- it's outlook.
lkp-tests does have the compare scripts
https://gitee.com/wu_fengguang/lkp-tests/blob/master/sbin/compare https://gitee.com/wu_fengguang/lkp-tests/blob/master/sbin/ncompare
However they work on data from local files, which does not scale. So we created
https://gitee.com/openeuler/compass-ci/blob/master/sbin/compare
to work on data from ElasticSearch database.
Some compare functions can be reused as base for detect/bisecting performance regression. However we still need add more functions to actually detect regression and bisect good/bad state. That is the current state.
Hi Fengguang, how are you?
We are about to start implementing our tool for detecting regressions. In this respect, I'd like to make sure I've got the following important point right. So let me sum up the situation with my words and ask you whether I'm right.
Functionalities for detecting regression are not mature yet in Compass-CI, whereas 01.org does report performance regressions. So, compass-ci and 01.org are both based on LKP, but 01.org guys have already implemented their own code for regression detection and reporting. This code has not to do with the compare functionalities your are developing.
Am I correct?
Thanks, Paolo
Thanks, Fengguang
Hello Fengguang,
I hope this email finds you well. We dug into the regression detection tool and discovered things like lib/compare and reports such as [1] clearly showing that LKP is able to do some regression analysis.
TBH, I'm a little confused as to what the regression detection tool we discussed is meant to do given that the functionality is already present in LKP. I recall you mentioning that CompassCI doesn't do regression detection so I might have misunderstood your statement or something got lost in translation. Our intention is to create a benchmark data analysis and regression tool that can be used by both CompassCI and LKFT.
Let me know what you think please. We can always jump on a call to sort this out if need be.
-- anmar
[1] https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/V725VQJIASSE2NY...