On 12 June 2018 at 16:05, Tim Jaacks <tim.jaacks@garz-fricke.com> wrote:

Hello everyone,

I know from the LAVA documentation how to add metadata to jobs and test suites. When I look at test results, I see that test cases have metadata, too. E.g. https://validation.linaro.org/results/testcase/9759970 shows the following metadata:

case: linux-linaro-ubuntu-lscpu
definition: 0_smoke-tests-lxc
result: pass

That data is retrieved directly from the test case. case is the string passed to lava-test-case as the test case ID. definition is the currently running test definition (so an integral part of how the test case is reported) and then there is the result.

Actions can supply metadata, e.g. https://validation.linaro.org/results/testcase/9759948

This metadata comes from the Python3 code running the action and covers a range of dynamic data which is not otherwise recorded. This data is not modifiable.

Test suites gain metadata from the operation of the test job itself. The content of the Lava Test Shell Definition is not used, other than to identify the test defiinition.

The user-controllable metadata needs to be in the Test Job Submission, to then appear in the top level results:

e.g. in

https://validation.linaro.org/results/1882533

path: health-checks/dragonboard-820c.yaml

source: https://git.linaro.org/lava-team/refactoring.git

come from the test job submission.

Is there a possibility to add custom metadata to test cases?

Only as a URL: https://validation.linaro.org/static/docs/v2/writing-tests.html#recording-test-case-references

Once the test job has completed, bug links are also available for test jobs, test suites and test cases.

Extra data can be recorded as separate test cases using the test case name, e.g.

https://staging.validation.linaro.org/results/testcase/5557198

(Although I really should tweak that to drop the Python3 string handling artifacts in the test shell scripts. It should read code-version-2018.5+12261.50e347aca )

We are considering new features for the Lava Test Shell Definition but changes won't be made any time soon, it's likely to be 2019 before we have a clear plan for that work.

Neil Williams
=============
neil.williams@linaro.org
http://www.linux.codehelp.co.uk/