Hello everyone,
we have a quite large test suite (> 150 testcases, and counting, devided into ~50 groups) which we want to run on our devices. This is required for each software release and would be nice for nightly builds as well.
Our deployment method flashes kernel and root filesystem onto the internal flash memory, which takes ~3 minutes. Booting the OS from RAM or using remote filesystems (NFS) is not an option for us. We need to run all tests on a device booted completely from its internal flash memory. Ideally our OS image should be deployed once and then all the tests should run on top of that deployment.
According to the LAVA documentation, best practice is not to put too many tests into one job file, as this would be hard to maintain and logs would become huge and difficult to read.
What is recommended in such a scenario? My intentional tendency was to create a job for each test group (each containing ~3 testcases average), which occurs reasonable to me. However, this would result in 50 deployment cycles with 3 minutes each, resulting in 2,5 hours spent with basically unnecessary work. This, in turn, does not seem reasonable to me.
Is there a possibility to combine jobs to run on the same device subsequently, so that the images need to be deployed only once? Or can jobs be nested somehow, so that one job does the deployment and contains sub-jobs which perform the actual tests? Or are there any other recommendations for this?
Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks(a)garz-fricke.com
www.garz-fricke.com
SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz
Hello to everyone,
I need to add some scripts to the
/etc/lava-server/dispatcher-config/device-types.
It says to contact Lava mailing list in such a case, to get some guidance.
Says here:
https://staging.validation.linaro.org/static/docs/v2/device-integration.html
*"Please* talk to us *before* starting on the integration of a new device
using the Mailing lists
<https://staging.validation.linaro.org/static/docs/v2/support.html#mailing-l…>
."
The device I need to add is Renesas iwg20m:
https://mp.renesas.com/en-us/boards/iW-RainboW-G20D-RZG1M_RZG1N_QsevenDevKi…
I have on the device working MLO and U-Boot with some U-Boot environment
which boot mmc, I also have working ser2net in Lava VM, it works
seamlessly. So these are all the prerequisites for adding the device type,
as my best understanding is?
My Lava is upgraded:
||/ Name Version Architecture Description
+++-======================-================-================-==================================================
ii lava-dispatcher 2018.2.post2-1+s amd64 Linaro
Automated Validation Architecture dispatche
ii lava-server 2018.2-1+stretch all Linaro
Automated Validation Architecture server
I have beaglebone-black
<http://localhost:8080/scheduler/device_type/beaglebone-black> device type
and added to it bbb01 device, which I finally made working with Lava. I
updated the beaglebone-black-jinja2 script, and created in
/etc/lava-server/dispatcher-config/devices bbb01.jinja2, added this device
to beaglebone-black device type.
I am wondering what should I do else, besides to write iwg20m.jinja2 which
inherits base-uboot.jinja2 script???
iwg20m is similar to Beaglebone Black (it is, after all, armv7
silicon/platform based). And I can make iwg20m.jinja2 as similar analogy to
beaglebone-black.jinja2 .
And, btw, how to add the device-type? Using GUI? CLI? Any description?
Thank you,
Zoran
I ever encountered this issue recently.
In my experience, it caused by failing to start postgresql service.
Have you changed from Jessie to stretch? You can refer to this page:
https://github.com/WindRiver-OpenSourceLabs/lava-base/blob/master/Dockerfile
Regards,
-Yang (Young)
-----Original Message-----
Message: 1
Date: Tue, 13 Mar 2018 17:36:27 +0000
From: Conrad Djedjebi <conrad.djedjebi(a)linaro.org>
To: Lava Users Mailman list <lava-users(a)lists.linaro.org>
Subject: [Lava-users] LAVA V2 | 500 Internal Server Error
Message-ID:
<CAF9r90RH4h3Z-=FZkcd6J6dV0V8nH+sW=tyDrZcHN0WLPR08Bg(a)mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Good morning everyone,
I would like to know if someone here already face the following message while opening a LAVA results page or LAVA alljobs page :
I am looking around LAVA documentation to check if it is a known issue. But if someone here already knows something about it, it could help me also.
regards,
Hi Senthil,
>It throws an error for me when trying to submit via the UI "Job submission error: u'device_type'".
>
>Similarly, when trying with command line I get a similar error:
>
><snip>
>$ lava-tool submit-job
>https://senthil.kumaran@staging.validation.linaro.org/RPC2/
>/tmp/multinode-lxc.yaml
>ERROR: <Fault -32603: "Internal Server Error (contact server administrator for details): u'device_type'"> </snip>
>
In my instance it works via the UI as well as via lava-tool.
>BTW, what is your lava-server and lava-dispatcher version?
tim.jaacks@A048:~$ apt-cache policy lava-server lava-dispatcher
lava-server:
Installed: 2018.2-1+stretch
Candidate: 2018.2-1+stretch
Version table:
*** 2018.2-1+stretch 500
500 https://images.validation.linaro.org/production-repo stretch-backports/main amd64 Packages
100 /var/lib/dpkg/status
2017.7-1~bpo9+1 100
100 http://ftp.debian.org/debian stretch-backports/main amd64 Packages
2016.12-2 500
500 http://ftp.de.debian.org/debian stretch/main amd64 Packages
lava-dispatcher:
Installed: 2018.2.post2-1+stretch
Candidate: 2018.2.post2-1+stretch
Version table:
*** 2018.2.post2-1+stretch 500
500 https://images.validation.linaro.org/production-repo stretch-backports/main amd64 Packages
100 /var/lib/dpkg/status
2017.7-1~bpo9+1 100
100 http://ftp.debian.org/debian stretch-backports/main amd64 Packages
2016.12-1 500
500 http://ftp.de.debian.org/debian stretch/main amd64 Packages
Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks(a)garz-fricke.com
www.garz-fricke.com
SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz
Hi Senthil,
thanks for your reply.
>> I want to set up a multinode job containing an LXC device. Unfortunately I always get the following error:
>>
>> Missing protocol 'lava-lxc' in ['lava-multinode']
>
>This is because each role should have a lava-lxc protocol defined.
Even real hardware devices?
>Going through your job, I could see you are requesting a single device
>(role) in your multinode job, which is not the use-case which multinode caters. There should be more than one device (roles) requested as part of the job. For a single device, single node jobs are the way to go.
I know, this was deliberate in order to strip the error down to a minimal example. I started out with one LXC device and one real hardware device (which is actually the setup I want to use), which led to the above error. Then I removed parts of the job for simplification until (I hoped) the error would disappear. But it did not, even with a single LXC device. I assume a multimode job should work even if it only contains one single device, shouldn't it?
>See https://staging.validation.linaro.org/scheduler/job/213532 - which is a sample multinode job run with two lxc devices requested as part of the job. You can see how each role has a lava-lxc protocol defined for it.
>
>The job definition is available here -
>https://git.linaro.org/lava-team/refactoring.git/plain/lxc-multinode.yaml
Thanks, that pointed me the way. I had to add an additional "remote" key under "protocols/lava-lxc", so the job looks like this:
job_name: lxc-pipeline
timeouts:
job:
minutes: 15
action:
minutes: 5
priority: medium
visibility: public
protocols:
lava-multinode:
roles:
remote:
device_type: lxc
count: 1
timeout:
minutes: 6
lava-lxc:
remote:
name: pipeline-lxc-test
template: debian
distribution: debian
release: stretch
mirror: http://ftp.us.debian.org/debian/
security_mirror: http://mirror.csclub.uwaterloo.ca/debian-security/
actions:
- deploy:
timeout:
minutes: 5
to: lxc
os: debian
role:
- remote
- boot:
prompts:
- 'root@(.*):/#'
timeout:
minutes: 5
method: lxc
role:
- remote
- test:
timeout:
minutes: 5
definitions:
- repository: git://git.linaro.org/qa/test-definitions.git
from: git
path: common/dmidecode.yaml
name: dmidecode
role:
- remote
This job runs without errors, even though it has only one single device in a multinode context. From here I can build my actual multinode job by adding my real device to it.
>HTH. Thank You.
>--
>Senthil Kumaran S
>http://www.stylesen.org/
>http://www.sasenthilkumaran.com/
>
Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks(a)garz-fricke.com
www.garz-fricke.com
SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz
Hello,
I want to set up a multinode job containing an LXC device. Unfortunately I always get the following error:
Missing protocol 'lava-lxc' in ['lava-multinode']
My LXC device works with singlenode jobs. I used this example job for reference:
https://github.com/Linaro/lava-dispatcher/blob/release/lava_dispatcher/test…
I only changed "release" from "sid" to "stretch" and then this job works correctly.
For a minimal multinode job I added the "lava-multinode" protocol, defined a role and assigned this role to each action:
job_name: lxc-pipeline
timeouts:
job:
minutes: 15
action:
minutes: 5
priority: medium
visibility: public
protocols:
lava-multinode:
roles:
remote:
device_type: lxc
count: 1
timeout:
minutes: 6
lava-lxc:
name: pipeline-lxc-test
template: debian
distribution: debian
release: stretch
mirror: http://ftp.us.debian.org/debian/
security_mirror: http://mirror.csclub.uwaterloo.ca/debian-security/
actions:
- deploy:
timeout:
minutes: 5
to: lxc
os: debian
role:
- remote
- boot:
prompts:
- 'root@(.*):/#'
timeout:
minutes: 5
method: lxc
role:
- remote
- test:
timeout:
minutes: 5
definitions:
- repository: git://git.linaro.org/qa/test-definitions.git
from: git
path: common/dmidecode.yaml
name: dmidecode
role:
- remote
With this job definition the above error appears. Am I missing anything or is this a bug in LAVA?
Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks(a)garz-fricke.com
www.garz-fricke.com
SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz
Hello everyone,
My targets are available via ssh interface and I don't need to update them
(install kernel, rootfs, etc) before testing.
My test case is: copy test binary at target device, run the test and check
the result.
As a first step I want to check the ssh connection: log into the device and
check the promts.
My LAVA 2018.2 instance failed to check the connection with "KeyError:
'deployment_data'" traceback: https://pastebin.com/2rsgBvx3
Job definition: https://pastebin.com/GJymcx3z
device dictionary: https://pastebin.com/x08HgSqQ
Is it the LAVA bug or lack of my job definition and device dictionary?
Best Regards, Ivan Arishchenko
I am currently evaluating if LAVA is the right tool for testing our custom flavour of a Raspbian Linux distro. The DUT is going to be a Raspberry Pi3 and I was trying to set it up in a minimal way using another Raspbi3 as a Server & Dispatcher (although first it seemed to work fine, when I followed the LAVA documentation and added the stretch backports sources in the apt list, my whole system crashed because of the depedencies).
What is the best way to get a minimal setup running? I am also interested to know the hardware process (ideally everything we want to be automated). So, should the DUT be connected via a USB2serial cable?
https://goo.gl/D5ZwxU
Other than that should that be connected to an SD multiplexer that burns the resulting .img after Jenkins finishes checking and generating it?
Documentation is a bit tricky to go through as it covers many cases and there's not a simpler minimal tutorial. Also all the online blogs/posts seem to work for LAVA V1 only as a few things have changed.
So it will look something like this:
Linux .img (to be tested) -> SD Multiplexer(?) - DUT - USB2Serial -> LAVA Server & Dispatcher
|
LCD Screen
Are there anywhere some sample tests so I can get an idea of what I will be able to do? (Had already a look at the linaro .git but those seem 'too' basic; probably I've missed smth).
Thank you all for your help.
Thanks again, Neil.
>Future proofing means keeping up with upstream and, hopefully, contributing
>the support back to upstream. A new Strategy class is a requirement for
>inclusion upstream, to be able to isolate this bootloader from future
>changes to better support U-Boot in ways that your bootloader cannot manage.
I get your point and agree with you on that. So does this mean you would accept contributions for a new bootloader, even if it is a proprietary one?
However, for a quick check how LAVA works I was able to go the hacky way and abuse the uboot class to send commands to our own bootloader. This led me to further questions:
1. The TFTP deployment strategy does not seem to support deploying a rootfs. Is that correct? If yes, why?
2. Our bootloader supports download via HTTP as well, so actually we could get the files from the web to the device directly. The test system would not need to download any files then. LAVA does not seem to support such a scenario. All deployment strategies download the files first, before performing further steps.
How could I handle this case in LAVA? Obviously I could remove the deploy action from the job completely and handle everything in the boot command sequence. Still I would need some possibility to specify the URL of the image within the job, which the boot command sequence then should use. Is there a way to achieve this? I assume the correct way would be to implement a new deployment strategy for this, which does not do anything. However, this seems a bit weird to me. Is there another way? Or would the whole idea work against the general LAVA workflow?
3. As far as I understand, LAVA needs to apply an overlay tarball to the image before executing tests, which is usually performed directly on the image file before it is deployed to the target. This would not work in the case of (2). But I saw there is a boot action "transfer_overlay" to do this after the actual deployment. Is it correct, that I could use this action in such a case?
Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks(a)garz-fricke.com
www.garz-fricke.com
SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz
I finally tracked down what is going on here:
Here is the point of the problem while running test script:
Press SPACE to abort autoboot in 10 seconds
end: 2.4.2 bootloader-interrupt (duration 00:00:02) [common]
start: 2.4.3 bootloader-commands (timeout 00:01:58) [common]
bootloader-commands: Wait for prompt ['=>'] (timeout 00:01:58)
Script does detect "Press SPACE to abort autoboot in 10 seconds", and
issues a " ". Then it does NOT detect: Wait for prompt ['=>'].
I replaced old prompt ['U-Boot'] with ['=>'], but neither way it
detects U-Boot prompt.
Anyway, I do not understand the jinja2 command:
{% set bootloader_prompt = bootloader_prompt|default('U-Boot') %} ???
Hasn't => been always U-Boot prompt?
Thank you,
Zoran
_______
On Fri, Mar 2, 2018 at 11:44 AM, <magnus(a)minimum.se> wrote:
> I believe your issue is that LAVA doesn't detect your U-boot prompt
> correctly and does not try to issue any commands. Meanwhile, your kernel is
> autobooting and at line 541 of your pastebin, LAVA gives up waiting for a
> U-boot prompt.
>
> You may need to tweak your u-boot prompt settings and/or uboot interrupt
> sequence. I am not sure if LAVA is doing substring matching of the
> interrupt-prompt, so "Press SPACE to abort autoboot" _may_ not match a line
> that reads "Press SPACE to abort autoboot 10 seconds". But I'll leave that
> to the LAVA devs to answer.
>
> BTW, you should avoid making copies of base-uboot.jinja2. Instead you can
> override these things from your device type config, for example to override
> the interrupt-string you can do:
>
> {% set interrupt_prompt = 'Press SPACE to abort autoboot 10 seconds' %}
>
> and it will override the default value specified in base-uboot.jinja2. Same
> goes for your TFTP commands,
>
> {% set uboot_tftp_bootcmd = '- your list of uboot commands here as a
> yaml-list' %}
>
>
>
>
> On Fri, Mar 2, 2018 at 10:58, Zoran S <zoran.stojsavljevic.de(a)gmail.com>
> wrote:
>
> Some additional data to the use case from the initial email. I have
> changed /etc/lava-server/dispatcher-config/device-types/base-uboot.jinja2
> (changed u-boot massge before prompt, since I am using U-Boot 2017.12,
> and also hard-coded SERVER_IP since I have two networks inside VM -
> one public - 10.0.2.0/24, second private - 192.168.15.0/24).
>
> https://pastebin.com/T8Q7bhWj
>
> Zoran
> _______
>
> On Thu, Mar 1, 2018 at 4:56 PM, Zoran S
> <zoran.stojsavljevic.de(a)gmail.com> wrote:
>
> Hello,
>
> I have very interesting problem: I would like to do Lava testing of
> BBB01, but I am not suceeding.
>
> Simple and plain, my U-Boot scripts gets somehow rejected, they are
> not executed.
>
> I do download correctly, I have all the correct ingredients in place,
> but the U-Boot scripts are not activated, at the time of U-boot
> prompt.
>
> Here is my output, from the testing:
> https://pastebin.com/hTQQSLU1
>
> Rather, the thing gets executed from SDCard (from /boot on rootfs).
>
> I have no idea why this is not executed.
>
> Help appreciated!
>
> Thank you,
> Zoran
>
>
> _______________________________________________
> Lava-users mailing list
> Lava-users(a)lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lava-users
