lava-users

lava-users@lists.lavasoftware.org

2 participants
944 discussions

Unable to detect the reboot triggered by watchdog or software updates in LAVA

by sai.sathujoda＠toshiba-tsip.com

Hello everyone, I would like to open thread of discussion to understand about LAVA test framework support for some of the use cases where I’m facing issues. While testing a reboot scenario in CIP (https://gitlab.com/cip-project/cip-core/isar-cip-core) where reboot is triggered by watchdog. LAVA is unable to do successful reboot. Following are the steps: device_type: qemu job_name: qemu x86_64 software update testing timeouts: job: minutes: 20 action: minutes: 10 actions: power-off: seconds: 60 priority: high visibility: public notify: criteria: status: finished recipients: - to: method: email email: sai.sathujoda(a)toshiba-tsip.com context: arch: x86_64 lava_test_dir: '/home/lava-%s' # ACTION BLOCK actions: - deploy: timeout: minutes: 15 to: tmpfs images: system: image_arg: '-drive file={system},discard=unmap,if=none,id=disk,format=raw -m 1G -serial mon:stdio -cpu qemu64 -smp 4 -machine q35,accel=tcg -global ICH9-LPC.noreboot=off -device ide-hd,drive=disk -nographic' url: ######.wic.xz compression: xz firmware: image_arg: '-drive if=pflash,format=raw,unit=0,readonly=on,file={firmware}' url: ###### # BOOT BLOCK - boot: timeout: minutes: 5 method: qemu media: tmpfs prompts: ["root@demo:~#"] auto_login: login_prompt: "demo login:" username: "root" password_prompt: "Password:" password: "root" # TEST_BLOCK - test: timeout: minutes: 5 definitions: - repository: metadata: format: Lava-Test Test Definition 1.0 name: sample-test description: "check reboot version" run: steps: - lava-test-case uname --shell uname -a - cd /home - wget --no-check-certificate #### - lsblk - swupdate -i cip-core-* - reboot from: inline name: sample-test-1 path: inline/sample-test.yaml - boot: timeout: minutes: 5 method: qemu media: tmpfs prompts: ["root@demo:"] auto_login: login_prompt: "demo login:" username: "root" password_prompt: "Password:" password: "root" - test: timeout: minutes: 5 definitions: - repository: metadata: format: Lava-Test Test Definition 1.0 name: sample-test description: "check partition switch" run: steps: - lsblk from: inline name: sample-test-2 path: inline/sample-test.yaml context: arch: x86_64 lava_test_results_dir: '/home/lava-%s' A reboot is triggered by watchdog following the reboot done in the test action due to failed case. The reboot triggered by watchdog failed with timeout error at login stage which can be interpreted that last boot action in the above job definition failed to give the assigned login prompts. I have already received some opinion about this from LAVA users community that LAVA does not support the board being rebooted outside it’s control ( whether by a watchdog or a package ). However, CIP extensively uses LAVA as test framework to regressively perform many kinds of tests on CIP supported hardware. Testing the watchdog is an important use case in CIP. Since LAVA is supposed to be test framework which can help to test many type of hardware. We as CIP project member would like to understand LAVA community future plan to support this use case. Thanks and Regards, Sai Ashrith

2 years, 2 months

lava-publisher not publishing

by Milosz Wasilewski

Hi, During last couple of weeks I had at least 2 occasions when lava-publisher stopped working. There is nothing in the logs that suggest a failure. The only symptom was that no events were published. Restarting the service fixes the issue. Is this a known bug? I'm running 2023.10 release. Best Regards, Milosz

2 years, 3 months

How to fail a test run on kernel warnings that happen after the boot action?

by Florian Bezdeka

Hi all, I'm basically repeating [1] here as there was no reaction for some months now. Maybe I used the wrong communication channel, let's see... We have a testsuite that is able to trigger a RCU WARNING inside the Linux kernel. My expectation was that whenever a kernel warning / oops / call stack dump / ... occurs the LAVA job is marked as "failed". This assumption seems to be wrong. It took some time to realize that we have a real problem as manual inspection of test logs only happens from time to time. After scanning the code my understanding is that the output of the connection (serial connection in my case) is only parsed during kernel boot (until the login action takes over). That is not sufficient for detecting problems that happen during test execution. Is there a way to scan the full log for the same patterns that are used by the boot action? If so, how to configure that? Whenever a kernel problem occurs my test run should be marked as "failed". Any ideas? Did I overlook something? Best regards, Florian [1] https://git.lavasoftware.org/lava/lava/-/issues/576

2 years, 3 months

How to set ssh password to autologin with ssh

by irreallich＠126.com

Hi, the test job : https://gitlab.com/lava/lava/-/blob/master/tests/lava_dispatcher/sample_job… device types: https://gitlab.com/lava/lava/-/blob/master/etc/dispatcher-config/device-typ… device: {% extends 'ssh.jinja2' %} {% set ssh_host = 'localhost' %} {% set ssh_user = 'test' %} lava version : 2023.08 The job failed because of missing passwd. log as below: test@localhost: Permission denied (publickey,password). Connection closed end: 2.1 scp-deploy (duration 00:00:00) [common] It would be very grateful if you could tell me how to autologin ssh server with user password.

2 years, 3 months

Test Jobs always going in submitted state for QEMU devices

by Ankit Gupta

Hi All, I am trying to setup LAVA 2023.08 lava-server and lava-dispatcher on a single machine, I successfully installed and was able to add workers and a QEMU device. From logs, it seems the workers are communicating with the lava server, I am trying to execute a simple QEMU sample job and it always going in the submitted state. I verified the device dictionary is correct. As per my understanding if communication is happening between lava-server/worker then a device should be assigned that is configured on the worker, but in my case, no device is assigned to the test job and it always keeps in a submitted state. Can someone let me know if any special settings are required for running a test job on a QEMU device? or share any docs link s to resolve this, any input will be appreciated. My Testjob YAML: https://docs.lavasoftware.org/lava/examples/test-jobs/qemu-amd64-standard-s… Thanks, Ankit

2 years, 3 months

Resting the ECU between test cases

by gemad＠outlook.com

Hello, Is there a standard way to reboot the ECU by calling the reboot command defined in the device type (hard_reset_command), between test case, so we are sure the tests are not impacting each other? Thanks.

2 years, 3 months

[SSL: CERTIFICATE_VERIFY_FAILED] self signed certificate

by Ankit Gupta

Hi All, I am setting an HTTPS instance of LAVA, I can access the LAVA UI and am able to log in as well. I am using a self-signed SSL certificate. I have added *URL="https://172.16.60.178/ <https://172.16.60.178/>"* in the /etc/lava-dispatcher/lava-worker file and restarted the service. It's giving me the below error. *2023-10-30 07:10:47,383 ERROR -> server error: code 5032023-10-30 07:10:47,383 DEBUG --> HTTPSConnectionPool(host='172.16.60.178', port=443): Max retries exceeded with url: /scheduler/internal/v1/workers/debian/?version=2023.10 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1123)')))* Do we need to configure some extra settings in case we are using a self-signed certificate? any help will be appreciated. Thanks, Ankit

2 years, 3 months

Bug using SSH as primary connection

by Sebastian Lucas

Hello everyone, There seems to be a bug in LAVA. I was on version 2022.04 and have also tried 2023.03. Both versions have the same bug. The same configurations works in a 2018 build of LAVA on an old machine. I am trying to connect to an always on board via ssh. The healthcheck is failing with this error : lava-dispatcher, installed at version: 2023.03<https://10.1.52.17/scheduler/job/8857#L1>start: 0 validate<https://10.1.52.17/scheduler/job/8857#L2>Start time: 2023-04-12 14:07:00.373707+00:00 (UTC)<https://10.1.52.17/scheduler/job/8857#L3>Traceback (most recent call last): File "/usr/lib/python3/dist-packages/lava_dispatcher/job.py", line 198, in validate self._validate() File "/usr/lib/python3/dist-packages/lava_dispatcher/job.py", line 183, in _validate self.pipeline.validate_actions() File "/usr/lib/python3/dist-packages/lava_dispatcher/action.py", line 190, in validate_actions action.validate() File "/usr/lib/python3/dist-packages/lava_dispatcher/actions/deploy/ssh.py", line 81, in validate if "serial" not in self.job.device["actions"]["deploy"]["connections"]: KeyError: 'connections' <https://10.1.52.17/scheduler/job/8857#L4> validate duration: 0.00<https://10.1.52.17/scheduler/job/8857#results_244238>case: validate case_id: 244238 definition: lava result: fail <https://10.1.52.17/results/testcase/244238><https://10.1.52.17/scheduler/job/8857#L6>Cleaning after the job<https://10.1.52.17/scheduler/job/8857#L7>Root tmp directory removed at /var/lib/lava/dispatcher/tmp/8857<https://10.1.52.17/scheduler/job/8857#L8>LAVABug: This is probably a bug in LAVA, please report it.<https://10.1.52.17/scheduler/job/8857#results_244239>case: job case_id: 244239 definition: lava error_msg: 'connections' error_type: Bug result: fail<https://10.1.52.17/results/testcase/244239> The health check looks like this: job_name: SSH check timeouts: job: minutes: 10 action: minutes: 2 priority: medium visibility: public actions: - deploy: timeout: # timeout for the connection attempt seconds: 30 to: ssh os: oe - boot: timeout: minutes: 2 prompts: ['root@(.*):~#'] method: ssh connection: ssh - test: timeout: minutes: 5 definitions: - repository: metadata: format: Lava-Test Test Definition 1.0 name: smoke-tests-basic description: "Basic smoke test" run: steps: - lava-test-case linux-linaro-ubuntu-pwd --shell pwd - lava-test-case linux-linaro-ubuntu-uname --shell uname -a - lava-test-case linux-linaro-ubuntu-vmstat --shell vmstat - lava-test-case linux-linaro-ubuntu-ip --shell ip a from: inline name: smoke-tests-basic Any ideas ? Best regards, Sebastian

2 years, 5 months

Which kind of connection should I choose?

by liuzacharyu＠gmail.com

Hello everyone, I hava a worker in docker container, and the container runs in a Ubuntu x86 PC. now I want to make the Ubuntu x86 PC as a device and connect device to the worker run on it. Then I can execute job test actions commands in device. Which kind of connection should I choose?

2 years, 5 months

How to connect with serial port?

by irreallich＠126.com

I am newer , and I try to make new device. there is no uboot, kerner ... on my system. for boot, we just need run cmd on host "echo "reset 1" > /dev/ttyUSB0" , it is the mcu serial port. then I can get boot log from /dev/ttyUSB1 , it is thesoc serial port. the following is the job device_type: orinshort job_name: for orinshort device timeouts: job: minutes: 20 action: minutes: 15 priority: medium visibility: public actions: - deploy: timeout: minutes: 10 to: flasher images: kernel: url: http://10.19.207.190/static/docs/v2/contents.html#contents-first-steps-using The following is the my device, it is very sample: {# orin test short #} {% extends 'orinshort.jinja2' %} {% set connection_list = ['usb1'] %} {% set connection_tags = {'usb1': ['primary', 'telnet']} %} {% set connection_commands = {'usb1': 'telnet localhost 10009'} %} {% set flasher_reset_commands = ['/tmp/test.sh'] %} {% block body %} actions: deploy: methods: flasher: commands: {{ flasher_reset_commands }} {% endblock body %} I don't know how to set boot with serial port . 1. about cmd echo "reset 1" > /dev/ttyUSB0 0.1 when I input the cmd on the host shell : echo "reset 1" > /dev/ttyUSB0 , soc can be reset 0.2 when I add it to test.sh and run it with the following code in device, soc can be reset too actions: deploy: methods: flasher: commands: {{ flasher_reset_commands }} 0.3 when I use {% set flasher_reset_commands = ['echo "reset 1" > /dev/ttyUSB0) '] %} , it doesn't work. {% set flasher_reset_commands = ['echo \"reset 1\" > /dev/ttyUSB0) '] %} , it is doesn't work. So how to echo cmd to /dev/ttyUSB0 by flasher: -> commands? 2. how to set boot with serial port in device jinja2 file? if I add the fowllowing code , the test can't run boot: connections: serial: usb1 3. I think the normal process is to configure deploy boot and other operations in the job. but I don't know how to do with my case. just run cmd on host "echo "reset 1" > /dev/ttyUSB0" to start system, how to select the method? how to run flasher_reset_commands in job yaml file? 4. If the job can't start job state is Submitted and can't start test, how to debug? where is the log? I am newer and my system is special , it just like docker, there is now uboot kernel, we need special cmd to deploy and boot so it is hard for me, please give me a hand for it.

2 years, 5 months

← Newer
1
...
5
6
7
8
9
10
11
...
95
Older →

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

lava-users