Hi!
Anibal and I put together a list of steps that can be used to 'fastboot
boot' to a ramdisk or nfs root filesystem in LAVA issue 271[1]. The idea
here is to have LAVA create a boot image at runtime based on a kernel,
kernel modules, optional ramdisk, and a dtb file. Embedded in the boot
image is the kernel command line necessary to boot to a ramdisk or to an
nfs root filesystem. This boot image is then run with 'fastboot boot'
directly - no flashing required.
This is based on discussions related to dragonboard-845 implementation,
and would make such boards compatible with kernelci (for example), but
also simplify kernel test jobs that don't require a system image to be
flashed to the board.
We started to look at what would be needed on the LAVA side to implement
this, but need help defining the semantics and requirements for the LAVA
implementation. I'm also not sure which parts should be in an LXC/docker
type context, and which parts should be directly supported in LAVA. I
can imagine the rest of the semantics will be very similar or the same
to the tftp deploy type.
Feedback and LAVA implementation suggestions welcome. Perhaps this can
be a topic at the next LAVA design meeting (can I get an invite?).
Thanks,
Dan
[1] https://git.lavasoftware.org/lava/lava/issues/271
Hi,
There is an idea of device type 'alias' in LAVA. I don't quite
understand what the use case for the current implementation was [1]. I
tried using it but it wasn't very useful. My use case is that I need
to submit jobs to a device type with different device type name. This
is used to align device type naming between different labs in a bigger
project (kernelci.org in this case). So the questions I have about
current implementation:
- is there anyone using current implementation?
- if current implementation is used, how much trouble would it cause
to change the behaviour?
Change in behaviour is quite intrusive and will require database migration.
[1] https://master.lavasoftware.org/static/docs/v2/glossary.html#term-alias
Regards,
milosz
Hi folks,
We held our regular weekly design meeting today via Hangout. Summary
of brief discussion:
## lava-lxc-mocker + db845 booting kernel/dtb/modules [Matt]
* Kevin and Matt: how to change binaries before flashing with fastboot?
* Should be possible using lxc and lxc://
* See [lxc-sdm845-mtp-test](https://validation.linaro.org/scheduler/job/1912316/de…
* Kevin: issue with lava-lxc-mocker inside docker container
* [Milosz] Working for [lavasoftware/lava-dispatcher](https://hub.docker.com/u/lavasoftware/lava-di… docker image
* Kevin to compare his docker container to lavasoftware/lava-dispatcher
## Replacing lxc protocol by docker (without protocol) [Rémi]
* New feature for the near future
* Allow for simple deploy.fastboot and boot.adb actions
* No need for lxc protocol anymore
* With the option (per job) to use a custom docker container
## u-boot commands: nfs vs ramdisk [Rémi]
* After [!467](https://git.lavasoftware.org/lava/lava/merge_requests/467), u-boot jobs booting from nfs without a ramdisk are failing. See [meta-lava](https://git.lavasoftware.org/lava/meta-lava/-/jobs/38813).
* [!492](https://git.lavasoftware.org/lava/lava/merge_requests/492) fixes this but breaks [cubietruck](https://federation.lavasoftware.org/lava/scheduler/job/361)
* Jobs using both ramdisk and nfs will fail (every job booting debian)
* Adding a new command: "ramdisk-nfs"
## lavafed and KCI [Rémi]
* Presenting lavafed
* [Kevin] will this be a problem with multiple dispatchers?
* [Rémi] lavafed will see the lab as many sub-labs (one per dispatcher)
* [Kevin] is it possible with docker dispatchers?
* [Rémi] yes. Install docker-cli and bnd the docker socket in the container. LAVA will then be able to call docker run from inside the container.
Rémi to send a mail with the details to kevin
## 2019.04 release [Rémi]
* When do we release?
* Next Monday
* What can be moved to the next release?
* See GitLab milestone
## REST API filtering - issue with package dependencies [Milosz]
* [failing pipeline](https://git.lavasoftware.org/mwasilew/lava/pipelines/3220)
* Looks like a problem when building the debian package
* Steve to look at it.
============================================================================
The LAVA design meeting is held weekly, every Wednesday at 13:00 to
14:00 UTC using Google Hangouts Meet: https://meet.google.com/qre-rgen-zwc
Feel free to comment here or join us directly in the meeting.
Minutes from this and previous meetings are also stored in the LAVA wiki:
https://git.lavasoftware.org/lava/lava/wikis/design-meetings/index
Cheers,
--
Steve McIntyre steve.mcintyre(a)linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
Hi folks,
We held our regular weekly design meeting today via Hangout. Summary
of discussion:
1. JS and CSS minifications [Rémi]
* Is minification really useful?
* Make building more complex
* Make debugging difficult
* Add potential failures (bugs in the compiler)
* Does minify save a lot of bandwidth?
If the resources are compressed (using gzip or brotli) then the gain is not
significant and not worth the effort.
Sounds better to remove the minified version and ensure that the server
only send compressed versions whenever possible.
2. ramdisk vs nfs boot in u-boot [Rémi]
Some jobs (like [cubietruck](
https://staging.validation.linaro.org/scheduler/job/251164/definition))
provide both a ramdisk and an nfsrootfs while booting with nfs commands.
* Is the ramdisk of any use?
* Is it possible to remove the command that loads the ramdisk completely?
Dean to look at the jobs ARM ran to see any use of this.
============================================================================
The LAVA design meeting is held weekly, every Wednesday at 13:00 to
14:00 UTC using Google Hangouts Meet: https://meet.google.com/qre-rgen-zwc
Feel free to comment here or join us directly in the meeting.
Minutes from this and previous meetings are also stored in the LAVA wiki:
https://git.lavasoftware.org/lava/lava/wikis/design-meetings/index
Cheers,
--
Rémi Duraffort
LAVA Team, Linaro
Dear LAVA team,
in my LAVA setup I need to connect my DUTs to certain interfaces on the worker. The information which of my DUTs is connected to which interface has to be available in my test shell somehow. The documentation gives the impression that LAVA is capable of this:
https://validation.linaro.org/static/docs/v2/admin-lxc-deploy.html?highligh…
It says: "Other static devices which are accessible over the network can be made available to a test shell in the LXC through lava test shell helpers".
Browsing through the code shows that this is implemented only for one single use-case, though, that is energy probes (via helpers "lava-probe-ip" and "lava-probe-channel"). There does not seem to be any generic mechanism to make static device info available in the test shell. The documentation is quite misleading here in my opinion, since it does not reflect the actual implementation.
I have asked about this on the lava-users mailing list, and Neil confirmed my investigation:
https://lists.lavasoftware.org/pipermail/lava-users/2019-March/001741.html
He answered my question, whether there is a generic mechanism to supply static_info from the device dictionary in the LAVA test shell, with "Not currently".
Is such a feature planned? If not, would you accept code contributions which add such a feature?
I actually do not see why the two helpers for the energy probes have been hard-coded in the first place. Why hasn't this been implemented in a generic way from the beginning?
In my idea, there would be only one single function "lava-static-info" which outputs the complete static info array in some form. The caller can then extract the info he needs from there. Is there any reason against such an extension?
Alternatively, are there any other suggestions on how to implement my use case? To make it clear once again: I have four USB-RS232 converters, available on the worker as /dev/ttyUSB0 to /dev/tty/USB3. I have four DUTs, each of which is connected to one of the converters. The test shell has to know which of the TTY devices to use for the test, depending on the actual DUT chosen by the LAVA scheduler.
Hope to get some useful replies here. Thanks in advance.
Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks(a)garz-fricke.com
www.garz-fricke.com
WE MAKE IT YOURS!
Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz, Marc-Michael Braun
Hi folks,
We held our regular weekly design meeting today via Hangout. Summary
of brief discussion:
1. [Steve] Connect sessions planning
1. Bootloader testing
2. lavafed
3. users forum
As lots of us will be at Connect this coming week, the design meeting
for 3rd April will not be happening via Hangout.
============================================================================
The LAVA design meeting is held weekly, every Wednesday at 13:00 to
14:00 UTC using Google Hangouts Meet: https://meet.google.com/qre-rgen-zwc
Feel free to comment here or join us directly in the meeting.
Minutes from this and previous meetings are also stored in the LAVA wiki:
https://git.lavasoftware.org/lava/lava/wikis/design-meetings/index
Cheers,
--
Steve McIntyre steve.mcintyre(a)linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
Hi folks,
We held our regular weekly design meeting today via Hangout. Summary
of discussion:
1. [Rémi] cycle planning
1. better at Connect?
2. we should make sure that all things are discussed/mentioned on
the -devel list before we start them
2. [Steve] Connect sessions
1. Bootloader testing
2. lavafed
3. users forum
1. Slides about last features and planned work for next 6m
3. Debian backports updates
1. More desired python modules uploaded into backports
2. lavacli 0.9.5 shortly
============================================================================
The LAVA design meeting is held weekly, every Wednesday at 13:00 to
14:00 UTC using Google Hangouts Meet: https://meet.google.com/qre-rgen-zwc
Feel free to comment here or join us directly in the meeting.
Minutes from this and previous meetings are also stored in the LAVA wiki:
https://git.lavasoftware.org/lava/lava/wikis/design-meetings/index
Cheers,
--
Steve McIntyre steve.mcintyre(a)linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
Hi folks,
We held our regular weekly design meeting today via Hangout. Summary
of discussion:
1. [Rémi] Feature flags (https://martinfowler.com/articles/
feature-toggles.html)
1. Useful for:
1. Schema validator deployment
2. MultiNode on/off for multiple parallel schedulers
3. Store logs in filesystem or ES
4. Other use cases?
2. Per-user or per-instance?
1. Per-instance:
1. Simpler
2. Cover most usages
3. Quicker to implement
2. [milosz] device type alias
1. Does anyone know if it’s used anywhere in the current form?
2. The request from KernelCI/LKFT is to have alias for device type
that can be used in submissions so we can resolve conflicts in
naming schemes between labs
3. ACTION: Milosz to ask about it in the ML
3. [milosz] JS/CSS in the current master doesn’t work for me. Links in
the top bar do nothing. Is this my setup issue or some general
problem?
1. There was a missing link to bootstrap-3.4.1.min.js. I guess this
is my Lsetup issue
2. File is present in lava-server docker image
1. Looks like a local issue
4. [Steve] docs.lavasoftware.org is online, with no content at the
moment
1. /lava, /lavacli, /vland - any others, shout!
2. just going to have the last released version available; if people
complain we can revisit
5. [Dean] Feasibility of targeting specific devices in job definition
1. We are getting asked if this is possible frequently.
1. Why?
2. Currently have to put a tag on with the device name on each
device if this is required, which is not ideal.
3. Really don't want to do this - this was a deliberate change from
v1 to v2. Would need to make modifications to the scheduler to
allow it. Tags are the right answer!
6. [Dean] Question about docker device type. Is there a download action/
deploy step that can be used on a docker to download extra binaries
into the container in the lava definition, then proceed to the test
step which can use those binaries?
1. There isn't, no - use the test shell
7. [Fathi] Alpine based dispatcher and server images
1. How to test those images in the CI pipeline?
1. How often do we test ?
2. Do we build/publish the docker images?
3. [Steve] Very little context on this - what/why/when/where etc.
needed
4. Can we get Fathi to come to next week's meeting and talk about
it?
============================================================================
The LAVA design meeting is held weekly, every Wednesday at 13:00 to
14:00 UTC using Google Hangouts Meet: https://meet.google.com/qre-rgen-zwc
Feel free to comment here or join us directly in the meeting.
Minutes from this and previous meetings are also stored in the LAVA wiki:
https://git.lavasoftware.org/lava/lava/wikis/design-meetings/index
Cheers,
--
Steve McIntyre steve.mcintyre(a)linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
Hi folks,
We held our regular weekly design meeting today via Hangout. Summary
of discussion:
1. [stevanr] LAVA auth revamp
1. Restricted for authenticated users use case
1. With current design, everything should be open (visible
permission) by default; user, group, is_public fields removed
and permissions assigned by groups
1. [dean] would be useful to restrict to only authenticated
or per group devices/testjobs
2. So if someone wants to restrict visibility to let’say only
authenticated users, it’s a bit tricky to support it with the
current design
2. Device owner
1. My plan is to remove every way of having some arbitrary field
mess with authorization other then permissions. Does anyone
see a problem with removing this field? (physical_owner will
still remain in place but it has no say in auth)
1. [Steve] Check with Dave?
2. Do they use device owner?
3. [Rémi] how to allow one group to update one device object?
1. Is it of any use?
4. [Rémi] code/design/schema available somewhere?
2. [Steve] Future plans - when/how/where do we notify of upcoming
changes?
1. lava-devel by default for most things
2. lava-announce for breaking changes, and give notice
1. how much notice?
1. 1m? 3m? agree on 2 months for things like BD migrations
2. 2019.05 will have some migrations, then no more yet planned
until .08 or .09
3. [Steve] When will schema validation become rigid? .08 or .09 as well?
1. Print warnings before then
2. Schema validation in strict mode will complain about most jobs
(80%?), so we can't turn that on!
3. For now, allow submissions. Also run the validator again in
strict mode and print its output as a warning on submission. Can
we put the same output into the test log too? Should we?
1. Some users may look at the LAVA job results page
2. Some may just grab the logs
3. Maybe start mailing admins with a daily summary of warnings?
Add a management command to check for warnings.
4. Talk to other people (e.g. Matt about kernelci)
4. Maybe look into versioning of schemas and support validation of
supported versions? Lots of work… :-/
5. Not complete yet, we still want people running schema checks and
letting us know about any problems found
1. lavacli jobs validate <job-def.yaml>
2. share/lava-schema.py job <job-def.yaml>
4. [Rémi] doc canonicalization
1. https://moz.com/learn/seo/canonicalization
2. Add links back to https://docs.lavasoftware.org/lava in our help
information
3. Need to get docs.ls.o set up!
5. [Steve] Stevan being added as a reviewer/admin
1. Sort out details offline
============================================================================
The LAVA design meeting is held weekly, every Wednesday at 13:00 to
14:00 UTC using Google Hangouts Meet: https://meet.google.com/qre-rgen-zwc
Feel free to comment here or join us directly in the meeting.
Minutes from this and previous meetings are also stored in the LAVA wiki:
https://git.lavasoftware.org/lava/lava/wikis/design-meetings/index
Cheers,
--
Steve McIntyre steve.mcintyre(a)linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
Hi folks,
We held our regular weekly design meeting yesterday via
Hangout. Summary of discussion:
1. [Neil] Job action timeouts
a. Downgrade current change to not reject XMLRPC API job submission [Rémi]
1. lots of jobs on LKFT need changes, likely to be indicative
of a wider problem
b. Implement the XMLRPC check and get lava-schema.py out to
people in 2019.02 [Rémi]
c. Announce the new schema
d. Leave fatal exceptions until a future release
e. Confirmed. the schema validation itself is not yet part of lava_scheduler_app submit.
2. [Neil] Aarch64 gitlab-runners
a. Initial config available, needs optimisation, especially
concurrency per machine vs cores per runner
1. optimisation to be done during 2019.03 cycle.
b. [Steve] Mustang machine not booting - investigating
1. could be an issue with upgrade to buster.
3. [Neil] wisdom of running unit tests inside docker builds in the
ci-images project?
a. guarantees that the new image won't break lava.git master.
b. another item to be added to the docs of how our CI
operates. [Neil] needs an issue.
4. [Neil] Remi to authenticate the GnuPG fingerprint for
4E9995EC67B6560E0A9B97A9597DCC10C0D1B33D to enable lavasoftware.org
ansible password_store
a. Now fixed via keys.gnupg.net and pgp.earth.li
5. [Dean] Feasibility of upgrading django-ldap-auth to version 1.7?
a. https://tracker.debian.org/pkg/django-auth-ldap
b. To sync LDAP groups into Django auth.
c. We want to be able to mirror LDAP groups as groups in LAVA
(details of this:
https://django-auth-ldap.readthedocs.io/en/latest/permissions.html#group-mi…)
but examples of this given in 1.5 docs (oldest I've found) don't
appear to work in 1.3
d. I believe 1.7 is in Buster, so is it a case of moving to buster?
1. not urgent to migrate to buster now.
6. [Steve] location for documentation
a. Needs an issue to track this.
1. avoid the rabbit hole of optimising what is left.
b. certain elements need to go into the website from lava-server-doc
1. Release docs
2. Development process
3. Design overview
4. Keep development-intro and update links.
c. update all the links in lava-server-doc.
d. website will move to Sphinx instead of Pelican.
e. a lot of docs are using Sphinx RST.
1. there will be some conversion needed for items which are currently in Google Docs.
f. move design meeting doc to the wiki?
1. no - interactive shared-editing feature is very useful
2. we can simply copy text to the wiki after the fact
3. Page per meeting in the wiki with index links
4. Consider the new document as public by the end of the meeting.
7. [Remi] MuxPi rPi zero. build raw images.
a. Guestfish based on GuestFS
b. inside docker? need /boot and /lib/modules volumes. Easy scriptable
way to do things.
c. Should we publish our images and a way to rebuild them?
1. Let's not do this as an automated public build
a. Neil to close https://git.lavasoftware.org/lava/functional-tests/issues/7 (Use
GitLab CI to auto-build functional test image files for
files.lavasoftware.org) on the basis that we are not an image
building service.
2. Must document how our images are built
a. files.lavasoftware.org already includes copies of the
scripts which were used to create the Debian standard image files.
The LAVA design meeting is held weekly, every Wednesday at 13:00 to
14:00 UTC using Google Hangouts Meet: https://meet.google.com/qre-rgen-zwc
Feel free to comment here or join us directly in the meeting.
Cheers,
--
Steve McIntyre steve.mcintyre(a)linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
