The 2017.7 release is available from the lava-repositories for
jessie-backports and has been uploaded to Debian unstable. Support for
stretch-backports is pending. (The production-repo/jessie-backports
can be used in the meantime.)
https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
jessie-backports "sloppy"
==================
This release has been uploaded to Debian unstable. The period after a
Debian stable release is turbulent in unstable and the LAVA packages
may not migrate into buster (testing) smoothly which will delay
uploads to the official backports archive (for jessie backports and
stretch backports).
To update jessie-backports, the upload will need to be the version
from buster, rebuilt for jessie and sent to jessie-backports-sloppy.
Once that is done, I'll post again to the this list about how that
becomes available to admins running Jessie systems.
Jessie continues to be the preferred release for LAVA until such time
as we've done more testing on Stretch. We don't anticipate any
problems with lava-server or lava-dispatcher on Stretch as this has
been tested frequently. However, third party packages and scripts
still need testing with real instances using Stretch.
V1 support in buster
===============
Some unit tests in the V1 support have had to be disabled due to bug
fixes in django 1.11 LTS in buster and sid affecting the atomic
transaction handling. The remote worker design in V1 has always had
issues with atomicity and these unit tests have failed intermittently
for a while. This is one of the many reasons to drive all instances to
V2. These tests are now failing consistently with 1.11, so have been
disabled if django 1.11 or newer is detected.
V1 is no longer run on any of our development instances (running
buster or sid), only in production instances (running jessie) which
are in preparation for the migration. We can only assume that the
failure of the unit tests would make V1 unreliable on buster or sid.
However, with the pending removal of V1, there will not be any fixes
for any problems discovered when using V1 with django 1.11 or newer.
LDAP support in buster and sid
=======================
python-django-auth-ldap has changed dependencies, from python-ldap to
python-pyldap. When upgrading python-django-auth-ldap, python-ldap
will be removed and python-pyldap installed.
Instances *must* restart the lava-server-gunicorn service after this
change or users will experience HTTP 500 errors.
lava-server manage check --deploy
=========================
This release adds some checks to the django system checks. When admins
update health checks or device-type templates, it is worth running the
deployment check which does a quick sanity check on the instance. This
can help to spot invalid health check submissions amongst other
issues.
Note that on development instances, some issues will always be
reported - e.g. when Debug is set to True for django using
/etc/lava-server/settings.conf
Also, for developers, some of the improved security features in django
may get in the way of signing in to http://localhost by enforcing
requirements for https:// which is unusual on localhost.
Check the django documentation - to enable localhost, you may need to
disable at least these security defaults by adding to
/etc/lava-server/settings.conf:
"CSRF_COOKIE_SECURE": false,
"SESSION_COOKIE_SECURE": false,
As a result of these changes, installing 2017.7 can generate a prompt
to update /etc/lava-server/settings.conf. In general, keep your
existing configuration but look at the differences and migrate the
changes across. Remember to restart lava-server, lava-master and
lava-server-gunicorn services whenever /etc/lava-server/settings.conf
is modified.
Skipping 2017.8
============
There won't be a full 2017.8 production release of lava-server or
lava-dispatcher as a number of the team are away at the annual Debian
conference, DebConf17 in Montreal.
https://debconf17.debconf.org/
If you are going to be there, let us know.
lava-server changes
==============
a9390fc9a Remove debug hacking session from doc examples
13edd8f12 Skip certain V1 unit tests on django 1.11 and later
0577447fc Update query script for python3 and to output CSV
0b3b0beb7 device-type: addresses fix for armada-3720-espressobin
ed1e402e5 device-type: addresses fix for armada-3720-db
c382a5aa0 device-type: fix at91sam9m10g45ek
54a3b7a46 Apply change for distinct test case links to all results
1824a18fb Adding uefi-menu tftp method
71aac817e LAVA-363 - set the scheme in api/help from settings
2c840debc base-uboot.jinja2: don't load DTB if append_dtb is set
1b13a0e3e Fix extra keyword in api/help view
2882529bc Provide an example script for using queries
7f4cfe569 logs: continue to poll the log when canceling
25eaab176 XMLRPC: add offline_info argument to list endpoint
38f7570e3 XMLRPC: fix inconsistency between code and doc
8c134839d LAVA-972 test case should use ID in URL.
99725606c Remove references to OpenID and Crowd
95b959602 XMLRPC: add a function to download job definition
ebf80c266 XMLRPC: fix crash when device.worker_host is None
b1ec14c89 LAVA-861 separate base-fastboot.jinja2 template
c4fd79d57 Remove unused file
7c066195a Remove deprecated django configuration
520ac8778 XMLRPC: remove unused function
3b8ab7d61 XMLRPC: do not raise an error when canceling twice
cce1bdea6 XMLRPC: allow to force health check for a device
218408db6 Handle the new log level called "input"
b49d9fb96 Adding uefi parameters for vexpress
4a52ba861 Allow overriding django system check values from settings as
documented
d2067a5f3 LAVA-987 fix log parsing
bbc39489c results_app: API: add the 'run_query' function
8746edb6b results_app: API: add the 'make_custom_query' function
4fd429ac2 Add note on apache2 for V2-only workers
641e61dcb Explain the handling of measurements and Decimal
bf8f53bf5 Count going offline as busy in overview table
06febd188 Expand on essential roles, results and sync
815d41b52 Tighten advice on prompt strings in docs
351f6fe41 Add documentation for login_commands in auto_login
5a7d4be95 Avoid warning for health checks of retired devices.
5713f5295 Document transfer_overlay deployment differences
8f04c6097 Fix typo in multinode api documentation.
fe2093994 device-types: add meson-gxl-s905x-khadas-vim
853ee80e3 Move action fields templates to table files.
12de946eb Allow bare except and override pep8
dd4faacaf Enforce schema rules for metadata
352049204 Add missing check for null key|value in metadata
b77731b31 device-types: rcar-gen3-common: handle bigger kernels
d2acb94a5 LAVA-983 - Helper for DeviceStateTransition
b4208116c LAVA-980 - Fix django deployment warnings
778fd4a8b Fix link to glossary term
af85e5225 LAVA 931 - documentation.
212e98a3a LAVA-976 Use the Django system check framework
e9c7ad477 Adding commands for booting whatever the precanned defaults are
f78c58811 Added device type for Cavium ThunderX
lava-dispatcher changes
=================
d8dabe63 Switch the print to a test in unit test
d1e22b67 Revert "Replace invalid characters in test_case_id."
b3b4f5ac Update the example to a working test job with prompts.
504ec2bc LAVA-777 last log line should be lava.job result
ad55acb9 Ensure overlay unpacking is clearly logged
2e7c9598 Add pipeline_ref update for test skipped by lavabot
ed936cd3 Simplify the power control actions
78dcc1b0 Fix dispatcher configuration
89880ce1 LAVA-96 Check for errors if a tftp file exceeds 4G
6fd2702e LAVA-989 add extra logging for actions
df86ed30 LAVA-990 add logging for VLANd + MultiNode support
bbfb1817 Stop the unit test outputting noise
c7839a98 Add a new log level for the DUT inputs
2502b8af Destroy LXC containers properly.
957734cf Add a unit test for Vland actions
68468abe Handle a bad url cleanly.
713d4e6e LAVA-645 - summarise slave logging
96a5effe Do not raise exceptions in validate
f45106fb Catch errors in testdef from handler cleanly
f34f41e2 LAVA-984 - LXC protocol timeout to report job result
e31722b5 Allow ci-run with updated pep8
175d8e84 LAVA-931 - MultiNode lava-sync should report results
fd944543 EOF in a test shell is an InfrastructureError
429691d9 Support power control for CMSIS devices
049c0227 lava-slave: add the right configuration for the host
1e92bf38 lxc: improve error message
b2f25cf0 Add support for booting from UEFI shell
67a1e1c6 ramdisk: only pass to bootx if has u-boot header
f4a4abd9 LAVA-967: check namespaces usage
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
As with previous releases since 2016.12, the 2017.6 release is only
available from the lava repositories:
https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
However, Debian Stretch is due to be released very soon. 2017.6 may be
the first release to be uploaded to Debian unstable since the stretch
freeze started with 2016.12. There may be a short delay whilst
unstable settles down after the usual round of extra uploads. Once
packages migrate into the new testing (buster) and stretch-backports
becomes available, packages will be uploaded for stretch-backports and
jessie-backports.
Device dictionaries moved to the filesystem
===========================================
LAVA-757 - all existing device dictionaries will be migrated to the
filesystem (/etc/lava-server/dispatcher-config/devices/) during the
installation of the 2017.6 production release. The dependency on
python-django-kvstore has been dropped. This means that there is no
longer a need to update the device dictionary in the database, simply
install the new file into the directory and the next test job to be
assigned to that device will use the new configuration. Unlike
device-types, there are no default device dictionaries provided by
LAVA, so a fresh install will have no device dictionary files in
/etc/lava-server/dispatcher-config/devices/
The documentation has been updated. There is now no need for
lava-server manage device-dictionary, so this command has been
dropped. Instead, admins who have permissions to run lava-server
manage can simply copy a device-dictionary .jinja2 file into
/etc/lava-server/dispatcher-config/devices
During installation, this migration will take place:
Applying lava_scheduler_app.0027_device_dict_onto_filesystem
Database entries for all device dictionaries are replaced with files.
If a device can support V2 but has no device dictionary, it will be
marked as Skipped.
This change means that lava-server no longer depends on the
python-django-kvstore package.
Feedback support with namespaces
================================
Devices may output messages to the serial console as a result of
operations performed within an LXC and this output is now logged as
part of the test job log file as 'feedback'. See the documentation for
more information on feedback output.
Architecture is optional for LXC
================================
Some LXC deployments support identifying the deployment architecture
based on the host system. Some instances may use workers of different
architectures and test jobs should be portable between instances.
Therefore, the arch: field of the lava-lxc protocol has been made
optional. Note that Fedora and Ubuntu deployments still need the
architecture to be specified for x86 machines to distinguish between
32bit and 64bit x86 deployments.
The architecture value can still be specified in test jobs but the
test jobs will be more portable if omitted.
Action names use hyphens, not underscore
========================================
If you are using action-specific timeouts in your test jobs or in
customised device-type templates, note that the names of some actions
have *changed*. This is to make the usage of hyphens and underscores
consistent. Actions with names containing underscores are now invalid
and all underscores in names have been replaced with hyphens.
'pdu_reboot' becomes 'pdu-reboot'.
lava-server changes
===================
daf1e2a lava-master: fix init script
b6b5fe7 Reduce device configuration dynamic connections
ebf4463 Fix a crash in the migration when the db is empty
cdfeb12 Fix index out of range exception
33092ef Adding usb deployment method to vexpress device template
3257ee2 XML-RPC: remove unused imports
55dd014 XML-RPC: fix crash when called by anonymous users
a3d4718 XML-RPC: allow every user to get templates/dict
ddaf693 Add documentation of feedback support.
eaae89a Set a default connection timeout for lava-test-shell
f305e47 publisher: fix umask
c25686f logrotate: lava-master should be owned by lavaserver
2da8e74 logs: except more exceptions
14f5f6c Update docs for changes in 2017.6
a486d12 Tidy up of debian support docs
b05712d Preserve original comments when resubmitting
19bd2db XML-RPC: add an api to manage aliases
30973a0 Log an error when saving device configuration fails
763239d Use --arch for fedora LXC example
8f26102 XML-RPC: add a method to show tag details
19a69f7 fix location of example test job files
480a6f6 XML-RPC: add an api for handling jobs
356667d Update installation docs for Stretch release
da74ae5 XML-RPC: add a decorator for is_superuser check
49f370c XML-RPC: add a clean api for device-types and aliases
c6d503d commands/devices: extend devices command to add tags
1655dbf Move set commands outside body block
87b8c01 XML-RPC: add a clean api for workers and tags
351c16b XML-RPC: add a clean api for devices
c4dec1f device-types: add bcm2837-rpi-3-b
9ca0e0d Adding combined vexpress device template
67f531f Make lava_scheduler_app.api a module
d3021a3 xmlrpc: add a system.version function
3ef0157 linaro_django_xmlrpc: remove unused properties
e994852 Use render shortcut instead of the loader
e51ca21 linaro_django_xmlrpc: rework the application
4bbe17c LAVA-959 run lava-master as lavaserver user
b283288 Fix migrations from Debian Jessie to Stretch and ensure smooth upgrade.
bc25204 LAVA 329 - Add old job URL to metadata when resubmitting a job.
271723d LAVA-782 - Change error type for incorrect handling of custom queries.
368903e notify: Don't send verbose mails when the job is cancelled
71c514d Fix broken link in unit test.
1f48674 LAVA-955 - Remove 'arch' parameter from lxc protocol.
3b36ba0 Add auto_login test case in test_pipeline unit test
e23bb17 Fix typo which causes XML-RPC error when saving device dict
9a82aee XML-RPC: allow to have sub_modules
b0c7a5b Document reboot and apply-overlay elements for fastboot deploy method.
8a93049 LAVA-935 - Provide feedback output from connections
d71f267 Fix typos in proxy configuration documentation.
c7227b3 Fix rebase error in grub hikey test
9253497 Only the master should create log directories
1ee736b Allow every u-boot devices to use minimal boot
e816c17 LAVA-832 allow to manage users from the command line
eb83cbe Add grub interrupt prompt and interrupt character to constants
ee79de2 LAVA-937 show job visibility and fix group visibility
73b6d86 Prevent key error in metadata handling.
f89e840 Improve job output migration helper
eb1ad02 device dict: do not show exclusive flag
3900276 Restore date-based subdirectories
1faa9f0 LAVA-855 - ART CI: Intel NUC device integration for LAVA v2
d195b01 Allow to revert (dummy) the migration
35f0802 Fix device dict export
da7ebe2 device dict: fix web rendering
4a3daa9 Allow override of root for installed mustang.
12c0508 Expand zmq_client to handle publisher socket
34f3625 Trivial whitespace changes in glossary
6246a5d Improve data export docs
2af8e72 Allow for non-Const values in device dictionary
59742fe Extend examples to use the DNS support for NFS
c514296 Fix change in import paths for V1 and V2 scheduling
765901e Add notes on advanced features for submission
fd0905c Remove deprecated commands
0237b85 Use CommandError whenever possible
a1cc08b Improve device-type templates
05123cc Missing conditional in device-type alias handling
17af14e Move commands to lava_server app
5bab54d Remove reference to unavailable boot method 'fastboot-boot-image'.
78e9048 Add mediatek dt for v2
49f27fa Allow in-place YAML include in V2 job submission.
c2be83f Grub support for HiKey
ae4d477 LAVA-757 Move device dictionaries to file system
52eb1e1 Remove unused model JobPipeline
9d0a5ae results: make the link unique
55bf39f Add a callback_url functionality to notification schema.
0324915 Add login_commands to the auto_login schema
ce6d0f7 Don't reload template from disk on each access
lava-dispatcher changes
=======================
38da5d8 LAVA-974: Fix init script and allow to set hostname
0fd687e Don't use os.path.realpath() on device names
d84dab7 Modifying vemsd deploy action to use ResetDevice
d21385e testdef: get 'revision' from the right variable
5ea16fe Rework test shell feedback timeouts
510c4e5 Declare the feedback timeout
b994757 Use pyudev device links as well as device nodes
e609c22 Fix namespace usage in power commands.
39e2e7e Read from feedback connection during long operations
7be926a Allow tar.gz firmware bundles
20e66c1 overlay: Raise JobError if root_partition is None
4e27fcd LAVA-935 - Provide feedback output from connections
831c71c Fix bug #3032 - Running an LXC/Hikey job without a test action ...
84e7378 Fix namespace issue in NFS deployment action
f1f8d71 Add login_prompt valid test case
7dc48aa Issue lxc-stop before destroying a container.
3e1cb07 Changing udev checks to only look for FS label
d08db39 Fix #3010: set logging timezone to UTC
1b664e5 LAVA-955 - Remove 'arch' parameter from lxc protocol.
8a4dbc7 Increase LXC finalize timeout to 90 seconds similar to multinode.
3823c49 Use one debug log call to log the same kind of message in lxc protocol.
5db313c Creating vemsd mount points
ea03ef1 Fixing boot test issues using NFS deploy action
d1f4a30 Fix rebase error with grub prompt constant
b2aef8b device-types: add sun6i-a31-app4-evb1
8ceca07 Make grub use interrupt prompt and character from constants
b495a35 device-types: add ttyAT0 to Atmel devices to support multiv7_defconfig
3a89027 device-types: fix at91sam9x25ek to use bootz
9e00895 Set download max_retries in the constructor
959480c Add minimal boot action
31fd021 User command: do not clean if it was not executed
26572a2 Fix keys and label that were outdated
67dd60e Remove unused functions
0c100ff Remove boot-result common data
342039a Fix RetryAction inside a RetryAction
9ce430f Fix check on LXC for default usage.
0576754 Fix typo: Use pixel job create function for testing pixel device.
e7a4422 Fix typo: remove visibility added twice.
9e67b6d LAVA-855 - ART CI: Intel NUC device integration for LAVA v2
dd5d874 Ensure all deployments can add LXC devices except QEMU
e72d490 Fix pipeline refs for change in monitor retry
cb57d2b LAVA-942 UEFI Menu and command lists
e57013b Start adb daemon before attempting adb commands.
ac389ad lava-slave: remove tmp dir when END_OK is received
632a0ff Fix device configuration
0cffce4 Fix validation of boot auto_login parameters and add a unit test
7e40701 Allow unix line separators for some UEFI devices
095798a Avoid calling cleanup in validate
f98dade Allow updating all pipeline_refs at a time.
31b62dd LAVA-928 - HiKey issue when switching between AOSP and OE
51f40ef Make soft reboot optional LAVA-846
4d43d02 HiKey Grub EFI support
874f889 Avoid logging the same message twice.
3be27df lxc protocol: simplify the tests
75126ff Do not set self.errors when Action.run fail
ce32bfb Do not use self.errors in Action.run()
a5d9eba Fix detection of missing ssh_host value in validate
234eef9 Fix calling of protocols after LXC change
477a36b tests: make ShellCommand.logger a DummyLogger
62867f4 Use images.validation.linaro.org files for unit tests
1868936 Silence logging in more unit tests
609c783 Fix download action name
b170197 More tweaks to silence messages from the unit tests
29efa26 Adjust for pep8 checks in jessie
0733f89 Make sure the test_character_delay is used for all commands in
test shell
78d2509 Account for empty environment string.
ea09b19 Take namespace into account when counting test stages
734688c Declare namespace of the test suite in results
60561c2 Parser: remove unused context
49a6b92 Drop noisy info log message.
8ed8467 lava-test-monitor: use TestMonitorRetry action
4fd5d4a Drop unused imports and unused variables
678b216 Rework the removable action to allow sd cards
3b42122 Fix bug #2975 after qemu-nfs introduction
7462ea6 Expand LXC support to add devices from all sybsystems
b853865 Fix missing check for u-boot commands parameters
8293693 Replace invalid characters in test_case_id.
a94e0f2 Fix preseed/late_command appending
9647d55 Run login_commands if provided in auto_login
db03483 Fix Action names (use - instead of _)
9e836e6 Add a Command Action
bbd4ba2 Make the parser stricter about the block names
65b3a56 Allow to pass integers to run_command
b41326b Raise JobError instead of NotImplementedError
073f3b3 update gitignore
00f5d33 log: limit the length of lines send other zmq
3f99cc3 Add missing super call to WaitUSBDeviceAction
5b2b074 LAVA-889 Fix handling of multiple test blocks
5436658 Fix indent typo
3bfdcdb Add support for use_xip
8df17dd Add support for append_dtb
3c110ca Remove unused imports
4847bf1 Put device rebooting info message at the correct place.
6b85382 Fix call to lava-lxc protocol for pre-os-command
176a4d4 Add new utility function infrastructure_error_multi_paths
bdb4d12 Fix test_lxc_api unit test
dc008eb Export the full version string.
e9df320 LAVA-920 - Workaround ptable issue in firmware
7b4096c Allow LXC protocol to call pre-os-command
b33d04b Fix OE image support for HiKey.
d714a9b Add missing test shell helpers
daebfad Extend secondary connection fix to support primary
9518e74 Add missing import
e97801d Add optional deployment complete message list to secondary
media deployments
fae0ced Move unnecessary constants into base jinja template.
18f745b device-types: add kirkwood-db-88f6282.conf
815e518 device-types: add at91sam9m10g45ek.conf
6ca0731 device-types: add at91rm9200ek.conf
003b53d device-types: add at91-sama5d4_xplained.conf
9b2371c device-types: add armada-3720-db.conf
5a76733 device-types: add armada-xp-gp.conf
8b05093 device-types: add kirkwood-openblocks_a7.conf
f5801d3 device-types: add alpine-v2-evp.conf
19e2027 device-types: add sama5d34ek.conf
49da704 device-types: add armada-385-db-ap.conf
ed7aa6b device-types: add armada-370-db.conf
9a10432 device-types: add at91-sama5d2_xplained.conf
ddb34ce device-types: add alpine-db.conf
f7642f5 device-types: add armada-375-db.conf
0d70c7d device-types: add at91sam9x35ek.conf
a642d45 device-types: add armada-xp-db.conf
2a5ff9e device-types: add armada-7040-db.conf
1a43999 device-types: add orion5x-rd88f5182-nas.conf
ad855a6 device-types: add armada-xp-linksys-mamba.conf
58308ca device-types: add armada-388-gp.conf
6a2898b device-types: add armada-370-rd.conf
cd1b4f4 device-types: add armada-398-db.conf
51925fb device-types: add sun8i-a83t-allwinner-h8homlet-v2.conf
4976d06 device-types: add sun8i-a33-sinlinx-sina33.conf
681480d device-types: add sama5d35ek.conf
db74d60 device-types: add sun5i-r8-chip.conf
40736b9 device-types: add sama5d36ek.conf
433be0a device-types: add imx6q-nitrogen6x.conf
ed4c82e device-types: add at91sam9x25ek.conf
ce7a488 device-types: add at91sam9261ek.conf
f87cdb4 device-types: add armada-xp-openblocks-ax3-4.conf
634442c device-types: add armada-388-clearfog.conf
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
This is a long mail but there is a lot to cover...
Stretch
======
Debian Stretch is expected to be released on June 17 2017:
https://lists.debian.org/debian-devel-announce/2017/05/msg00002.html
As has been mentioned on this list before - admins MUST check the apt
sources for all machines *NOW* to ensure that all your Debian apt
sources are pointing at 'jessie' NOT 'stable' if you are to be able to
manage the upgrade yourself.
What you want in /etc/apt/sources.list and /etc/apt/sources.list.d/* is:
deb http://mirror.bytemark.co.uk/debian jessie main
deb http://mirror.bytemark.co.uk/debian jessie-backports main
(The mirror you choose is up to you.)
If you find any apt sources referencing 'stable' or 'testing' CHANGE
THESE NOW or those systems could automatically upgrade to Stretch or
Buster respectively on 17th June.
As with all such releases, production machines are recommended to
allow some time for final testing before doing the upgrade. The
Cambridge LAVA lab is expected to upgrade from Jessie to Stretch
during August 2017. The LAVA software team have been doing testing of
this upgrade for some time, without issues. However, real machines
with extra packages and local admin configuration cannot be tested in
this way, so each lab should take some time to plan when and how to do
the upgrade. (Hint: have a backup and check that it can be restored
correctly before starting.)
https://validation.linaro.org/static/docs/v2/admin-backups.html
There has been a fix for lava-server in stretch to smooth the upgrade
for those users who have not installed lava-server from
jessie-backports. Those already running 2017.* are not affected by
this change and can continue using jessie-backports and the LAVA
repositories as before. Stretch will release with lava-server
2016.12-2 and python-django 1:1.10.7-2
Once Stretch is released, Debian testing will be called 'buster',
Debian stable will be 'stretch' and Debian oldstable will be 'jessie'.
At a later date, jessie-backports will be opened again and a new
stretch-backports will become available. New LAVA production releases
will be uploaded to unstable to migrate into testing (buster) and then
be backported to stretch-backports and jessie-backports. The LAVA
repositories will be updated as well.
Support for jessie-backports will continue after stretch-backports
support is added but cannot continue indefinitely. At some point, all
instances will need to have moved to Stretch to continue getting
updates of LAVA. (Prior to the process of releasing Debian Buster, the
jessie packages will be dropped from the archive, including
jessie-backports.)
Changes in LAVA repositories
=======================
2017.6 will be released in the same way as previous releases in 2017 -
to the LAVA repository for use on jessie using jessie-backports.
At a later point, 2017.6 will be uploaded to Debian unstable for
migration into testing (buster). Only developers should install that
version as Debian unstable can be problematic in the first few weeks
after a Debian stable release.
There are changes in how the lava packages are built on jessie versus
on stretch, so the lava repositories will be split. I'll be updating
the documentation in 2017.6 to cover the changes required to help
admins use the new suites. There will be a new jessie-backports suite
and a new stretch-backports suite. The existing 'sid' or unstable
suite will have no uploads after 2017.6.
This will be covered again in the 2017.6 release email and again with
2017.7. For now, no changes are required.
Once the new suites are announced, admins who choose to upgrade to
Stretch would use the stretch-backports suite from the LAVA
repository.
deb https://images.validation.linaro.org/production-repo stretch-backports main
Those who will upgrade later, would use jessie-backports from the LAVA
repository.
deb https://images.validation.linaro.org/production-repo jessie-backports main
The purpose of these suites is to allow admins to choose to upgrade to
the latest production release for jessie or stretch without waiting
for the migration of the packages into Debian testing and the
subsequent uploads to backports. The packages are otherwise the same.
Django
======
Longer term, there have been a lot of discussions within Debian about
managing django releases. The plan is that only django LTS releases
will be available in Debian unstable, testing and therefore stable
after the Stretch release. Non-LTS releases will only be available in
Debian experimental. This means that lava-server will migrate from one
django LTS to the next django LTS with future Debian releases. The
LAVA software team will manage how deprecation cycles within django
affect lava-server.
This does have the issue that the version of django in Debian stable
will be towards the end of the upstream support cycle but it is better
than having a non LTS django release in Debian stable. Synchronising
releases across multiple upstream teams is not feasible.
The next django LTS release after 1.8 is 1.11 which is expected to be
in the next Debian stable release - buster.
https://www.djangoproject.com/download/
Python3
========
The last django LTS release to support python2.7 will be 1.11 LTS
which retains upstream security and data loss bug fix support until at
least April 2020.
The LAVA software team have already been testing lava-dispatcher V2
with python3. Once the V1 code is removed from lava-server, testing
will proceed for lava-server too. We fully expect to be able to
support lava-dispatcher V2 and lava-server V2 on python2.7 and python3
for a limited time.
There will never be python3 support in lava-dispatcher V1 or lava-server V1.
Python2.7 support for lava-dispatcher V2 and lava-server V2 will be
removed at some point prior to the release of the next django LTS
after 1.11 which is 2.2 LTS, due in April 2019.
Once django 2.0 (non-LTS) arrives in Debian experimental, the LAVA
software team will begin testing lava-dispatcher V2 and lava-server V2
with python3. For some period in 2018, both python2 and python3 will
be supportable but only when building local developer packages. LAVA
production releases will stay on python2.7 during testing with
python3. Once we are happy that python3 support is ready, there will
be an email to this list and the following production release will
provide only python3 packages. This change will be transparent for
most instances, the upgrade will simply pull in a lot of python3
packages and the python2 packages will be left unused.
Reviews and contributions to lava-dispatcher V2 already run the unit
tests with python3 and once the V1 code is removed, this will start to
happen with the lava-server code as well.
LAVA can move to python3 at any time once the V1 code is removed - all
the necessary support is already available in Jessie. LAVA will have
to move to python3 before django 2.2LTS is the only version of django
released in Debian stable.
Details of how this will be managed are yet to be defined. We expect
to continue parallel support of python2.7 and python3 for a limited
time.
It is likely that support for jessie-backports would need to cease
before support for both python2.7 and python3 is included, just to
manage to matrix of builds and tests. LAVA can support python3 using
django 1.11LTS but must drop support for python2.7 to support django
2.2LTS.
Summary
=======
17th June 2017 - expected date of Debian Stretch release
August 2017 - migration of Cambridge lab to Debian Stretch instead of Jessie
November/December 2017 - removal of V1 codebase
Q1 2018 - testing with django 1.11 LTS
Q2 2018 - testing with python3, likely end of support for jessie-backports
Q3/4 2018 - add python3 support
Q1 2019 - drop python2.7 support.
April 2019 - release of django 2.2LTS
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
jessie-backports
============
New package update required from jessie-backports.
sudo apt -t jessie-backports install python-voluptuous
This package is used to check the submission schema of V2 test jobs.
It is already installed for V2 support but 2017.5 requires an updated
version which already exists in jessie-backports. For a smooth upgrade
ensure this package is upgraded as above before installing 2017.5.
As with previous releases since 2016.12, the 2017.5 release is only
available from the lava repositories:
https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
New device-type templates
===================
2017.5 includes a lot of new device-type templates and a number of V1
device-type configurations. These have been contributed to LAVA, as
part of the support for KernelCI, by community members. Many thanks to
all those who have contributed device-type support for this release.
Action names and effects on timeouts
===========================
2017.5 makes the individual names of actions in the source code
consistent across all classes and deployments. Where previously
power_off was available as an action name which could be specified in
timeouts and protocols, that action name now uses a hyphen instead of
the underscore, power-off. The same change has been made for all
classes across the V2 lava-dispatcher codebase. Future changes will
only be accepted using a hyphen to separate words, not underscores or
other characters.
This change affects V2 jinja2 templates as well as V2 test jobs (if
those jobs override the action timeout or call an action using
protocols.
The full list of changes is below:
lava-server
========
8ec2d6736 Fix jinja2 templates for default string handling
eb39ec0fd Extend bbb template check for ssh_host support.
55a251b87 Fix documentation example test job and remove unused
7769be718 Update docs for change in submit behaviour.
c10bf6197 Add more index items and detail on namespaces
b9686e5b1 Mark V1 XML-RPC functions as deprecated.
a0ac9894b Prevent health check warning when disabled
78f796eee more silencing of unit test logging
064bfabb9 Add a template for frdm-kw41z and delete a duplicate for k64f
a08df108d Silence logging in more unit tests
97dbd95b1 Extend power-off timeout for b2260
0e39966d4 device-types: add Hardkernel meson8b-odroidc1 board
b6151b003 Add support for aliases in device-type management
15589b8d1 Fix some typos in development documentation.
f476a351c device-types: base-uboot: use run bootcmd
6911b8502 Expand notes on reviews
cbbdcfcf4 Expand notes on code analysis around reviews
970e52230 Drop confirmation page on job submit for V2 jobs.
4a5050691 Device commands are allowed to be lists
cc60805db Adjust hikey template to allow target_mac and ip support
9d4878c40 Avoid forcing the date path immediately
68497b557 Remove the character_delay block override for d03
5aeed4aef Tags: fix HTML syntax errors
a0ae8a62d Update doc for adding a pipeline worker.
2f6287e71 Add Raspberry pi devices
69a8f40c4 Add collection of Exynos 4 and 5 devices
e31953519 Add more Tegra124 based devices
b843bfaec Add more r-car generation 3 devices
24d21cdb0 Add a note on https repositories and apt-transport-https package.
c3b852798 Expand notes on portability
c57c9f111 Fix doc to explain unprivileged containers and DUT interaction.
2018f705c Show the requested device tags in the job log page
96999537e Fix Action names (use - instead of _)
f0a272b4b Extend recent job support to requested device type
9040de9ca Add XML-RPC call to obtain job level metadata
a04086dd7 api: add get_recent_jobs_for_device
8bbd298f7 Set the documented flash_cmds_order for hikey
c9b9453d1 Migrate many U-boot devices to v2 configuration
613d17fb5 Tweak the developer workflow to skip devices/
c59cc6c31 Add a unit test for some of the new UBoot support.
e43e244b0 Remove unused imports and unused variables.
e2b3784f4 Fix pep8 error
af3cc6d42 Schema: Allow boolean variables in parameters
4d5861607 Allow is_valid check to operate correctly.
294bb62bd ensure device_type is checked
b8ce5ba49 Add a note on developer branches
11725bc3e doc: fix a small gap about test suites
6280b94d7 Add "sd" for removable media
21a5ecf6e Add "command" action to schema and device template
8db43278a Add schema validation for test/monitor/name in job definition.
0a5b1eaa3 tweak gitignore
e3f003f63 templates: remove duplicated blocks
f1331ef2e Exclude retired devices from Device Health table
790b39b9d master: use yaml.CLoader that is way faster
749a1081c Add notes on load balancing different bootloaders
26b822d67 Add note on how pyudev replaces / with _
cc9e4ea2c Allow to override U-boot bootcmd command
9a007665c Fix 500 when output.yaml is invalid
b3c2d162b Make it easier to spot incomplete test jobs
fe6af63a7 Improve job and device schema validation
4a568e261 Fix directory and file permissions
f33661ae4 env: fix comments about default values
fd6fe12b6 Fix scheduling when putting a device into looping
99b35ba79 Export the full lava-server version
da87efaa6 base-uboot.jinja2: add support for append_dtb and use_xip
babaef51d lava_scheduler_app: api: Add pipeline information to get_device_status
abe787872 Add a note on installing lava-dev
2b6ff1dca MASTER_CERT of lava-master should use secret key
d9e7e2c08 add recipients in notifications.yaml
44eb1b75b Move job outputs to sub-dirs based on submit-time
839b3ff19 Create directories with 0o755 by default
9459aae36 lava-master: call job.output_dir to get the path
8c9af4897 Fix description for devices and workers
541478930 Fix health-check tests by testing None and ''
68b33d572 Use job.output_dir whenever possible
d93df5f4f Add a management command to remove old jobs
bfd57121d Move unnecessary constants into base jinja template.
lava-dispatcher
===========
3be27df4 lxc protocol: simplify the tests
a5d9ebad Fix detection of missing ssh_host value in validate
234eef91 Fix calling of protocols after LXC change
477a36b7 tests: make ShellCommand.logger a DummyLogger
62867f49 Use images.validation.linaro.org files for unit tests
1868936c Silence logging in more unit tests
609c783c Fix download action name
b1701978 More tweaks to silence messages from the unit tests
29efa263 Adjust for pep8 checks in jessie
0733f89f Make sure the test_character_delay is used for all
commands in test shell
78d2509f Account for empty environment string.
ea09b19a Take namespace into account when counting test stages
734688c5 Declare namespace of the test suite in results
49a6b92c Drop noisy info log message.
4fd5d4af Drop unused imports and unused variables
678b216c Rework the removable action to allow sd cards
3b42122a Fix bug #2975 after qemu-nfs introduction
7462ea62 Expand LXC support to add devices from all sybsystems
b853865f Fix missing check for u-boot commands parameters
8293693c Replace invalid characters in test_case_id.
a94e0f24 Fix preseed/late_command appending
db03483a Fix Action names (use - instead of _)
9e836e64 Add a Command Action
bbd4ba25 Make the parser stricter about the block names
65b3a563 Allow to pass integers to run_command
b41326bc Raise JobError instead of NotImplementedError
073f3b34 update gitignore
00f5d33f log: limit the length of lines send other zmq
5b2b0746 LAVA-889 Fix handling of multiple test blocks
3bfdcdb6 Add support for use_xip
8df17dd7 Add support for append_dtb
3c110cae Remove unused imports
176a4d49 Add new utility function infrastructure_error_multi_paths
dc008eb5 Export the full version string.
daebfad1 Extend secondary connection fix to support primary
fae0ced5 Move unnecessary constants into base jinja template.
18f745bc device-types: add kirkwood-db-88f6282.conf
815e518d device-types: add at91sam9m10g45ek.conf
6ca0731f device-types: add at91rm9200ek.conf
003b53dc device-types: add at91-sama5d4_xplained.conf
9b2371cf device-types: add armada-3720-db.conf
5a767331 device-types: add armada-xp-gp.conf
8b050938 device-types: add kirkwood-openblocks_a7.conf
f5801d38 device-types: add alpine-v2-evp.conf
19e20276 device-types: add sama5d34ek.conf
49da704f device-types: add armada-385-db-ap.conf
ed7aa6b1 device-types: add armada-370-db.conf
9a104327 device-types: add at91-sama5d2_xplained.conf
ddb34cea device-types: add alpine-db.conf
f7642f58 device-types: add armada-375-db.conf
0d70c7dd device-types: add at91sam9x35ek.conf
a642d451 device-types: add armada-xp-db.conf
2a5ff9e4 device-types: add armada-7040-db.conf
1a439995 device-types: add orion5x-rd88f5182-nas.conf
ad855a62 device-types: add armada-xp-linksys-mamba.conf
58308ca1 device-types: add armada-388-gp.conf
6a2898bf device-types: add armada-370-rd.conf
cd1b4f47 device-types: add armada-398-db.conf
51925fb1 device-types: add sun8i-a83t-allwinner-h8homlet-v2.conf
4976d068 device-types: add sun8i-a33-sinlinx-sina33.conf
681480d5 device-types: add sama5d35ek.conf
db74d60e device-types: add sun5i-r8-chip.conf
40736b95 device-types: add sama5d36ek.conf
433be0a6 device-types: add imx6q-nitrogen6x.conf
ed4c82e6 device-types: add at91sam9x25ek.conf
ce7a488e device-types: add at91sam9261ek.conf
f87cdb49 device-types: add armada-xp-openblocks-ax3-4.conf
634442ca device-types: add armada-388-clearfog.conf
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
For 2017.5, we are planning on changing all of the action names
(self.name) to use hyphens to separate words instead of the current
mix of hyphens and underscores. This is to make future support more
consistent.
All V2 templates currently in upstream will pick up these changes as
part of the review process.
Any V2 templates which are not upstream and which reference timeouts
for specific actions may need adaptation, however, note that 2017.5
also moves a lot of the timeout settings to base.jinja2 anyway.
If you have test jobs which reference specific actions, these will
need changes too.
We're implementing a change in the dispatcher codebase which verifies
that underscore is not used in the self.name of any class inheriting
from Action, so all code reviews will also need updating.
https://review.linaro.org/#/c/19027/
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
This interim release includes a small set of fixes aimed at the LKFT
instance and HiKey support using LXC.
If your instance uses the pre_os_command support for fastboot with LXC
(not just pre_power_command), then note that this support now needs to
be called from within the LXC protocol.
See https://lkft.validation.linaro.org/scheduler/job/885/definition#defline77
for an example.
Changes:
lava-dispatcher (2017.4.post1-1) unstable; urgency=medium
.
* Production hotfix release for LKFT support.
3f99cc3 Add missing super call to WaitUSBDeviceAction
5436658 Fix indent typo
4847bf1 Put device rebooting info message at the correct place.
6b85382 Fix call to lava-lxc protocol for pre-os-command
bdb4d12 Fix test_lxc_api unit test
e9df320 LAVA-920 - Workaround ptable issue in firmware
7b4096c Allow LXC protocol to call pre-os-command
b33d04b Fix OE image support for HiKey.
d714a9b Add missing test shell helpers
9518e74 Add missing import
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
V2 healthcheck changes
======================
https://lists.linaro.org/pipermail/lava-announce/2017-March/000027.html
This support has now been delivered in 2017.4 using the lava-server
manage migrate-health-checks command. If used, the --clean option will
remove the old definition from the database for the device-type.
https://staging.validation.linaro.org/static/docs/v2/healthchecks.html#inde…
There is also a new migration status page to help admins track which
devices still need to be migrated to V2.
Branding message
================
The branding support now includes a message which can be added to the
home page for the instance. For example, a short message to describe
the main purpose of that instance.
Reminder about the LAVA repositories
====================================
Until Debian Stretch has been released and the new testing suite
(called buster) is open for new uploads, new production releases of
LAVA will only be available from the
images.validation.linaro.org/production-repo repository.
See https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
See also: https://lists.linaro.org/pipermail/lava-announce/2016-December/000023.html
Changes
======
A longer list of changes this time as 2017.3 was skipped to allow for
the Linaro Connect BUD17 conference.
lava-server
===========
7e36443dd Restore UEFI boot method for HiKeys.
6679d548e Fix dragoboard410c boot sequence.
5082adc80 Move anchoring fix for navbar to base template.
187c8520a Allow to override U-boot tftp command
0d28de652 Handle OSError as well as IOError in metadata store
be04868ff Update migration status page for disabled healthchecks
5f689e930 Improve job page
ed4dee272 Expand the docs on arbitrary device_info elements
35a7378e0 Remove references to deprecated health_check_job
5e5e6890c Add a link to the migration page
84f88a4fc Allow to override U-boot usb command
043bfeea4 LAVA-916 - Restore adb connectivity with hikey in V2
4211de051 Don't force {BOOTX} macro in U-boot template
571d71313 Don't force dhcp in U-boot template
b0184585c Add a section on which file is what in /etc/
9701ad6a0 Adding timeouts to Juno template
b2b8d8851 Add content on developer workflow
e36889a81 Revert change for debug filter button
c06ed4a22 Fix bug 2927 - typo in OPTIONS for $MASTER_CERT
cfa0c02c5 Fix navbar problem with in-page anchors.
bd554a2c4 Add docs on boot commands.
624c79ee1 Fix bug #2925 - LDAP Configuration demo mismatch distro.py
1ddde696b Another parent timeout needed for qemu
c3f356c0b Allow boot_qemu_image action timeout to be overriden
b24fe09af Use standard paths to keep the unit tests running
7433bb4e4 Add note on installing developer builds
a0e7756c7 Add a migration status page
ce59699ad Tweak the device type docs
adf670e7d Update copyright year for the docs to 2017
b2428ad8c Small updates to the docs for test development
6610ce36f Allow to import token from another instance
d57e5217b Cleanups for advanced installation docs
0119fc57f Fix job submission page
0c4e68769 Dedupe the index page and add more entries
ec0aa40ad Improve device page
912c174b1 add notes on the new lava-tool features
a662f0ed6 Add a standard NFS test job for panda
e80ed1e13 Update docs on transfer_overlay
b0ae30b56 Update migration for job_status_trigger for backwards compatibility.
f28af7fbe Fix mustang doc examples and add uefi-menu
a755b4b12 LAVA-768 allow to disable hc for a device type
7a05b7941 Add support for a branding message
b18e8d293 log: add a link to the top and to the doc
0fd8c985e fix typo in rst header
75d411f14 Do not send a spurious POST submit request
d7f8d6322 Update docs for change in behaviour of log file UI
acd068173 LAVA-112: display status and role for sub jobs
bf144047f Check V2 devices have valid config before reservation.
0a2da3f92 Skip retired and obsolete device types
c106b17c0 LAVA-913 - Documentation for lxc persistence
14c733d30 Add another index entry for the example first job definition
b8629e0e3 Expand simple admin to include administration outline
43bb9da87 V2 jinja2 template for armada-8040-db
ca05a93d0 Add a command to migrate health checks to the fs
ad55286d6 doc: fix label
0b17a0496 Add qemu NFS docs
a8bf524f4 Update of doc examples
7bc903e70 LAVA-736: move health checks to the file system
1de7ea5b3 LAVA-912 - Document verbose for lxc protocol
643c23efc Remove deprecated (and unused) functions
91cff154b Add notes on when to add more workers
69df43a50 LAVA-904: Improve job page layout
3e3075f2f Reorganise the scheduler app test files
639600952 LAVA-910 allow iteration if vland tag check fails
1ebe9d217 Improve readability of testcase results
b7792c56e fix typo in index syntax
143b5f742 Allow boot_interface override
b85bc07a6 Update default interrupt_* in hi6220-hikey device type jinja2 template
a561326bc Add a test case for secondary connections
d6a441c16 Add Acer Chromebook 13 CB5-311
7ded576cb Add Rada Rock 2 Square
65e2b1e22 Add Renesas R-Car Starter Kit Pro
61edf0180 Add Samsung Chromebook 2
068005a9a Add i.mx6q sabre lite board
a84a509e4 Allow u-boot-interrupt timeouts to be overridden
b7a23dbfd Fix test job timeouts and fix doc
c94238613 Ensure extra_kernel_args is used from job context
77693f080 Clean up error handling in is_deprecated_json method.
8b5fa8130 Add a 'full docs' link to the front page
9ab49a8e9 Allow unit tests to run without kvm module
e47c5c005 Change job_status_trigger field type to a non-deprecated one.
6ec411a55 Tweak the authentication configuration docs
7ee493bb1 Tweak the authentication configuration docs
baa9f5ebb Update references for a permanent location.
da79c4be7 Add available architectures for qemu 'arch' validation.
65e4c35ec Prevent intermittent unit test failure.
7f9b8760d LAVA-215 QEMU NFS support (server side)
3f2f414e2 LAVA-619 document changes for bootz to zimage
8bbbd425f Ensure missing font is available for packaging.
0f2d8a4ff X-axis attribute in charts now use NamedTestAttribute table.
93cc20aa6 Fix export custom queries feature.
a8d6cab43 Display results sorted naturally by key in job logs.
e7aa614c5 Update the default length of the job result table.
e2c07c1ac Fix doc example YAML syntax to read from file
ce59d6b94 Fix sphinx build errors in manpage
63d5f6fa5 Add a check on all templates in source tree
d9eb3e1f3 Ensure character delays are supported by overdrive
75efcb00b Allow repeating test cases
93d620a53 Explain how connection-namespace parameter works.
9041b0dc0 Begin a section on common YAML syntax errors.
6521cc0f8 LAVA-894 - Document lava lxc device add / wait command
ee3e5964f Fix documentation - punctuations, links, bullets.
988d89a34 Add server side support for mustang EFI Grub.
525d9ed47 Fix code blocks in Boot Action documentation.
106f7f4fc LAVA-890 - Allow DUT to stay in Android OS
887175d82 Remove unused images
9f9e8d304 Fix unknown test case handling in tables
bff6ea8ff LAVA-888 - Download button in Device Dictionary page
3314966de LAVA-887 - Introduce fastboot sequence in device type templates
1a1be1ff2 LAVA-867 - Allow fastboot options in device dictionary
d876d1463 Remove legacy nexus deploy document from V2 docs.
2750804af Updating Juno jinja2 template to accommodate vemsd deploy action
2b6d3d277 Deprecate "pipeline-worker" in favor of "workers"
587bc8931 Add a "details" command to "device-type" and "workers"
96761e533 Add a command to manage devices
3b2f27ee0 Encode maintenance state machine in unit tests
730b57b81 Do not put into maintenance retired devices
c0adfd2c2 Add a command to go into maintenance
49abd8bf7 Fix state transition when OFFLINING a device
f337d7d1c Go from OFFLINING to RUNNING when a job exists
39b50ec15 Extend docs on simplistic testing
8e56315eb Workers: allow to set properties
944caa4fd pylint improvements
c42629220 Use continue in a loop, not return
61b6886f9 add notes on debugging multinode as admin
2ed1f35c7 Device tables: add more fields to select_related
d95ad2d5b job logs: Highlight all failing results
ace40b01a Generate less database queries
7fcbd7c68 Allow to use last version of Django Debug Toolbar
61069528d LAVA-862 - Integrate Google Pixel into LAVA V2
e7925e7a1 Update docs on publishing and attaching to tests
da99f08f0 Fix a typo in nexus5x device template used for unit test.
ba5a106db More stdout changes to clean up the test output
c84741460 Skip panda_lxc template check if lxc not installed
6a0a73d36 Fix a typo in documentation.
79ea7cedd Fix looping bug in template
466f1506a LAVA-841 - Document device_info attribute
d420b9d5f LAVA-856 - Integrate nexus5x into LAVA V2
de33008ec Autologin doc update.
lava-dispatcher
===============
7aac7ae5 Fix problem with self.host on secondary connections
530ca366 Ban / in test case names, replace with underscore.
31ee0012 If allow_fail is not set, ensure the command fails
e53a280f Remove old device_type config files which are unused.
80fb3997 LAVA-916 - Restore adb connectivity with hikey in V2
c57a102b Add debug output to the LXC device operations.
7deea0fb Add missing calls to super
341ac593 Writing connection info to namespace data
3ee2af87 Upgrade the boot commands output to info
a1a87daa Fix bug in checking return value of action run_command output.
604323b0 Add a unit test for primary ssh connections
561f78a6 LAVA-914 - Action run_command should return log on allow_fail
a8fb0d9d device-types: add Armada 8040 DB board
98fc1d4d LAVA-911 transfer overlay to the rootfs
e85af0a2 Report versions of critical software on the worker
bdc97bd5 LAVA-901 - Allow lxc debug output if requested in job
93aab7ab Add systemd packages as default for debian and ubuntu templates.
d01f01a3 Allow params for install git repos support
eb821dc8 testdefs: rename the extra results keys
f262b21c testdefs: fix TOCTOU issues with test definitions
07affb55 Print warnings in validate and not in populate
3de0aac9 Make empty revision message more friendly
685eea49 Do not populate append when using tmpfs GuestFS
22a7ba93 Add missing imports
a1baf85e Add missing JobError import.
9e8c3c2f Improve validate and job summary
b1100080 Catch error in secondary connections
7428a9ae Improve git error logs
949b3fcd Rework the timeout settings in job and device conf
6933ac63 download: improve error message
304c39f7 LAVA-852 - LXC fastboot support for persistent devices in V2
fc28aafe Add an error type to the job result
522946b8 Make '/sys/class/misc/kvm' a constant.
f9a1b381 tests: use the DummyLogger whenever needed
ee11c796 tests: use permanent urls
538ff35f Context architecture validation for qemu.
bd88adab Check for enabled kvm module on workers.
28c66090 Fix bug #2898 - Job doesn't exit on fastboot-deploy failure
2879c019 Handle qcow2 conversion errors
7daf65d1 Action: use LAVAError base exception class
603fd049 shell: remove unused exception handling
da97077d Add missing import and remove uneeded one
beccc4da LAVA-215 QEMU NFS support (dispatcher)
c2e3ee34 Use DummyLogger for grub and add .target()
ed3519d2 Get uboot header length from utils.constants
7df2904d If no kernel type given in deploy, use the boot type until deprecated
82237860 Ensure the deployment data shell is exported.
108ef4a3 Add the commit-id to the lava testdef results
1e3dd3ce run: improve error handling
41b08257 lava-dispatch: use the new LAVAError base class
171e8b65 device-types: add Allwinner A23 EVB board
5898747b device-types: add OrangePi PC board
4ffb4c1e Simplify lava-test-shell by removing stdout.log
a5222296 validate: improve error handling
c2df0b20 Remove the message from cleanup()
78c334c2 Simplify test/shell.py check_patterns signals.
68f1c822 test-runner: remove redundant functions and files
3fe2105e Fix bug #2888 - adb root command from job breaks connection to device
22157cd2 Add a base class for LAVA exceptions
44af91c3 Support using UEFI Menu to load Grub using PXE
b667f2fd Use LAVA exceptions
1690ed95 Pylint updates for deploy actions
e538b5d8 LAVA-890 - Allow DUT to stay in Android OS
bbcf76fd Raise a TestError when the test installer fails
0d45d8fa test-runner: use the current shell
27593024 Blacklist brltty on workers
e2593966 Allow reusing of serial connection from another namespace.
cfe9821b LAVA-867 - Allow fastboot options in device dictionary
d5094f28 Fix unknown usage command in lava-test-reference
381eca8d LAVA-881 - Sequence list for fastboot pipeline
e84c00b1 action: be consitent when printing result duration
82d7f82f Print the result of a test run only when avaiable
1d09ece0 Catch python3 errors to clean up output
833d7207 python3: only show requests logs above warnings
5fb1e8a0 rpcinfo: also capture the output
496c7181 Sanitise default mkimage_arch and kernel handling
a6838ecb LAVA-862 - Integrate Google Pixel into LAVA V2
d2f968e3 Follow up to stabilize dragonboard 410c.
8ab18e05 rpcinfo check needs to specifically check NFS service
61777bb0 Refactor wait usb device action to stabilize dragonboard 410c.
e7649e64 Remove references to device_path in device configuration.
3fd11d81 LAVA-856 - Integrate nexus5x into LAVA V2
d970298d Adding new action for monitoring USB mass storage devices via udev
94876fd7 pyocd: allow multiple images to be flashed
26528b0c Detect bootloader resets.
09650ab1 Adding a Versatile Express Firmware deployment action
605fe529 Detect errors during auto login.
00306b79 Job: log the job result as a logger.results
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
2017.4 will deliver support for migrating V2 health checks out of the
database and into /etc/lava-server/dispatcher-config/health-checks/ on
the master. This has several benefits for admins.
* Health checks from 2017.4 onwards will be based on the device
configuration, not the device type. This applies to V1 devices and V2
devices.
* Multiple instances can all have the same V2 health checks controlled
in VCS using configuration management tools.
* V2 devices of the same device-type can have different V2 health
checks according to the Jinja2 template specified in the extends field
of the device dictionary. Submissions to such devices would need to be
managed through device tags.
* Devices which are V1 only (is_pipeline is False in the django admin
interface) will continue to use V1 health checks from the database,
based on the device-type.
* Devices which support V2 (whether these devices are exclusive to V2
or not) will only use V2 health checks from the filesystem, based on
the device dictionary of each device.
This does mean that devices which are currently both V1 and V2 will
need to have a V2 health check from 2017.4 onwards or the device will
no longer run a health check at all. Admins may choose to submit and
monitor regular V1 testjobs until all test jobs for those devices are
V2 but those test jobs will not be able to take the device offline
automatically.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
There is a bug in python-lockfile available from jessie-backports:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858835
This affects fresh installs of LAVA when LAVA is installed directly
from jessie-backports without first being installed in jessie.
The underlying problem is with apt which pulls in the updated version
of packages which have a backport available even if the requested
package does not explicitly ask for the backport of that package.
The problem shows up with lava-coordinator first - the package fails
to install cleanly because the daemon cannot be started correctly.
To fix this problem, simply instruct apt to install python-lockfile
from jessie, not from jessie-backports:
apt install python-lockfile=1:0.8-2
lava-coordinator will then install correctly.
This change will persist over later upgrades as apt does not consider
jessie-backports as upgrades of jessie - except in the case where the
package is not yet installed.
Other daemons also now need to be restarted:
$ sudo service lava-server restart
$ sudo service lava-master restart
$ sudo service lava-slave restart
Future builds for the staging-repo and production-repo will force only
version 1:0.8-2 of python-lockfile as these builds are done on and for
jessie.
Systems already running Stretch or unstable do not seem to be affected
and can safely use python-lockfile at version 1:0.12.2-2 - this is a
particular problem inside jessie-backports. However, this does mean
that until this bug is fixed, users running Stretch or unstable should
build their own developer packages instead of using the LAVA
repositories on images.validation.linaro.org. We currently do not have
capacity to build separate packages for jessie and stretch.
(LAVA does not recommend using Stretch or unstable for production
systems until Stretch has been released and stretch-backports becomes
available.)
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Reminder about the LAVA repositories
====================================
Until Debian Stretch has been released and the new testing suite
(called buster) is open for new uploads, new production releases of
LAVA will only be available from the
images.validation.linaro.org/production-repo repository.
See https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
See also: https://lists.linaro.org/pipermail/lava-announce/2016-December/000023.html
2017.2 release
==============
More updates to the jinja2 templates in this release and some new templates.
diffstat output:
lava_scheduler_app/tests/device-types/apq8016-sbc-uboot.jinja2 | 3
lava_scheduler_app/tests/device-types/arduino101.jinja2 | 5
lava_scheduler_app/tests/device-types/b2260.jinja2 | 8 -
lava_scheduler_app/tests/device-types/base-uboot.jinja2 | 26 ++--
lava_scheduler_app/tests/device-types/base.jinja2 | 11 +-
lava_scheduler_app/tests/device-types/d02.jinja2 | 4
lava_scheduler_app/tests/device-types/d03.jinja2 | 1
lava_scheduler_app/tests/device-types/frdm-k64f.jinja2 | 30 +++++
lava_scheduler_app/tests/device-types/hi6220-hikey.jinja2 | 7 -
lava_scheduler_app/tests/device-types/highbank.jinja2 | 2
lava_scheduler_app/tests/device-types/juno-uboot.jinja2 | 1
lava_scheduler_app/tests/device-types/mustang-uefi.jinja2 | 3
lava_scheduler_app/tests/device-types/nexus10.jinja2 | 1
lava_scheduler_app/tests/device-types/nexus4.jinja2 | 1
lava_scheduler_app/tests/device-types/nexus9.jinja2 | 1
lava_scheduler_app/tests/device-types/nrf52-nitrogen.jinja2 | 9 +
lava_scheduler_app/tests/device-types/nucleo-l476rg.jinja2 | 23 ++++
lava_scheduler_app/tests/device-types/nxp-k64f.jinja2 | 1
lava_scheduler_app/tests/device-types/nxp-ls2088.jinja2 | 12 ++
lava_scheduler_app/tests/device-types/overdrive.jinja2 | 1
lava_scheduler_app/tests/device-types/r8a7791-porter.jinja2 | 11 ++
lava_scheduler_app/tests/device-types/stm32-carbon.jinja2 | 11 +-
lava_scheduler_app/tests/device-types/x15.jinja2 | 55
++++++++++
lava_scheduler_app/tests/device-types/x86.jinja2 | 1
24 files changed, 178 insertions(+), 50 deletions(-)
Changes in the 2017.2 release are described below, including the
short git commit hash, include:
Use the git commit hash to go directly to that commit using a URL like
https://git.linaro.org/lava/lava-server.git/commit/?id=266b2e633
Commit messages including a reference to LAVA-NNN relate to projects.linaro.org
append the reference to https://projects.linaro.org/browse/
e.g. https://projects.linaro.org/browse/LAVA-854
lava-server - https://git.linaro.org/lava/lava-server.git/
-----------
9df8b354b Remove whitespace from case names
9354cd585 Uboot parameters fixes
297f27900 Fix documentation for auto_login
64b549e9a Fix group visibility and hidden device types
b211434de Document the use of a list for bug 2870
eb8a7bec3 LAVA-854 - Device type template for x15
ad81f33fb Add device template for renesas r8a7791-porter board
8c4d44ae4 Add device template for nucleo-l476rg
112ec6487 Remove send_char, and fix some broken character_delay settings
8f615cd6b Update reviewers handling
0fe482ffb b2260: remove duplicate ip_args from extra_kernel_args
a72c86732 LAVA-847 protect metadata store IOError
53a3c778e If a text_offset is used, quote it to ensure it is a string.
51351e18c Add documentation of the dispatcher-config support
7a1bfb649 Tidy up the Device-type templates for IoT devices
0b5de4dcb Allow lava_scheduler_app to unittest logs to stdout
bc807def4 Fix sizelimit warning handling for V2
13b691615 Summarise job timeouts with example and notes
b5c2ea1a7 Improve notes on portability.
65ae7d644 Fix internal server error due to unpacking TestJobUser.
83e8fa075 Migration doc improvements
9eb145050 Add a unit test for the ethaddr support in b2260
91eb07a36 Creating and restoring backups for V2
479a6295f Add support for setting ethaddr
52b5cf06d Add triage documentation for power failures
526a6e298 LAVA-840 - Remove documentation for USB_DEVICE_WAIT
3415d5319 Allow .py files in examples/sources to be packaged
be33389bf Allow notifications for jobs in Running status.
4d6afd607 Fix blacklisted bug in notifications.
58b48a8f0 Catch all errors during resubmission.
40b010238 Tag admin clean support must return name
14b15f40b LAVA-734 compatibility docs
dd18c26e1 scheduler: do not cascade deletion for some objects
5415eafa7 Do not create unused TestJobUser
2f3cd8389 Bug 2796. Add submitter username to job_details API.
08cd6c01f Revert "master: set the master identity"
be28c8c7b LAVA-814 - Explore pyudev for usb device wait
9150c9973 LAVA-832: Rework device-types management command
70e81493e LAVA-762 link level in timing to pipeline definition
c276df83f LAVA-728 extend ZMQ example to submit and wait
f6a0456a1 LAVA-731 document remote worker issues
41c67ae3b Add device template for NXP-ls2088
ee8964c7b Remove spurious call to save()
2a79a622d Ensure base_ip_args can be overridden
affa68b23 Add a command to add and list workers
10e6ea970 tokens: allow format as CSV
b0b5a1d30 Remove unused config value LAVA_CONFIG_VERSION
69138ac89 Add a command to manage tokens
e09beb1a4 Remove unused custom commands
905a0f9be Fix multinode job definition page
bb0fdc7a8 job logs: Highlight failing results
e1a125aa8 master: set the master identity
b7cb28e49 Fix device schema and test for arduino template
d411416bf LAVA-707 update result case and metadata views
15fc5c62f LAVA-140 - update running job calculation
9fcf92cfd LAVA-515 restore line numbers to YAML definitions
b7266712b LAVA-824 skip unused device types in running table
de6a141aa Expand docs on namespace
c54a5e262 events: fix documentation
lava-dispatcher - https://git.linaro.org/lava/lava-dispatcher.git/
---------------
ba779908 testdef: set "set +x" to avoid duplicating SIGNALS
ae2ab424 Enforce a download order to produce predictable pipelines
55a39b92 Remove unnecessary boot action in fastboot x15
81b178fc Add a unit test for db410c pipeline generation.
aa96dd75 LAVA-839 - Integrate X15 into LAVA V2
14acd2e2 Fixup u-boot prompts sanity
829fa380 Support CMSIS-DAP devices that do not reset after being flashed
d09a7a74 Simplify send_char support
031ba8e8 Ensure dispatcher.yaml is available for admins
aa6142dd Detect pdu controlled device using availability of power command.
c54891b9 Properly isolate stdout and stderr in unit tests
2d8b3110 Ensure lava-test-reference is available for release
555b6be0 Skip one test if lxc-info is not installed
fc42bd89 Fix support for persistent NFS
57c96b4b Remove duplicate prompt handling.
b0d39204 LAVA-844 allow string and dict for install: git-repos
76998b75 LAVA-830 Use the configured TFTP directory
1f01283f Partially revert a1658c by using a DEALER socket
9045292b Allow boot commands in the job definition
af34dfcb Extend the timeout unit tests to boot actions
9135c5aa Add support for CMSIS-DAP IoT devices and cleanup PyOCD
96dcc7d7 Only match udev events when device is added
a1658c7e slave: use a ROUTER socket to connect to master
35937826 Fix prospector warnings
e4e2fb97 etc/lava-slave: fix documentation of WARN log level
b45cb005 Remove unsed imports
93a282bf Job: log known error status when leaving
ee404039 lava-dispatch: allow to run without zmq logging
26daca49 lava-dispatch: work around a variable clash
c62e1ee9 LAVA-814 - Explore pyudev for usb device wait
989cd97b LAVA-827 ensure the case name is a valid URL
570c1b1e Do not reset the timeout before each connection
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
