Reminder about the LAVA repositories
===========================
Until Debian Stretch has been released and the new testing suite
(called buster) is open for new uploads, new production releases of
LAVA will only be available from the
images.validation.linaro.org/production-repo repository.
See https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
See also: https://lists.linaro.org/pipermail/lava-announce/2016-December/000023.html
2017.1 release
===========
This release includes substantial updates to the jinja2 templates. If
you have made local changes to these files, you will be prompted
during installation.
/etc/lava-server/dispatcher-config/device-types/apq8016-sbc-uboot.jinja2
/etc/lava-server/dispatcher-config/device-types/arndale.jinja2
/etc/lava-server/dispatcher-config/device-types/b2120h410.jinja2
/etc/lava-server/dispatcher-config/device-types/base-uboot.jinja2
/etc/lava-server/dispatcher-config/device-types/base.jinja2
/etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2
/etc/lava-server/dispatcher-config/device-types/cubietruck.jinja2
/etc/lava-server/dispatcher-config/device-types/highbank.jinja2
/etc/lava-server/dispatcher-config/device-types/juno-uboot.jinja2
/etc/lava-server/dispatcher-config/device-types/mustang.jinja2
/etc/lava-server/dispatcher-config/device-types/panda.jinja2
/etc/lava-server/dispatcher-config/device-types/qemu.jinja2
Changes in the 2017.1 release are described below, including the
short git commit hash, include:
Use the git commit hash to go directly to that commit using a URL like
https://git.linaro.org/lava/lava-server.git/commit/?id=266b2e633
lava-server - https://git.linaro.org/lava/lava-server.git/
--------------
266b2e633 LAVA-795 - Expose lava-publisher event socket settings.
f4b76e48e Fix ordereddict failure
9d3ae86cc Use dpkg-query which is available on all systems
080c3184f Update the device-type context help page
46a055a3b Fixup some jinja2 templates
cecefbfef Timing: improve the description
5bef1d877 LAVA-821 - add support for lava-test-reference
3b26fb234 LAVA-820 record lava-server version in job metadata
ff71a42a6 Append to existing metadata store results
0c9abf643 Fix RST whitespace typo.
05b96af3c Merge timing page into the job page
edc4431f1 Restore access to chart Action buttons
69709881e Improve the timing page and port to the new log format
a00c02aa8 LAVA-535 Make devicetags case-insensitive
36617b955 LAVA-743 - handling complex test operations
50084f1b5 Omit summary button for V2 definitions.
5c8bf4643 Upgrade the dispatcher v2 parser interface.
22409d543 master: send the dispatcher config to the slave
lava-dispatcher - https://git.linaro.org/lava/lava-dispatcher.git/
----------------------
64ae9131 Report the bootloader load addresses used
2085ffb8 Skip parsing of kernel messages for ssh connections
00fa2d24 Improve allow_fail fault handling
dce680f3 Simplify the connection session handling
ea18594a Use dpkg-query which is available on all systems
2cc64b70 Reinstate git version_tag in version.py
5c05bdae Improve logging
50236943 LAVA-821 - lava-test-reference support
a118e943 Add missing wait definition in base class
2dc955f6 Allow timeout during Linux Kernel message parsing
0404a419 LAVA-819 - record lava-dispatcher version in logs
542effdf Raise an Infra error when download fail in run()
29a232c1 Fix Download retries crashes
12e5789f monitor: make test a bit faster
a90a5184 Fix computation of action duration
ffd1d176 Fix retry logic (partial revert of 7bbd706d)
36ca3c21 LAVA-818 - errors should set the job as Incomplete
f5ac0328 Skip generation of invalid test_case_id
9e5e56c6 Re-enable support for nose tests
bb89772f Merge and re-use BootloaderCommandOverlay and
BootloaderCommandAction
f3864c2f job: fix cleanup when validation failed
8dd450c5 run_actions: clean error messages and exceptions
ec44db64 Properly compute the timeout
f3058a12 atexit is not needed anymore for cleanup
7bbd706d Improve error handling and reporting
a5e0e678 Fix issue when downloading two files with the same name
0d3773ea Call finalize directly in job.cleanup
af608eb2 Remove unused setter, properties and __call__
f946a21d validate: do not extend action.errors
4504aeb9 slave: use the dispatcher configuration
925019c8 Use the dispatcher config to get the dispatcher ip
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Enable the LAVA repository
====================
2016.12 will be the last chance to update of lava-server and
lava-dispatcher into Debian Stretch before the release as Debian 9.0.
This means that users of jessie-backports should now enable the LAVA
production repository - see
https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…https://lists.linaro.org/pipermail/lava-announce/2016-November/000019.html
Until Debian makes the 9.0 release, jessie-backports cannot receive
updates of the LAVA releases in 2017, so all updates will only be
available from the LAVA repositories. Shortly after the release, a new
stretch-backports suite will become available and jessie-backports
will be opened again.
In accordance with the Debian freeze policy, it will take 10 days for
the new uploads to migrate into stretch, not the previous 5, so an
upload to jessie-backports will be made at that time.
Note: this may be an issue for armhf or arm64 workers - if you are
using such workers, let us know so that we can prioritise the changes
required to build lava-dispatcher for those architectures.
Updates from jessie-backports
======================
A few updates are required from jessie-backports with 2016.12: lxc and
python-pyudev
The lxc update ensures that jessie and stretch containers can be
correctly created on jessie.
The pyudev support enables lava-dispatcher to monitor udev activity
for certain operations.
These updates can be applied before installing 2016.12, if desired:
$ sudo apt install -t jessie-backports lxc python-pyudev
lava-tool
======
A new version of lava-tool is also available, including a new way of
handling the authentication tokens and the ability to use shortcuts
(like "local" instead of "http://localhost/RPC2").
https://tracker.debian.org/pkg/lava-toolhttps://tracker.debian.org/news/825902
The new version of lava-tool is available via the LAVA staging and
production repositories and the package available in jessie-backports
will be updated in due course.
Unfortunately, it proved not to be possible to export the existing
authentication secrets into the new backend or to support both
backends simultaneously. All users will need to re-add all their
authentication tokens to use lava-tool after this upgrade. The change
has several benefits, including the ability to list and remove
authentication configurations using the new backend and the removal of
a whole host of problems related to using the old backend in
environments where lava-tool was not running in a full terminal. In
addition, this new backend allows lava-tool to support shortcuts which
prevent problems with repetitively entering long and complex URLs.
On installing the new upgrade, users will see the NEWS message:
https://github.com/Linaro/pkg-lava-tool/blob/master/debian/NEWS
lava-tool has changed the backend for token storage.
This means that existing authentications will not be usable to this
new version. This change has been made to fix persistent problems with
the python-keyring support, including:
* DBus errors when used over SSH
* Interaction with gnome-keyring causing authentication failure
* Inability to list or remove authentications added to the keyring
* Inability to work with other keyring solutions.
The new backend is able to list and remove authentications. The new
support also removes the need for a default password on the user
keyring, so lava-tool will no longer pause waiting for password
entry.
2016.12 release
============
Changes in the 2016.12 release are described below, including the
short git commit hash, include:
Use the git commit hash to go directly to that commit using a URL like
https://git.linaro.org/lava/lava-server.git/commit/?id=b19b9648a
lava-server - https://git.linaro.org/lava/lava-server.git/
-----------
541ba7b0a Enable ssh secondary connections on juno-uboot
2d6bb2fcc Handle errors which cause invalid description data
b19b9648a Fix template_mismatch check to use extends
d7f71eae3 Update ipxe x86 nfs to use base_ip_args
fa6b8b3ab Add a note on release-notes on lava-announce
d007d1819 Fix a crash when a device does not have a device dict
d46d6e4fd Protect against invalid parameter submissions
431fd2fc0 Allow pipeline jobs to be used with XML-RPC job_output
f3bf855ba Add Copy to clipboard functionality to MultiNode
19d814e0c Fix some failing tests due to new namespace changes
86950e985 LAVA-745 - expose device_path for all types
083d67d54 Add device-type template for arduino101
fd99c576e Fix UI submission error.
708a38a23 Add SSH support for overdrive
e838de3ba Update Linaro lab links to actual site
d8e127b74 Allow flash_cmds_order to be set
92eefe4aa Adding extra boot commands for secondary media on Juno
74310353c LAVA-792 - add shorthash to changelog
8f860a1e9 Keep result blocks on a single line with MultiNode
3b1d6654e LAVA-499 - Device type template for dragonboard-410c
5c46ea4ab Provide a default fastboot flash commands order.
a61ab2501 Download results XMLRPC support.
347fb3fd3 Allow to forward the messages to additional sockets
4d0753c3a Allow validate_pipeline devices to filter by device type name.
a859e5ede publisher: improve logging and fix logfile owner
f0002efb1 Fix sphinx indentation error
9f9bd71b9 Correctly expose TestSet data in results
c6f9b1df8 Handle decimal conversion error in test case
1769d8b6b Expand hints on new device-types
eec1ca1d0 master: expect a protocol version with HELLO messages
cd8d109f7 master: add the job id to the log messages
a0ac2d66a Add absolute_url to job_details API.
951ab9f68 master: device configuration is empty for secondary media
c0575503d master: improve error reporting when reading env file
f551daa49 Add notes on unattended upgrades
1d2e25d1d Allow override of nfsroot options, not arguments
edaad5419 Update docs to reflect reality of from: support
09503a482 Manually revert d31a8462: errors are already saved
3516b856d add some notes on packages needed for LXC usage
b5ee513bb Rename 'usb-showup' to 'usb-device-wait'
406832582 Drop misleading hostname from device metadata
6a6ee6f4d Handle change to git.linaro.org frontend
d85f99e1b Fix crash when description.yaml is empty
37a94aafc Fix prospector warnings
080499d18 Remove references to wsgi
267d22302 Paste job definition to clipboard as text and not as HTML.
ef09a9698 add a link to the device tag glossary entry
08bd011a4 Update aliases migration for review change
7010775b6 Move metadata extra content to files
95ed9a3ba Expand notes on debugging test jobs and test cases
6d772bb0d Update for pycodestyle pep8 failures
dd58a1182 LAVA-798 - lookup device-types by alias
104feb593 Enable LXC for existing device-types.
376a80021 LAVA-797 - Document changing USB_SHOW_UP_TIMEOUT in job
3e8dbfeca all_devices() api call to do one big query instead of many small ones.
d27f7a4d5 Fix for Debian reproducible builds.
ac8bb8e2a Fix bug #2596 - No result collected for multinode jobs
7b3adad0a All jobs will have a definition, so show it by default.
1e6992f6b Set boot_character_delay in x86 template
5cc0a4746 Extend x86 unit test to check character_delay support
ad4f8780f Combine interfaces block for juno and vland
bb2ecf716 Allow changing the timeout for u-boot-interrupt
19de59978 Add stm32-carbon jinja2
lava-dispatcher - https://git.linaro.org/lava/lava-dispatcher.git/
---------------
19fd79a9 Update preseed support for namespace changes
ef352abf Update zephyr sample jobs to use LAVA functional test urls
1301258a LAVA-498 - Support dragonboard-410c with lxc
1e0b60d2 Ensure pyudev version is >= 0.21
7d6a8134 Update the DFU boot code for namespace changes
161e8b47 Ensure installer tests share the final connection
567ddf0d remove duplicate call
e49bd971 Fix namespace error in flash_cmds_order retrieval.
cb04549d Set connection for secondary media deployments
dcd75e7b Use correct suffix for secondary media deployments
2a7b386f Declare the original ramdisk as compressed
fd73e937 Support DFU devices
6aa7280c LAVA-504 - Dynamic fastboot partition support in pipeline
9228901a LAVA-809 - Show HiKey kernel boot messages
774c7557 Extend namespace fixes for secondary connections
dc5bec22 Action: Always call super() in validate()
9b983cdf GitRepoAction: do not call validate twice
bd112aab Make namespaces common to all dynamic data operations
2e83e2ca Prevent invalid measurements being reported
1768c181 Fix testlevels for multi-test LXC jobs
0a49da8e Give namespaces to all actions and support simpler code flow
2423a123 slave: send the protocol version with HELLO message
983d3775 Ensure fixupdict is checked in test_case_result
08f79918 slave: writing an empty string to a file is valid
c6f382ef Move cleanup_actions from Action to Job
bc56ca24 Remove unused Action, Pipeline and Job members
1256f31e Run.sh should use set -x
a726be8f Allow install-steps to use the git-repo directly
b0ca553a Create a useful exception message
077aca26 slave: don't crash when lava-dispatch can't start
543ed4af Honor USB device wait timeout and add device before returning
connection
0e07089f Fix problem with duplicate test shells failing
f14747c1 Use dnf instead of yum on Fedora version greater than or equal to 22.
efdaabc1 More fix for usage of LXC with a device.
2d188baa Limit repetitive kernel message log entries
c3455a11 Allow newline line separator to be overriden per bootloader
01a482f6 Fix for Debian reproducible builds.
62c07465 Indicate that character delay is in milliseconds
c1f79680 Remove hardcoded character delay in ipxe
e43552b3 Fixing secondary media deployment
e885c881 device-types: add Renesas r8a7795 Salvator-X board
1374289b device-types: add Amlogic meson-gxbb-odroidc2
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Everyone using LAVA should be aware of V1 and V2 and that V1 is going
to be turned off. The plan for how to do that has been discussed
within the team and within Linaro. The most recent presentation on the
migration can be seen here:
http://connect.linaro.org/resource/las16/las16-503/
(Note: the video will auto-start, be aware.)
The slides are available on slideshare:
http://www.slideshare.net/linaroorg/las16503-lava-v2-migration
The dates for each step in the migration are not finalised but will
occur during 2017, this is advance notice.
The highlights of the migration are:
0: During 2017, regular announcements will be made on this list before
each major change. In addition, the regular release announcements will
include details of the major changes in that release.
1: The first major change will be that a new release of lava-server
and lava-dispatcher will *disable* V1 submissions entirely. Any queued
V1 test jobs will become invalid upon this upgrade,
2: A later release will remove V1 documentation and start removing the
V1 source code.
3: A release will be made which *forcibly deletes V1 test jobs,
bundles, filters and image reports*. This is required to complete the
removal of the V1 source code.
4: The final step of the migration is a release containing no V1
source code at all. Only V2 support will remain. This is essential as
the V1 source code is currently blocking some useful additions to V2
as well as plans for future development. The data cannot remain
accessible without the V1 source code. There is not and can not be any
support for converting V1 data to V2.
Please take this chance to read up on the V2 documentation, including
how Results, Queries and Charts replace the deprecated V1 support from
filters and image reports.
Work has started upstream on making sure the migration runs smoothly,
especially when deleting large numbers of V1 test cases.
Remember: Once the first major step is released, upgrades will stop
running any V1 test jobs. Subsequent releases will include changes
which force the deletion of your V1 test data.
If you want to preserve your V1 data, consider setting up a read-only
reference instance which is then pinned at a particular release of
LAVA to prevent deletion of the V1 data.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
There is a known bug with QEMU 2.7 which is affecting installations of
LAVA using QEMU from jessie-backports.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840853
The problem manifests in the apply-overlay-guest action because QEMU
is unable to process the GuestFS operations due to a missing file
which is in the seabios 1.9 package but not in seabios 1.7 - so V2
QEMU test jobs hang at this point.
If you are using QEMU from jessie (version 1:2.1+dfsg-12+deb8u6) then
there is no problem and nothing needs to be done.
If you are using QEMU from jessie-backports (version
1:2.7+dfsg-3~bpo8+2 currently) then there are two simple remedies.
0: You can find the updated seabios package in the LAVA production
repository. See
https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
If you have the production repository already available, you can
simply apt update and apt upgrade to install the new seabios package.
1: If you are not using aarch64 QEMU tests or the QEMU UEFI support
for LuvOS, then the version of qemu in jessie will work just fine. To
downgrade, specify the version string to apt:
$ sudo apt install qemu-system-x86=1:2.1+dfsg-12+deb8u6
qemu-system-common=1:2.1+dfsg-12+deb8u6
It is also worth downgrading other qemu packages at the same time.
Check your current install with:
$ dpkg -l | grep qemu
The relevant bug has been re-opened and a backport of seabios has been
uploaded. An updated backport of qemu has also been requested which
would require the installation of the backported seabios 1.9. So if
this bug is not troubling you immediately, you can simply wait for
seabios 1.9 to be available in jessie-backports and install it.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
2016.11 includes a move away from libapache2-mod-uwsgi and
libapache2-mod-wsgi to gunicorn support. This has been done to make it
possible for lava-server and lava-dispatcher to migrate into Stretch
ahead of the Debian 9 Stretch release. (see
https://lists.linaro.org/pipermail/lava-announce/2016-November/000019.html).
This should allow us to upload 2016.11 to jessie-backports in due
course.
See: https://lists.linaro.org/pipermail/lava-announce/2016-October/000017.html
for details of the changes for gunicorn support. In particular, if you
are using apache, before installing this upgrade, ensure that these
commands have been run on the master:
$ sudo a2enmod proxy
$ sudo a2enmod proxy_http
$ sudo service apache2 restart
These commands are safe to run before upgrading to 2016.11 but will
need to be run before 2016.11 will actually run.
This release includes all changes in master since the 2016.9 release.
The release has been uploaded to Debian unstable, it should appear on
the official mirrors shortly. It has also been uploaded to the
production repo.
We are anticipating that there will be an interim release ahead of
2016.12 for additional LXC support, more details will follow via this
list.
lava-server (2016.11-1) unstable; urgency=medium
.
* New production release aimed at Stretch.
Release details available at:
https://projects.linaro.org/browse/LAVA-789?filter=12078
* Drop dependency on libapache2-mod-uwsgi and libapache2-mod-wsgi.
Replaced by dependency on gunicorn.
* Require sphinx 1.4 or later for correct theme support.
* Downgrade linaro-image-tools and fuse to Recommends
* Update compat version.
* Highlights of upstream changes:
Note about removing adb and fastboot packages in LXC admin doc.
LAVA-789 - Document LXC support in V2
LAVA-788 - link to sub_jobs_list from results
Expand user notification documentation.
Move jinja template tests to unittest
Adding target interface entries to juno-uboot.jinja2
Add docs on notifications
Update home page documentation links
Improve documentation on metadata and job_name
Fix up glossary page
Tweak docs handling standard test jobs
Example jobs need auto_login support
Fix missing create_device_database ref
Port the documentation change for essential roles
Tweak the doc build options
Add notes on multiple hacking sessions
Update debugging docs for tmp file changes
LAVA-65 document recording measurements
LAVA-780 stop referring to wheezy images
Avoid crash in master on early failure
Remove beautify.js and fix yaml URL submission.
Update screenshot for cancel button fix
Fix visibility of the Cancel and Admin buttons
Update docs to reinstate web UI submission
Add a helper to add devices from cmd line
Remove old link about LAVA packaging which contains stale information.
LAVA-749 - Add actions column to results pages.
Ensure lava-server-gunicorn restarts cleanly
results: do not crash if the description is empty
Job submission UI for v2.
Add copy to clipboard button for job definition.
Turn all action durations into result measurements
Updating juno-uboot.jinja2
Add a log file for gunicorn logs.
Fix the LAVA logo
Ensure the V2 layout is included on jessie.
bootstrap: do not use the minified CSS
Results: fix HTML syntax
Use sensible default for interrupt string in HiKey jinja.
LAVA-749 - Improve query pages
Fix unit test to have unique test definition names
Add pre_* commands to base.jinja2 that will be applied conditionally.
Remove unused dependendies on lava_dispatcher
Fix a variable redefinition in list comprehension
Update contents and organise toctree
Allow changing the timeouts for auto-login-action
Add documentation on QEMU standard kernel tests
Remove old mention of JSON
master: set the failure_comment
Add a template for highbank using base-uboot
Remove hardcoded values from base
Fix profile page error due to wrong device health history url reference.
metadata: fix a crash if the description is invalid
Cover lava-coordinator in changes for V1 workers
LAVA-522 Link in existing docs on replacing vmgroups
Extend the qemu options docs
Fix index links and debugging doc links
LAVA-735 describe anonymous access to JIRA
qemu.jinja2: use qemu-system-i386 for i386
Extend docs for disabling V1 on a master.
Fix HTTP 500 on query views when a group is used.
LAVA-765 V1 MultiNode sub_id correction
Fix jinja2 syntax issues in base-uboot
Fix sub_id assignment in V2
Improved debugging docs
Add content for growing your lab.
Improve first device and first job pages
Update apache docs for gunicorn
Update hacking sessions and lava-network docs
Enable vland for overdrive
Fix uWSGI configuration file
Add notes on removing V1 from dispatchers
Add a helper to add device types from cmd line
DeviceType admin: fix has_health_check computation
Replace ip=dhcp with {{ base_ip_args }}
timing: add the corresponding timeout
Fix typo in the title of the result download link
Add ip=dhcp for juno with a base default
Fix page to show health history of my devices and not my device type.
Fix error on 'Devices Health History' page in django 1.10
Fix user profile page error in django 1.10
Create a base uboot jinja template
Use Gunicorn instead of uWSGI
wsgi: use the Django public API
index: add the missing title
Mark support for LAVA in Debian testing.
Expand the context schema for extra_options
Add the new LAVA logo to the front page of the V2 docs
Warn if a pipeline device does not have a template
master: don't generate the description
Add nxp-k64f jinja2
Add nrf52-nitrogen jinja2
Add drafts of new logos with SVG.
Extend base timeouts and convert to minutes
Handle inline definitions using only install steps
job details: removed unused template variable
master: don't print too many logs in debug
Fix build errors and simplify index pages
Add support for overdrive device type
Fix bad lookups into the glossary
Switch ASCII art to the existing SVG
Update dispatcher actions
Improve timing warning message
Document the principles of a CI Loop
master: remove the need for the ERROR message
Expand job_details doc string with available keys
Replace 1.10 deprecated get_field_by_name with get_field method.
Add support for extra QEMU options
LAVA-747 - Report parameters as result metadata
Switch from using png to svg for the architecture diagram
Add a page to display the pipeline timings
Expand glossary and add sections on parameters
Add D03 device type to pipeline
Add an introduction to results in LAVA.
Add docs on the standard test jobs
Simplify and clarify the "first install" docs
LAVA-584 - expand for sub-types of device-types
LAVA-706 - codebase structure
Port submission priority support to V2
Adopt a similar contents layout to the django docs
Device dictionary output
Extend qemu jinja template for cortex-m3
Pipeline log: use a better icon for download
Update bootstrap from 3.1.1 to 3.3.7
Fix relative url for suite results
Add nbdroot as optional parameter to the schema
Update qemu/kvm templates to allow overriding of guestfs interface
LAVA-93 mark some roles as essential
lava-dispatcher (2016.11-1) unstable; urgency=medium
.
* New production release aimed at Stretch.
Release details available at:
https://projects.linaro.org/browse/LAVA-789?filter=12078
* Update compat
* Highlights of upstream changes:
Ensure namespace is available to the test shell.
cpio output is not an error
Downgrade message match failure to a warning
Fix up usage of LXC with a device
Only attempt lxc-destroy if the container exists based on exitstatus.
Drop ShellLogger __del__ command
Improve accepts for monitored QEMU
Strategy needs to check methods, not device_type
Support parallel (pbzip2) compressed bzip2 files
Allow vendor U-Boot builds using Ctrl interrupts
Improve cpio extract command line and error message
slave: do not send None as error code
slave: send a SIGINT to lava-dispatch to cancel
Making target IP and MAC addresses available in lava test shell
TestShell: allow multiple test shells
Fix kernel message detection for panic
testdef: remove unused common data
Fix the logic regarding the pipeline building
shell: make testdef_index common data a list
device/target: allow multiple boot control chars
Add 943907AEVAL1F to pyocd deploy supported devices
Check if container exists before destroying.
Do not generate results.yaml
Fix uefi_menu pipeline for fastboot.
slave: move tmp files to /var/
Add device to lxc only when it is available.
device-types: add Amlogic meson8b-odroidc1 board
LAVA-755 - use UUID for guest, not label.
slave: send an error code when err is not empty
Fix git-repos reference in inline test definition.
Allow a different bootloader interrupt character
Make pipeline logging python3 compatible
slave: fix a crash under python3
device-types: add Amlogic meson-gxbb-p200 board
device-types: add TI da850-lcdk
bootloader: add ramdisk_raw option
Detect invalid characters in testdefinition name
Fix timeout inheritance from job definition blocks
Set the right permis when creating the ramdisk dir
ci-run: exclude the .eggs directory
tests: Close the file descriptors
Fix root directory of V2 apache config
LAVA-760 - Trim LXC log output
Require a summary and description to validate
Additional changes for removable media usage
Fix extra_options to always be a list of strings
Do not set kernel type for all 'kernel' YAML keys
Action: re-raised the exception without modification
LAVA-748 ensure test definition names are unique
Change sample jobs urls to images.v.l.o
Allow testdefinitions without run steps
Generate the pipeline description on the slave
slave: fix the END message parameters
Use the QEMU extra options where available
slave: resend the END message if the ack was lost
slave: send the error file along with the END message
Move tmp files to /var/lib/lava/dispatcher/job_id/
Only extract the nfsrootfs once.
Use the uboot interrupt prompt from templates
Improve LXC HiKey support.
Don't end test shell at invalid testcase name
SignalMatch: simplify and fix exception message
Shell: improve report formatting
Shell: factorize handling of measurements
RetryAction: log errors
Shell: factorize signal handling
Move override and log_results functions into Action
Initial support for PyOCD
Use \r\n for newlines
Allow guest drive interface to be specified
allow-modify functionality added
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
The 2016.11 release is currently in final preparation and is targeted
at Debian 9 Stretch. We've tried to get the documentation and support
up to a point where users of Stretch have a usable platform before
later upgrading via stretch-backports.
I expect to get 2016.11 into jessie-backports about 5 days after that release.
A few reminders on how the Debian release process (which starts in
January 2017) will affect things, most of which are the same as during
the jessie release process. More details will be given on this list as
the Debian release process continues.
0: New lava-server and lava-dispatcher uploads to Debian won't migrate
into Stretch once the freeze starts. This means that these will not be
available in jessie-backports either as the presence of the same
version of the package in Stretch is required for an upload to
jessie-backports. I expect to get 2016.11 and 2016.12 into
jessie-backports.
1: Once Stretch is released, stretch-backports will be opened and
uploads to jessie-backports can resume. Debian makes the release when
the release is ready, so there are only estimates of when Stretch will
actually be released. The key information is that the freeze to
prepare the release starts in January 2017.
2: I expect to keep jessie-backports updated at least for the first
few releases. This list will carry more information about when there
may have to be changes in this area. Essentially it depends on
movement within Debian after the Stretch release.
3: During this period, uploads will remain available on
images.validation.linaro.org built for jessie - lava-dispatcher will
only be available for amd64 from those repositories. If you are using
armhf or arm64 for the dispatcher, please let us know. Use the
production-repo to get updates of lava-server and lava-dispatcher
during the Stretch release process until jessie-backports is open
again.
4: At some point after the release, the production-repo will start to
be built on and for Stretch. stretch-backports and jessie-backports
will need to be open before this change is made. More details of that
will appear on this list before it happens.
5: Once Stretch becomes Debian stable, Jessie becomes Debian
oldstable. Debian testing gains a new name: Buster. Get used to the
names.
6: It is worth checking *now* that all your apt sources on all your
machines (including VMs) specify jessie and not stable. Otherwise, as
soon as you do an apt upgrade after the Stretch release, your machine
will start the upgrade from jessie to stretch. Make sure you are ready
for that upgrade and that it doesn't happen until you are ready for
it. Also check the images you use in test jobs and the build tools you
use to prepare those images.
7: We have been building images based on Stretch ahead of the release
for test jobs using QEMU and similar. We have also updated the
documentation in the next release to use stretch images.
8: At some point after the Stretch release, there will come a point
where support for jessie is dropped and uploads to jessie-backports
will cease. The upgrade to stretch will be required at some point to
continue getting updates to LAVA. Prepare for it ahead of time and
everything will go more smoothly. Further details of the timing will
be announced on this list.
9: The LAVA software team have been testing with stretch for some time
and will continue to do so. We do not expect any issues with LAVA
packages when upgrading from jessie to stretch. Nonetheless, all
admins are recommended to prepare for the new Debian release and to
perform the upgrade under their own control. Make sure your other
services will work after those machines upgrade to Debian 9 Stretch.
More information: https://release.debian.org/stretch/freeze_policy.html
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
$ sudo apt-get install -t jessie-backports python-sphinx
To solve some problems with the build of the documentation, we have
had to move to the version of python-sphinx available in
jessie-backports. (Essentially, our documentation is now so large that
it is causing problems for the theme we've chosen and we've had to
make some changes to keep improve the navigation within the pages.)
If you are building lava-server in sbuild, there is an addition option
required to sbuild:
--build-dep-resolver=aptitude
The build script in the lava-dev package uses this option already.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
There have been problems with using uWSGI for lava-server, including
preventing updates of 2016.6-2~bpo8+1 in jessie-backports.
The LAVA software team have completed a change which will be included
in the 2016.11 release which removes the need for the
libapache2-mod-uwsgi, libapache2-mod-wsgi and uwsgi-core packages. The
replacement is the gunicorn package.
This does involve changes to the apache configuration of all sites
which upgrade to 2016.11.
Ahead of installing 2016.11, two apache modules need to be enabled.
https://staging.validation.linaro.org/static/docs/v2/installing_on_debian.h…
$ sudo a2enmod proxy
$ sudo a2enmod proxy_http
$ sudo service apache2 restart
These will not affect older installations.
Then install 2016.11 and your instance will continue to operate.
If you have made changes to
/etc/apache2/sites-available/lava-server.conf then the changes to the
upstream file will need to be merged into your changes.
If you choose to remove libapache2-mod-uwsgi after completing the
install, be aware that there is a Debian bug which may affect you.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839666
If your apache server fails to restart after removing the
libapache2-mod-uwsgi package, you will need to workaround this bug by
executing:
$ sudo a2dismod uwsgi
$ sudo apache2ctl restart
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
2016.9 includes a rewrite of each of the daemons used by LAVA,
lava-server, lava-master and lava-slave. When installing the upgrade,
admins need to accept the maintainer's version of the scripts in
/etc/init.d/ for these daemons. Where previously changes were made in
the scripts in /etc/init.d/, the same parameters can now be changed in
associated configuration files to prevent these conflicts in the
future.
/etc/init.d/lava-slave - change /etc/lava-dispatcher/lava-slave
/etc/init.d/lava-master - change /etc/lava-server/lava-master
/etc/init.d/lava-server - change /etc/default/lava-server
The prompt would look something like this:
Configuration file '/etc/init.d/lava-server'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
*** lava-server (Y/I/N/O/D/Z) [default=N] ? y
See https://validation.linaro.org/static/docs/v2/pipeline-server.html#configuri…
References in the changelogs, commit history and announcements which
start with LAVA- followed by a number relate to the planning stories
which the LAVA team use for new features. The LAVA planning has now
been opened up to anonymous access by appending the reference to a
stub URL: http://projects.linaro.org/browse/, e.g.
http://projects.linaro.org/browse/LAVA-62
The planning stories do not support comments or changes other than by
the LAVA software team. If you have comments on the stories, please
use the lava-users mailing list.
lava-server changes
Show measurements and units of testcase detail page
Drop lxc schema - no support for alternatives
Configure logging for linaro_django_xmlpc calls
Fix link to lava-tool context-help
Fix the "Debian-based distribution" link grammar
Improve monitor test type, allow multiple tests
LAVA-494 allow searching for device tags in tables
Add support for LXC with multinode
Fix broken ref link for unit_test
Fix NoReverseMatch from ImageReports2.0 editing.
Fix typo in ./share/validate.py --hostname
Update hikey jinja2 templates for LXC
Fix occasionally failing multinode tests.
Add a configuration file for lava-master
Update lava-slave and lava-master docs
LAVA-374 - drop versiontools
Tidy up some build errors and reformat consistently.
Improve documentation describing inline test definitions
Major improvements for the "Writing MultiNode" page
Add docs for the lava-vland-names change
Using BOOTX in Juno bootcmds
Correcting default uboot commands in juno device template
Document the code locations for developers
Ensure multinode description is generated correctly
Allowing overrides to connection and action timeouts
master: Fix log rotations
Remove filtering by filter id for available testcases in image reports.
Init: remove the right lock file and simplify
Adding uboot jinja2 templates for all Juno flavours
LAVA-739 - declare sub_id and job.id for multinode
master: don't validate jobs on the master
select_device only accepts pipeline jobs
Remove legacy SyntaxHighlighter from job definition pages.
lava-master: simplify log file handling
Fix title disappearance on image reports.
Ignore retired devices in validate call
Don't add anchors to logs for running jobs
Don't crash when log contains invalid test names
Schema: accept 'parse' for inline test definitions
Use libYAML when loading logs
LAVA queries use cases.
Bug link fix for url regex.
Add LogEntry for BugLinks.
Update the init scripts
Remove the entry point as it's no longer needed
Ensure test-case name is valid
Extend vland unit tests to cover assignment
Expand on the device dictionary and templates
Expand the notes on using jessie-backports
Add a Contents page
Rewrite lava-server command line from scratch
events: set umask to a restrictive value
Bug links port.
LAVA-200 - track admin actions made in the UI
Add device tags support to V2
Allow the monitor test type
Update qemu command line options for arm64
lava-dispatcher changes
Catch the edge case where serial line corruption means we miss the testrun
Add support for alternative Test actions
Update manpage of lava-slave for configuration change.
Allow configuring the slave from file
git: use -C instead of --git-dir
Allow setting the envrionment on most systems
Handle exceptions from subprocess from compression
download: don't expand the path when retrying
Grab finalize from the root pipeline
Log exception as strings and not objects
LAVA-740 - export vlan names and interface names
Reopen log file when rotating the logs
LAVA-699 - Support hi6220-hikey board with lxc
Init: remove the right lock file and simplify
pipeline:vland: raise JobError if switch_id or port_id is None
Remove redundant assignation
Allow logging at the validate stage
Force the shell to produce an initial prompt
Handle errors if the testrun start is omitted.
Pipeline parser: sort and reverse in one call
logs: remove unused support for local logs
slave: put all tmp files into the same directory
Power: improve logging message
Exclude patterns from description
Port lava-slave to Python3
ApplyOverlayTftp: remove duplicated untar_file
ExportDeviceEnvironment: only source valid shell_file's
add LEDE to available distributions
Always log the command to be run beforehand.
There is currently a delay on uploading lava-server to
jessie-backports due to problems with uwsgi in Debian. A backport will
be made available once these problems are fixed. In the meantime, the
same build as will later appear in jessie-backports is available in
the production-repo:
https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…https://tracker.debian.org/pkg/lava-serverhttps://tracker.debian.org/pkg/lava-dispatcherhttps://qa.debian.org/developer.php?email=pkg-linaro-lava-devel%40lists.ali…
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
A number of the configuration files which manage the daemons used by
lava-server and the lava-slave daemon for lava-dispatcher are being
updated in 2016.9. This has implications for those who install the
upgrade, especially if this is normally done by automation.
Configuration file '/etc/init.d/lava-slave'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
*** lava-slave (Y/I/N/O/D/Z) [default=N] ? y
Installing new version of config file /etc/init.d/lava-slave ...
dpkg will check the checksums of the init files against the current
and previous packages and halt installation with the above prompt in
case of doubt. Admins may need to install the update manually
(directly on the command line).
The recommendation is to accept the package maintainer's version but
all admins may want to check the difference between the two.
Changes in this release include fixing the log rotate handling of
lava-master using reload and a rewrite of the lava-server CLI script
to drop the use of entrypoints which changes the syntax of the daemon
options.
The most common reason for init files to be modified is to change the
loglevel. This can be done in /etc/default/lava-server:
# Additional options that are passed to the Daemon.
DAEMON_OPTS="loglevel=debug"
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
