http://annex.debconf.org/debconf-share/debconf14/slides/lava/


 #  Automated Validation in Debian using LAVA

 Linaro Automated Validation Architecture

 Continuous Integration Testing

 Operating system deployment testing

 Kernel boot testing

 Test across multiple nodes over serial

 Extra hardware required for certain tests

 Results tracked over time

 Summary reports created by users.

 Data export for further analysis.


 #  Current architecture support

 Physical ARM hardware (ARMv7, ARMv8)

 Physical and virtual x86 systems

 Emulated ARM, MIPS and PowerPC systems

 Virtual ARM systems on physical ARM hardware

 Dummy devices like schroot and ssh

 # Current testing in LAVA

 Boot test kernel CI builds

 Android testing

 Test bootloader (UEFI, u-boot)


 #  LAVA features

 - test jobs written using shell, YAML and any utility,
   binary or harness which can be built or installed
   inside the test environment.

 - produces results based on test deployment and the tests
   defined by the test writer, including support for pass/fail,
   measurements and units. Some test harnesses will need 
   regexp parsers.

 - provides a standard LAVA overlay of data and shell scripts
   to provide a common interface across different device types

 - does not define what can be run and makes no assumptions
   about device capabilities beyond serial connectivity.

 - allows for tests which could brick the device without
   manual intervention.
   

 #  Extending LAVA to support Debian

 Testing the Debian ARMMP kernel

  - Find the overlap between regular boot
    reports and DTBs in the ARMMP build
  - http://status.armcloud.us/boot/
  - https://packages.debian.org/sid/armhf/linux-image-3.14-2-armmp/filelist

  * Beaglebone Black
  * iMX.53
  * iMX6q-wandboard
  * panda
  * panda-es
  * cubie2
  * cubietruck

  - check all hardware is actually operational
  - Other devices available through LAVA can be tested


 #  Work to do for ARMMP support

 - scripting to build an initramfs with ARMMP modules
   http://linux.codehelp.co.uk/?p=49
   vmdebootstrap, d-i daily builds.
 - Decide on which DTBs to support / drop
   exynos, armada-xp, tegra, n900?
 - Provide links to LAVA results for reports of tests
   of each supported DTB in the Debian wiki.

 - Add a LAVA install on DSA hardware if one of each 
   supported board is available in the same location.

 - Use XMLRPC submission to existing instances.

 - Create tests to run on the booted kernel. LTP?
 - Check board-specific hardware support, USB, ethernet,
   SATA, hwclock, GPIO, I2C, wifi, bluetooth.
 - Test ARM installer images (needs preseeding)


 #  Testing package combinations

 - Create a LAVA test job which reproduces a complex
   scenario, including:

   : across multiple nodes. (client : server)
   : across multiple architectures
   : across virtualisation barriers

 #  Testing image creation tools

 - devices may need support for supplementary media

 #  Test distribution upgrades

 - including tests with dirty images
 - including tests on any supported architecture

 #  Whatever else someone thinks up ....


 #  Current refactoring

 - Modular components
 - idempotent actions
 - diagnostic intervention on error
 - simulation of the test job flow
 -