# gobby.debian.org/debconf17/bof/ci_autopkgtest


updates
-------

- lintian now complains if your package does not have autopkgtests (INFO level)
- ci plans
  - qemu support: in the near future
  - ppc64el should be added soon
  - arm64 stopped because boxes were unstable


autopkgtest questions/issues
----------------------------
* How to setup a chroot / what's the recommended workflow?
  autopkgtest --apt-upgrade -B $(debc --list debs) . -- lxc --sudo autopkgtest-sid-amd64
  autopkgtest --apt-upgrade -B $(debc --list debs) . -- qemu --sudo autopkgtest-sid-amd64
  
* How is the chroot created?
  With the helper tools that come with autopkgtest:
  autopkgtest-build-{lxc|lxd|cloud} or vmdebootstrap or debci setup -b qemu
  
* There is no user created by the typical helper tools
  Indeed, debci creates a user. Also many tests require root.

* Simon has a new bootstrapping tool, vectis: https://bugs.debian.org/843486

* ci.debian.net/doc has documentation regarding how to setup and run tests.

* How to test fonts with autopkgtests.
  The test could be a LibreOffice document that you open and take a screenshot of.
  The data (the LibreOffice document) should be shipped inside the source package.
  
* test suite triggers
  Currently autopkgtest runs for the reverse dependencies when a package is
  uploaded. Future work is to use test suite triggers, which would allow to 
  specify other packages that might trigger the tests.
  
* Manual tests
  Not currently possible. Future work.
  
  --shell-fail option!
  
* How about network access?
  Some environments allow this, but in general it sounds like a bad idea.
  It might be better to setup a local web service and connect to that.
  
* Problem to distinguish tests that have special requirements
  - upload to add it to control was reverted
  * solutions: out-of-band keeping track or fail quickly/special error code
  
* How do we test GUIs?
  GNOME is using the accessibility bus to test. This has the advantage of also
  testing accessibility.
  Ubuntu is doing installer tests of the point-and-click installer.
  Tails does integration testing, by booting and doing some picture based
  comparison, this is painful because pictures change. Now moving to using 
  the accessibility module.

* How about centralizing some great examples of some types of tests
  e.g. webapps, GUIs, fonts, libraries...
  -> This list should be in some wiki. (which?)
  
* We will probably discover duplication in the tests. But we shouldn't let this 
  become an overabundance of helpers.
  
* How about the automatic tests for perl packages?
  autodep8 is called on runtime and creates the tests on the fly. More things 
  can be added to autodep8.

ci questions/issues
-------------------

need to handle testsuite-triggers