Participants
============
knowledgejunkie
gregoa, fsfs, axhn, carnil, knowledgejunkie, ntyni, alexm, olly

Work done
=========

* update libtext-reflow-perl
* update libcatalyst-perl
* update libcatalyst-view-csv-perl
* update libcbor-xs-perl
* update libmojolicious-perl
* update libparams-validationcompiler-perl
* discuss RC bugs, esp. the "random test failure" ones
* look over other RC bugs
* discuss team sprint 2017

Subject: Please clarify release team's policy on random FTBFS

Package: release.debian.org
Severity: normal

Hello release team,

the pkg-perl team has trouble with a few packages that randomly FTBFS
from errors in the test suite,
where randomly means a failure rate of 10 percent or often much less.
Naturally, these are hard to debug and since time's running up, we'd
like to find a solution for bugs like

  #848408: libapp-termcast-perl: FTBFS randomly (failing tests) - https://bugs.debian.org/848408
  #848409: libchi-driver-redis-perl: FTBFS randomly (failing tests) - https://bugs.debian.org/848409

other than not shipping them in stretch.

It has been suggested in #841662 random FTBFS is not considered RC
anyway, do you agree to that position?

Otherwise, we'd like to suggest the following policy (or: guidelines):
Packages that randomly FTBFS at ten percent or less may still enter
testing. That of course is not the nicest thing to have, but in our
understanding the importance of that story is, rebuild for security,
point releases and downstream should not fail - but it is an
acceptable burden if they might need a second attempt (but a second
attempt only). Else we'd rather work around it by disabling the test suite
entirely or in major parts like it was done in #841662. Nobody probably would enjoy that, and it's just
hiding problems instead of dealing with them.

bugs from https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=ftbfs-randomly;users=sanvila@debian.org

#845167 [S|  |  ] [src:lemonldap-ng] lemonldap-ng: FTBFS randomly (failing tests)                                                                    
#848060 [S|  |  ] [src:libx11-protocol-other-perl] libx11-protocol-other-perl: FTBFS randomly (failing tests)
#848408 [S|  |  ] [src:libapp-termcast-perl] libapp-termcast-perl: FTBFS randomly (failing tests)
#848409 [S|  |  ] [src:libchi-driver-redis-perl] libchi-driver-redis-perl: FTBFS randomly (failing tests)
#834912 [S|  |↝]  [src:libfile-tee-perl] libfile-tee-perl: FTBFS randomly (Failed 1/2 test programs)
#843052 [S|u|↝]   [src:libterm-filter-perl] libterm-filter-perl: FTBFS randomly (failing tests)

other similar bugs:
#846667 libpoe-loop-event-perl: FTBFS: Test failures

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844264#35
jcristau:
| The common rule is that FTBFS is serious.  Then there's the real world,
| where e.g. some tests are timing-dependent and so don't always fail, but
| having them is still better than not, so if the failure rate is low
| enough I think a lower severity can make sense.

21:34 <gregoa> ok, so could go through them, forward upstream where not already happened, disable if the failure is 
               somehow probable. lower severity for the sporadic cases with a reference to msg #35 in this bug and add 
               that we're also fine with stretch-ignore
21:36 <fsfs> gregoa: sounds good
[next day]
10:43 <Dom> so what's the current thinking on randomly-occurring FTBFS being RC?
10:45 <fsfs> Dom: I think we should make sure to have those failures forwarded upstream, and then it's either downgrading 
             the bugs or disabling the test. I'd prefer downgrading, and my feeling is that the release team does so, too
10:48 <Dom> Yeah, downgrading is much better than disabling the test
10:59 <carnil> ack yes
11:00 <carnil> I don't have much to say, sicne was quite inactive regarding recent FTBFS bugreports, but it feels not 
               right to just going to disable the testsuite because rare/small failing probability