+------------------------------------+ | Debian i18n Sprint - brainstorming | +------------------------------------+ Goals ===== * Migrate to new machine (lower urgency due to fixed HW) * VCS reorganization and conversion * Have a better DDTP in the future * Redraw faw's piece of paper? symoon volunteered * Plan Administration - i18n.git * Reorganization of VCS - Subversion to git - Inclusion of scripts and cronjobs - Changes in filesystem layouts and internal structure * Where to put documentation? Documentation ------------- DSA, Machines, Guest Accounts, Backup, Reboots reorganized and moved to i18n Sprint 2012 - Documentation. Dependencies ------------ * Services pointing to i18n.debian.net - PTS - www (intl/l10n pages) Hosting our stuff ----------------- we'll use debian-l10n git, more having a working copy of the VCS than packaged stuff * ddtp.git with ddts, ddtss, ddtc * dl10n.git with bubulle did put in the package * compendia.git We leave packaging stuff in SVN - translate-toolkit - pootle - virtaal SVN viewvc: http://anonscm.debian.org/viewvc/debian-l10n/ debian-l10n Alioth project -------------------------- Need "cleaning" out project members. Done (bubulle). I left: faw, nekral, br0ther-guest, kleptog-guest, grisu, elric-guest (Omar Campagne, maintainer of virtaal), taffit Need cleaning out SVN Need importing SVN in git Identify & remove data (!= scripts) and too big files from git repo Cleanup of VCS: * Alioth: /srv/cvs.debian.org/cvs/debian-l10n.delete no more needed -> we can just delete it ourselves PostgreSQL ---------- * Database is 2.2GB. - Isn't a big part of it disposable? - Do we need to keep archived releases in the database? . Yes, maybe for translation memory . No, Translation files are static and translation memory can be stored in a different way * Better using a PG instance on a dedicated machine (name?) - We will start with that * What version will it be? 8.4 or 9.0 (I asked for 9.* series) Various scripts to move ----------------------- have a "material" place where l10n material extracted from the mirror is located (/srv/i18n.debian.net/www/debian-l10n-material) -->this is pointed by www.debian.org -->this is used by some scripts and robots * gen-material * l10n status pages (aka "robots") * l10n compendia * statistics * PTS data * BTS usertags (where is the script?) Current organisation of scripts in churro ----------------------------------------- /srv/dl10n-stuff/svn/dl10n/ - SVN workcopy /srv/dl10n-stuff/svn/dl10n/data - status (l10n + nmu) data generated here /srv/dl10n-stuff/bin/ - cron jobs + utils + config /srv/dl10n-stuff/log/ - cron jobs logs (.log / .err for gen-testing / gen-unstable / spiderbts / dl10n-nmu) /srv/i18n.debian.net/cron/ - cronjobs (uses /srv/dl10n-stuff/bin/ scripts) (cron.d / cron.hourly / cron.monthly) /srv/i18n.debian.net/mail/ - not used currently. archive of l10n mailing lists for robot. rotation broken. /srv/i18n.debian.net/tmp/gen-compendia - TMP dir for gen-compendia (clean) /srv/i18n.debian.net/tmp/gen-material - TMP for gen-material (clean) /srv/i18n.debian.net/www/debian-l10n-material - contains the database with the translation status of the archive (data/unstable.gz / data/testing.gz); and the extracted material (menu / po / templates or testing/menu / testing/po / testing/templates) and the associated tar balls. This is used internally by other tools (e.g. dl10n-txt) or by www.d.o /srv/i18n.debian.net/www/debian-l10n-coordination - web pages for l10n teams /srv/i18n.debian.net/www/debian-l10n-coordination/l10n-nmu - web page for NMU campaign /srv/i18n.debian.net/www/l10n-pkg-status - data for the PTS /srv/ddtp.debian.net/ - script for handling DDTP /srv/ddtp.debian.net/ddtss/ - scripts for handling DDTSS /var/www/ddtp/ddt.cgi - main DDTP script /var/www/ddtp/ddtss/index.cgi - main DDTSS script Issues in current organization ------------------------------ cron jobs not in VCS (as templates) utils not in VCS (as templates) config templates not in VCS no log rotation DDTP/DDTSS web scripts running are not those in the working directory Data generated in VCS working copy No atomic change of data (data continuously updated in /srv/i18n.debian.net) --> in packages.d.o, rsync-mirror using hardlinks from the place where data are produced to the publication place Corruption prone scripts No backup integration of testing in the file hierarchy a bit fuzzy. Impact on www.d.o expected /srv/i18n.debian.net/www/debian-l10n-coordination/english to be removed? /srv/i18n.debian.net/www/debian-l10n-coordination/l10n-nmu to be located elsewere Hardcoded list of languages (dl10n scripts and www.d.o) Proposed organization for scripts --------------------------------- /srv/i18n.d.o/git/ - VCS working copy same as currently move compendia to ../../ [DONE] cron from bin directory [DONE] etc from bin directory [DONE] html renamed htdocs-static [DONE] pootle to be removed [DONE] /srv/i18n.d.o/tmp - TMP directories /srv/i18n.d.o/log/ - log directories gen-testing/gen-testing.YYYYMMHH-HHMM ... /srv/i18n.d.o/data/ status -> status.YYYMMDD-HHMM material/testing -> testing.YYYYMMDD-HHMM material/unstable -> unstable.YYYYMMDD-HHMM stats /srv/l10n.d.o/htdocs/ coordination/ french (generated) ... static (link to ../../git/htdocs-static) stats compendia /srv/i18n.d.o/htdocs/ material/testing -> ../../data/material/testing material/unstable -> ../../data/material/unstable nmu-radar l10n-pkg-status -> l10n-pkg-status.YYYYMMDD-HHMM Questions: cronjobs? compendia? www.d.o and pts.d.o to be informed churro & i18n.d.o to run in parallel i18n.debian.net/debian-l10n-material/testing i18n.debian.org/material/testing Convert to git -------------- Authors meta data: mapping the svn account to a From line to put in the git log. barbier = Denis Barbier bubulle = Christian PERRIER elric-guest = Omar Campagne faw = Felipe Augusto van de Wiel fzt = Nicolas François grisu = Michael Bramer jfs = Javier Fernandez-Sanguino Peña kleptog-guest = Martijn van Oosterhout mquinson-guest = Martin Quinson nekral-guest = Nicolas François pmachard = Pierre Machard taffit = David Prévot themill-guest = Stuart Prescott thuriaux-guest = Thomas Huriaux pootle-guest = Pootle guest root = The knights of nicky-nicky (no author) = The knights of nicky-nicky +---------+ | PENDING | +---------+ * git conversion + br0ther is working on that, we are pretty much ready to start tests +------+ | DONE | +------+ +--------------+ | NEEDS ACTION | +--------------+ * Ask for removal of ddtp project on Alioth https://alioth.debian.org/projects/ddtp/ Current admins: grisu, otavio, zobel - Mail them to confirm the exclusion of DDTP on alioth.d.o * PostgreSQL - Waiting DSA (zobel) to create cluster/database and gives us access