Update on the Auto/Assisted Landing System

Almost a week since the post introducing the design attempt for auto/assisted branch landings via Bugzilla and Try and guess what? We re-wrote everything!

The details are in the wiki, bugs have been filed, code is being written.  We are working on making this system use a message queue and also see if we can work with mozillapulse to get information on bug changes from Bugzilla.

I'd love to tell you more about it but you can read the wiki and I'm excited to get back to my SchedulerDBPoller component.

Assisted/Automated Landing - Designing the Systems

Ehsan's blog post wishing for assisted landings on mozilla-central started a lot of people talking about this being a very desirable and useful tool for developers, where they could set a flag in Bugzilla and then be free to do other work until the results of their push were posted back to the bug. As part of enhancing the Tryserver I was already working on a way for users to signify in their try-syntax that the results of the push should go to the bug and these two ideas started to fuse into a dreamworld where someone could attach a patch to Bugzilla and have it be tried and pushed to trunk all with some magical bot automation.

After doing a very short survey of developers and their try usage I have observed that there are two very different stakeholders here and both of them need separate-but-related tools:

Developers	The Bot (automation)
ease of use better reporting (less email anyone?) option to post to bug(s) *after* a try run has indicated success	queuing of patches culled from a flag in Bugzilla automatically apply to tip of repo push, and report back with results

After soaking in the survey feedback and a first attempt with a whiteboard yesterday, I woke up this morning with some clearer ideas on how to take a first run at creating this system.  It involves creating several new tools, one new database, and enhancing our existing buildapi.

New tools for Developers:

• Adding more Try syntax options:
• include list of the bug(s) that you would like your try results posted to (however many make for a complete run on your push, this can be one linux build or a complete ~186 builder try: -a buildset) 
• turn off email notifications
• Adding functionality to the self-serve api view for a revision (eg: https://build.mozilla.org/buildapi/self-serve/try/rev/de8ea75bc48e) that will better show your results for that push and provide a button which will post the patch(es) to a specified bug
• Auto-landing from a bug in Bugzilla using the [autoland-try] whiteboard tag where any attached patches which are not obsolete, and have nothing set for 'r' are applied to the current tip of mozilla-central, pushed to try and those results are returned to a comment in the bug

New tools to Automate landings (bot or script):

• Crawl Bugzilla for bugs where [autoland-$branchname] is in the whiteboard and automatically push to tip of named branch, get the results, and return them to a comment in the bug (stripping out the whiteboard tag on completion)
• bot will grab all non-obsolete, r+ patches (if $branchname != 'try') 
• interdependent bugs will not be handled in this first swipe at a working system
• pushes will have autoland-$bugnumber as the reason for the build in schedulerdb so that the results can be watched for, aggregated, and reposted to the bug on completion
• Watch results coming back for one or two oranges (we can set a threshold) and re-triggers those, watching for the second set of results - to attempt catching intermittent oranges
• Backout patches where even with a rebuild on an orange, there still remain orange results
• LDAP authentication checking for bugzilla patch author -> hg commit permissions and being able to ensure that only people with the right credentials can trigger automatic landings. This may mean checking the reviewer too before allowing a patch to be applied & pushed.

The next step is to get this design organized into bugs so that we can parcel out the work involved and start testing/completing segments and features as we work towards the whole. We have a RelEng intern this summer, Marc Jessome (Another Canadian in RelEng!), who will be doing a lot of the work between now and the end of August. Stop by anytime to say "Hi" to Marc and to chat with either of us about the project - feedback is always appreciated.  I'm happy to say that 52 people filled out the Try Usage survey just from posting it on Yammer. That was super helpful, thank you.