Thursday, September 23, 2010

Get more green with TryChooser!

Our fabulous intern Anamaria Stoica has been working hard the past few months helping to pull data from buildbot's scheduler DB that allows us to learn all sorts of new information about how the release infrastructure is being used, and how it could be optimized. Among other things, she's generated an end-to-end report so we can see at a glance how long the complete build cycle for any push to mozilla-central took.  Now, on top of that, she's added a way to see who is using the awesome new TryChooser syntax.

In the coming weeks I'll be emailing folks who are using try but not the TryChooser syntax and encouraging them to give it a shot. In my stock email letter I will say something along the lines of:
When you use the TryChooser syntax to make requests for only what you need, you allow precious CPU cycle time to be used elsewhere by those who need it.

This is helpful for two reasons:

1. Those testing fixes to blockers of the next beta/rc/release can get try results quickly and get their code landed on trunk
2. Overall wait times for try results are reduced
What's interesting in looking at the reports though is that more of the pushes to try that use TryChooser are green. Here's a snapshot of those who are using TryChooser:

The number of build requests is an aggregate of all the builders triggered so all build types, platforms, test and talos builders run on that push.

Now for comparison, here's a snapshot of folks not using the TryChooser:

I have no way of knowing from this snapshot which of those pushes were intentionally not using the TryChooser and perhaps trying to get all 176 possible builds. Some of them may be using custom mozconfigs as a way to kill off unwanted builds (p.s. you get burning with that).

If you haven't tried it yet, please give TryChooser a go and see if getting a smaller set of builds (ie: what you need, not just the default) and perhaps getting more green on you try pushes gives you a boost.

3 comments:

Mossop said...

Well I would have thought it was a given that the less tests you run the less chance you have of hitting a failure, intermittent or otherwise.

What would be really awesome is a simple webpage that creates the TryChooser text for you based on ticking a few checkboxes.

Lukas Blakk said...

Indeed, it does lower your chances of hitting a failure when you run less builds/tests against a push. However it also looks like people are using the trychooser to get incremental results against their work on particular platforms or test sets and that doing so is not only saving try resources but helping with regression testing and confirming what is an intermittent orange vs. patch.

Lukas Blakk said...

Beginnings of a syntax-builder webpage:

http://people.mozilla.org/~lsblakk/trychooser/trychooser.html

code:
http://hg.mozilla.org/users/lsblakk_mozilla.com/trychooser