Almost a week since the post introducing the design attempt for auto/assisted branch landings via Bugzilla and Try and guess what? We re-wrote everything!
The details are in the wiki, bugs have been filed, code is being written. We are working on making this system use a message queue and also see if we can work with mozillapulse to get information on bug changes from Bugzilla.
I'd love to tell you more about it but you can read the wiki and I'm excited to get back to my SchedulerDBPoller component.
Monday, May 23, 2011
Tuesday, May 17, 2011
Assisted/Automated Landing - Designing the Systems
Ehsan's blog post wishing for assisted landings on mozilla-central started a lot of people talking about this being a very desirable and useful tool for developers, where they could set a flag in Bugzilla and then be free to do other work until the results of their push were posted back to the bug. As part of enhancing the Tryserver I was already working on a way for users to signify in their try-syntax that the results of the push should go to the bug and these two ideas started to fuse into a dreamworld where someone could attach a patch to Bugzilla and have it be tried and pushed to trunk all with some magical bot automation.
After doing a very short survey of developers and their try usage I have observed that there are two very different stakeholders here and both of them need separate-but-related tools:
After soaking in the survey feedback and a first attempt with a whiteboard yesterday, I woke up this morning with some clearer ideas on how to take a first run at creating this system. It involves creating several new tools, one new database, and enhancing our existing buildapi.
New tools for Developers:
After doing a very short survey of developers and their try usage I have observed that there are two very different stakeholders here and both of them need separate-but-related tools:
| Developers | The Bot (automation) |
|---|---|
|
|
After soaking in the survey feedback and a first attempt with a whiteboard yesterday, I woke up this morning with some clearer ideas on how to take a first run at creating this system. It involves creating several new tools, one new database, and enhancing our existing buildapi.
New tools for Developers:
- Adding more Try syntax options:
- include list of the bug(s) that you would like your try results posted to (however many make for a complete run on your push, this can be one linux build or a complete ~186 builder try: -a buildset)
- turn off email notifications
- Adding functionality to the self-serve api view for a revision (eg: https://build.mozilla.org/buildapi/self-serve/try/rev/de8ea75bc48e) that will better show your results for that push and provide a button which will post the patch(es) to a specified bug
- Auto-landing from a bug in Bugzilla using the [autoland-try] whiteboard tag where any attached patches which are not obsolete, and have nothing set for 'r' are applied to the current tip of mozilla-central, pushed to try and those results are returned to a comment in the bug
- Crawl Bugzilla for bugs where [autoland-$branchname] is in the whiteboard and automatically push to tip of named branch, get the results, and return them to a comment in the bug (stripping out the whiteboard tag on completion)
- bot will grab all non-obsolete, r+ patches (if $branchname != 'try')
- interdependent bugs will not be handled in this first swipe at a working system
- pushes will have autoland-$bugnumber as the reason for the build in schedulerdb so that the results can be watched for, aggregated, and reposted to the bug on completion
- Watch results coming back for one or two oranges (we can set a threshold) and re-triggers those, watching for the second set of results - to attempt catching intermittent oranges
- Backout patches where even with a rebuild on an orange, there still remain orange results
- LDAP authentication checking for bugzilla patch author -> hg commit permissions and being able to ensure that only people with the right credentials can trigger automatic landings. This may mean checking the reviewer too before allowing a patch to be applied & pushed.
Wednesday, May 4, 2011
A PyStar Supernova in the Sky
The first Bay Area PyStar event has come and gone. I'm finally getting a moment to regroup and ponder all the trial and error of being the organizer of this event as well as having time to look at some of the statistics we gathered. Just from an organizing perspective here are a few items I'd like to share about the process.
Things to do differently next time:
* When creating the Eventbrite event, add questions like "What level do you want to learn at?" "Meat or Vegetarian?" "Operating System?" to the registration so there's no need to send out blanket emails to attendees to try and get that information after the sign up.
* Only do one day workshop instead of Friday night installation and Saturday workshop. I think that for many people the setup could have been done in the first hour and the rest of the day been spent learning instead of having a night session that only is needed by a handful of people.
* Have the teachers/assistants already assigned to a particular level of instruction - prepare topics, tutorial materials, and class size ahead of time so that on the day of the workshop there might only be a handful of late arrivals to place and the other attendees will already be set up in the right learning level as requested in the sign up.
Things that really worked this time:
* Eventbrite! They have amazing tools, stats, emailing options, charts, and also a way to see where your sign ups come from which showed us that we got a TON of views from Tweets which apparently was an impressive number (I am told by one of our attendees who is an Eventbrite employee)
* Mozilla! By sponsoring the event - providing the space and food - being able to let people/groups spread out and work in our various conference rooms as well as having lunch on site was very much appreciated by attendees (and of course by me!)
* CodeChix! This peninsula-based group of women coders accounted for 30% of our attendance and also netted some teacher/assistants for the workshop. CodeChix co-sponsored the event and helped get word out as well
Attendance
There was something odd happening with the Eventbrite signups. In a couple of short bursts, a ton of tickets were being snatched up by names that seemed slightly suspicious. Now the event has passed and I've checked in all the attendees as well as accounted for the no-shows (almost all of whom took a moment to send in their regrets so the tickets could be freed up for another person - very sweet!). It looks to me like about 40% of our attendees were fake accounts. Julie (who works at Eventbrite) and I took a look at the numbers and she's kindly offered to look into it further to see if there is indeed something fishy happening. All that aside, we had 47 people! That feels like great attendance to a first workshop, on a Saturday, in Mountain View.
Speaking of Mountain View - we had attendees come from all over Northern California. I love this view of how spread out geographically we all were:
This graph is useful for seeing how my own promotion attempts were successful. The original spike of page views is obviously when I first announce the event link. CodeChix, Baypiggies, and Devchix were the mailing lists I sent emails to with the link. While that got the ball rolling, it was the tweets and emails sent out almost 3 weeks later - a week before the event where the event got lots of attention. It probably helped that PyStar Minneapolis was happening then too so #PyStar got lots of tweets (sorry to the person who's twitter nick is @pystar).
Can I just say that I am so thrilled with the amount of people who volunteered to teach/assist? Seriously. Amazing. I love that there are people out there who really enjoy getting newbies involved, who can share their skills, and who will give their time to events that grow community.
Finally, here's a breakdown of where we got ticket "sales" from via Eventbrite. This is another reason they rock - they help you promote your event! As you can see here the Twitter share link definitely got us the most eyes even though direct invitation resulted in more actual signups. For next time I would send the link to a few more mailing lists like SF Python Meetup, Systers, and also next time we'll be able to invite the folks who came to the first one as well as those who couldn't make it.
In follow-up posts I will post and analyze some of the survey results of both the PyStar Bay Area and the PyStar Minneapolis. I need to go learn how to create charts from Google doc spreadsheets. Also we need to figure out how to set up our site and materials to be easily updated and adjusted by a distributed team without having to break off into separate sites. Finally, the curriculum needs an overhaul. We kept an etherpad during the event to track issues so that I can go through post-workshop and take advantage of all the feedback to improve our offerings.
What's Next?
The next PyStar I plan to organize will be in late July or early August and I'd like to do that one in SF. Following that I'm going to plan one in Toronto for mid-to-late October. What we did this past Saturday is only the beginning. I'll be working with all the folks in the pystar group to get this program shaped up into a much more modular system for learning Python and Django in stages (badges) and also will be setting up sub-groups for things like hack nights, code-masters (think toastmasters but writing code in front of people), and I have this idea of taking the PyStar lessons into women's prisons as a way to get marketable skills into the hands of people who need them badly.
Anyway, first we'll get more material prepared and digest/incorporate all the excellent feedback. Then we can take over the world :)
I hope I'll see you at future events. Thanks to everyone who helped make this a great day!
Things to do differently next time:
* When creating the Eventbrite event, add questions like "What level do you want to learn at?" "Meat or Vegetarian?" "Operating System?" to the registration so there's no need to send out blanket emails to attendees to try and get that information after the sign up.
* Only do one day workshop instead of Friday night installation and Saturday workshop. I think that for many people the setup could have been done in the first hour and the rest of the day been spent learning instead of having a night session that only is needed by a handful of people.
* Have the teachers/assistants already assigned to a particular level of instruction - prepare topics, tutorial materials, and class size ahead of time so that on the day of the workshop there might only be a handful of late arrivals to place and the other attendees will already be set up in the right learning level as requested in the sign up.
Things that really worked this time:
* Eventbrite! They have amazing tools, stats, emailing options, charts, and also a way to see where your sign ups come from which showed us that we got a TON of views from Tweets which apparently was an impressive number (I am told by one of our attendees who is an Eventbrite employee)
* Mozilla! By sponsoring the event - providing the space and food - being able to let people/groups spread out and work in our various conference rooms as well as having lunch on site was very much appreciated by attendees (and of course by me!)
* CodeChix! This peninsula-based group of women coders accounted for 30% of our attendance and also netted some teacher/assistants for the workshop. CodeChix co-sponsored the event and helped get word out as well
Attendance
There was something odd happening with the Eventbrite signups. In a couple of short bursts, a ton of tickets were being snatched up by names that seemed slightly suspicious. Now the event has passed and I've checked in all the attendees as well as accounted for the no-shows (almost all of whom took a moment to send in their regrets so the tickets could be freed up for another person - very sweet!). It looks to me like about 40% of our attendees were fake accounts. Julie (who works at Eventbrite) and I took a look at the numbers and she's kindly offered to look into it further to see if there is indeed something fishy happening. All that aside, we had 47 people! That feels like great attendance to a first workshop, on a Saturday, in Mountain View.
In follow-up posts I will post and analyze some of the survey results of both the PyStar Bay Area and the PyStar Minneapolis. I need to go learn how to create charts from Google doc spreadsheets. Also we need to figure out how to set up our site and materials to be easily updated and adjusted by a distributed team without having to break off into separate sites. Finally, the curriculum needs an overhaul. We kept an etherpad during the event to track issues so that I can go through post-workshop and take advantage of all the feedback to improve our offerings.
What's Next?
The next PyStar I plan to organize will be in late July or early August and I'd like to do that one in SF. Following that I'm going to plan one in Toronto for mid-to-late October. What we did this past Saturday is only the beginning. I'll be working with all the folks in the pystar group to get this program shaped up into a much more modular system for learning Python and Django in stages (badges) and also will be setting up sub-groups for things like hack nights, code-masters (think toastmasters but writing code in front of people), and I have this idea of taking the PyStar lessons into women's prisons as a way to get marketable skills into the hands of people who need them badly.
Anyway, first we'll get more material prepared and digest/incorporate all the excellent feedback. Then we can take over the world :)
I hope I'll see you at future events. Thanks to everyone who helped make this a great day!
Subscribe to:
Posts (Atom)