So you want a new Talos suite, eh?

This quarter Alice and I have focused on trimming the list of pending test suites and where several new ones (419776, 524089, 515540, 506772) have been turned on in production.

The process for getting a new suite in has becoming a lot clearer, so we gave a presentation at the recent all-hands to help the developer know what to do on their end and what RelEng can do for them once their test suite is ready for staging.

Here's what a developer needs to do:

• Download and install Standalone Talos to test their suite in
  
• Once they have established that the test works on at least one platform, write a patch against talos in cvs
  
• File a bug against RelEng in the General component and provide the following information:
  
  • Contact person who will work with us on getting the test suite enabled
    
  • What the test does, what the expected output should be, long name, short description
  • Which branches and platforms you want the test run on

RelEng will create the buildbot patches that enable the tests, insert the tests into graph server, and work with the contact person while the tests are in staging to make sure the expected outcome is reached. Once the tests run as expected we can turn them on in production. Perfect world turnaround for this process is about a week and a half and involves a short Talos downtime. The rest of the time allotted to our presentation was spent discussing where we should be setting our sights for Talos improvements. This looks to involve two relatively large undertakings:

1. While it recently underwent some much needed improvements, the graph server still needs to be faster, more stable, scalable, and able to handle our ever-growing data sets. The blocker here is that no one really owns graph server and it's hard to know who should.
2. Talos is barely holding up under the current load of tests, hardware, and infrastructure. It also works in such a way that a lot of manual involvement is required to add new tests. It would be awesome for it to work more like unittests where once individual tests are checked in, they would go into production immediately. It would then be possible for a developer to not only write a unittest for any bug fix, but also a performance test to go along with it.

Now this brings up the problem of what performance we want to measure and how we want to approach performance metrics in the long run. Alice made a great point when she stated that folks who are not trained and accustomed to doing QA might be challenged by trying to generate tests that actually create a good metric for the performance they wish to be testing. It's entirely possible to have tests that seem interesting on the surface, when you drill down, don't provide any useful data for actually improving anything.

Do we want per-bug performance tests as we do with unittests? While it looks like this is a way to make a developer more accountable for their code, it's pretty obvious that this model wouldn't scale well at all with our current hardware and turnaround expectancy. Imagine as many individual pageload tests as there are mochitests...I suspect no one wants to see that.

Performance testing would be better and more useful if it was targeted at specific features or areas of the product where someone is actually tracking the improvement/regression ranges on them as they are developed. That's a key area of Talos - that a human is actually accessing the data, finding it useful, and making improvements on their feature/area as a result of this information.

While brainstorming with Aki on the potential of the graph server data, one idea really got me excited. Open up the data.

There's been a lot of hype lately about opening up data. In February of this year Tim Berners-Lee encouraged us to start thinking about open, linked data and how it could be the next round in how the Web helps us re-frame our world. In Canada the city of Vancouver opened up its data in the hopes of "improving liveability and governance" in the Metro area.

What if the Talos graph data was made available to the community and a challenge was created in the spirit of the marketing design challenges where we ask people to help us find new ways to view the data? I'd be really curious to see what kind of visualizations would come out of the larger community. RelEng doesn't have a very large community outside of employees, so this could be a great way to start working on creating one.

Upcoming improvements to Talos documentation and test suite creation

This quarter I'm going to be joining Alice in trying to improve the system for adding new suites to Talos.  The current system involves a lot of hackery on our side and slows down the ability for us to get Talos suites up and running as quickly as might be desired.

So with John's help to create a prioritized list of suite requests, we will be doing a lot of communicating with developers in the coming months to get them up and to improve the process and documentation at the same time.  Currently there are 10 new suite requests waiting that are known and there may be others.  

Part of the issue with adding new suites is that there is a lack of documentation and tools for developers.  Our new system will look more like this:

* A request is made for a new suite and a developer is attached to the request who will be the lead person for working with us to get the suite into production

* The dev will be able to use tools we provide (standalone talos, corral of staging-talos slaves) to do proof of concept on the suite so that it works and is ready to go up in staging when it's handed over to RelEng

* RelEng will enable the test suite in staging and verify that changes in staging work fine with the other existing jobs being run on the same machines. Once all is well, then rollout to production would happen

As we progress through the suite requests, this process should get easier for all parties and more streamlined.  We hope that by the time we reach suite #10 it will be much easier and faster for developers and RelEng to get the proposed new Talos suites into production.

I mentioned the developers will have tools provided by us. We need to do a bit of work to make these tools usable by developers and the first place to start is with our documentation of what Talos is and how it works.  Following this we will have discussed having boilerplate code for creating each of the two styles of tests startup or pageload.  Also, it might be beneficial to have a coral of Talos machines that can be loaned out to a dev for a limited time in order to test a suite during creation and debugging.  This coral could then be re-imaged and passed along to the next suite developer.

Here is the current documentation page.  Doesn't give you much to go on, right?

Well this is about to change.  Given my complete lack of Talos knowledge, I will be writing up what I learn about Talos as it's happening so that hopefully a more complete set of docs will exist for the Talos neophyte and folks who want to work with us to add new suites will benefit from this as well.

Here's the current list of the docs to be created based on what we think you might want to know:

* How Talos works and an overview of the development from past to present

* What preferences Talos runs with

* A description of each test suite, what each runs

* What the numbers mean

These are the things I don't know - is there anything you don't see listed here that you want to know more about?  Feel free to make suggestions in the comments.

Mozilla Service Week - Toronto Event

Yesterday I dropped off posters at the Parkdale Library for our Mozilla Service Week event which will take place on Monday September 14th from 2 - 6pm. The Parkdale Library is a really lively branch, with about 10 computer stations that the neighbourhood folks use constantly. Parkdale is the oldest Toronto neighbourhood and though it has a reputation for being a "bad" neighbourhood, it's been heavily gentrified in the past 6 years or so. There's still a lot of people here who live below the poverty line though, and for whom the digital gap is a very real thing. It's also a neighbourhood full of recent immigrants who depend on the library for connections to learning english, finding work, accessing resources for new Canadians, and keeping in touch with family and friends in their home countries.

When I first moved to Parkdale in early 2001, I relied heavily on access to their computers for my internet needs since I didn't have a computer. It was always a stampede to get in the door and sign up for a time slot when the library opened its doors in the morning. They've recently undergone some renovations and now have added wireless as well as a few more stations. The Parkdale librarians are super friendly and encouraging of community (and noise, in a library!) and the building itself is used often for local activities and grassroots festivals. I'm excited that 8 years since arriving here, I'm in a position to give something back to this vibrant place.

Our event involves a table set up near the computers - "Ask a Geek" - where we can field questions about anything that will help improve their interactions with the web. I believe there will be lots of people interested in picking our brains. Of course, I'm also bringing lots of Mozilla swag to draw people over to the table and to use as ice-breakers :)

Anyone in Toronto who wants to participate - please come on by.

Details about this and other Toronto events HERE

Adding choices to Try Server web interface

Just put in my patches on bug 473184 which will allow folks who submit patches through the try server web interface to select if they want a build or unittest run and what platforms it should be run on.

Looks like this:


I've tested it successfully in the staging environment and I hope to get this rolled out in production before the end of the quarter.

New Branch Timeline: Electrolysis

Now, a week or so later - we are setting up a new project branch. Here's the break down:

2009-06-26 11:55 PDT

• Bug requesting the branch [500755] was created
  

2009/07/09 11:13:41 PDT

• Created a tinderbox page for the branch to report to [Tinderbox Page]
  

2009-07-14 17:44:27 PDT

• Patches to add Electrolysis branch to buildbot are written and put up on staging-master. There are several compile fails.
  
  

2009-07-22 06:12:49 PDT

• After testing and scheduling downtime, the config files are checked in and P-M is reconfigured - last minute patch to turn on debug builds is added as well when we realize that is not in the default template for turning on a new branch.
  

2009-07-22 13:31:59 PDT

• Need graph server machine table updated for new branch - we did a bunch all at once in [504435]
  

2009-07-23 06:25:18 PDT

• Add Nagios monitoring support file is checked in and bug filed requesting IT turn it on. [505986]
  
  

What remains is to check in the debug builds patch, this should happen in a downtime next week at which point the project branch request bug will be closed.

New Branch Timeline: Places

A brief rundown of what was involved setting up the Places project branch. This is based on the time since the branch request was given the go-ahead, not when the bug was filed since that happened quite a bit earlier.

2009-04-29 17:17:43 PDT

• Bug requesting the branch [459269] was re-opened
  

2009-05-18 16:31:46 PDT

• Created a tinderbox page for the branch to report to [Tinderbox Page]
  

2009-05-19 09:16 PDT

• A separate bug was filed requesting a repo [493745]
  
  

2009-05-26 16:32:18 PDT

• Repo is created and [493745] is closed as FIXED
  

2009-06-24 10:22 PDT

• Patches submitted to update config files for Staging-Master and Production-Master
  

2009-06-30 12:51 PDT

• After testing and patch updates, the config files are checked in and P-M is reconfigured
  
  

2009-07-01 08:49 PDT

• Add Nagios monitoring support by filing a bug with IT [501710]
  

2009-07-08 08:04:40 PDT

• [493740] is fixed to deal with the scheduler not picking up the new Places poller after a reconfig, only after a stop/start
  

2009-07-09 14:36 PDT - 2009-07-10 12:50 PDT

• Patches submitted to turn on talos and graph server support for the new builds. The first set were not patches to Talos-Pool so a second set was required.
  
  
• New row in graph server added for branch (bug 459269. IT (justdave) ran the INSERT statement against the production database)
  

2009-07-13 11:18 PDT

• Patches submitted to turn on leak testing debug builds. This was checked-in and P-M was reconfigured the same day.
  

2009-07-13 14:57 PDT

• Bug closed - project branch is up and running on P-M
  

Celebrating Firefox 3.5 with sparkly accessories...

A month or so ago when the Firefox 3.5 was close to launching I got in touch with some artist friends of mine who have a small jewellery making business to create some custom accessories for me in celebration. They have been making these awesome belt buckles, cuff links, magnets and many other items for years and I have several of their pieces including a custom "fancy deluxe" belt buckle with my hound dog on it surrounded by shiny Swarovski crystals. They did a great job with the Firefox logo and I am now the proud owner of

A belt buckle:

And cuff links (though I only have one shirt that uses them):


I'm looking forward to wearing them about town and spreading the word about Firefox 3.5. Many of my pals outside of tech circles have been excited about the new Firefox because they see how excited I am about it. For anyone else looking for a custom buckle or accessory of their own get in touch with the folks at Barbie's Basement Jewelry. I hope you enjoy flashing the Firefox logo around in new and fashionable ways - it's always a conversation starter.

Mozilla Service Week - making a difference in your community

Mozilla just announced Mozilla Service Week which will be held September 14 - 21, 2009. This week is a push to connect people who can help make the web work better for someone in the community with people and organization who need that help.

Coming from the non-profit arts sector prior to my job at Mozilla, I will be spreading the word with many arts organizations in Toronto who would be wise to sign up for help from such a talented pool of volunteers. Of course I will also donate my time that week even though it's the kind of work I do all the time already. I can't even count the amount of time I've spent setting up routers and networks for less technical folks in my life, or helping them set up their new computers and teaching them basic skills (all teaching sessions include installing and setting up the latest Firefox of course).

The best part of this week, in my opinion, is that it's an opportunity to get hands-on with local users in the community. The library is a great place to start. It wasn't that long ago (around 2003) when I was using the library computers as my primary access to the internet. I'd love to go in now and make sure that their computers are up to date, and write up how-to manuals and helpful hints for beginners. Even better, get some folks to translate those manuals or tip sheets. At my local library I'm certain that there are many folks who would appreciate localized information sheets.

Two areas that are of particular interest to me with regards to the organizations I know in Toronto:

1. Bring their websites over to an open-source CMS like Drupal. Many of the sites are hand-coded php (or god forbid Dreamweaver-created sites) with no administrative back-end and keeping the site's information up to date is a difficult/dangerous task for non-technical staff.


2. Take their FileMaker Pro databases over to MySQL or PostgreSQL so that they are no longer locked in to expensive, proprietary database software that requires additional hosting costs. Three organizations I have worked with are on three different versions of FMP and none of them the latest. Upgrading is painful for them and their hosting costs are ridiculous (especially the ones who are on older versions).

If you're technically inclined, go to the site and sign up. If you've got an organization in mind, tell them to sign up. Let's make this event a success so it will inspire more weeks like this in the future.

iPhone tethering - how to restore after updating to 3.0b5 and iTunes8.2b10(13)

Breaking my extended blog silence (I've been busy, you know, starting my full time job at Mozilla!) to tell you about the quagmire I went into when I tried to set up tethering on my iPhone.

So here's the story - with links that will help you avoid what happened to me.

I purchased and ADC membership so I could download and install the iPhone 3.0 SDK and OS. When Beta 5 came out I eagerly updated since my phone had been crashing regularly and went into restore mode. As soon as the 3.0b5 update went live on my phone, iTunes said it needed 8.2 to work so I downloaded the most recent version from ADC. This turned out to be 8.2b10(13) which disables the ability to change carrier settings and thus, to enable tethering if you didn't already do it before the update.

So - after many (failed!) attempts to go back to 3.0b3 and 3.0b4 so that I could use iTunes 8.1 - I did eventually (10 hours and many blog tips later) work out my tethering issues with this extremely simple set of steps:

Simple Steps to restoring tethering:
Download: http://www.adammcnamara.com/CanadianCarriersTetheringPatch.zip
Download: iTunes 8.2b7(10) http://thepiratebay.org/torrent/4874487/

* Delete iTunes 8.2b10(13) from your computer, install the older version - as the torrent description points out, it says that it installs as an 8.1.1 but it is in fact 8.2.

* When you connect your phone in iTunes, Option-click the “Check for Updates” and pick the.ipcc file for your carrier (from the carriers patch you downloaded). It will update the carrier settings.

* Reboot (NOT RESPRING) the phone. You can now tether over USB or BlueTooth (unless you have a new Mac Unibody Mac where apparently the Bluetooth stack is broken and so you're relegated to only sharing via USB).

To activate tethering, go to “Settings->General->Network->Internet Tethering”. The first time you do so it may say “Contact Rogers to enable this feature”, but a few second later it should allow you to enable it. If not, restart the phone again.

Thanks to many blogs out there but more specifically to Adam Mcnamara where the bulk of the steps are from.

Tryserver now has unittests on all 3 platforms

Bug 445611 is now resolved fixed, you can see the unittest results on all three platforms here: MozillaTry.

There are hopefully more machines coming that will be allotted to tryserver, with the likelihood that more developers will now use the tryserver. I'm also hoping to start working on setting up the web interface to allow for the selection of which platforms you want to try your patch on and whether you want unittests run or not since at the moment they are turned on for all patch submissions by default.

This patch also required the landing of a patch for bug 479225 where we now call the reftest/crashtest suites and the mochitest suites with make from the top level directory which helped us get rid of a bunch of extra workarounds we had to do for Mac OS since it needed to know which .app file to look for and we're changing that name so often (Minefield, Shiretoko, etc). Now all three platforms are even closer to being alike. Many thanks to Ted for making this possible.

No More Passwords Please

This is the tentative title for my upcoming white paper, which is the major deliverable for the btr820 course on Research Methodologies and Writing. I'm excited to be doing this paper because a) I love writing and b) I'm looking forward to learning more about my topic which is essentially looking at solutions for the future of authentication on the web.

As users of web sites and applications, we are now subject to having to authenticate ourselves multiple times a day - I read somewhere than an average is 13 but for some of us who spend more time online it's probably twice that. Having your passwords remembered for you by the site or by your browser helps, but that is not a great solution for folks who are on multiple computers. Besides our passwords aren't even that safe to begin with (my bank won't let me use more than alpha numeric characters) and some sites make you change them regularly for extra security (a lie) and so as users we are caught up in a game of constantly trying to stay on top of the latest password for which site and please stop the web now, I want to get off.

What I want to look at is open, decentralized authentication identifiers that go beyond passwords with regards to actual security, that could be in your browser itself, and that would move with you easily no matter what computer you are on.

So I have some questions.

What are the implications of a web browser incorporating an open authentication protocol out of the box where the identifier is the browser itself?

What other options are coming down the pipe in terms of built-in browser features that help users deal with authentication? Is there something better than a decentralized open authentication protocol?

Do browser providers have to stay neutral and leave it up to web application providers to decide how users authenticate on the web or can they step in and lead the charge towards a certain protocol and influence sites instead?

While Weave is an excellent way of syncing your profile across various computers - is it really scalable? What other options are there for having an easy, portable profile which would be able to contain your identity as you move between computers, countries, even to your mobile device?

Thanks for reading this, I look forward to your thoughts on this issue.

There are no new ideas...

So I'm in a MacOS and iPhone development class this term. It's my last term in the Bachelor of Software Development degree program at Seneca College. I've now done some major project implementation, interface design, web programming, business plans and marketing exercises - a bunch of useful skills have been developed to be sure. However, today I am having a moment of doubt.

Here's the issue:

I'm a pretty new programmer/developer. One of the main reasons I chose this degree was to become more able to help people who are not so into computers and programming, to be someone who could help link the non-tech people (mostly artists) with technology in ways that are comforting and useful not scary and burdensome.

So.

I had this thought that for my MacOS programming assignment I would make a little app to help my freelancing friends keep track of their time spent on contracts and print up nice little invoices at the end. Simple, useful, something I myself would need from time to time when I do side projects. I have a free app called Khronos which sort of does this but creates ugly invoices so I was inspired to improve on it.

Thing is, better (not free or open source) apps already exist. And they're good. As a pretty green programmer (and someone with 7 weeks of classes left) I can't touch these apps for their functionality. I would like to think I would do slightly better on the design, but that's not going to mean much if I can't come close to implementing the kinds of features the other guy has.

Now I wonder how to even a) get motivated and b) set reasonable expectations for myself in this assignment. I am feeling daunted by the fabulousness of these other programs (which I realize took years and a team of developers to accomplish) and it's hard to see how I can dip my toe in the water.

If anyone has read this far - advice is much appreciated. What do you do when you want to write something and find out it's already out there? How do you scope out an app for your first version when there are apps that already do more than your 1.0 could?

Unittests on Try Server - Linux HG Builds is Go!

So today we deployed a unittest builder for the Linux platform on Try Server. You can see on the waterfall that there is now a column for Linux unittest results. The builds resulting from that builder will not be uploaded anywhere.

The Mac unittest builder is close and I hope it will be ready to go later today, if not then very early next week. Windows is of course being a pain and isn't production ready yet.

This is a big win in that developers can now test their patches in the Try Server and see the unittest results before committing, it's unfortunate though that the current implementation of this functionality is going to take 2 builders per platform per patch. We only have 4 slaves per platform on Try and so you can imagine that this will slow down Try significantly. A Linux unittest run only takes ~ 1hr though so that's good.

Anyway, enjoy the unittests. Watch for more soon.

Learning from Lab 1 in Mac OS and iPhone development

Just finished my Lab 1 for Peter's Mac class and I want to keep track of the little bumps along the way.

Things I learned:

1. If you want the keyboard to hide after you are finished inputting text into a field - basically once the field loses focus you need to do this:
   [txtControl resignFirstResponder];
   thanks to iCodeBlog for that one. That site also has the best tutorial that I could find for getting started on iPhone apps.


2. It's much easier to do a View-based application with the iPhone SDK template, but this means you have fewer UI objects to play with. I still have not figured out why this is. In the regular application template there is a wide variety of inputs, pickers, etc. But in that template you have to create custom views and though I read up on this a bit, it seemed like overkill for Lab 1.


3. Looking at the existing apps in the iTunes store is a good way to see what UI designs are working already in the real world. I like to think I'm pretty good at laying out apps, but it's amazing how much better they can be and it's very helpful to look at what people are doing in the store since they have probably gotten a lot more feedback on their UI.

The amount of information out there about iPhone code, especially for beginners is really sparse still. It seems like every time I google about an issue I'm hitting the same few blogs/sites. I'll post the code for my apps when the due date has passed so that there's just a little more out there. Remind me if I forget.

Holiday boredom leads to memes...

Taking this from Dolske:

1. Open this link, select-all and copy it.
2. Open up the Error Console.
3. Paste into the Code field at the top, and click Evaluate.

10 random Wikipedia URLs out of 85...

1. Polaroid
   
2. Mötley Crüe
   
3. Letterpress printing
   
4. Morphology (linguistics)
   
5. Eicar test file
   
6. Settlers of Catan
   
7. The Wire (TV series)
   
8. Transport Layer
   
9. Explorers (film)
   
10. Jeremy Hammond