Crashing Into Open Source (without a paddle)

Growing the company, structuring volunteerism: My response to David Eave's community lifecyle audit

2012-01-26T18:00:00.000-08:00

Last Wednesday David Eaves presented the results of the multi-tiered contributor lifecycle audit (watch the video). A few points really grabbed my attention and as someone with a background in arts & education non-profits I feel the need to share my experiences alongside my reactions to this talk.

David pointed out that as we are growing, we can hire people to solve problems, so what exactly do we need volunteers for? Survey results showed that contributors often don't feel their contributions had much impact on the project and that as our paid staff pool grows in size, there is less clarity about what exactly a volunteer's importance is to the critical path of Mozilla's mission. I wish we had this data prior to the 1+ year push to releasing Firefox 4. My hand-waving hypothesis would be that as we were cramming to get a product out the door we forgot to be leaders of volunteers and instead unconsciously pushed them aside so that things could get done on a schedule. It was a stressful time, but now that we've moved on to regular, rapid releases perhaps we paid staff can all take a step back and really assess how we incorporate the work of volunteers into our individual areas.

For many Augusts I have worked at a women's music festival in the woods of Michigan and at that festival there is a kitchen that has pumped out 3 vegetarian meals a day for 5 days to 4000-8000 attendees depending on the year. This festival relies on a core group of 'workers' who are in fact volunteers but let's pretend that group of 500-600 people is like the paid staff of Mozilla. The main kitchen work crew gets about 30 workers to run the kitchen. You might think to yourself "but 30 people cannot produce enough food for 4000-8000 women" and you'd be very right. The way it works is that all attendees of the festival are asked to do two 4 hour workshifts during the week they attend the festival. The majority of workshifts center around the main kitchen and creating the meals which range from burrito night to pasta putanesca to nutloaf (a vegetarian version of meatloaf). All these meals involve preparation of pounds upon pounds of vegetables as well as cooking of pasta, beans, sauce. Did I mention this was all in the woods, over a massive firepit? Yup, so 30 women are in charge of making that entire process work and they do so by wrangling hundreds of volunteers per day into shifts around each meal and constantly leading and dividing up the work so it can be done by many hands yet results in one huge meal - 3 times a day!

So let's go back to people who are volunteering not feeling clear about how Mozilla works and whether their time and effort has impact. How can we make sure that the smallest task makes that person feel like they've contributed? Some areas of Mozilla do this very well to name a few; AMO, SUMO, QA, and Personas. These teams manage tons of volunteers and have tasks with measurable outcomes (tests run, filing bugs, approving an add-on, answering a question in the knowledge base, shrinking the queue of pending Persona submissions) and sometimes they can use scoreboards or themed days to get volunteers mildly competitive for the respect of their peers and accomplish larger goals in a short burst. I'd encourage people interested in having volunteers participate in their team's work think about how to make sure their volunteers have tools to measure their impact from the get-go. In Release Engineering I would measure the number of bugs that we have yet to fix, many of them labelled "simple" or "old". If I had volunteers doing RelEng work I could keep track of their progress and post reports (as blog posts) of who fixed what when and how as the number of bugs shrank.

Another interesting result from the survey: older folks (34-55!) aren't on-ramping as much as younger ones. At first I wondered how much of this was about access to the muscle memory of being a student. I think it's fair to assume that many students/youth can have a lot more time to contribute to projects like Mozilla as certain adult responsibilities may not be required of them yet. Over the past week though, I have started to visualize another take on this. In the arts & social justice organizations I have been involved with there have been plenty of adult volunteers but those organizations have a different need for volunteers. The music festival I mentioned would not happen if it wasn't for volunteers. The fact that the volunteers want the community of the festival to exist for them every year becomes the carrot drawing women of all ages to come to the woods of Michigan and build a music festival every year. People quit jobs, take unpaid leave, and make plenty of other sacrifices of their time to participate in creating this community. The thanks for this 2-4 weeks of donated work is an amazing live/work experience camping with 5000 women on private land, having workshops, parties, and dances all in a very natural, safe, and ad-free environment and watching incredible performances all day and night for 5 days.

In a different example, let's look at a the Toronto International Film Festival. Volunteers have to make it through the rigorous screening and application process in order to 'get' to volunteer for the festival. They are rewarded with behind-the-scenes access to the festival, sometimes a quick run in with a movie star, and free tickets to festival screenings. The festival shows entirely world-premiere films so this is a huge deal for a volunteer. Several films will see theatrical releases after the festival but seeing the premiere, often with the star in attendance and with a director Q&A post-screening really sweetens the experience. The volunteers also get thanked before every screening with a cute trailer from the sponsor for the volunteer program and there's always a fantastic party post-festival as the final thank you.

At Mozilla we do thank our volunteers with tshirts and sometimes bringing them to summits, MozCamps, or other parties. For older volunteers though, I wonder if that's enough. What does it take to get someone in the 34-55 range to donate their time? I'm really interested in this one since I fall in that demographic as well. For me, I need the time donated to do at least one of the following things:

be social, meeting new people in the community I chose to volunteer in is a big part of why I'd do it in the first place
provide a networking opportunity (similar to above, but might apply to folks on the job market a bit more accurately)
get me free access to an event I wouldn't attend if it wasn't
be something my friends are also doing so we are visiting with each other while doing something interesting
be for a very good cause so I feel good just helping that cause regardless of the tasks I perform

Let's look at that last one. The good cause is certainly in the eye of the beholder but I can honestly say that sometimes I have no idea why I would want to encourage a friend who volunteers for hospice care, homeless shelters, AIDS awareness, or other non-profits where the staff is small and the operating budget miniscule to come and contribute to Mozilla. In the circles I travel in outside of my job Mozilla seems right up there with Google, Apple, and of course Microsoft. Sure, a lot of people don't know we're a non-profit. I tell people that all the time and while it's of interest to them, it doesn't generate an "Oh! Can I volunteer there then?" kind of response. We compare ourselves sometimes to the Red Cross or Boy Scouts - large organizations with volunteer bases - but I think we should start to re-think ourselves more like the film festival or the music festival. We pay competitive salaries to our employees, we throw amazing events, and we don't have to write grant applications every year to the government (like in Canada) or to private funds (like in the US) to ensure we can keep operating on a shoestring budget. So even though Mozilla is a GREAT cause, I don't think that's the bait for the on-ramping of volunteers - especially non-students and people in the 34-55 age range.

What's going to encourage 34-55 year olds to engage with Mozilla? In my opinion it's going to happen with targeted events where they can do something in a few hours that leaves them feeling connected and fulfilled and even better if it makes something in their life easier. A while back, in Toronto, we did a day of service and set up an info table at the local library so people could come and ask anything about Firefox and even though by some ironic twist the library's internet died we still had an amazing day just conversing with people and answering questions about Firefox, the web, security, and even general computing questions. According to David "having good experiences in the project helps one to want to find others and pull them in" and "age and gender have no impact on the willingness to on-ramp, but the longer you volunteer with Mozilla, the less you on-ramp". My approach with trying to on-ramp then, in light of this, would be to look at getting a lot out of that short interaction. Help someone help themselves on their computer. Teach them a few keyboard shortcuts or how to install an add-on that helps them do what they already do faster and with more confidence. Then encourage them to come back and teach someone else what they learned. This can spread like a pyramid scheme and it's no longer about getting a single volunteer to stick around for a long time, it's just about having a good experience and carrying that forward. Potential volunteers can be motivated by the mission and/or by practical considerations it's important to remember both have tremendous value to the Mozilla project. I think it's important to encourage 34-55 year olds by believing it's OK for a contributor to do a one-off in a few hours as long as they walk away happy.

In conclusion of this very long post, I want to circle back to measuring. If we are going to make community a core part of what we do then we need to measure it we currently do not have an institution-wide measurement of contributions, volunteer performance cannot be evaluated, there is no structure for including volunteer engagement during strategic or operational planning. Until we require Directors to create annual and quarterly goals that include measurable goals around volunteer growth, retention, participation, and effectiveness we will only see people (like myself) trying to do this "off the corner of their desks" which means it's not a part of your paid work and thus less likely to be sustainable and effective. The Toronto International Film Festival is a great example of what we could do here. They have paid staff who organize the volunteers each year. They have a clear path for volunteers to follow to be accepted - training sessions are mandatory. Each year returning volunteers are given roles depending on performance from previous years. The record kept of each volunteer's performance allows paid staff to request certain volunteers for specific tasks based on that information and a volunteer's history with the organization. Teams of volunteers will sometimes have "Captains" who are also volunteers but with more experience and they are thus given more responsibility. Each area of Mozilla that can accommodate volunteers should keep an eye out for their current or potential "Captains". David also suggested that, while we avoid it, we should really look head on at guidelines for when to rely on volunteers and when to not - this seems to fly in the face of open source "free for all" mentality but if we compare ourselves to other non-profits like TIFF there is proof that having some structure for volunteers allows staff and teams to develop stronger relationships and the work gets done smoothly, which was really the point all along.

Don't forget to throw them fabulous parties, share with the world the importance and impact of their contributions, and remember you can never thank a volunteer too much.

OccupediA - Women Contributing to Wikipedia (the first of many such events)

2012-01-10T17:58:00.000-08:00

Last Thursday night about 8 women arrived at Noisebridge to learn how to contribute to Wikipedia. Several things led to this gathering:

An article in the New York Times back in October drew attention to the lack of women contributors to the Wikipedia knowledge base and that got me thinking.
Having organized other spontaneous "women get together and learn stuff" events I figured I could take the same approach to Wikipedia contributing, get some women together to create accounts, generate content, learn how to stop vandalism and see what would stick.
Recent participation in activism around the Occupy Wall Street movement also inspired me to try and reach out to communities I am in who are not as technical, to encourage people to come first with knowledge and interest in topics Wikipedia could benefit from and let the tech come second.
A month ago Elsa and I were talking casually about all the the above mentioned things and we decided to just go for it and pick a date, throw it up on the Noisebridge (local SF hackerspace) calendar, and see what we could make happen.

We took over a small makeshift classroom space at the back of Noisebridge. It had one lamp as the primary source of light because the fluorescent holders above were missing their tubes. A man was near the back working on a dress for fashion school, several other hackers were up front working on their various projects. Noisebridge was a wonderful place to have this event. It feels like anything is possible in a space like that.

I was happy with the turn out - we had a mix of artists, educators, and tech workers. Also as a bonus one of the attendees, my coworker Boriss, was a seasoned Wikipedia contributor who was able to really detail the ins and outs of the different levels of participation. I can't stress enough how amazing it was to have her and her knowledge there because there are lots of misconceptions about Wikipedia (I definitely had some) and her first-hand knowledge was inspiring to me.

So the beginning of the meetup went well enough, and as you might expect. We introduced ourselves, talked about why we had come to the event and what we were hoping to get out of it. We started in on learning how to set up an account if one didn't already exist and we looked at discussion/history/edit and other basic navigations of Wikipedia space. There were a lot of questions about what belongs in Wikipedia, neutral tone, citations. The conversations were lively and I found them quite enjoyable.

Here's what I didn't expect: Getting folks interested and excited about Wikipedia becomes REALLY HARD in practice. Unlike learning Python where the participants can hammer out some code on their own computers in minutes and feel accomplished, there is a lot more complexity to Wikipedia. There is a lot of confusion about their UI, their purpose, who can do what and when. Very quickly it seemed that the women who had come to the event feared adding anything new to the knowledge base and they were also incredibly intimidated by the UI of the site. It wasn't even clear enough how one would create a new article when none existed.

From this event I learned a lot about organizing and about the intentions of future events like this and I did a little braindumping while we were meeting so I could remember to list them later in this very post.

Things that would help newcomers:

Having a "new to wikipedia" moniker next to their nickname for the first N activities on the site (we have this on our Mozilla bugzilla) so that hopefully older and wiser participants would be extra nice to them
Find a way to make some of the simpler tasks that help Wikipedia (typos, reverting vandalism, categorizing articles) into a game that a new arrival could play that would start easy and then move more toward the real-life workflow of working on Wikipedia - as a way to warm them to the UI
Encourage newcomer to write a straight-up article and have a place for these things to be dumped for inpection/linkage/categorization and otherwise Wikipedia-fying the knowledge dump. My partner is an English professor and can certainly write good content for Wikipedia but everything about the site is intimidating. There should be a page where she could copy/paste or upload a document of her article and then let people who know wiki syntax and the other requirements an article needs come along and finish it up
Make it way easier to find the "adopt a user" program that I hear exists but no one would know to find that from the Wikipedia home page

I will continue to organize these events, perhaps once a month. More reports as they happen.

Want more women in Open Source? Donate to Ada Initiative today!

2011-12-14T13:26:00.000-08:00

Short version: If you love women, or even like them just a bit, go right now and donate to the Ada Initiative to show the women in your life that you value their contributions past, present, and future to the wonderful world of Open Source. I'm going to make a donation in my grandmother's name this year and I know she'll be happy to have supported such a valuable project.

Long Version

I love Open Source.

When it first came to my attention, in the first year of my degree in software development at Seneca College, I knew we'd be a good fit. There's something about the spirit of Open Source that instantly clicked with my existing guerilla activist sensibilities. The way that you just take what you want and make it happen. That you create and give away. That you work with other passionate people to make cracks in the surfaces of monopolies that only want you to be able to do things through their (usually financially) gated communities. It reminded me of how I had approached being a filmmaker - taking $50 of Super 8 film and developing it myself in 16L bucket in a dark bathroom then submitting the results to a prestigious film festival and being accepted. Having my work shown alongside films with budgets bigger than the cost of a house was an amazing experience and taught me that not everything has to be polished to be valued.

Open Source is like that to me, the diamond in the rough.

While I was working on my degree I of course noticed (and was not surprised by) the lack of women in my classes. I was surprised when I started to get involved in Open Source to discover that there were less women in FOSS than in proprietary software companies. That seriously BLEW MY MIND. I mean, if Unlocking the Clubhouse is to believed (and it is very thorough research) then technical women want to do work that is meaningful and helps people. Why that sounds a lot like Open Source doesn't it? So why aren't there more women in Open Source? I'll let you Google that question to your hearts content, there's a lot written on the subject and so much more could be. The point though is that the Ada Initiative is a new project that is here to take on that very question through ACTION. They will DO things to get more women in Open Source. Women don't have to be dragged into FOSS kicking and screaming. Trust me, after seeing the overflowing wait list for the Grace Hopper Celebration of Women in Computing's FOSS day, there are a ton of talented and smart women interested and able to do work in Open Source. We (all of us who have already drank the Kool-Aid) need to help them get integrated and feel comfortable staying in FOSS.

When I first met my future team at Mozilla in April of 2008 there was a woman on my team (!) and she self-identified herself to me as a feminist within the first 5 minutes we were together. As someone who was coming in as a student with zero experience in professional tech workplaces I was so thrilled to have an immediate feeling of relief, trusting that if she was respected there I would be too. She also introduced me to wonderful internet properties such as GeekFeminism and Sociological Images both of which helped me start connecting with other feminists in tech fields. Almost three years later I am starting to feel like I've been successful in building the community in FOSS around me that I want to be a part of. It's a wonderful mix of the talented people I work with at Mozilla, the folks I'm working on planning the next Dare 2B Digital with, the programmers I organize PyStar workshops with, the Women Who Code, the Women 2.0, and of course - The Ada Initiative.

I'll leave you with their own words about why you should go straight to the donation page:

We’re proud of what we’ve accomplished already.  Since our founding in
early 2011, we helped over 30 conferences and organizations adopt an
anti-harassment policy, organized the first AdaCamp unconference,
provided free consulting on high-profile sexist incidents, wrote and
taught two workshops on supporting women in open tech/culture, and ran
two surveys, among other things.

http://adainitiative.org/what-we-do/

We need your help to achieve our upcoming goals.  The Ada Initiative
is funded entirely by donations.  Without your financial support, the
Ada Initiative will have to shut down in early 2012.

http://supportada.org/donate

Your donations will fund upcoming projects like: Ada’s Advice, a
comprehensive guide to resources for helping women in open
tech/culture, Ada’s Careers, a career development community, and First
Patch Week, where we help women create and submit their first open
source patch.  You can learn more about how the Ada Initiative is
organized and operated on our web site and blog:

http://adainitiative.org

Whether or not you can donate yourself, you can help us by spreading
the word about our fundraising drive.  Please tell your friends about
our important work.  Email, blog, add our donation button to your web
site, and tweet.  Here’s how:

http://adainitiative.org/support-us/spread-the-word/

You don’t have to stand on the sidelines any longer.  You can help
women in open technology and culture, starting today.

My First Startup Weekend: Women 2.0 Startup Weekend

2011-11-20T13:06:00.001-08:00

On November 18th, 2011 I jumped into the deep end of the Bay Area startup culture I have been lurking on the periphery of for the past two years of living here. After going to my first Geek Girl Dinner at Microsoft a month ago, and preparing to talk about women in open source at the Grace Hopper Celebration of Women in Computing, it seemed very much up my alley to sign up for the Women 2.0 Startup Weekend held in SF at The Hatchery. Originally Angie had asked me to be one of the mentors which, while incredibly flattering, seemed beyond my current skill set. I do always have ideas for new projects/apps though and I've been trying to get even more full on development experience under my belt so it seemed like a deal to get to spend 54 hours working on a startup idea for $50.

[tangent]
I love the immersion-as-classroom experience, btw. I made my first Super 8 film in 1999 at G.I.F.T.S under similar conditions where I lived with my other new-to-filmmaking cohort in a couple of trailers-turned-bunkhouses over on the beautiful island of Galiano and for one week we did nothing but eat, sleep, and breathe guerrilla filmmaking. We shot, hand-developed, transferred, and then edited our work, cranking out an entire short film in just one week. I left that experience filled with confidence that I could make a movie a week for the rest of my life!*

What I was hoping for out of my first Startup Weekend was an up-to-your-armpits code-a-thon and what I got was...very much NOT that. Here's what I got out of Women 2.0 Startup Weekend instead:

Pitching 101

Some people had come prepared. They had read an email I missed, knew what was supposed to be in a pitch, perhaps even had some code or a site name or some idea of what they would need to take the next step into their imagined company. I had none of these things. I had 2 ideas, one of which had occurred to me the week before on a bike ride. I jotted down my ideas quickly and 'pitched' them to a couple of women I knew from other local events (like my CodeChix pal, Vicki). Both of my ideas seemed to get people interested and with the help of a few very kind listeners, I chose which one I would officially pitch and worked on naming the imaginary app as well as figuring out what salient points I wanted to get across. It seemed wise to me to get into the early round of pitchers, little did I know that there would be about 67 pitches. I was #6 and so it took a long time to get to the point of being able to move about the room chatting people up, which I am sometimes not so good at

What I did: I pitched it, was too shy to really reach out to strangers and try to woo them to my idea, I hid my sign for a while only taking it out when people asked me about it, I got 7 sticky-note 'votes' for it (which was amazing to me), but I already knew that I would not be working on this project over the weekend and I was shopping around for a team that I would be excited to spend my next 54 hours with.

What I will do next time: Work more on my pitch ahead of time, have a clear idea of WHO I would like to join me, go around the room and find people who match those roles, have more research about my 'market' ready to help with the business side of things.

What I wish Startup Weekend organizers could improve: Help people match up by roles - so have all the designers go to one corner, all the marketing folks, all the developers, etc. Give us a visual of who's there looking to do what so that we can more easily go around and network. It seems less efficient to me that I would have to go chat up 10 people and perhaps discover that none of them are a match for what I'd be seeking. Even putting this info on people nametags would help - especially for folks who have multiple skills they want to highlight.

A Team is Formed

The eliminations were happening and I already knew I was going to put my idea aside for another time, so I had to figure out where I would lend my energy for the 54 hours to come. I'd been interested in a project called Safe Steps whose goal was to help independent women set a timer on their travel to ensure safe arrival at their expected destination. I spoke briefly with the woman who pitched it, and I had already learned from a conversation with a volunteer that the pitcher was a seasoned pro at marketing. I felt like I would learn a lot in that team but I was still checking around for other ideas. In the chaos of the eliminations I ended up behind a pillar with 4 people (one is a coworker at Mozilla) and two of them I had met briefly when they accosted me, they were looking for people who could program in C (and though I did it in school 4 years ago, I was not about to claim any proficiency). I asked if they had found what they were looking for and inquired about what they were planning. Judy explained her pitch about doing an educational project with the Kinect to teach language to children. I have experience teaching technology to both youth and adults, so working on anything that helps make educational materials accessible to all types of learners, as well as the possibility of doing hands-on Kinect-hacking for the first time, was all it took to sweep me up into this team that was bouncing off the walls and repeating those magic words: "Kinect" and "education".

Team Roles

We had 54 hours to come up with a demo of our 'company' for a panel of judges to evaluate based on marketability, creativity, and feasibility so when we got our workspace assigned to us at 9:30pm that Friday night we went straight to work. Introductions all around, describing our experience and what drew us to the project, came first and then we divided up into the technical team: James and I, and the Business/Strategy/Research team: Judy, Elsa, and Jen.

Our technical idea seemed simple at first - Grab the Kinect motion data and send it to Processing.js so users could interact with a language learning flashcard game that was one of many 'decks' our 'platform' would support. The initial deck would be a simple game with a bear where the bear calls out a verb, enacts it, then waits for the user to imitate both the motion and the word. I really did go into this thinking that was simple. Am I crazy? Turns out none of that was within our reach in a 54 hour period. The challenges are too many to list completely but here's a few: both James and I were completely new to Kinect-hacking. While open source Kinect hacks exist there were lots of library conflicts, documentation gaps, and finicky installations that led to failure on several frameworks we were trying and build on. I could get the Depth.js example to work in Chrome for a second (but never again for unknown reasons), but couldn't compile the native google plugin from the depth.js project so couldn't write new code for the extension. I couldn't build the OpenNI Sample-NiUserTracker after altering it to add a network tunnel so that it would report data to a node.js server (though I'm happy to have now touched Node.js even just a bit!). By Saturday late evening we had nothing to show except an intimate knowledge of library linking errors and compile failure messages. There still isn't a ton of material online about how to work with the Kinect data in a usable way. This actually gets me excited for future projects but at this point in Startup Weekend, we had to get ourselves a demo for Sunday's judging.

We decided to move on to the Kinect SDK that Microsoft provides, we installed Visual Studio 2008 Express and an open source gesture recognition library which allowed us to capture a movement and assign it to a saved gesture namespace. In the end, our demo was created in a few hours by James using those tools (and a bit of C#) while I came up with some very quick animations objects and put together our landing page.

Needless to say, the weekend was nowhere near being a code-a-thon. It was surprising to me how long it took to try and get a development environment setup and what I take away from this experience is that when the time comes to work on my own idea, if I bring it to a Startup Weekend, I should have the beginnings of an implementation already and have settled on a framework to build on that I am familiar with so that I can spend more time being creative about the idea and less time fiddling with configurations and installations of unfamiliar code.

Oh ya, but we won!

I should mention that the whole time we were having our ups and downs with the technical side Judy, Elsa, and Jen worked hard at analyzing all the angles of language learning by doing. I listened in at one point on a very helpful discussion with Cindy Alvarez who asked great questions about "what next?". Sure, verbs and kids are easy and lots of language-learning stops there - how would we push the envelope and take people to other levels? We had lots of mentors come by, and all of them poked and prodded at the research and story-shaping that the business end of the team was doing. At the end of the weekend our team won first place with a demo that had very little custom code in it, but I think we did well because we told a great story and had an extremely well thought out marketing strategy. When our demo was complete the judges were silent at first. Finally one of them asked the question we had prepared for "so, after learning verb with bears- what next?" to which I answered that we could build a platform for AI interactions in WebGL 3D space with the Kinect. Yes, I like to promise technology that doesn't really exist yet. It sure is exciting to imagine it though.

Some final thoughts

Startup Weekend, to me, felt a lot more like a school project than 'real life'. This is most likely due to the fact that I have a really great full time job right now and am looking at startup ideas mostly as learning and hobby and not necessarily something I would do for money for a few more years (at least). All the reading I have done about startups gets me thinking that I would likely go the way of bootstrapping and working on my scalable project in my spare time instead of trying to get a big VC investment and leave a steady job for the unknown. In terms of working during the weekend there are lots of ways to fall down rabbit holes and lose focus when you are working on something that is completely new. I love getting to learn about new technologies but there was this time pressure that kept us coming back to a general goal of having something to show at the Sunday evening presentations. The Startup Weekend environment isn't one for coding/development efficiency. It's distracting to have other people and their ideas/surveys/questions coming around a lot and to be working out in the open with your entire team instead of under noise-cancelling headphones as I normally do. It's not bad, it's just not a focused environment and it's good to know that for next time. I think it would help me set my expectations differently. It was important for me to learn that the goal of Startup Weekend is not necessarily to have a working application at the end but to have a really well thought out idea and story about your company's goal.

Speaking of story, come on out to TEDx Women next week where Elsa from Words With Bears will be presenting ours! Most importantly, I want to say that Words With Bears was a great team to work with. I heard stories of teams falling apart or losing team members, none of that touched us at all. We started strong and we ended strong. We're continuing to stay in touch and aim to develop this idea into something bigger.

* This is not what ended up happening, but I will always carry with me the knowledge that with little else than enthusiasm, a couple of rolls of film, and willing friends, a tremendous amount of creative output is possible in a short time with no budget.

New to IRC? Never tried it?

2011-10-26T17:46:00.000-07:00

Do you shy away from IRC because it seems daunting, complicated, or completely foreign to you? A pretty large chunk of the Mozilla online community lives and breathes in IRC so I encourage you to come give it a(nother) try. I recently updated https://wiki.mozilla.org/IRC#Getting_Started to help get you started. Hit me up with any questions here or find me as lsblakk in IRC :)

I'll re-visit the instructions soon to write about screen & irssi to keep a perpetual connection going in an attempt to make that option more accessible to folks who might feel it's too technical.

Where Are My $project-branch Nightly Builds?

2011-10-26T09:57:00.000-07:00

Did you know that we don't build a fresh nightly on a branch unless there's fresh code? Well, now you do! In the interest of saving even _more_ resources and network bandwidth so that we can accommodate even _more_ project branches we have added this little bit of logic to our nightly build scheduler. It makes sense, right? I mean, why have a nightly that is just like last night's nightly?

In other news, Ben just added 4 more twigs (aka Disposable Project Branches) for side project work and one of them is still available for booking.

Want to help? Encouraging community contributions to Release Engineering

2011-09-26T16:04:00.000-07:00

In a timely confluence with Mozilla's new Steward initiative, I'm preparing to get some community contributors engaged with some of the projects we work on in Release Engineering. A fair amount of our production infrastructure has to be locked behind VPN and sekrit passwords (we have 400+ million users to protect) but there are more and more RelEng side projects. We provide tools to the larger developer community and solve interesting scalability challenges with our unique (and massive) automation systems that can be worked on by any interested person in their own local test environment and then integrated into our /build repos. My personal goal is to try and get 2 or 3 regular community contributors to come work with us on tackling these.

In order to solicit contributions I have been working with David Boswell. We added Release Engineering to the mozilla.org/contribute 'areas of interest' page and I have created the beginnings of a RelEng-specific contribution page. The first two areas that I think would be a great introduction to working with RelEng code & tools are the TryChooser and our upcoming Autoland system. For the latter, our intern Marc Jessome is sticking around this fall as a contributor to carry on the amazing work he put into this system over the summer. He'll be continuing to debug the code and improve the portability of it so that we can get it into a beta testing stage by the end of October. As that work is being done we also need someone to help us write the API functionality that will allow sheriffs and developers to write tools that utilize this new hands-off landing queue. We'd also be happy to have people work on the issues that come up when we take Autoland to the next level - auto-landing on a production branch. To do this we'll want some automated backing out, bisection, and the ability to wait on getting patches reviewed before continuing.

Another great area for someone interested in helping out Firefox developers is working on the TryChooser syntax and features. There is a whole tracking bug dedicated to try_enhancements and most of those bugs are ones that can be worked on in a local staging environment. It's a chance to get your feet wet with buildbot and our custom scheduling setup. Some of these smaller bugs would be short on time commitment and high on developer appreciation if you fix them. That can be a winning combination for a new contributor, I speak from experience on that :)

So, if you're reading this post and you or someone you know is interested in dipping their toes into becoming a Mozilla contributor and these projects make you curious then come find me and we'll get you set up with a staging environment so that you can start fixing real world tools and automation bugs in no time.

Mozilla Seeks Program Manager for Open Web Innovation Incubator

2011-07-25T12:16:00.000-07:00

Ok, I'm a little biased - full disclosure: I work for Mozilla. But even if I didn't I suspect I'd be impressed with the amount of amazing innovation and hustle that Mozilla's community puts out towards making the open web more accessible to everyone. Recent projects like Popcorn and Butter are changing the way we work with video on the web. Hackasaurus is reaching out to kids, getting them to move beyond consuming the web (read-only mode) to being able to build and design their own web experience (read/write). Addons SDK, Open Web Apps, Browser ID, the list goes on and on and I'd better stop now or I'll lose you before getting to the good stuff.

Mozilla may be a huge brand but we're actually a relatively small group of people doing this work all around the world. Which is why we now have WebFWD, a program to help innovators of the open web get a chance to hook in to the resources of Mozilla (space, mentors, public reach, food and housing, and more) to help bring their products to the world. With this rolling program of projects Mozilla can be a driving force in getting more open web to the people who need them.

Right now we need a visionary and hard working Program Manager to become the leader of this movement. I'm attaching the posting below, get in touch with p at mozilla dot com with your resume for consideration.

/// WebFWD - Program Manager

Mozilla, the organization behind the Firefox Web browser, is looking for 
an all-star to join our new accelerator/incubator program WebFWD 
(http://webfwd.org) which aims to do for the open Web what organizations 
such as Y Combinator, TechStars and Seedcamp have done for startups.

As the Program Manager of WebFWD you are charged with leading the 
overall program - from designing and managing the curriculum, supporting 
the selected teams locally as well as globally, working with our 
ever-growing list of mentors and partners to organizing events.

If you're passionate about the Web, want to help people build amazing 
products and are willing to roll-up your sleeves, then this position is 
for you.

Primary responsibilities:
* Design and manage the curriculum for both the Fellow program as well 
as the Bootcamp
* Work with and support teams in the program (both locally and remote)
* Work with our mentors and partners
* Coordinate community and press outreach on a worldwide basis
* Create and run events (locally and remote)

Requirements:
* Excellent written and verbal communication skills
* Experience with organizing and running events
* Experience working with startups, entrepreneurs, venture capital and 
incubators / accelerators a huge plus
* Proven ability to work independently and in cross-functional teams
* Familiarity with Web technologies
* Passionate about helping people and solving problems
* Enjoys learning and teaching others
* Works effectively in a fast-paced, start-up environment
* 3-5 years of relevant job experience
* BA/BS, or equivalent experience

Try results to the bug(s) of your choice upon completion

2011-07-18T16:59:00.000-07:00

The TrySyntax helper and TryChooser wiki docs have both been updated to reflect the new option when pushing to try where you can now ask to have your complete summary of results (and a link to the tbpl page for your revision) posted as a comment to the bug on completion. Here's a live example to check out:

Sample comment in a bug when using --post-to-bugzilla in your syntax.

Now you have more control over how you get your try results and how noisy a try push is.

Please send comments and issues to the bug tracking this work. Thanks for trying it out!

A quick morning rant about "gender" and data collection

2011-07-06T07:03:00.000-07:00

This morning I read that Google+ is going to make your name and "gender" required to be public if you want to participate. This bothers me for several reasons:

Web sites and forms notoriously say "gender" when they mean "sex" and only put M/F or Male/Female as options. When this type of choice is required but called "gender" it erases many people who do not feel that those options cover their gender since that is actually something way more mutable than your assigned sex at birth. Solutions: Call it "sex" which is really what those two categories are or don't make something that is not in fact binary into a required choice of two options.

Google are so proud of being all scientific and data driven and I'm frustrated that they would not take the opportunity on their new potentially game-changing social platform to re-vamp data collection. Don't they have the processing power to allow people to put in whatever they like as "gender" and let the power of the search sort things out in the end? If a small number of people want to put "jedi" or "dog" let those people find each other! Who cares if there are some people who don't feel like Male/Female defines them? Why Google? Why do you want to act like two boxes can cover the breadth of human experience as it relates to gender in this world? Why can't you innovate on the small things as well as the big things that affect human interactions?

I'd really like to see a shift in how we collect data where there is more trust that the user knows who and what they are and that they want to share this information at their comfort level and that those on the other side, let's call them advertisers (cause isn't that what it all comes down to?), be the ones to deal with the outliers and uniqueness of human experience instead of trying to bash everyone into a two-party system.

Sidenote: When I have collected data recently for PyStar and allowed the gender field to be a text box I have found that the expected percentage (98%) of people entered "typical" information like woman, girl, female and that those who needed to express a different response appreciated the ability to do so by entering something else. Leaving this field to user input choice did not result in a messy, chaotic list of random words or unidentifiable descriptors. I fear not that most people will suddenly start to be something else when given more autonomy on forms.

Tree Closing Downtime Notice - 4am - 8am PDT Thursday June 16, 2011

2011-06-14T09:55:00.001-07:00

Trees will be closed for downtime so that we can land the following:

1. https://bugzilla.mozilla.org/show_bug.cgi?id=662396 -- Fix time on dm-wwwbuild01

2. https://bugzilla.mozilla.org/show_bug.cgi?id=600980 -- Set journal_mode = WAL for dirty places profiles -- This mean new performance numbers will start on Thursday morning after the downtime

3. https://bugzilla.mozilla.org/show_bug.cgi?id=649123 -- Run ANALYZE on dirty places.sqlite files -- This mean new performance numbers will start on Thursday morning after the downtime

4. https://bugzilla.mozilla.org/show_bug.cgi?id=663568 -- reboot the DNS and DHCP servers in scl1 -- Rebooting these servers has been shown to burn builds in the past, requires a short (~5min) outage to reboot these servers to allow updates to take effect.

5. https://bugzilla.mozilla.org/show_bug.cgi?id=663963 -- change LDAP to see if that speeds up mercurial -- This change should be entirely transparent. Hg processes that are running at the time that the change was made will have already loaded the NSS LDAP module and will continue to use it until they exit. The only issue to be aware of is that changes to hg access (group membership, or the creation of a new account) will not automatically propagate to the hg servers the way they do now. If any hg access changes need to be pushed urgently, we can do that manually.

If anyone has a reason not to proceed with this downtime, please let me know.

Thoughts on cultivating an "Everyone is Remote" attitude

2011-06-14T02:12:00.000-07:00

As I write this I am working from Paris and our team timezone spread looks like this:

Rangoria, New Zealand: UTC (+12)
Bucharest, Romania: UTC (+3)
Istanbul, Turkey: UTC (+3)
Paris, France: UTC (+2) <--- ME!
Ottawa, ON: UTC (-4)
Toronto, ON: UTC (-4)
Philadelphia, PN: UTC (-4)
Clifton Park, NY: UTC (-4)
Chicago, IL: UTC (-5)
San Francisco, CA: UTC (-7)
Mountain View, CA: UTC (-7)

I'm going to go out on a limb here and say this: Release Engineering does a good job of working remotely with each other. We are 15-16 people (with a few more contractors/fte on the way) and it doesn't matter where you live for you to work with us. Here we are in our meeting yesterday:

Releng Weekly Meeting - June 2011

Quite the impressive Brady Bunch layout, right?

Here's what we do that I think works well for working remotely:

* We meet once per week as a whole group on Mondays. This starts the week off with a status update on our major projects and also a chance for individuals to speak up about anything they're working on that they'd like people to be aware of.

* We are always having conversations in IRC amongst ourselves and with others in several channels. We use #mozbuild as a backchannel for our inter-team discussion, #build for access to a larger group of fellow Mozillians (like philor, Kairo, and ted for example, who often need to liaise with us), #developers is also a place we frequent and then there are some IT/mobile/QA/release-specific channels we hang out in as needed. I think this helps us have a presence in many areas of engineering/dev/IT and even with some of the non-technical teams at Mozilla where inter-team communication needs to happen. It keeps us in the loop on what various teams are up to and also provides the IRC equivalent of being able to overhear water-cooler chat and participate as well.

* We keep wiki pages for most everything. From "how-to" pages for our own release process, automation details, and project planning all the way to pages for outside-releng folks like the Try Syntax. While I find wikis frustrating the minute the information is out of date, the fact that I can update them and find them in my awesomebar quickly when I need them is very valuable to me.

* We email our group with important notices and changes to how things are done. There are not often times when someone will say "Oh I didn't know about that" and the response is "It came up in the hallway when I was talking with so-and-so". More often than not, the person driving a particular upgrade or change to current practices will send out an email to the group with details of : a) what the change is b) what it means going forward c) how the message has been disseminated to a wider audience (if needed) and finally d) where the wiki pages (and bugs, if needed for reference) can be found. This allows any of us to find the information N time units later when the change actually comes up in your daily work and you're wondering "What was I supposed to do when trying to use the new X again?"

* We all meet up face to face approximately once per quarter. Twice a year for Releng work weeks and twice a year at Mozilla all-hands/summit gatherings. We take these as opportunities to discuss larger topics with lots of brainstorming, whiteboard scribbling, and animated opinion-sharing. Notes from meetings like this turn into wiki pages (often during the meeting itself) and those can become specs for projects/bugs to carry the work that needs doing to the next level.

I think that gives a good idea of our team practices. Now here are some thoughts I've been having about lately with regards to working remotely in Mozilla as a whole. It helps that I'm currently working in Paris right now and am pretty much completely opposite of the PDT work day but some of this was on my mind even when I was in SF.

I think Mozilla has an amazing opportunity to set trends in how to work with distributed teams. We already have people in every time zone! Even with the incredible advancements we've made with our use of video/audio/irc tools (airmozilla/vidyo), there are some ways in which MV is still the eye of Mordor for the company.

I would like to see us shake that up so I think we should try:

* Not having meetings in large groups in MV (except at all-hands). Instead, put small groups of people in various rooms around the building so that "we are all remote" is a reality for everyone so that the clarity of the communication channels are taken seriously. This means we all become just as invested in the quality of audio/video feeds, using tools like Etherpad for public collaboration, and advocating best practices for the speakers/presenters as those who are not in MV. I bet we'd see an increase in contributions to new tools & meeting practices if we were all experiencing meetings remotely on a regular basis.

* Rotate the hosting of the Monday meeting so that over a series of Mondays it would be run from various remote Mozilla offices and this would mean that it moves in time (which could be scheduled in advance) but it also means that all offices get a chance to feel special and be the center of attention. We'll have an opportunity to get to know our co-workers from other offices better as they present the meeting and I even imagine some friendly competition could develop for who can run the most energetic and engaging meeting.

I'm really interested in trying that second one. The most MV-centric thing we do is have our 11am PDT meeting on Mondays be a locked-down time. What if it rotated around each week and just happened somewhere in the 9-5pm spectrum of your timezone? We could create a schedule for it so folks could have lots of notice for scheduling their other Monday things around it. Also, maybe sometimes you might miss one Monday meeting because it's just not at a good time for you but that's something some of our remote workers might say is just par for the course.

I know the idea needs more work, but there's the nugget of it. Curious to know what others think. I'll be continuing to talk this up - maybe we can have a larger discussion at the all-hands in September. Eventually I'd like to see us get to a point where we all think of ourselves as remote since if you look at Mozilla as a whole there does not really need to be a "hub" where one would be "local" compared to everyone else - there's just planning for timezones/meetings and then all the people we work with doing their amazing stuff.

Use Try? Read this.

2011-06-10T02:35:00.000-07:00

Two updates to Try are about to go into effect which enforce asking for what you want using the try syntax and configuring how much email you want to get with your results. Read more below.

Bug 661409 - Now that this has landed, a push to try only generates email about a particular try builder's results if it does not succeed. You can adjust this to be more verbose by adding a -e/--all-emails to your try syntax if you miss getting over all those emails, or you can just shut off the emails completely with a -n/--no-emails in your commit syntax. Note that you must be using the "try: " syntax for these email flags to be picked up which leads quite handily to...

Bug 649402 - Try syntax use is about to be mandatory as soon as this bug is fixed and the hg hook is enabled on the try repo. We're doing this to encourage developers who use try to take an extra moment and request only the resources they absolutely need on their push. This should reduce the test/talos load that has been increasing wait times across all branches during busy periods. One additional psychological change is that the "try: -a" syntax has been removed and in order to ask for a mozilla-central matching run you must be more explicit: "try: -b do -p all -u all -t all". I've updated the docs to reflect this change as well as the TryChooser syntax helper webpage. We're really not trying to make your life harder with this change, approximately 50-60% of pushes to try currently use the try syntax and if you push to try without it you will get a helpful message pointing you to docs and syntax builder. Check with #developers for tips and tricks from the folks who've been using this since the beginning, I know they have many including using the newly-minted Mozilla-Inbound repo where a push will get the complete set of tests/talos if you'd like to let your patch bake for a bit after doing a selective try run.

Update on the Auto/Assisted Landing System

2011-05-23T15:52:00.000-07:00

Almost a week since the post introducing the design attempt for auto/assisted branch landings via Bugzilla and Try and guess what? We re-wrote everything!

The details are in the wiki, bugs have been filed, code is being written. We are working on making this system use a message queue and also see if we can work with mozillapulse to get information on bug changes from Bugzilla.

I'd love to tell you more about it but you can read the wiki and I'm excited to get back to my SchedulerDBPoller component.

Assisted/Automated Landing - Designing the Systems

2011-05-17T16:43:00.000-07:00

Ehsan's blog post wishing for assisted landings on mozilla-central started a lot of people talking about this being a very desirable and useful tool for developers, where they could set a flag in Bugzilla and then be free to do other work until the results of their push were posted back to the bug. As part of enhancing the Tryserver I was already working on a way for users to signify in their try-syntax that the results of the push should go to the bug and these two ideas started to fuse into a dreamworld where someone could attach a patch to Bugzilla and have it be tried and pushed to trunk all with some magical bot automation.

After doing a very short survey of developers and their try usage I have observed that there are two very different stakeholders here and both of them need separate-but-related tools:

Developers	The Bot (automation)
ease of use better reporting (less email anyone?) option to post to bug(s) after a try run has indicated success	queuing of patches culled from a flag in Bugzilla automatically apply to tip of repo push, and report back with results

After soaking in the survey feedback and a first attempt with a whiteboard yesterday, I woke up this morning with some clearer ideas on how to take a first run at creating this system. It involves creating several new tools, one new database, and enhancing our existing buildapi.

New tools for Developers:

Adding more Try syntax options:

include list of the bug(s) that you would like your try results posted to (however many make for a complete run on your push, this can be one linux build or a complete ~186 builder try: -a buildset)
turn off email notifications

Adding functionality to the self-serve api view for a revision (eg: https://build.mozilla.org/buildapi/self-serve/try/rev/de8ea75bc48e) that will better show your results for that push and provide a button which will post the patch(es) to a specified bug
Auto-landing from a bug in Bugzilla using the [autoland-try] whiteboard tag where any attached patches which are not obsolete, and have nothing set for 'r' are applied to the current tip of mozilla-central, pushed to try and those results are returned to a comment in the bug

New tools to Automate landings (bot or script):

Crawl Bugzilla for bugs where [autoland-$branchname] is in the whiteboard and automatically push to tip of named branch, get the results, and return them to a comment in the bug (stripping out the whiteboard tag on completion)

bot will grab all non-obsolete, r+ patches (if $branchname != 'try')
interdependent bugs will not be handled in this first swipe at a working system
pushes will have autoland-$bugnumber as the reason for the build in schedulerdb so that the results can be watched for, aggregated, and reposted to the bug on completion

Watch results coming back for one or two oranges (we can set a threshold) and re-triggers those, watching for the second set of results - to attempt catching intermittent oranges
Backout patches where even with a rebuild on an orange, there still remain orange results
LDAP authentication checking for bugzilla patch author -> hg commit permissions and being able to ensure that only people with the right credentials can trigger automatic landings. This may mean checking the reviewer too before allowing a patch to be applied & pushed.

The next step is to get this design organized into bugs so that we can parcel out the work involved and start testing/completing segments and features as we work towards the whole. We have a RelEng intern this summer, Marc Jessome (Another Canadian in RelEng!), who will be doing a lot of the work between now and the end of August. Stop by anytime to say "Hi" to Marc and to chat with either of us about the project - feedback is always appreciated. I'm happy to say that 52 people filled out the Try Usage survey just from posting it on Yammer. That was super helpful, thank you.

A PyStar Supernova in the Sky

2011-05-04T19:58:00.000-07:00

The first Bay Area PyStar event has come and gone. I'm finally getting a moment to regroup and ponder all the trial and error of being the organizer of this event as well as having time to look at some of the statistics we gathered. Just from an organizing perspective here are a few items I'd like to share about the process.

Things to do differently next time:
* When creating the Eventbrite event, add questions like "What level do you want to learn at?" "Meat or Vegetarian?" "Operating System?" to the registration so there's no need to send out blanket emails to attendees to try and get that information after the sign up.
* Only do one day workshop instead of Friday night installation and Saturday workshop. I think that for many people the setup could have been done in the first hour and the rest of the day been spent learning instead of having a night session that only is needed by a handful of people.
* Have the teachers/assistants already assigned to a particular level of instruction - prepare topics, tutorial materials, and class size ahead of time so that on the day of the workshop there might only be a handful of late arrivals to place and the other attendees will already be set up in the right learning level as requested in the sign up.

Things that really worked this time:
* Eventbrite! They have amazing tools, stats, emailing options, charts, and also a way to see where your sign ups come from which showed us that we got a TON of views from Tweets which apparently was an impressive number (I am told by one of our attendees who is an Eventbrite employee)
* Mozilla! By sponsoring the event - providing the space and food - being able to let people/groups spread out and work in our various conference rooms as well as having lunch on site was very much appreciated by attendees (and of course by me!)
* CodeChix! This peninsula-based group of women coders accounted for 30% of our attendance and also netted some teacher/assistants for the workshop. CodeChix co-sponsored the event and helped get word out as well

Attendance
There was something odd happening with the Eventbrite signups. In a couple of short bursts, a ton of tickets were being snatched up by names that seemed slightly suspicious. Now the event has passed and I've checked in all the attendees as well as accounted for the no-shows (almost all of whom took a moment to send in their regrets so the tickets could be freed up for another person - very sweet!). It looks to me like about 40% of our attendees were fake accounts. Julie (who works at Eventbrite) and I took a look at the numbers and she's kindly offered to look into it further to see if there is indeed something fishy happening. All that aside, we had 47 people! That feels like great attendance to a first workshop, on a Saturday, in Mountain View.

Speaking of Mountain View - we had attendees come from all over Northern California. I love this view of how spread out geographically we all were:

This graph is useful for seeing how my own promotion attempts were successful. The original spike of page views is obviously when I first announce the event link. CodeChix, Baypiggies, and Devchix were the mailing lists I sent emails to with the link. While that got the ball rolling, it was the tweets and emails sent out almost 3 weeks later - a week before the event where the event got lots of attention. It probably helped that PyStar Minneapolis was happening then too so #PyStar got lots of tweets (sorry to the person who's twitter nick is @pystar).

Can I just say that I am so thrilled with the amount of people who volunteered to teach/assist? Seriously. Amazing. I love that there are people out there who really enjoy getting newbies involved, who can share their skills, and who will give their time to events that grow community.

Finally, here's a breakdown of where we got ticket "sales" from via Eventbrite. This is another reason they rock - they help you promote your event! As you can see here the Twitter share link definitely got us the most eyes even though direct invitation resulted in more actual signups. For next time I would send the link to a few more mailing lists like SF Python Meetup, Systers, and also next time we'll be able to invite the folks who came to the first one as well as those who couldn't make it.

In follow-up posts I will post and analyze some of the survey results of both the PyStar Bay Area and the PyStar Minneapolis. I need to go learn how to create charts from Google doc spreadsheets. Also we need to figure out how to set up our site and materials to be easily updated and adjusted by a distributed team without having to break off into separate sites. Finally, the curriculum needs an overhaul. We kept an etherpad during the event to track issues so that I can go through post-workshop and take advantage of all the feedback to improve our offerings.

What's Next?
The next PyStar I plan to organize will be in late July or early August and I'd like to do that one in SF. Following that I'm going to plan one in Toronto for mid-to-late October. What we did this past Saturday is only the beginning. I'll be working with all the folks in the pystar group to get this program shaped up into a much more modular system for learning Python and Django in stages (badges) and also will be setting up sub-groups for things like hack nights, code-masters (think toastmasters but writing code in front of people), and I have this idea of taking the PyStar lessons into women's prisons as a way to get marketable skills into the hands of people who need them badly.

Anyway, first we'll get more material prepared and digest/incorporate all the excellent feedback. Then we can take over the world :)

I hope I'll see you at future events. Thanks to everyone who helped make this a great day!

Captain Destructo Breaks Everything

2011-04-22T17:04:00.000-07:00

Alternate title ideas: "It's not all s/Tryserver/Try" or "What I should have done, and didn't"

I bet you get the point by now. Today I caused a fairly lengthy, unnecessary downtime on Try. Now that I'm writing this, things are under control again and there's a few small niggly bits left but nothing that will keep me up at night.

It all started with a bug about graphserver posts from tests not getting through because they were looking for MozillaTry (the tinderbox name for Try) but instead the graphserver only knew about Tryserver (the branch name for Try) and nothing was using Try (except the repo for Try) which is what it ought to have been doing in the first place.

Now that I've been adding a lot of project branches in a short amount of time, certain things have become more streamlined and so I felt that the best option was to go through and rename Tryserver/MozillaTry to Try everywhere so that from the repo going forward, everything was the same. This has been working extremely well for our project branches and helps make setup a snap.

Here's where it gets all broken. I approached this bug with a quick swipe at this problem was superficial and ended up causing some preventable burning. I shall now list for you (and future me) what I did and what I should have done:

Did:

hg rename on configs for desktop
branch configs for s/tryserver/try
updated graphserver branch name to Try
a quick downtime window from 10am - 12pm in order to prevent builds from getting split into two different upload dirs

Should have done:

hg rename on configs for mobile
grep of buildbotcustom for "tryserver" as we have special casing for it in several files
log uploader and post_upload scripts to make sure everything about the try build was going to the right place
updated the dir permissions on ftp for the new upload location and ensure that the archive is on nfs mount
edited cronjobs on staging to catch the new try builds
updated graphserver machines table for each try platform's builder name
more notice for downtime, with a 4 hour window that would have allowed a test push to make sure everything was wired up correctly
updated the treeclosure hook to include the new tinderbox page

Some of the things I should have done didn't have an impact on the burning/try closure but it's fair to say that if I had done a staging round of all my plans first I would have caught more of the obvious things that I missed. I would have then planned the downtime better and been prepared to ensure the disturbance would have been minimal since this was, after all, a really low priority bug.

Aki told me that he had a manager who said "you don't learn til you break something". Well I broke everything try-related today and here's hoping that I have learned something because the stress of this whole day is not something I want to experience often. It's that feeling you get when you realize you've started something that you can't back out of and there's no way to go but forward, even though everything in front of you now appears hopeless and messy.

So here's some lessons to take away:

Staging is not to be underestimated even for just renaming things that are already working
Taking the time to search with grep/mxr and find the terms you are replacing before starting the upgrade in production will help find wiring you might have overlooked in your preparations
Prepare more thoroughly and have a clear idea of the env. you started in and what it will take to have that env. back when you're done. Leaving dangly bits is not ideal.

Happy Friday.
(and many thanks to Aki)

Hey BBC would you like to know how releasing software works?

2011-03-23T10:29:00.000-07:00

Dear BBC,

Today on the front page of your technology section you said that downloads for Firefox 4 have been lower than they were for Firefox 3 and that:

The lower figure may be explained by the widespread availability of pre-release versions of Firefox 4 in the months ahead of its launch.

First of all, you forgot that we've had 3.5 and 3.6 between those two and so we now have users spread out a bit across versions. Second, here's an overview of how we're organizing the release of Firefox 4:

We put out the RC and picked up users from outside of our usual beta testing pool in order to give our final candidate some solid tire kicking
Firefox 4 went live but our users on 3.5 and 3.6 are not offered the update automatically yet, they must "Check for updates" in order to be asked if they want to upgrade to Firefox 4
Once we have more coverage of the new release for a couple of weeks and are even more confident that we've got an amazing browser out there we will turn on the Major Update notification which will offer our 400+ million users the chance to come on up and experience the next level of the web

According to W3C school's stats(which are measured by visits to their site) the browser distribution of their visitors looks like this:

2011	Internet Explorer	Firefox	Chrome	Safari	Opera
February	26.5 %	42.4%	24.1%	4.1%	2.5%
January	26.6 %	42.8%	23.8%	4.0%	2.5%

2011	Total	FF 4.0	FF 3.6	FF 3.5	FF 3.0	Other
February	42.4 %	1.9 %	35.8 %	2.9 %	1.5 %	0.3 %
January	42.8 %	1.5 %	36.1 %	3.1 %	1.7 %	0.4 %

What this says to me is that our more than 8 million downloads since yesterday morning PDT only shows us how many people are paying attention to the fact that Firefox 4 has launched and is available for download. It's not representative of our 400+ million active daily user base (the people who just use the browser but perhaps don't read your blog or mine). These people will soon learn about Firefox 4 through their browser's update notification window. We'll be seeing a spike in downloads in a couple of weeks and I hope you'll report on that.

Bay Area Video Coalition - Teaching Open Video Part 1

2011-02-23T07:43:00.000-08:00

Last night was the first meeting of The Factory and Mozilla. The partnership if a result of work between Mozilla Drumbeat and Web Made Movies. Ben Moskowitz and Brett Gaylor invited myself and Atul Varma to what is to be the first of three sessions helping teens learn about the budding open web technologies that can be integrated with video.

About 13 kids streamed into a computer lab at 4:30pm and we began with some introductions detailing who we were and why this stuff is interesting. The group were very engaged and eager to dive right in with whatever we had for them. So we started with Atul's Web X-Ray Goggles so the students could see what exactly the web was made of. The idea was to just grab parts of whatever websites you liked and change them right on the page so you could see how easy it is to "hack" the web. Some of the teens went even further and started building their own pages with Etherpad by grabbing snippets of code from sites. About 4 kids said they had done a View Source on a page prior to this class and 30 minutes into this workshop they were all doing it like pros.

Once they had a chance to remix a web page we moved on to the next exercise which was to select 4 popular sites of their choosing (Etherpad democracy!) and those sites were printed out on paper, the teens were split into 3 teams, and each team did paper prototyping of a new site using elements from the originals. I was very impressed with how the students took the idea and ran with it. Each team worked fast (they had 7 minutes) and no one was hanging back keeping their opinions to themselves. The teams produced 3 new site mock-ups that each had a very simple look, with a video as the prominent element on the page but they also took time to add site navigation and social media integration by putting Facebook and Twitter in the sidebar or footer.

In the last 30 minutes of our time Brett and Ben demonstrated Butter also explaining how very "caveman" the technology is right now. With only a glance at the interface and a basic explanation of how it's wired up the teens jumped right in with suggestions and ideas about what they would like to be able to do:
* Hide popcorn elements when nothing is showing in them
* Be able to zoom in or out on Google maps while the video is playing
* Click on something in the video (example: coffee mug) and have it trigger an event

Ben made a really great point about how it's also important to look at something like Butter and think "How can you go beyond the interface?". How do you make your story more interesting from the beginning knowing you can use this tool throughout instead of just tacking on events and additional information to a completed video that is done in a standard format?

We'll be working with them again tonight, with chunks of a film they made last summer called "The List". More updates and more potential bugs and feature requests coming soon.

Also, if anyone is planning to teach a class like this you might want a few things in a "kit":
* Portable printer (and paper)
* Scissors
* Tape or glue
* Handout with links to the tools/sites

Just to save some time :)

Volunteers needed for upcoming HTML5/Open Video tutorial

2011-02-07T11:31:00.000-08:00

I'm hoping if you're reading this that you might be interested in volunteering this coming Saturday to help 12-16 year old girls at the upcoming Dare 2B Digital conference learn about HTML5 and open video. There's more information and background on what's happening on this wiki page.

Two kinds of volunteers needed:

1. Someone who is in the Bay Area and available this coming Saturday from 9-3:30pm to be on-site with us in Mountain View at the Computer History Museum and will work hands-on with the girls to demo Miro Converter, Universal Subtitles, and a little bit of Popcorn.js.

2. Anyone, anywhere, who can do translations to any language and who is available on Saturday anywhere in the 10:15am-3pm PST window to do some 'live' subtitling and show the workshop participants how amazing the universal subtitles project is.

Please get in touch if you are interested/available. Or sign up on the wiki.

Thanks in advance! I will be posting any demos, workshop materials, and an update post-event on how it went.

Automated Try Results Posted to Bugzilla - A request for input on what the comment should contain

2011-02-03T16:59:00.000-08:00

Lately I've been working on a a script which can check your try syntax for a bug number and a setting asking for --post-to-bugzilla. If you've provided both, your try server results can be posted directly to the bug. This is just part of a larger project to have patches submitted on a bug get automatically tried out, results posted, and at some point down the road they could even be pushed to trunk after a successful try run so "look Ma! No hands".

Today I have a script running in staging, polling for completed try runs and doing dry runs of posting to bugzilla in log format only so I can keep an eye out for unusual output. Already this has shown me a couple of bugs to sort out, and I anticipate having them ironed out very soon. However, before this lands I would like to get some feedback/ideas/suggestions on what the output to the bug should look like and there could be a couple of options even, with a setting in your try syntax.

In my early testing I posted to our landfill bugzilla and here's what a couple of results looked like:

Lots of success looked like too much info

So I took it out, and only printed the warnings and failures


Which works on small runs

From what you see here, I'm sure you can imagine what it would look like if 145 builds all had warnings/failure combos.

So - what do you want to know in the bug? Let's keep it simple, ok? We can add more later and it's important not to create a bug-spammer here that folks will clamor to turn off soon after it goes live.

Off the top of my head, and after talking with Catlee today about it, I think it should look like this:

Try run for $revision with the following comment:
$try_syntax line
S:# W:# F:# (results total) builds complete from N total requests
S:# W:# F:# (results total) tests complete from N total requests
For more information please see http://tbpl.mozilla.org/?tree=MozillaTry&rev=$revision

This gets you a quick glance at total builds/total requests so you can see that everything is accounted for, and where things might have gone wrong in builds vs. tests but doesn't list the failed/warning builder names so you have to follow the link to get them. Maybe there would be interest in printing what your try syntax request triggered but I'm not sure that's useful in the bug reporting even though it's requested for when a developer is pushing to try. What do you think?

It would be possible to break down the results further by platform instead of or as well as build/test. Any ideas on how to get that much information across without making a bug comment too verbose? All input appreciated and considered, I'll be trying to land this in the next couple of weeks so comment here, ping me in IRC (lsblakk), or comment in the bug.

Try Server Road Map - Q1 2010

2011-01-13T16:47:00.000-08:00

The other day this post by Google with slides detailing their Chrome release cycle speed up was going around and it mentioned how try and CI were key to their success. It got me thinking that it's time for another update about the upcoming improvements our try server automation. Most of my Q1 work will be on the try server, with some time on Fennec Beta releases, and a bit of time also working on making it much easier to spin up disposable project branches.

The road map image above shows how there are three areas of focus and here they are now in a more detailed list with bug number attached:

Improving Current Automation

Bug 617321 tracks adding two new try buildbot master instances to our other buildbot-masters in the Santa Clara colo. This gives us flexibility to have rolling downtimes on try (as we already have on the other masters) where we can update things behind the scenes and it also helps by adding redundancy to the try automation in case of a colo outage.
Bug 580346 is almost done and it adds xserves to try which gives us some faster macosx building power, that along with about 40 ix builders for win/linux builds will crank out more try builds faster.
Bug 594236 is key to getting TryChooser syntax turned on as a default. With an interactive hg prompt on push to try, you should be able to select what you want and/or have your syntax validated. Khuey started something to do this and if anyone is up for taking it to the next level before I can get around to it, please please do.

Add Features

Bug 430942 is what I am actively working on today and the rest of this week, I'm about to have a second draft ready for review and I expect you can watch for this to land in the next few weeks. With or without try syntax, you will be able to specify the bug number in your push comment and have the results of your try run posted as a comment on that bug. Hopefully this will help out development in letting people know where something is at if you are away when the results come in. It's also part of our old bug (pre-2010) smack down goal so finishing it will be one more step toward that carried-over goal being met.
Bug 615705 is tracking a few more tweaks to the try syntax that will give users more flexibility in the syntax and choice about what to build.
Bug 421895 is another old bug and Chris Atlee is close to getting it up for poking at. It provides a way for users to cancel their own try build requests without having to ping RelEng.
Bug 621681 addresses having better threading/headers since the current headers only help with threading for some clients. When I first wrote it I was testing with Thunderbird, where it works as intended but apparently Gmail and other clients need some help.

Future

Looking to the future right now we have bug 625464 which talks about setting up something to scan bugzilla for a flag on attachments that will trigger an automatic try run with that attachment and either tip of trunk or perhaps a user repo, which would require the other main future bug, 625463. With the ability to poll and run try on hg.m.o user repos we can have project branches (temporary branches that are loaned out to devs or teams to work on a particular project) toggle a setting that would have their pushes to the repo get run through try instead of the main mozilla-central automation. This could be handy when you want to limit the machines you are building/testing on with the TryChooser syntax.

I hope anyone reading this will find the upcoming try work as exciting as I do. Reading the Google slides, I couldn't help but sit up straighter at the mention of try being one of the reasons they were able to speed up their release cycle. I'm hoping we can get there too and that our try server will be more robust and ready to handle our soon-to-be-speedier release process too.

Please use TryChooser

2010-12-03T16:02:00.000-08:00

Recently there were some improvements to the trychooser and the landing of those changes led to a couple of bugs[1][2] being discovered and quickly fixed. It is thanks to those who are regularly using the trychooser that we are able to find bugs quickly and also continue to improve the tryserver.

Right now there are over 350 backed up test/talos requests for the tryserver and when I checked our report for trychooser usage it shows that the average number of users pushing with a try syntax has fallen below 50% where it used to be closer to 60%.

I encourage you to please use the trychooser syntax as much as possible. If you do not need every single try result for your patch, do not just push to try and use up all the resources needlessly. Take a moment to insert some try syntax into your commit message.

See https://wiki.mozilla.org/Build:TryChooser for details and http://people.mozilla.org/~lsblakk/trychooser/ for a simple try syntax builder.

Thanks in advance.

Impending TryChooser change you should know about

2010-11-23T10:26:00.000-08:00

On Monday November 29th there will be a bug fix landed on the TryChooser that will change how you use it:

Removed: -m,--mobile argument for requesting mobile platforms.
ALL platform selections will use the -p,--platform argument (as requested by developers)

This change is part of a larger back-end change which will improve try by making new platforms or test/talos suites added to mozilla-central dynamically available to tryserver. This also means the Mozmill test suite will be enabled for try when this patch lands on Monday.

I will update the TryChooser wiki and the trychooser syntax web helper. If anyone is using other tools to help with using trychooser, please update accordingly.

What will be useful once this lands is a way to give a user-friendly, and dynamically-generated, list of what platforms/talos/test options are available to try. Any ideas?

Playing with Audio Visualization

2010-11-21T18:33:00.000-08:00

After Dave's blog post put out the call for 50 visualizations using the new Audio API in Firefox 4 I decided it was time to stop thinking about learning processing and its .js version and time to start doing things with it. So I took a few hours on Saturday to poke at the code that Dave put up and after a lot of shots in the dark I came up with this but I was disappointed in how simple my results seemed after so much time spent. Today I went back to the book and learned a few more processing tricks which inspired me to hack on the audio visualizations again.

For my second attempt I really tried to understand more what the javascript functions were doing, how the actual audio data was being generated, and in what form it was being passed to the processing draw() function. I found a couple of pages on the Mozilla Developer Network that helped me understand the API a bit better. On my second attempt to visualize audio data I could really make it do what I wanted. The next round of learning I would like to do in this area is how to work with user interaction better. To respond with audio and video to user movements and key press events.

Second attempt at audio visualization

I'm really glad Dave put out the challenge, and that I made the time to teach myself something new.

Get more green with TryChooser!

2010-09-23T15:24:00.000-07:00

Our fabulous intern Anamaria Stoica has been working hard the past few months helping to pull data from buildbot's scheduler DB that allows us to learn all sorts of new information about how the release infrastructure is being used, and how it could be optimized. Among other things, she's generated an end-to-end report so we can see at a glance how long the complete build cycle for any push to mozilla-central took. Now, on top of that, she's added a way to see who is using the awesome new TryChooser syntax.

In the coming weeks I'll be emailing folks who are using try but not the TryChooser syntax and encouraging them to give it a shot. In my stock email letter I will say something along the lines of:

When you use the TryChooser syntax to make requests for only what you need, you allow precious CPU cycle time to be used elsewhere by those who need it.

This is helpful for two reasons:

1. Those testing fixes to blockers of the next beta/rc/release can get try results quickly and get their code landed on trunk
2. Overall wait times for try results are reduced

What's interesting in looking at the reports though is that more of the pushes to try that use TryChooser are green. Here's a snapshot of those who are using TryChooser:

The number of build requests is an aggregate of all the builders triggered so all build types, platforms, test and talos builders run on that push.

Now for comparison, here's a snapshot of folks not using the TryChooser:

I have no way of knowing from this snapshot which of those pushes were intentionally not using the TryChooser and perhaps trying to get all 176 possible builds. Some of them may be using custom mozconfigs as a way to kill off unwanted builds (p.s. you get burning with that).

If you haven't tried it yet, please give TryChooser a go and see if getting a smaller set of builds (ie: what you need, not just the default) and perhaps getting more green on you try pushes gives you a boost.

Try What You Want

2010-08-30T08:46:00.000-07:00

Last week bug 473184 saw some progress when we landed the try_parser which allows devs to put in their hg commit comments what try builds they would like on a particular push. This is a first step towards a two-part goal: 1) giving devs more self-serve tools for try and 2) lowering try wait-times.

Now you can try what you want instead of just pushing and getting the default set (currently default is all opt & debug builds, all unittests, no talos). You will get less emails and only the results you're looking for and everyone wins when you only ask for what you need since it will make more machines available for other builds.

To use this fabulous new TryChooser in your commit message (soon to be available in a committed config file as well) you do the following:

You've got your patch ready to try
Set the commit comment to what you would like to run on try
In this case, let's say you want a linux opt build, no mobile, no unittests, and only talos tp4 results

hg commit -m "Patch to fix a pixel placement try: --build o --p linux --m none --u none --t tp4"

Push to try
What happens? 'Linux tryserver build' builder is created and runs your opt linux build. If the build completes successfully then the talos tests are triggered and once the test-master gets the notice of the change it will read the hg commit comments and only kick off the one builder for tp4 'Rev3 Fedora 12 tryserver talos tp4'
You get a total of 3 emails and have used very little compute time - you rock!

please go to http://wiki.mozilla.org/Build:TryChooser for complete info and options as well as some answers to questions you make have and take the TryChooser out for a spin.

No really, it is easy to test your Python code

2010-08-11T21:02:00.000-07:00

This week is all about getting custom selection for tryserver submissions (bug 473184) ready for public consumption. The plan is to have a custom scheduler that calls a function which parses a commit comment (and later, the option of an .info file instead) and then only schedules the builders which have been requested. This will be awesome for our build machine resources since it will mean that if you don't want a particular platform/test/talos suite it will never even get triggered.

In today's work on the parser I wrote python unittests for the first time. I remember doing some unittests in school but that was way back when we used Java. The python unittest module is really awesome and easy to get up and running. Once I had my tests passing I then ran the coverage tool on both the test suite and the parser to see how I did in covering all my bases:

This tool is Frickin' Awesome! I can see at a glance that I need another test to see what happens if someone wants a subset of mobile build platforms.

Tomorrow I'll be getting my coverage even higher and then starting to stage this scheduler and see how it handles our large amount of possible builders to choose from. Keep your ears open - selecting your try build configuration is right around the corner.

Disposable Project Branches - aka Twigs

2010-07-02T10:36:00.000-07:00

Hey Firefox Developers - Want a place that's kind of like Tryserver but all yours for a short period of time? Release Engineering now has 3 'disposable' branches set up in our build automation that you can use for testing a mozilla-central clone or a clone of your own m-c-based repo. I call them Twigs because they are for temporary, small projects.

The Twigs (maple, cedar, and birch) each have their own tinderbox pages, run the full suite of unittests, and give you talos results by default. Like Tryserver, they also allow you to override the mozconfig with a custom mozconfig so that you could kill off platforms you are not interested in results for. Doing so will give you a cleaner tree and will allow our build and test resources to be used where they are most needed.

To use a branch simply grab a twig from the bookings page, file an IT bug to have your repo cloned over to your selected twig repo and then start pushing to the twig's repo to see your builds start. At the moment one small glitch is that the first push doesn't get you any builds (bug 562026) - so you need to push twice on a fresh repo clone and the second set will get picked up by the hg poller.

I look forward to hearing if this setup helps you out.

Tryserver 2.0 - Fine tuning and learning the hard way

2010-05-26T18:21:00.000-07:00

I really thought the try server as a branch was ready to roll out when I did it. Seriously. It took a couple of months to get it to the point where I felt it was ready for the public. So I pushed it live last week - detail here.

Immediately a few issues came up that ruined my vision of a smooth transition:

* Emails were going to the changeset author and not the person pushing the change. Turns out I had changed this default behaviour and didn't know that there was a reason why we used to grab the email address from buildbot sendchange info instead of from hg author.

* Packaged unittests were being merged. This was quite a big deal for the first full day of being live because people were getting all kinds of strange emails about results that weren't theirs. The reason this wasn't caught in testing was that there is no 1:1 mapping between a build and its packaged unit test results (except when you go look at tbpl). We should get one set up for MozillaTest where our staging results go in order to make sure something like this doesn't happen again.

* The URL included in an email regarding test results didn't contain a changeset - only the build and leak test builders were setting the information needed to grab this for the emails. Thanks to dbaron for catching that quickly and bringing it to my attention.

* The try-mac slaves got delayed on their way to the new tryserver's slave pool because of some glitches with puppet and so a huge backlog formed for OS X builds and people thought they didn't even exist. Because the backlog got so large (80+ full-length builds) I opted to restart the master, wiping those out of the queue in order to get the tryserver back to decent turnaround for all platforms.

I'm still ironing out some issues, tweaking the email results, and I've temporarily disabled the web interface as I work on getting it to use hg push so that all the inputs to tryserver arrive in the same way. I'm also trying to get more slaves to add to the builder pool since the packaged unittest builders are builder hogs.

Let me know if I've missed anything. It's my goal this quarter to make tryserver as helpful as possible in keeping mozilla-central green.

Tryserver 2.0 is Live

2010-05-20T12:35:00.000-07:00

This morning the quietly running in the background try-as-branch became the new try server and the try you've known for a few years now is about to be turned off.

You will now be getting opt and debug builds, as well as packaged unit tests for all current try platforms (64 bit are on the way). If you would like to read more about how the new try server works you can look to https://wiki.mozilla.org/Build:TryServerAsBranch where how to add a custom mozconfig and where to file bugs on tryserver issues is covered, as well as other info.

Now that this new tryserver is up and running, the remaining try slaves from the current tryserver are quickly being moved over and we will be back to full speed on try builds in the next few days.

Upcoming work on try will see 64 bit builds, sending the unit tests over to talos user desktop platforms for testing (just like mozilla-central has now) and also allowing you to select with more granularity which try builds/platforms you would like your patch to be done on. Also there will be a try branch for 1.9.2-based builds coming soon - tracking bug for that is 563822

If you have questions, come find me (lsblakk) in #build or #developers or you can file bugs on tryserver.

Speaking of bugs, this new try-as-branch will allow RelEng to close the following pretty much right away:

I really appreciate everyone's patience during this transition, I hope that the new tryserver gets us better results and helps keep trunk building green.

I'm much less interesting now...thanks to Facebook.

2010-04-22T06:57:00.000-07:00

Two days ago, my Facebook page threw me a very large popup explaining that now all my interests (music, books, etc) were to become links to Pages and that I could either accept turning all my interests into links to their respective pages or go and customize. I took a second to customize because I'm not necessarily interested in being attached to the Page for all my favourite things. Once I finished this step my profile's Info section was updated for me and now it looks like I'm only interested in 1 movie, 6 authors, and about 20 bands. This is so far from representing my actual interests. It's also very different than what I had originally taken time to enter into my profile back when I opened my Facebook account.

So here's what I've gathered so far about what Facebook is doing to me:

* My self-described interests were no longer valid and were removed - looks like you can still add things that don't have pages, but they'd better not have the same name as an existing page or you'll link to the wrong thing. I don't like that I have to go and re-enter my interests.
* If there's no existing page for something you list as an item a Community Page is created that states

"Our goal is to make this Community Page the best collection of shared knowledge on this topic. If you have a passion for {interest}, sign up and we'll let you know when we're ready for your help. You can also get us started by suggesting a relevant Wikipedia article or the Official Site."

* This new Community Page automatically aggregates any mention of the item - so a book I mention in my profile is cross-posted to the news feed for that page without my knowledge or consent even if that item is not in my interests!

From cnet:

"This is a really significant step for Facebook. For years we've been saying that FB is an open platform, but now for the first time, the likes and interests of my Facebook profile link to places that are not Facebook.com...My identity is not just definied by things on Facebook, it's defined by things all over the Web."

I don't think that this is putting your "open platform" where your mouth is. They are doing this with major partners like Amazon. Partners who bought into having "like" buttons on their sites. Facebook is streaming content from user profiles into pages for movies, books, bands, and television shows to promote those items without the user being aware, or interested in doing this kind of free advertising. Until now people signed up to be reviewers on sites like Amazon but now you're going to be one whether you want to be or not. I don't read "open" into this thinking. Open doesn't mean a free-for-all of data being tossed around the web without permission. Open should allow you to customize, and to look behind the curtain to make sure you know where information is going.

Here's my attempt at an analogy:

I buy a t-shirt for a band I like. It's got a nice design and that's why I chose it. I wear this shirt every day. Suddenly one day, I'm talking to a friend and I mention a band I saw the other night and my shirt design changes to that band's logo. I'm not controlling this change, I may not even notice that it changed but from now on I'm a walking billboard for whatever music I talk about. The bands get attention, the record companies get money, what do I gain from having a shirt that changes all the time and no longer just displays the design I selected it for?

Having just read danah boyd's SXSW talk yesterday about privacy and publicity I feel very sensitive to this new collapse of worlds that has been thrust upon me. She summarized the Facebook's news feed update as follows:

Consider the Facebook News Feed fiasco from a few years ago. What Facebook did was aggregate content in ways that made it more visible to users who could already access it. In essence, it made quasi-public data more public. Many users flipped. Why? There's a big difference from knowing that I just entered an "it's complicated" relationship by looking at my profile and getting that information in a stream of updates. In effect, Facebook publicized publicly available content, making it more public. Time and time again, this is what technology companies do. Can users adjust? Yes, and they do. But their behavior changes or they find themselves in a lot of trouble in ways that they weren't expecting.

So it looks like it's time to adjust again. I've certainly learned that changing my relationship status in Facebook gets broadcast to everyone instead of just hanging out in my profile for the curious friends I have to discover. Now I must learn that if I mention a band, book, tv show, or movie I'd better really like it.

My name is Lukas, I live on the Second Floor.

2010-04-13T10:25:00.000-07:00

Several times in the past few weeks I've been up on the third floor of our Mozilla office here in Mountain View and folks have asked me questions along the lines of "How long are you in town for?" or "You're moving here soon, right?". Well, here's the thing - I've been living in California since mid-October and working in our Mountain View office on the second floor this whole time.

When I was an intern in the summer of 2008 and we were at our previous offices I hardly saw folks from Building S because I was in Building K. Now I rarely see people from the third floor, and even the folks who sit in the other half of the second floor are strangers to me.

I could blame office layout, but the bricks aren't going to move any time soon. So it's up to me to do something about the lack of social interaction that my desk location provides me with.

This is the first job I've had with this many employees so I'm going to experiment a bit with ways to help me connect with the people I work with every day. Dria suggested going to sit up in Ten Forward (our huge gathering space) once a day. She said it would guarantee that I would run in to everyone eventually. I'll certainly try this, though not every day. Other thoughts I've had for ways to interact more with my co-workers include starting a lunch time Settlers of Catan game, making an effort to go for lunch with people who are not on my team, and doing a lap around the office (both floors) with the dog to give us both a stretch and to chat with whoever isn't too busy.

If you're ever on the second floor, behind the conference room, in the little pocket where Release Engineering and IT are situated - say Hi!

I got tagged - Open Source Contributors blog meme

2010-03-31T14:10:00.000-07:00

The rules:

Copy/paste these rules and questions into a blog post, answer the questions, then tag some other people (however many you like) and encourage them to do the same.
Include a link to the original post.
You don’t have to be tagged to take part — if you see this post and want to play, just dive on in. Simple!

The questions (and my answers):

How (and when) did you originally get involved with an open source project? Which projects have you contributed to?

I originally got involved with open source projects as a student in Toronto doing a 4 year bachelor's degree in Software Development. I was a mature student, looking for a 'career' path, and soaking up everything I could about this world of creating software. My experience prior to going to school was mostly teaching myself web design and some text games programmed in Basic when I was in high school. In the first year of the four year degree program I learned about open vs. closed source and was instantly drawn to the politics and principles of FL/OSS. I had been using Firefox already for a few years (cause IE 5 for Mac sucked), but didn't make the connection with open source until a ton of Mikes (and a Johnath) came to my school to talk to us about Mozilla right before FF 2.0 came out. They were so engaging, humourous, and full of excitement for the potential of this project that thousands of people worked on, and I knew right away that the Mozilla project was the one for me to get involved with. It combined technical superiority with an incredibly friendly user/contributor community and most importantly it's a product that helps a lot of people be safe on the web. In my third year I took two classes with Dave Humphrey that got me working on actual code for Mozilla. My contributions led to being offered an internship with the Build team in the summer of 2008 which resulted in a part-time contract while I did my last year of school and a job offer to go to full time with Mozilla's Build and Release team in May of 2009 when I graduated. Since getting involved in open source, I have contributed to the Miro and Drupal projects in small ways and I look forward to getting more involved as my experience and abilities improve.

Why did you choose to contribute to an open source project?

I was drawn to contribute because I could - even as a student with relatively little programming experience - I helped with documentation, support, anything I was able to do. It's great to be able to participate in something while learning at the same time and open source project models are great for this since they are largely volunteer-based.

What keeps me contributing is knowing that my contributions help people, and because open source software is something I believe in. My leftist, anti-capitalist, activist sensibilities fit well with a not-for-profit enterprise, a free software project. Oh, and I like sharing.

If you were to pick one or two people who have had a major influence on your involvement with open source, who would those people be? Why?

Dave Humphrey none of this would have happened like it did without Dave. If there is a church of open source, Dave is the preacher. He pushes his students to get involved and to jump in feet first. No coddling, Dave has high expectations and expects you to get really involved with open source, not just do the minimal amount of assigned work. In fact, his class is structured in such a way that you cannot do a mediocre job. In or out - it's your choice. He's amazing at matching up students with projects, and his enthusiasm for his own learning is contagious.

Ted Mielczarek is an incredible mentor and worked with me on my Mozilla student project of adding source server support to the Windows symbol server. He is very available in the student IRC channel outside of regular work hours, is willing to answer newbie questions, is a great communicator and patch reviewer who would help test my patches when that was needed, and he made me feel very welcome as a new contributor to Mozilla.

Angie Byron is a Drupal contributor I heard speak at the first Ontario Linux Conference that I attended. She spoke about being a woman in open source which interests me as a topic but more importantly she did what I hope to do - she got involved with Drupal through the Google Summer of Code program, had a great experience, and now she encourages other new folks to participate. Her testimonials are all over the GSoC wikis, and her positive reinforcement is really needed in an environment that can be intimidating or have barriers to entry for some folks.

How have you personally benefited from being involved with open source projects?

See question #1, I work full time for a project that I love and am excited about. I work with an extremely talented global community of people to make the web the best it can be. I love the web and I get to work on it every day. Also, I'm learning a lot about the process of creating and releasing software in the open and because we're open, I can share my learning with everyone. This isn't something I take for granted, from what I hear it's not like that everywhere :).

What advice and/or encouragement would you give to someone who is considering getting involved with an open source project?

Find a project that excites you because then you will honestly want to work on it, in any capacity. Don't be afraid to take on something you don't know how to do (yet). Work on whatever it is you have taken on regularly. I can't stress this enough. Picking up a bug, taking months to write some intensive fix, and then resurfacing looking for feedback will not likely not ingratiate you to the community. Blog, get on IRC, submit patches of your work in progress, ask questions - be a persistent contributor. Even if you fail at reaching your desired solution, by failing publicly you will have shown your skills in sticking to the problem, and will have made connections with the community in the process.

Tagging

Improving 1:1s

2010-03-23T14:36:00.000-07:00

Deb Richardson wrote an inspiring post recently about how to get the most out of the weekly 1:1s we have with our managers here at Mozilla. I've been very appreciative of the 1:1 time I get with my manager John O'Duinn because it's often the time when big issues can get some forward momentum. We always have lots to talk about and it can be hard to stick to the 30 minutes we dedicate to the meeting. I think Deb's format and structure ideas will really help us keep to the time window. I've always taken notes at the 1:1s up until now, but the new format made me want a new way to track the results of our weekly chats so I did a simple Google Docs spreadsheet and form to help me prepare and take notes.

I haven't made a form in a while and I was pleased to discover that they now have a theme option, neat. So with my form and a weekly calendar alert the day before the 1:1 reminding me to take a moment to gather up items for discussion, I feel ready to rock this new format.

Being a decision maker - Part Two

2010-02-23T14:25:00.000-08:00

In my last post I was exercising my search engine choice for kicks. Choosing a search engine is easy, you type in the URL and get directed to the home page of your favourite search engine's site. If you prefer one and want it to be your default search engine, Firefox has a handy pull down menu where you can manage your search engines and set your engine of choice.

This is just one of the many ways that Firefox, and Mozilla, promote choice. Another way we do it is by providing the world's best browser in over 70 languages, and on multiple platforms. We give users the choice of a browser that is built with them in mind. For security, accessibility, and extensibility users can count on Mozilla's Firefox to be doing its best to improve the areas that make the open web work.

The Microsoft Browser Ballot screen has started to roll out this week and there are people who are most likely not expecting, nor informed about what it means to make a choice as it relates to the web browser. It's important that we not forget that many people don't know what a web browser is.

I recently posted about how I suspect the design of the ballot screen will scare away people before they even get a chance to make a choice. For those that make it to the second screen (where you are presented with the 5 top browsers by market share) there is another obstacle: lack of information. The screen doesn't tell you why choosing your browser is important. It doesn't tell you which browsers are more secure, which ones work with screen readers, which ones can be extended to add custom functionality. These are important factors in making a choice. Otherwise "choice" is really "pick the pretty logo and see what happens". Or perhaps "choice" is "stay with what you know, cause change is scary".

Which web browser you use may seem trivial thing at first but when you look under the hood - it matters that your know the browser you choose will work with your assistive technology. It matters that your identity is safe, that a site's legitimacy is explorable before you make an online purchase, and that you can customize your web browser to maximize your efficiency. I've had several academics tell me they rely on Firefox add-ons to help them cite, bookmark, and make notes in the browser as they prepare class materials. Your browser can make viewing the web a comfortable, seamless, and efficient experience. Don't you want to have the information to help you make the choice that's best for you?

I hope that John Lily's letter, and other blog posts in the coming weeks will reach a wide audience and help supplement the lack of information that the ballot screen contains. Just as it would be odd to let a stranger pick your car out for you - with no information about your driving habits, family size, gas budget, style preferences - you should try as much as possible to make an informed choice about the tools you use on your computer to do your work and live your digital life.

It really does matter. Have fun exploring your options.

Being a decision maker - Part One

2010-02-22T22:27:00.000-08:00

While helping a friend with a wordpress site, I googled for image gallery plugins and was met with:

I've never seen this before so of course I started to search for the site in other search engines:

Interesting, kind of suggests that this site also uses Yahoo. That's impossible. Now how about Alta Vista, Dogpile, Lycos, and Ask.com?

Nope, no special greeting. I saved the "best" for last, Microsoft's Bing:

Ok, wait a minute. Decision maker? Why is everything else just a friendly greeting? Decision maker makes it sound like I've done something radical by using Bing. I'd love to know if Technically Personal is generating this box or if it's coming from somewhere else.

Browser Choice Screen slight of hand?

2010-02-22T06:06:00.000-08:00

Just reading over Microsoft's "What to Expect" post about the upcoming browser choice ballot. I tried to imagine I was a windows user seeing this for the first time.

Two things about this screen bother me right away. One is that the "Ok" is just a link, not the usual, and obvious call-to-action BUTTON. It's also on the left and I usually look to the right (or center) for things like "OK", "Next", "Continue", or "Agree" type action buttons. I notice that on the next screen (the actual choice screen) "Select Later" is also on the left so maybe it's just my own habits and not a mind-game Microsoft is playing with me.

Second, they 'unpin' IE from your task bar and then the last line of screen 1 is "Before proceeding, please confirm that you are connected to the internet." If I didn't know for sure that I was connected to the internet, I'd probably want to open IE to check. I find this user experience confusing and wonder how much work Microsoft's team did in trying to make it intentionally so. The scenario I picture is my friend's mom Janice. Janice still saves web pages to her desktop instead of bookmarking them so I don't think she's the majority use case here, but I thought of her anyway. I think she would fairly represent a certain group of computer users that are competent at doing daily tasks on their machines but are nervous about making changes to their systems.

I imagine that Janice sees this screen and has no idea about the ballot's history so she takes the time to read the first screen. What? Features? What is a feature for a browser? I connect to the internet with IE, what features do I need? You've unpinned my shortcut to IE? Where do I go to open it now? I need to be connected to the internet? How can I check that? How do I open IE now that you've taken away my shortcut? Janice clicks on the "here" link to find out how to re-pin IE to the taskbar and who knows what happens next (Microsoft's post doesn't show this) but I suspect that browser choice is put aside and this screen will not be run again.

What has Janice learned about browsers, choice, security, compliance, open standards? Nothing. She unfortunately is now maybe more afraid of running a Windows update than before, and life goes on.

Anyway, it's just one scenario that came to mind. I know we're going to see some really interesting stories, comments, and choices being made as this ballot reaches more and more people. I'm looking forward to watching this all go down.

FOSDEM 2010 Video - Women in Open Source and Free Software

2010-02-12T14:54:00.001-08:00

Click to Play

A quick 'n dirty vlog featuring some of the women attending this year's FOSDEM conference. I was really glad to see so many women attendees, a much higher ratio than any FLOSS events I've been to so far. It was challenging for me to get over my initial shyness about talking with them but I'm glad I did. As my friend Bevin has said "No one ever died of awkward". That in mind, I just put myself out there on the Sunday of FOSDEM and was pleasantly surprised to discover how eager women were to talk about their involvement. I hope to continue to do this kind of documenting at future events. I'd also encourage you, dear reader, to do the same if you can. We can compile a ton of testimonies from women in open source and free software describing what they do and why.

FOSDEM 2010 Reflections - Part One: Why I went

2010-02-10T11:49:00.000-08:00

This was my first time attending FOSDEM and my first time in Europe as an adult. Being able to attend this conference was very exciting because I got a whole new perspective on the Mozilla project's community outside of North America and I met many folks in the l10n community which helps remind me that there are people attached to the RelEng Buildbot columns. In my time at Seneca, my internship, and now as a full time employee of Mozilla I mostly interact with employees and contractors so it's really great to be around folks who aren't paid to do this and yet love it as much as I do.

It wasn't until the second day of the conference that it really started to sink in how the driving forces of community were very different across the ocean. People I met told me how they were involved with open source for political reasons, and one pointed out that he could watch TV or do something that mattered, so for him contributing to Mozilla was a way of doing something important. If more people in America put down the remote (or game controller) and followed suit, imagine how many new contributors we could have! I loved hearing that for some of our European contributors, their time spent on Mozilla projects isn't volunteerism; it is essential to their daily lives.

I suspect that coming from countries that have at some point been governed by autocratic or dictatorial leaders has spurred many free-thinking people to want to personally work on openness and freedom in many areas, but especially in areas relating to technology, where web browsers become an important tool for managing identity and privacy. It's activism, it's political, and it's one of the reasons I'm here too.

My early roots of activism looked a lot more like this:

Invigorating but prone to burnout. Being crushed up against riot shields and poked with billy sticks loses its appeal fast especially when police crowd control tactics get more and more violent with every protest. So now my activism is more about finding positive, measurable, and constructive ways to help people. Hopefully free of riot gear and pepper spray.

Enter WoMoz. For folks who don't know, WoMoz is a new group in Mozilla aimed at increasing the visibility of women at Mozilla as well as increasing the number of women contributors. The main reason I attended FOSDEM was to participate in two days of planning sessions with other WoMoz members in order to lay a road map for the rest of this year. Up until now the WoMoz participants have only interacted through IRC, wiki, and mailing lists. It's nearly impossible to get a decent big picture that way, let alone get to know each other.

In the next couple of blog posts I'm going to write up my thoughts and ideas about directions that WoMoz can take and my goals for this project. I'll also post a quick 'n dirty video I've made featuring some of the women in open source that I met at FOSDEM.

So you want a new Talos suite, eh?

2009-12-11T12:51:00.000-08:00

This quarter Alice and I have focused on trimming the list of pending test suites and where several new ones (419776, 524089, 515540, 506772) have been turned on in production.

The process for getting a new suite in has becoming a lot clearer, so we gave a presentation at the recent all-hands to help the developer know what to do on their end and what RelEng can do for them once their test suite is ready for staging.

Here's what a developer needs to do:

Download and install Standalone Talos to test their suite in
Once they have established that the test works on at least one platform, write a patch against talos in cvs
File a bug against RelEng in the General component and provide the following information:
- Contact person who will work with us on getting the test suite enabled
- What the test does, what the expected output should be, long name, short description
- Which branches and platforms you want the test run on

RelEng will create the buildbot patches that enable the tests, insert the tests into graph server, and work with the contact person while the tests are in staging to make sure the expected outcome is reached. Once the tests run as expected we can turn them on in production. Perfect world turnaround for this process is about a week and a half and involves a short Talos downtime. The rest of the time allotted to our presentation was spent discussing where we should be setting our sights for Talos improvements. This looks to involve two relatively large undertakings:

While it recently underwent some much needed improvements, the graph server still needs to be faster, more stable, scalable, and able to handle our ever-growing data sets. The blocker here is that no one really owns graph server and it's hard to know who should.
Talos is barely holding up under the current load of tests, hardware, and infrastructure. It also works in such a way that a lot of manual involvement is required to add new tests. It would be awesome for it to work more like unittests where once individual tests are checked in, they would go into production immediately. It would then be possible for a developer to not only write a unittest for any bug fix, but also a performance test to go along with it.

Now this brings up the problem of what performance we want to measure and how we want to approach performance metrics in the long run. Alice made a great point when she stated that folks who are not trained and accustomed to doing QA might be challenged by trying to generate tests that actually create a good metric for the performance they wish to be testing. It's entirely possible to have tests that seem interesting on the surface, when you drill down, don't provide any useful data for actually improving anything.

Do we want per-bug performance tests as we do with unittests? While it looks like this is a way to make a developer more accountable for their code, it's pretty obvious that this model wouldn't scale well at all with our current hardware and turnaround expectancy. Imagine as many individual pageload tests as there are mochitests...I suspect no one wants to see that.

Performance testing would be better and more useful if it was targeted at specific features or areas of the product where someone is actually tracking the improvement/regression ranges on them as they are developed. That's a key area of Talos - that a human is actually accessing the data, finding it useful, and making improvements on their feature/area as a result of this information.

While brainstorming with Aki on the potential of the graph server data, one idea really got me excited. Open up the data.

There's been a lot of hype lately about opening up data. In February of this year Tim Berners-Lee encouraged us to start thinking about open, linked data and how it could be the next round in how the Web helps us re-frame our world. In Canada the city of Vancouver opened up its data in the hopes of "improving liveability and governance" in the Metro area.

What if the Talos graph data was made available to the community and a challenge was created in the spirit of the marketing design challenges where we ask people to help us find new ways to view the data? I'd be really curious to see what kind of visualizations would come out of the larger community. RelEng doesn't have a very large community outside of employees, so this could be a great way to start working on creating one.

Upcoming improvements to Talos documentation and test suite creation

2009-10-23T16:03:00.001-07:00

This quarter I'm going to be joining Alice in trying to improve the system for adding new suites to Talos. The current system involves a lot of hackery on our side and slows down the ability for us to get Talos suites up and running as quickly as might be desired.

So with John's help to create a prioritized list of suite requests, we will be doing a lot of communicating with developers in the coming months to get them up and to improve the process and documentation at the same time. Currently there are 10 new suite requests waiting that are known and there may be others.

Part of the issue with adding new suites is that there is a lack of documentation and tools for developers. Our new system will look more like this:

* A request is made for a new suite and a developer is attached to the request who will be the lead person for working with us to get the suite into production

* The dev will be able to use tools we provide (standalone talos, corral of staging-talos slaves) to do proof of concept on the suite so that it works and is ready to go up in staging when it's handed over to RelEng

* RelEng will enable the test suite in staging and verify that changes in staging work fine with the other existing jobs being run on the same machines. Once all is well, then rollout to production would happen

As we progress through the suite requests, this process should get easier for all parties and more streamlined. We hope that by the time we reach suite #10 it will be much easier and faster for developers and RelEng to get the proposed new Talos suites into production.

I mentioned the developers will have tools provided by us. We need to do a bit of work to make these tools usable by developers and the first place to start is with our documentation of what Talos is and how it works. Following this we will have discussed having boilerplate code for creating each of the two styles of tests startup or pageload. Also, it might be beneficial to have a coral of Talos machines that can be loaned out to a dev for a limited time in order to test a suite during creation and debugging. This coral could then be re-imaged and passed along to the next suite developer.

Here is the current documentation page. Doesn't give you much to go on, right?

Well this is about to change. Given my complete lack of Talos knowledge, I will be writing up what I learn about Talos as it's happening so that hopefully a more complete set of docs will exist for the Talos neophyte and folks who want to work with us to add new suites will benefit from this as well.

Here's the current list of the docs to be created based on what we think you might want to know:

* How Talos works and an overview of the development from past to present

* What preferences Talos runs with

* A description of each test suite, what each runs

* What the numbers mean

These are the things I don't know - is there anything you don't see listed here that you want to know more about? Feel free to make suggestions in the comments.

Mozilla Service Week - Toronto Event

2009-09-11T08:07:00.000-07:00

Yesterday I dropped off posters at the Parkdale Library for our Mozilla Service Week event which will take place on Monday September 14th from 2 - 6pm. The Parkdale Library is a really lively branch, with about 10 computer stations that the neighbourhood folks use constantly. Parkdale is the oldest Toronto neighbourhood and though it has a reputation for being a "bad" neighbourhood, it's been heavily gentrified in the past 6 years or so. There's still a lot of people here who live below the poverty line though, and for whom the digital gap is a very real thing. It's also a neighbourhood full of recent immigrants who depend on the library for connections to learning english, finding work, accessing resources for new Canadians, and keeping in touch with family and friends in their home countries.

When I first moved to Parkdale in early 2001, I relied heavily on access to their computers for my internet needs since I didn't have a computer. It was always a stampede to get in the door and sign up for a time slot when the library opened its doors in the morning. They've recently undergone some renovations and now have added wireless as well as a few more stations. The Parkdale librarians are super friendly and encouraging of community (and noise, in a library!) and the building itself is used often for local activities and grassroots festivals. I'm excited that 8 years since arriving here, I'm in a position to give something back to this vibrant place.

Our event involves a table set up near the computers - "Ask a Geek" - where we can field questions about anything that will help improve their interactions with the web. I believe there will be lots of people interested in picking our brains. Of course, I'm also bringing lots of Mozilla swag to draw people over to the table and to use as ice-breakers :)

Anyone in Toronto who wants to participate - please come on by.

Details about this and other Toronto events HERE

Adding choices to Try Server web interface

2009-09-11T08:02:00.000-07:00

Just put in my patches on bug 473184 which will allow folks who submit patches through the try server web interface to select if they want a build or unittest run and what platforms it should be run on.

Looks like this:

I've tested it successfully in the staging environment and I hope to get this rolled out in production before the end of the quarter.

New Branch Timeline: Electrolysis

2009-07-24T11:00:00.000-07:00

Now, a week or so later - we are setting up a new project branch. Here's the break down:

2009-06-26 11:55 PDT

Bug requesting the branch [500755] was created

2009/07/09 11:13:41 PDT

Created a tinderbox page for the branch to report to [Tinderbox Page]

2009-07-14 17:44:27 PDT

Patches to add Electrolysis branch to buildbot are written and put up on staging-master. There are several compile fails.

2009-07-22 06:12:49 PDT

After testing and scheduling downtime, the config files are checked in and P-M is reconfigured - last minute patch to turn on debug builds is added as well when we realize that is not in the default template for turning on a new branch.

2009-07-22 13:31:59 PDT

Need graph server machine table updated for new branch - we did a bunch all at once in [504435]

2009-07-23 06:25:18 PDT

Add Nagios monitoring support file is checked in and bug filed requesting IT turn it on. [505986]

What remains is to check in the debug builds patch, this should happen in a downtime next week at which point the project branch request bug will be closed.

New Branch Timeline: Places

2009-07-24T10:57:00.000-07:00

A brief rundown of what was involved setting up the Places project branch. This is based on the time since the branch request was given the go-ahead, not when the bug was filed since that happened quite a bit earlier.

2009-04-29 17:17:43 PDT

Bug requesting the branch [459269] was re-opened

2009-05-18 16:31:46 PDT

Created a tinderbox page for the branch to report to [Tinderbox Page]

2009-05-19 09:16 PDT

A separate bug was filed requesting a repo [493745]

2009-05-26 16:32:18 PDT

Repo is created and [493745] is closed as FIXED

2009-06-24 10:22 PDT

Patches submitted to update config files for Staging-Master and Production-Master

2009-06-30 12:51 PDT

After testing and patch updates, the config files are checked in and P-M is reconfigured

2009-07-01 08:49 PDT

Add Nagios monitoring support by filing a bug with IT [501710]

2009-07-08 08:04:40 PDT

[493740] is fixed to deal with the scheduler not picking up the new Places poller after a reconfig, only after a stop/start

2009-07-09 14:36 PDT - 2009-07-10 12:50 PDT

Patches submitted to turn on talos and graph server support for the new builds. The first set were not patches to Talos-Pool so a second set was required.
New row in graph server added for branch (bug 459269. IT (justdave) ran the INSERT statement against the production database)

2009-07-13 11:18 PDT

Patches submitted to turn on leak testing debug builds. This was checked-in and P-M was reconfigured the same day.

2009-07-13 14:57 PDT

Bug closed - project branch is up and running on P-M

Celebrating Firefox 3.5 with sparkly accessories...

2009-07-09T14:09:00.001-07:00

A month or so ago when the Firefox 3.5 was close to launching I got in touch with some artist friends of mine who have a small jewellery making business to create some custom accessories for me in celebration. They have been making these awesome belt buckles, cuff links, magnets and many other items for years and I have several of their pieces including a custom "fancy deluxe" belt buckle with my hound dog on it surrounded by shiny Swarovski crystals. They did a great job with the Firefox logo and I am now the proud owner of

A belt buckle:

And cuff links (though I only have one shirt that uses them):

I'm looking forward to wearing them about town and spreading the word about Firefox 3.5. Many of my pals outside of tech circles have been excited about the new Firefox because they see how excited I am about it. For anyone else looking for a custom buckle or accessory of their own get in touch with the folks at Barbie's Basement Jewelry. I hope you enjoy flashing the Firefox logo around in new and fashionable ways - it's always a conversation starter.

Mozilla Service Week - making a difference in your community

2009-06-16T15:50:00.000-07:00

Mozilla just announced Mozilla Service Week which will be held September 14 - 21, 2009. This week is a push to connect people who can help make the web work better for someone in the community with people and organization who need that help.

Coming from the non-profit arts sector prior to my job at Mozilla, I will be spreading the word with many arts organizations in Toronto who would be wise to sign up for help from such a talented pool of volunteers. Of course I will also donate my time that week even though it's the kind of work I do all the time already. I can't even count the amount of time I've spent setting up routers and networks for less technical folks in my life, or helping them set up their new computers and teaching them basic skills (all teaching sessions include installing and setting up the latest Firefox of course).

The best part of this week, in my opinion, is that it's an opportunity to get hands-on with local users in the community. The library is a great place to start. It wasn't that long ago (around 2003) when I was using the library computers as my primary access to the internet. I'd love to go in now and make sure that their computers are up to date, and write up how-to manuals and helpful hints for beginners. Even better, get some folks to translate those manuals or tip sheets. At my local library I'm certain that there are many folks who would appreciate localized information sheets.

Two areas that are of particular interest to me with regards to the organizations I know in Toronto:

Bring their websites over to an open-source CMS like Drupal. Many of the sites are hand-coded php (or god forbid Dreamweaver-created sites) with no administrative back-end and keeping the site's information up to date is a difficult/dangerous task for non-technical staff.

Take their FileMaker Pro databases over to MySQL or PostgreSQL so that they are no longer locked in to expensive, proprietary database software that requires additional hosting costs. Three organizations I have worked with are on three different versions of FMP and none of them the latest. Upgrading is painful for them and their hosting costs are ridiculous (especially the ones who are on older versions).

If you're technically inclined, go to the site and sign up. If you've got an organization in mind, tell them to sign up. Let's make this event a success so it will inspire more weeks like this in the future.

iPhone tethering - how to restore after updating to 3.0b5 and iTunes8.2b10(13)

2009-06-04T05:59:00.001-07:00

Breaking my extended blog silence (I've been busy, you know, starting my full time job at Mozilla!) to tell you about the quagmire I went into when I tried to set up tethering on my iPhone.

So here's the story - with links that will help you avoid what happened to me.

I purchased and ADC membership so I could download and install the iPhone 3.0 SDK and OS. When Beta 5 came out I eagerly updated since my phone had been crashing regularly and went into restore mode. As soon as the 3.0b5 update went live on my phone, iTunes said it needed 8.2 to work so I downloaded the most recent version from ADC. This turned out to be 8.2b10(13) which disables the ability to change carrier settings and thus, to enable tethering if you didn't already do it before the update.

So - after many (failed!) attempts to go back to 3.0b3 and 3.0b4 so that I could use iTunes 8.1 - I did eventually (10 hours and many blog tips later) work out my tethering issues with this extremely simple set of steps:

Simple Steps to restoring tethering:
Download: http://www.adammcnamara.com/CanadianCarriersTetheringPatch.zip
Download: iTunes 8.2b7(10) http://thepiratebay.org/torrent/4874487/

* Delete iTunes 8.2b10(13) from your computer, install the older version - as the torrent description points out, it says that it installs as an 8.1.1 but it is in fact 8.2.

* When you connect your phone in iTunes, Option-click the “Check for Updates” and pick the.ipcc file for your carrier (from the carriers patch you downloaded). It will update the carrier settings.

* Reboot (NOT RESPRING) the phone. You can now tether over USB or BlueTooth (unless you have a new Mac Unibody Mac where apparently the Bluetooth stack is broken and so you're relegated to only sharing via USB).

To activate tethering, go to “Settings->General->Network->Internet Tethering”. The first time you do so it may say “Contact Rogers to enable this feature”, but a few second later it should allow you to enable it. If not, restart the phone again.

Thanks to many blogs out there but more specifically to Adam Mcnamara where the bulk of the steps are from.

Tryserver now has unittests on all 3 platforms

2009-03-16T15:19:00.000-07:00

Bug 445611 is now resolved fixed, you can see the unittest results on all three platforms here: MozillaTry.

There are hopefully more machines coming that will be allotted to tryserver, with the likelihood that more developers will now use the tryserver. I'm also hoping to start working on setting up the web interface to allow for the selection of which platforms you want to try your patch on and whether you want unittests run or not since at the moment they are turned on for all patch submissions by default.

This patch also required the landing of a patch for bug 479225 where we now call the reftest/crashtest suites and the mochitest suites with make from the top level directory which helped us get rid of a bunch of extra workarounds we had to do for Mac OS since it needed to know which .app file to look for and we're changing that name so often (Minefield, Shiretoko, etc). Now all three platforms are even closer to being alike. Many thanks to Ted for making this possible.

No More Passwords Please

2009-03-15T09:33:00.000-07:00

This is the tentative title for my upcoming white paper, which is the major deliverable for the btr820 course on Research Methodologies and Writing. I'm excited to be doing this paper because a) I love writing and b) I'm looking forward to learning more about my topic which is essentially looking at solutions for the future of authentication on the web.

As users of web sites and applications, we are now subject to having to authenticate ourselves multiple times a day - I read somewhere than an average is 13 but for some of us who spend more time online it's probably twice that. Having your passwords remembered for you by the site or by your browser helps, but that is not a great solution for folks who are on multiple computers. Besides our passwords aren't even that safe to begin with (my bank won't let me use more than alpha numeric characters) and some sites make you change them regularly for extra security (a lie) and so as users we are caught up in a game of constantly trying to stay on top of the latest password for which site and please stop the web now, I want to get off.

What I want to look at is open, decentralized authentication identifiers that go beyond passwords with regards to actual security, that could be in your browser itself, and that would move with you easily no matter what computer you are on.

So I have some questions.

What are the implications of a web browser incorporating an open authentication protocol out of the box where the identifier is the browser itself?

What other options are coming down the pipe in terms of built-in browser features that help users deal with authentication? Is there something better than a decentralized open authentication protocol?

Do browser providers have to stay neutral and leave it up to web application providers to decide how users authenticate on the web or can they step in and lead the charge towards a certain protocol and influence sites instead?

While Weave is an excellent way of syncing your profile across various computers - is it really scalable? What other options are there for having an easy, portable profile which would be able to contain your identity as you move between computers, countries, even to your mobile device?

Thanks for reading this, I look forward to your thoughts on this issue.

There are no new ideas...

2009-03-08T08:11:00.000-07:00

So I'm in a MacOS and iPhone development class this term. It's my last term in the Bachelor of Software Development degree program at Seneca College. I've now done some major project implementation, interface design, web programming, business plans and marketing exercises - a bunch of useful skills have been developed to be sure. However, today I am having a moment of doubt.

Here's the issue:

I'm a pretty new programmer/developer. One of the main reasons I chose this degree was to become more able to help people who are not so into computers and programming, to be someone who could help link the non-tech people (mostly artists) with technology in ways that are comforting and useful not scary and burdensome.

So.

I had this thought that for my MacOS programming assignment I would make a little app to help my freelancing friends keep track of their time spent on contracts and print up nice little invoices at the end. Simple, useful, something I myself would need from time to time when I do side projects. I have a free app called Khronos which sort of does this but creates ugly invoices so I was inspired to improve on it.

Thing is, better (not free or open source) apps already exist. And they're good. As a pretty green programmer (and someone with 7 weeks of classes left) I can't touch these apps for their functionality. I would like to think I would do slightly better on the design, but that's not going to mean much if I can't come close to implementing the kinds of features the other guy has.

Now I wonder how to even a) get motivated and b) set reasonable expectations for myself in this assignment. I am feeling daunted by the fabulousness of these other programs (which I realize took years and a team of developers to accomplish) and it's hard to see how I can dip my toe in the water.

If anyone has read this far - advice is much appreciated. What do you do when you want to write something and find out it's already out there? How do you scope out an app for your first version when there are apps that already do more than your 1.0 could?

Unittests on Try Server - Linux HG Builds is Go!

2009-02-26T13:21:00.000-08:00

So today we deployed a unittest builder for the Linux platform on Try Server. You can see on the waterfall that there is now a column for Linux unittest results. The builds resulting from that builder will not be uploaded anywhere.

The Mac unittest builder is close and I hope it will be ready to go later today, if not then very early next week. Windows is of course being a pain and isn't production ready yet.

This is a big win in that developers can now test their patches in the Try Server and see the unittest results before committing, it's unfortunate though that the current implementation of this functionality is going to take 2 builders per platform per patch. We only have 4 slaves per platform on Try and so you can imagine that this will slow down Try significantly. A Linux unittest run only takes ~ 1hr though so that's good.

Anyway, enjoy the unittests. Watch for more soon.

Learning from Lab 1 in Mac OS and iPhone development

2009-01-21T15:16:00.001-08:00

Just finished my Lab 1 for Peter's Mac class and I want to keep track of the little bumps along the way.

Things I learned:

If you want the keyboard to hide after you are finished inputting text into a field - basically once the field loses focus you need to do this:
[txtControl resignFirstResponder];
thanks to iCodeBlog for that one. That site also has the best tutorial that I could find for getting started on iPhone apps.

It's much easier to do a View-based application with the iPhone SDK template, but this means you have fewer UI objects to play with. I still have not figured out why this is. In the regular application template there is a wide variety of inputs, pickers, etc. But in that template you have to create custom views and though I read up on this a bit, it seemed like overkill for Lab 1.

Looking at the existing apps in the iTunes store is a good way to see what UI designs are working already in the real world. I like to think I'm pretty good at laying out apps, but it's amazing how much better they can be and it's very helpful to look at what people are doing in the store since they have probably gotten a lot more feedback on their UI.

The amount of information out there about iPhone code, especially for beginners is really sparse still. It seems like every time I google about an issue I'm hitting the same few blogs/sites. I'll post the code for my apps when the due date has passed so that there's just a little more out there. Remind me if I forget.

Holiday boredom leads to memes...

2009-01-04T22:45:00.000-08:00

Taking this from Dolske:

1. Open this link, select-all and copy it.
2. Open up the Error Console.
3. Paste into the Code field at the top, and click Evaluate.

10 random Wikipedia URLs out of 85...

Unittest Consolidation and You

2008-12-18T08:49:00.001-08:00

This morning was the culmination of months of work from RelEng. We have finally turned on unittests as well as a11y tests on the production Buildbot master which was already doing the nightlies and l10n builds.

Mostly this change is to make our lives easier. Now we have one instance of Buildbot to pay attention to (disk space, waterfall page) with one pool of slaves that can do several types of builds (nightlies, unittest, tracemonkey, l10n). This will make replacing burned out slaves incredibly simple. Doing this consolidation also led to a great unittest factory (thanks to catlee) that will make adding unittests to other branches much easier to deploy.

What does this mean for you? Well, if you are someone who looks at unittest output regularly you are probably familiar with something like this:

Where each unittest build has its own waterfall column and you can gauge how the unittests are doing by comparing them to each other (changeset, start time, colour).

The new way will be different:

There will only be one column on the waterfall, with no machine name since the unittest build will be coming from a pool of over 14 slaves for that platform. When you look at this waterfall column you will see the colour (thus result) of the most recent build, but there may be several builds that start within moments of each other.

More investigation will be needed to look at changesets and start times with this new method.

Hopefully people will adjust to this, and we welcome your feedback about the new setup.

It's a very big step for us, one that leads to hopefully parallelizing the unittest steps for quicker unittest results, and eventually doing unittests on builds independently - where right now the unittest run includes a build step every time, we would like to get to a place where unittests can be run on a completed build so that the suite of tests could be run repeatedly on the same build for example.

We'll wait a day or so to make sure that this consolidation is running smoothly and then the standalone production unittests will be turned off, the mac slave machines will be moved into the new pool and the linux/windows vms will be deleted and then new ones created for the production pool. Hopefully there won't be too much of a backlog on builds pending while we wait to get those new machines added.

Dubai and Dashboards

2008-12-08T12:15:00.000-08:00

Two things on my mind these days:

Item One:

I will be attending the Education Without Borders conference as a delegate from Seneca College. Five students were chosen to represent Seneca from several fields.

My goal was to try and get a spot presenting about Mozilla's partnership with the Open Source curriculum that we have at school, taught by Chris Tyler and Dave Humphrey. Sadly, I was not selected to present. There were about 1000 attendees from all over the world submitting their proposals so I trust there was lots of competition and that the selected presentations with be mind-blowing. Many of the attendees will be grad students presenting academic papers. It will be great to be there regardless, and I will certainly be pitching Mozilla development and sharing the Seneca teaching model to anyone who will listen.

Item Two:

Q4 is rapidly approaching its end and while I am working on the consolidation of build and unittest to the same buildbot master (when I am not studying for my exams), I am also looking forward to Q1 where I would like to spend some time gathering requirements for a meaningful dashboard of the unittest information that would be useful to developers.

I'm not sure how one gets this kind of conversation started. It would be great to hear from people who care about unittest results and have opinions on what they use the information for. Would a questionnaire be useful? Should I start a forum discussion?

I'll continue to think on that and ask around for ideas on how to do this right. Has anyone been successful in creating a dashboard that is used frequently?

Seneca and Mozilla in the news

2008-11-20T09:43:00.000-08:00

So Armen and I were interviewed a while back for the school paper - The Senecan - about our work with Mozilla through Seneca's open source courses. The article just came out, Download PDF

The TTC on the move...

2008-10-23T10:28:00.000-07:00

This is an awesome video made with OpenGL by the guys who just did a PK on myttc.ca - an amazing visual of the transit system in Toronto throughout the day:

TTC Weekday Service (HD) from Kieran Huggins on Vimeo.

Ready, Set, FSOSS

2008-10-23T09:01:00.001-07:00

Sitting in the workshop "ohai! art!" at FSOSS early on a Thursday morning. I'm downloading pure data (extended) to participate. Very curious to see what we'll do in this time.

Last night was the kick-off party at Mozilla for FSOSS speakers and also a time to do a dry run of the PK presentation that I'll be doing on Friday. I got a sneak-peek at Gale Moore's slides and then did a run-through myself. By setting the timing on the slides to 20 seconds I forced myself to stick to the format. It's not hard to do this when you're really just talking about your life. It's a whirlwind tour of what I call my "Zero to Hero Journey". Come by tomorrow and see it.

The best part of last night though was informal discussions about two topics I love; activism and enjoyable work environments. The former came up in discussion with Gale and Mike Hoye about the systems being used in schools. Seneca uses BlackBoard which is akin to being drawn and quartered on a regular basis. It's painful to use, ugly, has absolutely nothing to offer students, and even our teachers hardly touch it. Gale wondered why students weren't rebelling or otherwise agitating for a better system...Moodle for example. My take, granted that I am a bit different than the "average" student at Seneca, is that because of the decentralization of students - many live up to 3 hours away from campus - there is never going to be a decent level of student activism about this or any other issue because we don't spend time with each other outside of class.

On rare occasions, students may have time to compare notes with each other about their experience in their program - but without a common space for downtime, combined with an interest in making changes to the way things are, we will never join together to combat the system as it stands.

I'm 6.5 months away from being done school and it's not in my list of priorities to take on this kind of activity either. So no complaining about BlackBoard for me. I've got bigger fish to fry.

The second topic, the work environment, was fun because it wasn't just a Mozilla love-in (and there is so much to love) but instead I heard from someone in a completely different field how much their workplace supported and encouraged them, gave them a clear path to advancement and had a strong team with no weak links. I'm often griping about the team work that we are obliged to do in school, mostly because the teams are often nothing like anything found in the "real world" and then also due to the fact that the assignments are often fictitious and therefore don't give us as much of a challenge.

The open source class at Mozilla really raised the bar by giving us real work to do and that experience is invaluable. Instead of wasting time with teammates dividing up work and trying to manage other students' work ethics, we were project leaders on our Mozilla work and we pulled people into our teams as needed to get the work done. This kind of turns the model that other classes use on its ear. Next term I'll be doing a research and methodologies course where our deliverable is a significant research paper on some facet of technology. I intend to pitch an idea to research the current BSD program's project stream and how it currently leads to very little implementation. Hopefully with a little research, we can look at ways to turn that around and give the next generation of BSD students a level of satisfaction that will enable them to recognize this in their future workplaces.

Third time's a charm - Unittest Production moves tomorrow morning

2008-09-23T13:21:00.000-07:00

Wednesday September 24th at 6:00 PDT I'll be connecting the new slaves to Tinderbox:Firefox and taking down the old ones.

Fingers crossed that it really happens this time.

Postponed - Production MozCentral Unittest Moves - to the Build Network

2008-09-23T07:02:00.000-07:00

As you may already know, there is some major outage (power out at the San Jose colo) this morning - now dubbed "Black Tuesday" by me. So the switch is postponed again.

New time and date forthcoming when the network is up and stable again.

Take 2 - mozilla-central unittest production waterfalls moves to build network

2008-09-22T11:09:00.001-07:00

Hopefully things will still look good tomorrow morning because the new time of the move is now:

This will take place on Tuesday September 23th, at 7am PDT

The new production buildbot will already be up and running smoothly
(currently reports to the UnitTest tinderbox tree) so there should
hopefully be very little impact when this switch over happens.

Basically, the new unittest buildbot slaves will start reporting to
the Firefox tinderbox tree, and the current slaves will stop reporting
there.

This means you will be looking at new slave names. There are 2
buildslaves for each platform and their names are as follows:

Linux mozilla-central moz2-linux-slave07 dep unit test
Linux mozilla-central moz2-linux-slave08 dep unit test
MacOSX Darwin 9.2.2 moz2-darwin8-slave01 dep unit test
MacOSX Darwin 9.2.2 moz2-darwin8-slave02 dep unit test
WINNT 5.2 mozilla-central moz2-win32-slave07 dep unit test
WINNT 5.2 mozilla-central moz2-win32-slave08 dep unit test

Postponed - Production MozCentral Unittest Moves - to the Build Network

2008-09-19T07:08:00.001-07:00

Due to some glitches in the buildslaves, the move of the mozilla-central unittest build master has been postponed for now.

I will post again soon with a new time and date for the switch.

http://groups.google.com/group/mozilla.dev.planning/post

Production MozCentral Unittest Moves - to the Build Network

2008-09-17T14:50:00.000-07:00

So the last task in the move from QA network to Build network is moving the production mozilla-central unittest boxes.

This will take place on Friday September 19th, at 7am PDT

The new production buildbot is already up and running smoothly (currently reports to the UnitTest tinderbox tree) so there should hopefully be very little impact when this switch over happens.

Basically, the new unittest buildbot slaves will start reporting to the Firefox tinderbox tree, and the current slaves will stop reporting there.

This means you will be looking at new slave names. There are 2 buildslaves for each platform and their names are as follows:

Linux mozilla-central moz2-linux-slave07 dep unit test
Linux mozilla-central moz2-linux-slave08 dep unit test
MacOSX Darwin 9.2.2 moz2-darwin8-slave01 dep unit test
MacOSX Darwin 9.2.2 moz2-darwin8-slave02 dep unit test
WINNT 5.2 mozilla-central moz2-win32-slave07 dep unit test
WINNT 5.2 mozilla-central moz2-win32-slave08 dep unit test

Thanks to Jesse Ruderman and others for working so hard on bug #450637, as it means we are now running unit tests on win32 VMs with very consistent results.

As soon as this is done, and working well I can tie up the loose ends, close out some bugs, update the config files and start looking at some new projects...like the test results - and try server unittests.

Continuing saga of the 1.9 Unittest Move

2008-08-14T17:19:00.000-07:00

When we left off, there was a check error happening across all Linux slaves and a reftest failure on the Win32 ones.

Update #1: A bug (450637) has been filed on that win32 failure, and also I brought the physical boxes back from sleep to be up on the new 1.9 master alongside their VM counterparts. We should know in the next hour or so if the reftest failure is consistent on all of them.

Update #2: The check error on Linux was due to the placement of a simple .sqlite file bug-365166.sqlite to be specific. This file was in /tmp and not in the slave build dir and thus, escaped during chown. Being owned by buildbot instead of cltbld was the cause of the access denied errors. Huge thanks to Cesar and Sdwilsh for looking at that test with me and for catching this anomaly. I've filed a bug (450665)to remove the offending placement so that this doesn't happen again in the future. Files shouldn't be getting created outside of the build dir, creates a whole mess of problems.

Speaking of mess:

Ew. That's all I can say. I've been watching this waterfall obsessively (more than usual) as it has displayed a bruised variety of colours, mostly *Not* green.

In other news, something I noticed while upgrading the windows slaves:

Really? I didn't know that people _chose_ IE. I thought it just came with the OS. I wish they would choose their words more carefully.

Back to the unittest trenches.

Update on the Unittest 1.9 move

2008-08-13T13:26:00.001-07:00

In order to streamline the buildslave pool, the names of the following unittest 1.9 slaves were changed when we switched networks yesterday.

All of these machines now run Buildbot 0.7.7 and the latest Twisted & Python.

The Linux machines had their names changed and user changed - they are the same VMs as before:

qm-centos5-01 --> fx-linux-1.9-slave07
qm-centos5-02 --> fx-linux-1.9-slave08
qm-centos5-04 --> fx-linux-1.9-slave09

The Mac machines are the same ones as before, only a user change:

qm-xserve01 --> bm-xserve20
qm-xserve06 --> bm-xserve21

The two non-pgo windows machines are now VMs, the pgo box is the same VM that it was before - with a user change and a new 30GB fcal drive added for building on

qm-win2k3-01 --> fx-win32-1.9-slave07
qm-win2k3-02 --> fx-win32-1.9-slave08
qm-win2k3-pgo01 --> fx-win32-1.9-slave09

At the moment all three Linux boxes are experiencing errors in Check :

gmake[2]: Leaving directory `/builds/slave_new/trunk_centos5_8/mozilla/objdir/storage/build'
gmake[2]: Entering directory `/builds/slave_new/trunk_centos5_8/mozilla/objdir/storage/test'
../../_tests/xpcshell-simple/test_storage/unit/test_bug-365166.js: FAIL
../../_tests/xpcshell-simple/test_storage/unit/test_bug-365166.js.log:
>>>>>>>
*** Storage Tests: Trying to close!
*** Storage Tests: Trying to remove file!
*** test pending
[Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [mozIStorageService.openDatabase]" nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)" location: "JS frame :: ../../_tests/xpcshell-simple/test_storage/unit/test_bug-365166.js :: test :: line 22" data: no]
*** FAIL ***

<<<<<<<
../../_tests/xpcshell-simple/test_storage/unit/test_bug-393952.js: PASS
../../_tests/xpcshell-simple/test_storage/unit/test_bug-444233.js: PASS

And all three Win32 boxes are having the same 1 test fail in Reftest:

REFTEST UNEXPECTED FAIL: file:///E:/slave/trunk_2k3_8/mozilla/layout/reftests/bugs/212563-1.html

Please contact me if you have any ideas about what could be causing these.

-- Lukas

Welcome to Build, Ben says

2008-08-12T16:58:00.001-07:00

Today was a big day for the Firefox 3.0 unittest set up. Since QA and Build have become separated, I have been working towards lining up all out unittest masters on the Build network. What used to be 10+ master addresses will be narrowed to 2 - you're either on staging-master or production master.

Easy.

No. It's actually not that easy. What I estimated would be 2 hours of downtime has turned into almost 8 hours (and counting) for many reasons, including the following:

* All the slave VMs had to have a new user created, one that is consistent with all our other Build machines. It makes sense to do this all at once, but it takes some time to get all the permissions and paths and ssh keys and other little details to line up properly

* In switching networks and users, the linux boxes were unreachable by VNC for some time until it was discovered (thanks to bhearsum & joduinn) that the xstartup in ~/.vnc was configured differently than the other linux boxes. I think it took almost an hour to get the fix on this figured out

All in all there were many little trips and glitches that made this process go for so long, and the fact that it can take over an hour to see if a build & test run is successful sucks. Thank you very much to all the Build Team who helped during this process.

At the time of writing this, I am only waiting on the pgo box to come back up on the new network with a 30GB disk partition added, and looking into a few compiler warnings on Mac and Windows. The PGO box didn't have an fcal disk partition for building on and I wonder if the issues in this bug are related to that. It would be a pretty great bonus if this switch turned up the fix for that machine.

The good news is that we are in the process of streamlining and making things more efficient for the future. All the build machines are getting closer every day to being interchangeable. The time it takes to get a new linux VM running is miniscule - and hopefully the same will be true of the other two platforms soon.

Things still to do:
* post about the new machine names of these VMs
* make sure that Nagios is clear about what it should be reporting on
* update the cron job that does the rsync of the buildmaster logs to the TB share
* file patches for 1.9 unittest's mozconfigs, master.cfg, mozbuild.py and killAndClobber.py

Back to watching the buildbot waterfall for green.

Scheduled Downtime Tues Aug 12 - 8:00 am PDT for Unittest network switch

2008-08-11T10:54:00.001-07:00

Tomorrow there will be a ~2hr downtime starting at 8:00 am PDT as the 1.9 unittest master is moved over to the build network.

At the same time there will be a short interruption on the Mozilla2 production master.

If any issues arise, please comment in bug 450119.

Looking for suggestions on dealing with lots of data

2008-08-07T10:49:00.001-07:00

So I'm still plugging away at figuring out how to interpret the massive amounts of error log output that our unittest builds create.

As the test suites are being run, there is a steady stream of stdio being generated and logged. From this stdio, I gather up all the lines of output that contain "TEST-UNEXPECTED-FAIL" (thanks to Ted for unifying the output!).

Now I have files that look something like this:

linux-2 | 67 | 07/25/2008 | 06:40 | *** 61506 ERROR TEST-UNEXPECTED-FAIL | /tests/toolkit/content/tests/widgets/test_tree.xul | Error thrown during test: uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMWindowUtils.sendMouseScrollEvent]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: http://localhost:8888/tests/SimpleTest/EventUtils.js :: synthesizeMouseScroll :: line 273"  data: no] - got 0, expected 1
linux-2 | 67 | 07/25/2008 | 06:40 | *** 62352 ERROR TEST-UNEXPECTED-FAIL | /tests/toolkit/content/tests/widgets/test_tree_hier.xul | Error thrown during test: uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMWindowUtils.sendMouseScrollEvent]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: http://localhost:8888/tests/SimpleTest/EventUtils.js :: synthesizeMouseScroll :: line 273"  data: no] - got 0, expected 1
linux-2 | 67 | 07/25/2008 | 06:40 | *** 63084 ERROR TEST-UNEXPECTED-FAIL | /tests/toolkit/content/tests/widgets/test_tree_hier_cell.xul | Error thrown during test: uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMWindowUtils.sendMouseScrollEvent]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: http://localhost:8888/tests/SimpleTest/EventUtils.js :: synthesizeMouseScroll :: line 273"  data: no] - got 0, expected 1</pre>

Where the info is "|" delimited and goes like this:

<pre>PLATFORM | BUILD_NO | DATE | TIME | TEST-RESULT | TEST-NAME | TEST-OUTPUT

Approximately 7000 lines of error output for less than a month of constant testing.

I want to be able to know the following (at least):

* How many times has a particular test failed?
* On which platforms?
* How many times this week vs. last week?

That would be a start anyway.

I would love to be able to create a graph or something visual that shows peaks of test failures. Unfortunately I don't really know much about that area.

So I am asking for help/suggestions. If you had about 490,000 lines of errors (representing 3 platforms) in the above format - what would you do?

I can pretty easily add to the python script that greps for error output so that it creates sql insert statements instead of a text file and I would welcome tips that include creating/automating a database to hold all the error info. I've been thinking of setting something up with RoR to let people create their own views of the data depending on what they are looking for.

Looking forward to your advice.

Grovelling isn't so bad

2008-07-23T16:14:00.001-07:00

Been working on a couple of little utility scripts that I think are ready for public viewing. I'm interested in any tips on writing better code, or other ways to do what I'm doing that are more efficient.

The first one is cleanup.py which we need to be able to quickly get rid of old log files so that when we grovel through for errors, only the files of interest are being scanned.

Once you've got the old log files cleared out, you can use grovel.py to scan through for TEST-UNEXPECTED-FAIL. This script looks through each directory passed in from the command line, and prints all the failure lines to a .errors file for that directory - so the darwin log errors end up in a darwin_timestamp.errors file. The script also keeps a counter of TEST-PASS, TEST-KNOWN-FAIL, and TEST-PASS(EXPECTED RANDOM) and then prints the total tests run as well as these counters on the last line of the .errors file.

Next steps:

Add gathering up all the .errors files into a tarball

Set up a weekly cron job that will run these scripts and email the tarball

Create a database and insert results

Web interface for aforementioned db that will allow for searching

Even though these are pretty simple utility scripts, I'm excited because they will make my life a little easier and also because it's the first python I've written from scratch...oh, and it's not a school assignment :)

Discussing Data

2008-07-21T20:36:00.001-07:00

Some general thoughts on the discussion of data, inspired by Mitchell's blog post.

When I first started using the internet with some regularity, about 13 years ago, I was suspicious about entering any personal information whatsoever. This was before identity theft was a common occurrence, before I had any money to worry about losing, I don't think I even had a credit card yet. Some of the fears were based on run of the mill rebellion against "The Man" but some if it was just a reaction to something new.

For many years, whenever prompted for personal information, I would look for a way around having to enter it. If I couldn't get under it or over it, I would make stuff up...or leave. Creating false accounts gets tiring, because then you have to remember all your lies. Firefox wasn't around yet to help me keep track of all my phony accounts. I sure do appreciate the password manager and extensions like BugMeNot.

Skipping forward to the present, I still look for a way out of having to enter any identifying data wherever possible. Something that continually annoys me is being required to choose between male and female on a form when I am making a purchase. This should NOT be required to buy a sticker, test beta software or sign up for a social networking site. I'd like to see the end of generalized marketing based on gender and find new ways of triangulating what cat owners are doing that is different than dog owners.

Back to the data...

Even though I hate the thought of anyone assuming they know me because of a few hastily checked radio buttons, I also want the freedom to go about my business on the internet as easily as I do in real life - with my driver's license and a bank card. I have proof of who I am and I have money - what more do you want? I should now get to do whatever it is I'm looking to do with as few clicks as possible.

So if the future web browser allows me to safely keep all the important stuff handy, to know that I am who I say I am, and let me skip the 3 page sign up process, this is a Good Thing.

How can we get to that kind of level without talking about data and all the good/bad/lukewarm associations we have with it?

I tell people all the time that they should be using Firefox because it is the safest. People care about safety, and this is what they need to hear. If Firefox starts to work with data, I trust that we will do so in the best interest of the people who came to us for safety. I'm excited to talk about data and what we can do with it.

My hope is that data collection will become less of a top-down "Tell me this information or you can't access {fabulous service name here}" and instead will become the equivalent of the clerk at Best Buy asking you for your postal code and being able to say "No, I don't want to give that information to you, but I will still buy Rock Band from you".

Set the VNC Password for Mac's Remote Desktop in Terminal

2008-07-17T10:19:00.001-07:00

I was stuck in trying to access one of our xserve machines that just got moved from the QA network to the Build network. I could connect via ssh, and Justin could ping it but attempting to connect with VNC wasn't working. It wouldn't accept the usual passwords. Justin seemed to think that it was possible to change the VNC password through the command line, so I google it and read a post from 2 years ago.

Something I've learned from reading "how-to" blogs is that you should always read the comments first. That's where the most up to date information will be, if there is any. The person who wrote the post used strange template structure that made his idea hard to read and understand. Anyone who didn't read the comments wouldn't know that kickstart now takes plain text passwords.

The long and short? If you want to change the VNC passoword do this:

sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -clientopts -setvnclegacy -vnclegacy yes -setvncpw -vncpw [newpassword]

Apparently you can enable VNC access and set the VNC password via the kickstart command. It isn't terribly well documented, but since it now accepts plain text passwords, I think that's a step in the right direction.

Chasing rainbows is easier

2008-07-02T10:09:00.000-07:00

I was so thrilled to discover Splunk that I installed it on one of the buildbot masters - qm-rhel02 - without realizing that in fact, Splunk starts to quickly eat up disk space and hogs memory usage. Yesterday afternoon some Talos boxes started to go down because of this, and once I stopped the Splunk server everything started to right itself.

Lessons learned:

     Do not play with the buildbot master. 
     Do not look directly at the buildbot master. 
     Do not taunt the buildbot master.

So today's tasks include getting access to the samba share that was set up, creating a cron job that will rsync the buildbot master logs to said share and then finding a safe place to set up Splunk again.

We really need to have a way to look at data from the buildbot master over a long period of time - otherwise filing bugs on these intermittent failures is just a shot in the dark. Take yesterday for example. qm-win2k3-pgo01 is being "unreliable" and had the same errors in refest for two consecutive builds. I file a bug, and the response is "grab me a copy of the offending objdir so we can poke at it". Wouldn't you know that the very next build does not have the same error output - this time it has mochitest issues that are seemingly unrelated. This morning I check again and it's had a compile failure, an exception (the most hideous purple) and then a completely green run.

Intermittent failures == needle in a haystack

sha1sum on Mac OSX

2008-06-25T11:56:00.000-07:00

Getting ready to assemble the Firefox 3 CD and came upon the glitch that Mac OS X doesn't provide sha1sum tools. Quick Google search turned up a great comment on this blog post which suggests using openssl and by putting alias sha1sum="openssl dgst -sha1" in my .profile I can now do sha1sum $app_name.iso to my heart's content.

The Firefox 3 CD will now have all the supported locales on it, which is a step up from the Firefox 2 CD. Look for it in a Mozilla Store near you.

Splunk - Where IT's at

2008-06-19T14:40:00.001-07:00

So, you know that script I was working on to parse error logs? Well, it turns out that there is already an amazing, free, graphical program that does the work for me. Excellent.

It's called Splunk, John O'Duinn mentioned it in passing last night and today I got it running on the unittest-staging build master in about 2 seconds flat.

Installation (on linux) is a breeze, simply:

wget 'http://www.splunk.com/index.php/download_track?file=3.2.6/linux/splunk-3.2.6-38259-Linux-i686.tgz&ac=&wget=true&name=wget&typed=releases'

Then unpack and bin/splunk start --accept-license

It creates and starts a server on port 8000 which you can then access to use all the graphical features as well as an admin dashboard. The site is clean and well laid out. I look forward to going deeper into what this app can offer us.

I also installed it locally on my Mac and the steps are exactly the same. After starting up the Splunk server, simply point it to the directory where your log files live and Bob's your uncle.

See how it beautifully transforms log files into searchable fields with a graphical display? This will be extremely useful as we shift our machines around, play with the difference between VMs and hardware as well as put all our unittest machines up to Buildbot 0.7.7.

With only a few minutes on the dashboard I found it easy to navigate, add several input streams from the various build slaves that run on unittest-staging and also noticed that you can create and save specific search requests.

Can't wait to see how much this helps others, now off to install it on the other masters.

Robot War - Firefox 3 vs The Other Browser

2008-06-17T10:26:00.000-07:00

Just in time for Download Day, a little video produced and directed by Marcia Knous and edited by me. I'm posting it on several sites youTube, Flickr and blip.tv - please link to it, digg it, favourite it, whathaveyou - just pass it along.

What did I do today?

2008-06-11T17:19:00.001-07:00

Interning at Mozilla has so far provided me with many opportunities to learn (and re-learn) some of the finer points of the command line interface.

Today I have spent most of my time working on some scripts (both shell and python) that will assist in parsing error logs from the unittest builbot masters.

Here's how I would like it to work:

The script will live in the master directory and when run followed by the 1...n slave directory names it will create a folder with files containing all lines which match particular error messages that are in the stdio log files. Each file is specific to the pattern searched (eg: reftest, browser, check)

A python script then reads through each of the files and breaks the lines of error statements up into sqlite insert statements and throws them into a sqlite db

The DB will have a front end which will allow for easy searching and sorting of the data to see if there is a particular test that fails more frequently than others, on which platforms, etc...

This is an interesting side project for me as we wait for release to start bringing the unittest boxes up to Buildbot 0.7.7. I'm enjoying writing python (esp. compared to shell scripting).

Finally - many thanks to Chris Tyler for today's little success. I needed to put the time and date of the log file into the pattern matches so that the database can be more useful and I was having difficulty figuring it out, Chris is the expert at the one line solution which was ( >> means continued on next line):


for file in *-log*    # traverse all files in $DIRNAME
  do 
    grep -HnA 2 "$string1" $file >>
  | sed "s~^~|$(date -r $file '+$%D|%T|')~" >>
    > "$OUTPUT_FILE.reftest"
  done

What's new in Firefox 3? Check out this demo!

2008-06-06T15:59:00.001-07:00

A quick (< 4 minutes) overview of some of the new features in the soon-to-be-released Firefox 3. Check it out, then head over to http://www.spreadfirefox.com/worldrecord/ and sign up to be notified when the new version comes out!

read more | digg story

Parsing for errors in Buildbot log files

2008-06-05T16:27:00.001-07:00

The other day two of our Moz2 unittest buildbots - one Linux and one Windows - were both failing tests intermittently. We have all these logs but no way to parse the data to look for patterns and try to figure out what is going on. In an attempt to scratch the surface of this issue, I was tasked to look at the error messages and put together something for bug 435064.

It took a little while to come up with the right approach but in the end I had some grep statements that did the trick. Thinking that this will come up again, I packaged them into a little shell script to do the dirty work for me next time:


#!/bin/bash
# parseError.sh
# simple script for gathering up errors in log files

if [ -z "$1" ] 
then 
  echo "Usage: '$0' [directory]"
  exit 1
fi

string1="UNEXPECTED FAIL"
string2="ERROR FAIL"
string3="command timed out"
string4="FAIL"

DIRNAME=$1
OUTPUT_FILE="../output"

echo "Looking for UNEXPECTED, ERROR and command time outs..."
cd $DIRNAME
for file in *-log*    # traverse all log files in $DIRNAME
do 
  grep -Hn "$string1" $file >> $OUTPUT_FILE
  grep -Hn "$string2" $file >> $OUTPUT_FILE
  grep -Hn "$string3" $file >> $OUTPUT_FILE
done

# these two searches include 5 lines of context
echo "Looking for Check and Browser Fails...."
grep -HnC 5 $string4 *-log-check* >> $OUTPUT_FILE 
grep -HnC 5 $string4 *-log-browser* >> $OUTPUT_FILE

echo "Sorting......"
sort -n $OUTPUT_FILE | sed /--/d > "$OUTPUT_FILE.sorted"

echo "Search complete."
exit 0

Thanks to dchen and humph for helping with the finer points of writing shell scripts.

Bets on Canada going over 100,000

2008-05-30T09:54:00.001-07:00

Armen doesn't think that Canada will make it over the 100,000 mark - I would like to see him proven wrong, so please Canada - get on with the pledging!

This map is fascinating to watch and there's so many tangents you can follow with it. How amazing is it to see 14 pledges in Yemen? Why does Alaska get to be orange (50K + pledges) when you know that there are not that many Alaskans signed up for Download Day (prove me wrong). I think South Africa will be light blue soon, there's almost 1,000 pledges there.

It's a great distraction from buildbot...

Guiness Record for Most Downloads

2008-05-28T13:29:00.001-07:00

Want to help Mozilla set a World Record? Join in Download Day and pledge to grab a copy of Firefox 3 in the first 24hours of its release. By pledging, you'll get to see the number of pledgers in your country go up by one, and also you'll get a friendly reminder email on the day that FF3 is released.

Spread the word by putting an affiliate button on your site.

Learning advanced Bugzilla

2008-05-22T16:21:00.001-07:00

In our first year of Mozilla development at Seneca - we learned how to file basic bugs, how to upload patches and we followed a module owner or the like so we could see just how much bugmail a person can handle. Now that I'm a Build intern I am learning to use bugzilla on a few more levels.

This is just one (small) example:

These bugs will allow me to go through the process of creating, setting up and deploying 4 new VMs so that I'll be able to see if they can handle both build and unittest builds. In a perfect world, they will be able to do this and that means we will set up a pool of buildslaves for each platform and delegate work from both build and unittest as needed. The goal is to be scalable and to use as little hardware as possible.

The reason I have to test this is because it's possible that VMs cannot successfully run unittests as they are written now. We need to know that the unittests can and will run before choosing this method.

More on this soon. Back to bringing WinXP vms up to date with mozilla-build instead of ye olde Cygwin.

Vista Building with VC9

2008-05-20T16:06:00.001-07:00

If you've been banging your head trying to build Firefox on Vista with the newest Visual Studio 2008 express edition (VC9) - know that it is possible now, and with minimal bruising of your forehead.

Two things you need to remember:

1. You have to run start-msvc9 in mozilla-build as administrator (if you do not your builds will fail with messages about bad file numbers and such)

2. You have to put the following four lines in your .mozconfig (for now - keep your eye out for a new rev of mozilla-build which should fix - current at 1.2)

ac_add_options --disable-xpconnect-idispatch
ac_add_options --disable-activex
ac_add_options --disable-activex-scripting
ac_add_options --disable-accessibility

That's it - after bug 419665 is checked in, there should be no other obvious issues unless you forget to run as administrator or check out when the tree is burning.

Happy Vista building.

Clobbering buildbot run leaves no trace of history

2008-05-14T14:49:00.000-07:00

Sadly, in my excitement to get a Mac buildbot slave up and running yesterday, I have overwritten my profile with a lot of file:/// addresses in the awesomebar and little else. The fabulous part of searching in the awesomebar is so much less when you have NO history.

This is because the master.cfg is set up to clobber and thus wipes clean the default profile as it does it's thing. Not so bad for a VM unittesting machine, kind of terrible for a human with poor retention for URLS thanks to a dependence on the awesomebar.

/me writes on the board "Never use the default profile" * 1000

Week [1] - Learning to set up Buildbot

2008-05-12T15:14:00.000-07:00

Okay, it's time for another update as to my activities in MV.

By the time Robcee left last Friday to go back to the picturesque province of New Brunswick, I had come pretty close to having Buildbot installed and ready to be deployed on my CentOS VM.

Today in between wrestling with trying to build on Vista with VC9 (I must remember to "run as administrator" on the mozilla-build shell) I have been learning to deploy my own local buildbot master and a linux slave.

In getting to this:

there were a couple of roadblocks.

1. The current master.cfg and mozbuild.py are written for Builbot 0.7.5 and so I had to make a couple of small changes as per the documentation to account for import changes (no more step, using steps.shell or just steps instead) and also html.Waterfall is deprecated we should use html.WebStatus now.

2. It took a little while to figure out that the "force build" option that you can get when you click on a slave's name link in the waterfall is actually an option passed in to html.Waterfall as in html.Waterfall(http_port=2005, allowForce="true") and this gives you a nice little html form where you can force a build as you need one.

3. The last little glitch was just making sure that all the names matched up. The major lesson learned here was: Never follow directions. Just because someone says to do


buildbot create-slave slave localhost:9989 linux mozilla

doesn't mean that will work. You have to look beyond the literal instructions and realize that the slave name must match what is in auth.py and same with the password.

Next goal - learn how to kick unittest machines when they misbehave.

Wrestling with the CentOS ref platform and configurations

2008-05-07T17:29:00.000-07:00

Today was supposed to be "learn all about unit tests" day and instead it was "configure CentOS until you drop" day.

Here's what I was working with:
Ref Platform VM (CentOS-5.0)
Ref Platform set up instructions
Install scripts that are supposed to help make it all much easier

So the day looked a little like this:

* Wrestling with Python for about 2 hours - after much trial and error with the python path I ended up deleting my messed up ref platform vm, and starting up a clean one. The install scripts caused a clean VM with a python version of 2.5.1 to become a busted, can't finish make, 2.4.3 version. What?!

* Scrap the scripts, instead follow the instructions as above from the wiki. This brought up a couple of issues. First of all, all the versions on the wiki are a lot older than the ones in the install scripts. With a clean VM you first have to set a symbolic link to gcc-4.1.1 otherwise compiling zope-interface will not work:

ln -s /tools/gcc-4.1.1 /tools/gcc

Now the steps for installing zope-interface work fine. However there is an issue where every time a symbolic link is created, a symbolic link is then also created inside of the symbolic link as a link back to the original item. This is extremely confusing - I just deleted the broken links without looking too deeply into why that was happening. It happened with zope-interface and the twisted/twistedcore.

There are also a few spelling errors in the wiki for the PATH settings, I'll go in an change those soon.

Now that I am trying to write down what happened, all the glitches seem minor. I swear there was a lot of "Why is this happening?" and studying PATH, PYTHONPATH and PYTHONHOME to see if they were configured properly.

Probably the next step is to study those scripts and see how they can be more easily run on the VM anytime, anywhere because right now - they are hit and miss.

Getting set up on Day[0]

2008-05-05T15:00:00.000-07:00

First day of the internship, and things are going well. As my first build on the MBP is running in the background, let me lay out my set up so far. It will be familiar to many of you but I want a list for my own records.

Upon opening up the MBP with a clean install of Leopard, I am faced with the first task of downloading Firefox. What people choose to download is an interesting mix. While my supervisor John O'Duinn downloads the beta version when he needs a fresh build both Armen and myself leap headfirst into the latest trunk build, 3.0pre.

Next in line:
* Mozilla-VPN installer
* Tunnelblick
* Colloquy
* Quicksilver
* Chicken of the VNC
* Remote Desktop for Mac (beta)
* MacPorts
* XCode
* Thunderbird
* TextWrangler (a free Mac text editor that has command line integration - which is awesome!)
* VMware Fusion

I've done the build configuration and am just waiting to get my VPN set up properly. Everything here is a bug. As Sean from IT just told me - "If you think 'Should that be a bug?' that's probably a bug".

Speaking of bugs, I got assigned another small bite today, bug 432003, which is to enable source server on thunderbird windows builds. Two lines and a patch uploaded, nice to have such a minor bug to get back into the swing of things after my 2 week holiday from school.

Day[0] out.

Just for fun...

2008-05-04T08:13:00.001-07:00

Packed and (almost) ready...

2008-05-01T14:23:00.000-07:00

I've just finished attempt #3 to re-pack and prune what I'm bringing out west on Saturday. My bike is packed up in a box, I've booked an airport limo for 4:30 am and there are two 70lb bags of dog food in the basement to keep the hound in chow while I'm gone.

It's amazing to imagine 4 months living out of one suitcase. Kind of makes me want to donate all the rest of my clothes and shoes right now. I'll probably do a huge spring clean when I return and get rid of a ton of stuff.

It's been great to have Armen already out in Mountain View so that I can ping him and ask questions like "Is there an alarm clock in the apartment?". I need to know these things. Right now I'm praying there will be space in my carry on bag for a coffee grinder. Not because I drink coffee but because I used it to grind up flax, spices and oats for baking and smoothies.

There's not much going on in terms of open source work. The past couple of weeks have been filled with preparing to leave town, seeing friends and doing a bit of AMO editing. I'm happy with this past term I have to say - in the end I made the President's Honour list again. Now I just have to keep finding ways to turn that into potential funding for my last year. When I'm filling out applications it seems that the biggest factor is what kind of community and volunteer work you do.

Next year I'm going to volunteer with SOY where I would do 1 on 1 mentoring with a queer youth. It's hard to translate all the hours and effort that went into my time on the Trinity Square Video Board of Directors into funding applications and I've maxed out my 2 X 2 year terms. Mentoring a youth will probably look better and maybe I'll be matched up with a young'un who's into computers and/or film. The other options that interest me are to get involved with Sketch or Parkdale READ. I've been able to give at least 4 hours a month to TSV so hopefully I can make a similar comitment to the new organizations.

In other news, one of my films got into Frameline which is in San Francisco in June. Lucky for me, I'll be in the neighbourhood this summer and will be able to attend my screening. I was working on getting accredited as a distributor by representing my old workplace but as a filmmaker I will get treated much better and probably score some interesting shwag.

After the Images festival opening where G.B. Jone's feature premiered (I did a voice over for one of the main characters) I was inspired to make a feature on Super 8 film and I made a pact with two other filmmaker friends that we would each have a feature before ten years were up. It may be ambitious but my hope is that by the end of this summer I will have a working screenplay to start preparing a schooting schedule from. I'd prefer to have a feature finished in 3 years, not 10.

That's all the news fit to print right now. Time to take the dog out which I'm dreading because she ate 10 pull 'n peel licorices earlier. Ugh.

Final Demo of Source Server for Mozilla

2008-04-18T16:11:00.001-07:00

Click to Play

This is the demonstration I showed today for the final wrap-up of my DPS911 class - "Open Source Project".

The Source Server will be ready for public consumption

2008-04-11T17:56:00.000-07:00

Okay.

As I download VS Express so that I can test it in that environment, I've uploaded two patches in the continuing saga that is Source Server.

Here's what we've learned since my last post:

* cvs.exe that comes with mozilla-build has issues so it's necessary to point your path to a standalone version
* the tinderbox cvs_root uses private key access so we have to alter symbolstore.py to check for an environment variable of SRCSRV_ROOT which will be set in the tinder-config.pl file to the public :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot.
* the reason that %fnchdir% wasn't working in my VS was because the srcsrv.dll it was using was an earlier version. once I copied the version from WinDBG over to the VS devenv folder, it worked. Still don't know where %fnchdir% came from (googling it only turns up myself) but I am glad it's working now and hopefully VS2008 express comes with this newer version - otherwise the documentation will be instructing people to do all sorts of extra work
* the last piece of the puzzle - Why Doesn't the Code Show Up After Downloading? - turns out there's a little checkbox in the options for debugging - something about requiring the exact file match...uncheck that and VOILA! Source Server worked in Visual Studio!!!

This is awesome. I am so thrilled that when the two bugs are resolved, downloading a nightly and debugging it should be really really easy!

Time to update some screenshots :)

Tweaking locally - ftw

2008-04-09T18:27:00.000-07:00

So in order to get the current version of nightly builds to work for me, I had to add a srcsrv.ini file in the same place as the srcsrv.dll and devenv.exe - on my computer this is C:\Program Files\Microsoft Visual Studio 8\Common7\IDE

In that srcsrv.ini the only lines needed are:

[variables]

MYSERVER=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot

The reason for this is that anything in your local srcsrv.ini will override what's in the pdb file. In the current situation, the pdb files for the nightly debug builds have MYSERVER=:ext:ffxbld@cvs.mozilla.org:/cvsroot since that is what the computer they were built on used. That cvspath does not work without a key though, so it is no good to the average user of source indexed builds.

The second tweak is still making sure that the code is being located properly once it's downloaded. There is a way to change your registry to alter where Visual Studio looks for its cache of source code. I tried this and it worked but I'm not sure it was necessary. It's all still a bit of a grey area.

However, the silver lining? The patch does work - with the tweaks which are pretty simple - so source indexed builds exists and hopefully some people will start using them and familiarizing themselves with source server. In the long run, the more people using it, the more people who will be able to hack on it.

Thanks especially to this article which helped tonight as I tried to find the right combination of settings.

Now it's time to go write some things into the documentation.

Testing the indexed nightly

2008-04-09T16:13:00.000-07:00

So the fix worked and now the Mac/Linux |make buildsymbols| functionality is working again. As well, the nightly debug build from last night had source server indexing.

I downloaded the windows nightly to test it - in both WinDBG and Visual Studio 2005.

All looks good on VStudio - the symbols download and then when I try to break debugging I get this:
"This is great!" I think to myself and I happily hit "Run" several times to the various prompts for cvs commands.

Then...NO Source code can be found anywhere. Why? Well, it didn't download with that cvs command. See how it's got ":ext:ffxbld@cvs.mozilla.org:/cvsroot"? Well that doesn't get me any code, what it does get it timeouts and errors like this.

So now what? I was able to run the command with :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot and pull the code files to their default location - which by the way is not anywhere you would find it by accident. Here's what I found out about the location that SourceServer puts your code by default:

You may be wondering where Visual Studio puts the source server download cache. The default location is C:\Documents and Settings\Username\Local Settings\Application Data\SourceServer. This is great because with proper security in place only logged-in users can see that directory.

http://msdn2.microsoft.com/en-us/magazine/cc163563.aspx

So I ran the command in that directory and when VStudio is done asking me over and over again about running cvs commands - I do get the source file I checked out appearing in my VStudio solution.

What now? I don't know if this is something that needs to be changed in the mozilla code or if this is now about doing local changes with a srcsrv.ini file. Hopefully I will get some good advice about this soon so that I can demo it tomorrow without too much hacking and with a little bit of authority even :)

Linux headache...

2008-04-07T19:04:00.000-07:00

So my patch was backed out because it broke make buildsymbols on the Mac and Linux platforms. From looking closer at the error message and the patch, I deduce that it's GetVCSFilename containing the offending line - a return file where a tuple (two return values in python) should be returning, so I change it to return (file, None) and after testing the patch on a clean Mac build I am presented with the successful making of buildsymbols.

Now enter evil Linux (cue evil music)- the one platform I have yet to build on in any serious fashion. I should test this patch on Linux to make sure that it works there too. Here's the bullet form of what happened (as best I can remember at this point in head bashing on keyboard):

I ssh to 'Liberia', one of the Quad's running Fedora in the ORI lab
Check out, apply patch and build a debug build on it - fast! It takes ~3 mins to checkout and ~13 to build
Run make buildsymbols and I get "make: nothing to do for 'buildsymbols'" (thanks.)
Clobber build, tweak .mozconfig adding --enable-application=browser
Build - run make buildsymbols - still nothing to do for buildsymbols
After attempting to bike home, getting a flat tire from a massive pothole on Keele St and having to bus it I ssh into Liberia again and start from scratch. At this point both Armen and Dave start builds too, all of us using --enable-debug and other assorted .mozconfig settings
No one can make buildsymbols. Without the patch there are errors and with the patch - same errors.
In objdir/config/autoconf.mk I check and see that MOZ_CRASHREPORTER = 1 so make should be recognizing this.
I try a clean checkout and can't even build because of errors with dump_symbols.cc

Swimming in a sea of red herrings is not fun. Tomorrow's another day and another stab at getting this going. At this point I'm convinced that there's something wrong with locally building and not in fact with the patch since it does not touches dump_symbols.cc and since today's Linux symbols are up and fine (ted checked) which means that make buildsymbols does in fact work somewhere...just not for me (or Dave).

Build & Release - Part Two - more meetings...

2008-04-05T08:53:00.001-07:00

Continuing along with the education of future Build and Release engineers, Armen and I were introduced to automation with Robert Helmer. Rob's leaving Mozilla in a couple of weeks and it's too bad we won't get to work with him this summer. He's really got this all figured out and most of his presentation was way over my head. I said to Ben Hearsum later that it feels like once I'm actually touching the build system it will probably make more sense, but just talking about it? That's a bit confusing. In any case, Rob presented for about an hour on the automation process from past to present and then there was a lively discussion about what's coming next.

The next day Ben Hearsum talked about Try server and showed us the current set up. I'm really excited to learn more about Try server, and also to having permissions to use it myself. Considering that my recent patch had to be backed out for breaking buildsymbols on the Mac and Linux platforms the Try server could really help me out. I'll be talking with Ben about how I can get in on it. I have an LDAP account now...

Following Ben, Nick Thomas showed us the flow of releases from tinderbox to virus scanning to their final resting place on the ftp servers. In his diagram you can see pre Sept 2007, how it is currently and then on the right - what we're aiming for.

The last presentation of the day was Rob Campbell with his presentation on unit tests. Again we were treated to a thorough explanation of how things were done, how they're being done now and where things are heading.

The most exciting part of the week was realising that there is still a lot of room in Mozilla to have an impact on how things are done. On Wednesday morning we went for breakfast and Armen and I were lucky enough to be sitting next to Mike Schroepfer - VP of Engineering for Mozilla. He shared a lot of information with us and also pointed out how every single person at the table (the Build, Release and Automation team) has contributed significantly. That Mozilla has been able to grow as quickly as it has without falling down is a testament to all their hard work (and that of many others) and I am inspired to be part of that.

Thanks to Dave and to John for helping organize this opportunity. I think that my transition to interning this summer will be much smoother as I have met many of the people with whom I'll be working, plus the overviews of each area of Build were invaluable.

Build & Release - Learning about Talos

2008-04-01T12:15:00.000-07:00

Armen and I are in California attending Build and Release team meetings this week. Over the next two days we'll be introduced to the many facets of the Build and Release workflow.

Todays first session was about Talos with Alice.

Here is the diagram of Talos (copied from the diagram Alice drew - yes Talos is a robot):

Alice walked us through how Talos gets its information from browser builds by having buildbot read for new builds from quickparse which is a text file. Buildbot has a script that knows what to look for in order to find new builds and there is a 5 minute delay before Talos is deployed because the information can get into quickparse before the build is finished and so therefore does not technically exist.

Currently there are 30 Production and 20 Stage Talos machines running, this past December there was only 1 Production machine and the stage machines.

This huge increase of Talos machines has led to an insane amount of data being gathered and a database which is in serious need of some help.

After Alice's presentation we all tried a standalone Talos so we could see the tests at work.

Anyone can try them, just follow these directions. If you are using a recent nightly you might need to add security.fileuri.strict_origin_policy : false to your sample.config file preferences because of new security features. Also, you can comment out the tjss tests because those are kind of boring - the fun test is the svg since you'll see a lot of graphics tests running on your browser. This standalone runs on a new profile so it's okay if you already have Firefox running when you run this script.

So the information that's generated is good for recognizing regression like in this bug, where if you look at the graph you can see how the build was chugging along, something got checked in that affected performance and then it was backed out and the performance went back to normal.

Pic here, see bug #425941 for more info:

More information on Talos Machines.

Source Server Tweaks - Now with refactoring and a clean_root function

2008-03-28T18:39:00.001-07:00

Things to remember:
1. When you post your patch you select review ? and not review + (I had been confused about why I couldn't put my reviewer's email address in next to the +)
2. When your previously working code stops working suddenly and print statements galore are not helping, and you know what part isn't working but not why...Stop poking at the code and go to MXR -- Thank you MXR for helping me catch (through a line by line comparison) that I had accidentally deleted a key line thinking it was my own addition. I think this calls for an editor that does coloring on text not for syntax but for diff'd text.

Tomorrow should be a big day. I'll be in a little box office at the Royal Cinema all day and night working for Cinéfranco - if anyone wants to see a French movie, I can hook you up - and I'll be anxiously awaiting a review result because this is it people, these are the tweaks that should net me a test version of a nightly build.

Fingers crossed that one particular Build guy will be working on Saturday...

Some of my favourite things

2008-03-23T08:44:00.000-07:00

I'm still learning how to get the most out of my system set-up. Since I've recently become an Editor for AMO, I've started a list of add-ons that are of interest or could be useful to me. Here's the list so far, in no particular order:

GTDinbox (gmail functionality booster)

GButts (google services buttons - and a handy reminder to blog more often)

Firefox Accessibility Extension

Regular Expressions tester

Web Developer Tools - Finally updated for Firefox 3!

BugMeNot (sign into sites with user submitted passord/username)

That last one, BugMeNot, really impressed me because it's such a great idea. I complain all the time about how many times I need to create a user account with a site I may never visit again, or a site to whom I do not want to provide my information. BugMeNot is a simple idea that has a huge impact. I get to share the username/password of "nobody" with everybody.

As well as add-ons, I've installed Quicksilver
so that I can do more from the keyboard. I don't know why I never looked into this before but apparently you can go to the spotlight search with command + space...nice to know. Quicksilver is spotlight on steroids though, and I'm looking forward to exploring the capabilities. The only glitch I had on starting up was that the hotkey wasn't working how I thought it should. In the end, I set my preference for hotkey to modifier activation only, single, control. To pull up Quicksilver I just tap the control key twice. Perfect.

The last thing is BBEdit - I'm still trying to find a text editor that I feel comfortable with. For some reason XCode scares me...perhaps this summer I will try it again. My favourite editor on the PC was Crimson, which sadly has no Mac support. BBEdit looks like it's got everything I liked about Crimson, namely the ability to work on files over FTP/SFTP. This morning when I was working on a patch for tinder-config.pl though, it did some weird things to line breaks. So I'll have to check the preferences more closely before I use it for another patch.

Time to go fix that tinder-config.pl patch.

Patch updated - now with double slashes

2008-03-23T08:40:00.000-07:00

On the edge of my seat as I wait to see if my patch to tinder-config.pl will be checked in and a source indexed build will come down the pipe. I just tweaked the one line addition with double \\ in the file path. Small things hold up big things...

Soon I will be testing the nightlies, grabbing screenshots and putting together my demo/docs.

Crashing Into Open Source (without a paddle)

Growing the company, structuring volunteerism: My response to David Eave's community lifecyle audit

OccupediA - Women Contributing to Wikipedia (the first of many such events)

Want more women in Open Source? Donate to Ada Initiative today!

My First Startup Weekend: Women 2.0 Startup Weekend

New to IRC? Never tried it?

Where Are My $project-branch Nightly Builds?

Want to help? Encouraging community contributions to Release Engineering

Mozilla Seeks Program Manager for Open Web Innovation Incubator

Try results to the bug(s) of your choice upon completion

A quick morning rant about "gender" and data collection

Tree Closing Downtime Notice - 4am - 8am PDT Thursday June 16, 2011

Thoughts on cultivating an "Everyone is Remote" attitude

Use Try? Read this.

Update on the Auto/Assisted Landing System

Assisted/Automated Landing - Designing the Systems

A PyStar Supernova in the Sky

Captain Destructo Breaks Everything

Hey BBC would you like to know how releasing software works?

Bay Area Video Coalition - Teaching Open Video Part 1

Volunteers needed for upcoming HTML5/Open Video tutorial

Automated Try Results Posted to Bugzilla - A request for input on what the comment should contain

Try Server Road Map - Q1 2010

Please use TryChooser

Impending TryChooser change you should know about

Playing with Audio Visualization

Get more green with TryChooser!

Try What You Want

No really, it *is* easy to test your Python code

Disposable Project Branches - aka Twigs

Tryserver 2.0 - Fine tuning and learning the hard way

Tryserver 2.0 is Live

I'm much less interesting now...thanks to Facebook.

My name is Lukas, I live on the Second Floor.

I got tagged - Open Source Contributors blog meme

Improving 1:1s

Being a decision maker - Part Two

Being a decision maker - Part One

Browser Choice Screen slight of hand?

FOSDEM 2010 Video - Women in Open Source and Free Software

FOSDEM 2010 Reflections - Part One: Why I went

So you want a new Talos suite, eh?

Upcoming improvements to Talos documentation and test suite creation

Mozilla Service Week - Toronto Event

Adding choices to Try Server web interface

New Branch Timeline: Electrolysis

New Branch Timeline: Places

Celebrating Firefox 3.5 with sparkly accessories...

Mozilla Service Week - making a difference in your community

iPhone tethering - how to restore after updating to 3.0b5 and iTunes8.2b10(13)

Tryserver now has unittests on all 3 platforms

No More Passwords Please

There are no new ideas...

Unittests on Try Server - Linux HG Builds is Go!

Learning from Lab 1 in Mac OS and iPhone development

Holiday boredom leads to memes...

Unittest Consolidation and You

Dubai and Dashboards

Seneca and Mozilla in the news

The TTC on the move...

Ready, Set, FSOSS

Third time's a charm - Unittest Production moves tomorrow morning

Postponed - Production MozCentral Unittest Moves - to the Build Network

Take 2 - mozilla-central unittest production waterfalls moves to build network

Postponed - Production MozCentral Unittest Moves - to the Build Network

Production MozCentral Unittest Moves - to the Build Network

Continuing saga of the 1.9 Unittest Move

Update on the Unittest 1.9 move

Welcome to Build, Ben says

Scheduled Downtime Tues Aug 12 - 8:00 am PDT for Unittest network switch

Looking for suggestions on dealing with lots of data

Grovelling isn't so bad

Discussing Data

Set the VNC Password for Mac's Remote Desktop in Terminal

Chasing rainbows is easier

sha1sum on Mac OSX

Splunk - Where IT's at

Robot War - Firefox 3 vs The Other Browser

What did I do today?

What's new in Firefox 3? Check out this demo!

No really, it is easy to test your Python code