Well, on this my 50th post, I am excited to announce that I have been offered a summer internship with Mozilla.
I'm just waiting for the actual paperwork to set the wheels in motion. This is going to be one heck of a ride. I'm kind of hoping that once I get this Windows source server up and running, it might be possible to do some hacking on gdb and learn how to get it doing the same thing. Then anyone could debug mozilla without having to build!
When I first became involved with the Open Source class and Mozilla I never would have imagined that I would become involved at this level of the code, but I wouldn't trade the learning I have undergone for anything.
Talking with Ted the other day about the pdbstr.exe replacement and how it will be written in C++. I'm super rusty on my C++ so that will be challenging. I wonder how the pdb files are written to and read from? This will be the focus of my next week's work while I await feedback on the bugfixes that I've started. Everyone's busy locking down Firefox 3, so I may have to wait a bit.
Thursday, January 31, 2008
Monday, January 28, 2008
The Source Server Janitor
That's me, cleaning up the symbolstore.py patch one little scrap of python at a time.
Today's lesson was a brief introduction to String Interpolation.
In the bug, Ted had mentioned using string interpolation as one of the ways I could shrink down how many times I was writing to file. There are a couple of ways to do this, and apparently the best way, when there is only one variable to replace, is to do %s in the string and then a % variableName at the end of the string. Once I had done this however, 'make buidsymbols' was complaining that a float argument was required.
As it turns out, there are many places in the stream I am writing where % is followed by another character, and the first is a %f...so it was never getting to my %s way down the line. This is because Source Server does string interpolation as well and so I guess that means that I can't use the python string interpolation after all.
Unless someone has dealt with this before and has any brilliant ideas to share.
I've spoken with Ted on IRC and he will be looking at my revisions this week when ff3 beta freezes (man, they must be pulling out hair out in california) and so I'm looking forward to taking the next step. We spoke briefly tonight about the pdbstr.exe replacement. I wasn't sure what it should look like. Does it have to be a windows binary? Apparently, yes. That means some C++. Woot.
Today's lesson was a brief introduction to String Interpolation.
In the bug, Ted had mentioned using string interpolation as one of the ways I could shrink down how many times I was writing to file. There are a couple of ways to do this, and apparently the best way, when there is only one variable to replace, is to do %s in the string and then a % variableName at the end of the string. Once I had done this however, 'make buidsymbols' was complaining that a float argument was required.
As it turns out, there are many places in the stream I am writing where % is followed by another character, and the first is a %f...so it was never getting to my %s way down the line. This is because Source Server does string interpolation as well and so I guess that means that I can't use the python string interpolation after all.
Unless someone has dealt with this before and has any brilliant ideas to share.
I've spoken with Ted on IRC and he will be looking at my revisions this week when ff3 beta freezes (man, they must be pulling out hair out in california) and so I'm looking forward to taking the next step. We spoke briefly tonight about the pdbstr.exe replacement. I wasn't sure what it should look like. Does it have to be a windows binary? Apparently, yes. That means some C++. Woot.
Monday, January 21, 2008
Just a blip...
Well, I've broken the ice that formed on my project over the holidays. There's something that really intimidated me about going back to that code, that work flow. I don't know how to explain it except that I felt afraid of my own code and lack of experience, felt like I wasn't going to remember anything. So I hemmed and hawed but tonight I got back into it for a solid 3 hours and it feels good to remember what I was doing and why.
One small change that has made my life easier - I added srctool.exe to my path so I could call it from anywhere and this makes life good because when I want to test if my code is working I have to call make on the new symbolstore.py, then call make buildsymbols in my objdir, then go into the command window and call srctool.exe -r (a raw dump of the indexing info in a pdb) to see if it all worked. The path to a pdb file is something like C:\ff_clean\mozilla\objdir_debugInfo\dist\crashreporter-symbols\2007120422\accessiblemarshal.pdb\E82E8047412045539226FBBC0BE301974\accessiblemarshal.pdb
and so being able to go to that directory and just call srctool -r accessiblemarshal.pdb is way easier. So I'm happy about that.
I've submitted my changes so far back to the bug and am now awaiting feedback. If I can successfully handle at least one of the points that led to a - review, then I am moving forward.
Now it's time for some (light) reading about windows proprietary debug database formats.
One small change that has made my life easier - I added srctool.exe to my path so I could call it from anywhere and this makes life good because when I want to test if my code is working I have to call make on the new symbolstore.py, then call make buildsymbols in my objdir, then go into the command window and call srctool.exe -r (a raw dump of the indexing info in a pdb) to see if it all worked. The path to a pdb file is something like C:\ff_clean\mozilla\objdir_debugInfo\dist\crashreporter-symbols\2007120422\accessiblemarshal.pdb\E82E8047412045539226FBBC0BE301974\accessiblemarshal.pdb
and so being able to go to that directory and just call srctool -r accessiblemarshal.pdb is way easier. So I'm happy about that.
I've submitted my changes so far back to the bug and am now awaiting feedback. If I can successfully handle at least one of the points that led to a - review, then I am moving forward.
Now it's time for some (light) reading about windows proprietary debug database formats.
Friday, January 18, 2008
0.4 Release
This release is primarily research. After talking with Ted (ted), I have some documentation about pdb file formats and pdb "exploding" tools to play with. Also I am currently trying to make my laptop more friendly to the kind of development this project requires. In the last term I started using a MacBook mid-semester and was able to work with a Windows XP VM for the testing of my patch. This was a cumbersome task because I did not have (nor could I afford) the RAM upgrade which would make VM usage much less laggy. Yesterday I finally picked up the 2 X 2GB chips that have made my VM usage much speedier.
The bug I filed (bug 408134) will be my guideline and the goal of this project is to pass super review and have my patch (and whatever tools are needed to make it work) checked into mozilla trunk. In accordance with the specs of this class, here is an outline for how I will accomplish a 1.0 release of my project:
* All the syntax and minor logic issues will be cleared up first (by the next release), this means that I will save a copy of the filename variable right before the call to GetVCSInfo, the large block of text for the pdbstream will be rewritten as a triple-quoted string, and I will also figure out what portions of the pdbstream are unnecessary and remove them to streamline the source indexing
* The source indexing will be pulled out into a method called SourceIndex and will be located in Dumper, called from symbolstore.py
* Changes need to be made so that the source indexing is called from the command line , since it cannot (and should not) be called upon in anything other than Win32 platforms
* All hard coding will be cleared out
* A replacement program for pdbstr.exe (which is provided by Microsoft) will be written so that it can be distributable
The resources I am researching on this release were provided to me by Ted and are:
* http://www.jorgon.freeserve.co.uk/Other/pdb.htm
* http://www.rawol.com/?topic=41
* http://www.rawol.com/download/zip/win_pdbx.zip
The bug I filed (bug 408134) will be my guideline and the goal of this project is to pass super review and have my patch (and whatever tools are needed to make it work) checked into mozilla trunk. In accordance with the specs of this class, here is an outline for how I will accomplish a 1.0 release of my project:
* All the syntax and minor logic issues will be cleared up first (by the next release), this means that I will save a copy of the filename variable right before the call to GetVCSInfo, the large block of text for the pdbstream will be rewritten as a triple-quoted string, and I will also figure out what portions of the pdbstream are unnecessary and remove them to streamline the source indexing
* The source indexing will be pulled out into a method called SourceIndex and will be located in Dumper, called from symbolstore.py
* Changes need to be made so that the source indexing is called from the command line , since it cannot (and should not) be called upon in anything other than Win32 platforms
* All hard coding will be cleared out
* A replacement program for pdbstr.exe (which is provided by Microsoft) will be written so that it can be distributable
The resources I am researching on this release were provided to me by Ted and are:
* http://www.jorgon.freeserve.co.uk/Other/pdb.htm
* http://www.rawol.com/?topic=41
* http://www.rawol.com/download/zip/win_pdbx.zip
Thursday, January 10, 2008
Starting up again with Source Server
And, we're back for another semester of poking and prodding the Microsoft Source Server for use with the Mozilla debug symbols.
First order of business is to clean up as much of the symbolstore.py patch as possible and then the next step would be to start writing a pdbstr.exe replacement that can be distributed. Also, there's the issue of Mozilla 2 and mercurial...
Looks like there was an involved process in choosing the next generation VCS for Mozilla 2 and so now I am wondering whether I should be continuing to work on my 0.3 or start working on something that will make source server work with mercurial.
This week I'll be trying to pull together a solid list of what I hope to accomplish this term for my 1.0 release, i'll post more soon.
First order of business is to clean up as much of the symbolstore.py patch as possible and then the next step would be to start writing a pdbstr.exe replacement that can be distributed. Also, there's the issue of Mozilla 2 and mercurial...
Looks like there was an involved process in choosing the next generation VCS for Mozilla 2 and so now I am wondering whether I should be continuing to work on my 0.3 or start working on something that will make source server work with mercurial.
This week I'll be trying to pull together a solid list of what I hope to accomplish this term for my 1.0 release, i'll post more soon.
Quick moment of "wow"
Thanks to http://yichao.zhang.googlepages.com/blogsync-with-gui.tar.gz - I have now crossed over to blogger. I already had several blogs because of videoblogging and probably should have just started here in the first place, I like that I can customize my blog a little more. Anyway, hopefully I can get back on the planet now.
Monday, January 7, 2008
Open Source code reading/writing
Today was the first class of my BTP600 Design Patterns class. The teacher talked about how there are 3 ways to go about learning to write larger, non-trivial programs.
I like this class. It's very hard to see how I will ever write something large and non-trivial but I know that's what we will accomplish by the end of BTS and this course as well. When I look at the mozilla code base, I can't see how something like this came to be. What do you write first?
The teacher mentioned that Kernighan (sp?) and Richie write really great code and so I'd like to read some. Anyone know of a good example? I welcome tips on code that you may have found exciting to read.
- Try + Fail + Fix
- Read the code of other (superstar) coders to see how they did it well
- Design Patterns
I like this class. It's very hard to see how I will ever write something large and non-trivial but I know that's what we will accomplish by the end of BTS and this course as well. When I look at the mozilla code base, I can't see how something like this came to be. What do you write first?
The teacher mentioned that Kernighan (sp?) and Richie write really great code and so I'd like to read some. Anyone know of a good example? I welcome tips on code that you may have found exciting to read.
IRC Chat purpose and atmosphere
Something that I have come to appreciate about Mozilla is that its chat channels on IRC are incredibly positive, helpful and relevant. As there are so many areas in developing and using Mozilla projects, several channels exist that allow you to join a channel when it is specific to your area of discussion.
Recently I went to the Joomla channel because I am working on a project for a friend in Joomla. Now, this is my first time working with Joomla. I have attended a brief "Hands on" workshop at the Ontario Linux Conference and I've read some of Dan Rahmel's "Professional Joomla". I was having trouble understanding the way that Joomla uses plug-ins. Basically there's a plug-in that can recognize a snippet of custom code in an article - ie: {images view|gallery} or something like that. What I wanted to do was to be able to put that code in a module so that I could position it on the page in the way that modules can be positioned. I can't find a way to position articles.
So I go to the Joomla channel in the hopes that talking with real person will help - much as going to the mozilla channels has helped in the past. I know that the #seneca channel is special because it is targeted at students who are learning, but the Joomla channel said at the top "Be patient, as your question - if no answer, try giving more info" (paraphrasing). I assumed that I could ask my question and get at least one person to talk it out with me.
Wrong.
What I got instead was a big fat nothing. NO response. NO acknowledgement. I asked politely if I should provide more information, and someone (who had been talking away about someone being gay) said "Oh, i didn't see the question". So I rephrase it and ask again - then nothing again. What?!
In frustration, I ask "is anyone here knowledgeable about plugins?" and a half hour later, at which point I had moved on to other tasks since there was no reply yet again, I get a private message from someone who says "ya, what you want?". I didn't bother responding. I created an html workaround to what I wanted to accomplish and moved on.
Needless to say, this experience was disheartening for a moment. But then I saw the positive side. I am a small part of a channel or two on the mozilla irc that is friendly, helpful and welcoming. So that's nice to notice and appreciate.
Recently I went to the Joomla channel because I am working on a project for a friend in Joomla. Now, this is my first time working with Joomla. I have attended a brief "Hands on" workshop at the Ontario Linux Conference and I've read some of Dan Rahmel's "Professional Joomla". I was having trouble understanding the way that Joomla uses plug-ins. Basically there's a plug-in that can recognize a snippet of custom code in an article - ie: {images view|gallery} or something like that. What I wanted to do was to be able to put that code in a module so that I could position it on the page in the way that modules can be positioned. I can't find a way to position articles.
So I go to the Joomla channel in the hopes that talking with real person will help - much as going to the mozilla channels has helped in the past. I know that the #seneca channel is special because it is targeted at students who are learning, but the Joomla channel said at the top "Be patient, as your question - if no answer, try giving more info" (paraphrasing). I assumed that I could ask my question and get at least one person to talk it out with me.
Wrong.
What I got instead was a big fat nothing. NO response. NO acknowledgement. I asked politely if I should provide more information, and someone (who had been talking away about someone being gay) said "Oh, i didn't see the question". So I rephrase it and ask again - then nothing again. What?!
In frustration, I ask "is anyone here knowledgeable about plugins?" and a half hour later, at which point I had moved on to other tasks since there was no reply yet again, I get a private message from someone who says "ya, what you want?". I didn't bother responding. I created an html workaround to what I wanted to accomplish and moved on.
Needless to say, this experience was disheartening for a moment. But then I saw the positive side. I am a small part of a channel or two on the mozilla irc that is friendly, helpful and welcoming. So that's nice to notice and appreciate.
Thursday, January 3, 2008
New Year, New To-Do list
Today was the first day back in the hallowed halls of S@Y. I managed to clear up the conflict signing up for DPS911 and am now officially registered.
I've gotten some great feedback on the bug I filed for the source server so there's lots to get started on. First rewriting the patch better and then the grand task of writing a new pdbstr.exe that we can distribute because the Microsoft version cannot do so.
Anyway, that's the short, glossy version.
I'm looking forward to getting back in the game. As the last few days of the holiday whip by I'm also working on some websites for friends, training to be a tutor in the Learning Center, helping some friends paint and trying to finish another pair of socks.
Yesterday at a New Year's brunch I spent some time with a friend who has recently lost a large amount of her vision. She is almost (but not quite) legally blind and her vision will continue to deteriorate at an unknown rate. She's a visual artist as well as a teacher and we had a lively discussion about software and adaptive technology. I'm determined to find a way to develop simple, well designed, as open source as possible programs for the Mac. This is a challenge since a) I have been informed that there is no money in writing adaptive software and b) apple development is a mystery to me and c) I'm not sure if it's even possible to write open source programs for Apple.
Learning about this will be the sidebar on the to-do list for the winter you know, in my spare time :)
I've gotten some great feedback on the bug I filed for the source server so there's lots to get started on. First rewriting the patch better and then the grand task of writing a new pdbstr.exe that we can distribute because the Microsoft version cannot do so.
Anyway, that's the short, glossy version.
I'm looking forward to getting back in the game. As the last few days of the holiday whip by I'm also working on some websites for friends, training to be a tutor in the Learning Center, helping some friends paint and trying to finish another pair of socks.
Yesterday at a New Year's brunch I spent some time with a friend who has recently lost a large amount of her vision. She is almost (but not quite) legally blind and her vision will continue to deteriorate at an unknown rate. She's a visual artist as well as a teacher and we had a lively discussion about software and adaptive technology. I'm determined to find a way to develop simple, well designed, as open source as possible programs for the Mac. This is a challenge since a) I have been informed that there is no money in writing adaptive software and b) apple development is a mystery to me and c) I'm not sure if it's even possible to write open source programs for Apple.
Learning about this will be the sidebar on the to-do list for the winter you know, in my spare time :)
Subscribe to:
Posts (Atom)