A day that shall go down in infamy...

Today at 6pm I have finished the necessary changes to my Source Server patch so that it is ready to be checked in. This will be my first check in to the Mozilla code and I'm pretty psyched that after about 6 months of working on this, this particular chapter is about to close.

The last few changes today were quite interesting and good learning for me. Basically, I had to take my | if self.srcsrv | stuff and put it into a separate method. Ted pointed out that this was because it's win32 specific code, so there's no need for it to be in the general Dumper class.

Hello object-oriented programming? This is exactly what I need to learn this in an applied manner and I will remember this lesson better than having just read it in a book.

So now there's a declared SourceServerIndexing method in the Dumper, and then a detailed method of the same name in the Win32 Dumper - which is where all the action is. I wonder if there's a chapter somewhere down the line where I will try to take this SourceIndexing into the Linux and Darwin dumpers.

When I first pulled everything out into the method, I had to wrap my head around the way that Python calls methods. Basically, if I called it from within Dumper, I need to do self.MethodName so that it looks for the method inside the Dumper class. Without the self. in front, I got an error message saying that a global name "MethodName" was not recognized. Also, when I was running | make buildsymbols | I wasn't source indexing the pdbs. So I put in some print statements to see if all the right steps were happening. They were. Confusing.

Eventually I noticed that there's a |shutil.copyfile(file, full_path)| which is what puts the pdb files in the place that I want them to be for indexing and I was calling my method before that. Quick switch and the lights turned on, the music played and my pdb files were indexed.

This has been a great project for teaching me how to set up a work flow as efficiently as possible. Now I'm at the point where I can get right into working on the code and testing frequently with minimal effort. The closer I got towards getting it all working, the easier the work flow got. Right now I can test this patch with just MingW32, a command shell and my Crimson Editor. I can tell within the first few seconds of |make buildsymbols| running if it has worked and then a ctrl + C to interrupt it and get back to the tweaking of code.

It feels great to have put this up for approval. Can't wait to see what happens next.

Temporary Geek Home

Went to the Toronto Mozilla office today. Spent the afternoon reviewing add-ons and I also got to put some faces to names. Since last Friday when we got training from Alex Polvi to be editors for AMO, I've reviewed 14 extensions and I'm learning a lot about the process.

Ted said he'd be reviewing my patch tomorrow and we're hoping to land it in the next few days. As far as I can tell right now, the only issue should be that my error handling is currently just printed to stderr and doesn't actually change the srcsrv flag.

So fingers crossed - there might be source server on the debug builds by the end of the week. This means the rest of term can be devoted to pdbstr.exe. I have yet to communicate with Timeless who is the recommended contact to discuss things of this ilk.

Over reading week I should be able to do some more digging into those hex dumps as well as start testing the source server once it's on Tinderbox.

Keeping busy

Well as reading week approaches and I wait (again) for feedback on my patch, I am trying to keep busy in Mozilla activities.

One of my recent activities is helping out with Live Chat user support. It's amazing! A very small team of dedicated people are helping individuals in real time with their wide variety of issues. I've been learning tons about the issues that users face - right now firewalls and security updates seem to be a frequent problem. I'm also jumping headlong into another thread of the vast Mozilla community, this one mostly volunteers, and getting to know some folks. The EST hours for Live Chat are from 4pm - 9pm and from 10pm - 12am. I mostly have been able to pitch in on the later shift because that's about when I am tired of doing my homework :)

Getting involved with Live Chat was super easy - I read the documentation, created and account and was shadowing experienced helpers less than an hour later. One week later and I'm in on a phone conference to discuss what the priorities are for the next 3 months. Again, I say amazing.

I encourage all Mozilla or open source involved Seneca students to jump into Live Chat and try helping out people in real time. It's intense, and can be challenging (I think my success rate is only about 50%) but it's an eye-opening experience for how the other half lives. The half what don't have #seneca for all their questions.

Better late than never

Although my goal was to have the symbolstore.py patch approved for committal by week's end, I am very close. Today I spent about 5 hours tweaking the patch as per the most recent comments.

What was done:

* Makefile now checks for the environment variable of PDBSTR_PATH to be set before assigning the flag to source index
* Several minor logic changes to the symbolstore.py like removing redundant file path fixing
* I finally understood what Ted was saying about using a temp filename before checking for VCSinfo. This small change made it possible to remove 3 lines of work-arounds that I had been doing :)
* The hard-coding of the path to the pdbstr.exe executable is gone and has been replaced by whatever is in the environment variable

A few glitches:

* Making changes to the Makefile that resulted in errors left me with a makefile that couldn't update itself and Ted pointed me to doing a make -f client.mk configure which regenerated the Makefiles and cleared that up
* I had to look into how to get the environment variable in python - thanks to web search I found the os.environ.get() function

Other than that, it's all working and I'm very happy to be awaiting another review.

Smart Install Instructions

Today as I was setting up a mysql GUI I saw this:


It made me very happy to see that part about Ejecting and ridding yourself of the installer because a lot of folks still might not know that. I've definitely met some people who drag firefox to their dock from the installer instead of copying it to their Applications folder. Every time they open the application from the dock, the installer has to mount and someday down the line they might delete it and then not know why the application no longer launches from the dock. OS X just gives the ever so helpful "?" and the user is left to wonder what happened. That far down the line - it's not so apparent that they should have dragged to the Application folder and ejected the installer.

Most Mac installers these days do make it quite clear that you should drag to your install folder and I would love to see them add this tip too - just to help folks finish what they start.

Upcoming Source Server Demos

This is mostly a reminder for me, but if anyone is interested in seeing how the Source Server is working/coming along, my demos are scheduled for:

• Thursday March 13th at 1:30pm in ORI


• Thursday April 3rd at 1:30pm in ORI

The first demo should involve the symbolstore.py patch having been committed, there should be a debug build that can be downloaded and attached to VStudio and/or WinDBG where the source code is then pulled via the Source Server.

The second demo will hopefully be a look at a reverse engineered pdbstr.exe - something that can read and write to pdb files the source indexing data block.

Now on Windows

Using StraceNT I was able to get this output.

Here's a snippet:

IntellectualHeaven (R) System Call Tracer for NT, 2K, XP, 2K3.
Copyright (C) Pankaj Garg. All rights reserved.

Tracing command: ["pdbstr" -r -p:accessiblemarshal.pdb -i:am3.stream -s:srcsrv]
[T3600] TlsGetValue(1, 0, 2bfef8, 182020, ...) = 2c7778
[T3600] EnterCriticalSection(77c61b30, 2c7778, 2bfed0, 77c3a03b, ...) = 0
[T3600] LeaveCriticalSection(77c61b30, 2bfed0, 77c3a0fa, d, ...) = 0
[T3600] EnterCriticalSection(77c61b18, 2c7778, 2bfed0, 77c3a06c, ...) = 0
[T3600] LeaveCriticalSection(77c61b18, 2bfed0, 77c3a108, c, ...) = 0
[T3600] HeapFree(2c0000, 0, 2c7778, 0, ...) = 1
[T3600] TlsSetValue(1, 0, 0, 2bfef8, ...) = 1
[T2556] LeaveCriticalSection(2c1fdc, 6f1c0, 77c2d154, 4, ...) = 0
[T2556] LeaveCriticalSection(2c7718, 6f1d8, 77c3b967, 13, ...) = 0
[T2556] HeapFree(2c0000, 0, 2caa48, 1058d24, ...) = 1

The results are certainly a bit clearer looking than the Linux/Wine results. I am still clueless however to the deeper meaning. I've been told I need to talk to timeless on IRC, that he is the one with major knowledge on reverse engineering.

This is turning into quite the rabbit hole.