Linux headache...

So my patch was backed out because it broke make buildsymbols on the Mac and Linux platforms. From looking closer at the error message and the patch, I deduce that it's GetVCSFilename containing the offending line - a return file where a tuple (two return values in python) should be returning, so I change it to return (file, None) and after testing the patch on a clean Mac build I am presented with the successful making of buildsymbols.

Now enter evil Linux (cue evil music)- the one platform I have yet to build on in any serious fashion. I should test this patch on Linux to make sure that it works there too. Here's the bullet form of what happened (as best I can remember at this point in head bashing on keyboard):

1. I ssh to 'Liberia', one of the Quad's running Fedora in the ORI lab
2. Check out, apply patch and build a debug build on it - fast! It takes ~3 mins to checkout and ~13 to build
3. Run make buildsymbols and I get "make: nothing to do for 'buildsymbols'" (thanks.)
4. Clobber build, tweak .mozconfig adding --enable-application=browser
5. Build - run make buildsymbols - still nothing to do for buildsymbols
6. After attempting to bike home, getting a flat tire from a massive pothole on Keele St and having to bus it I ssh into Liberia again and start from scratch. At this point both Armen and Dave start builds too, all of us using --enable-debug and other assorted .mozconfig settings
7. No one can make buildsymbols. Without the patch there are errors and with the patch - same errors.
8. In objdir/config/autoconf.mk I check and see that MOZ_CRASHREPORTER = 1 so make should be recognizing this.
   
9. I try a clean checkout and can't even build because of errors with dump_symbols.cc

Swimming in a sea of red herrings is not fun. Tomorrow's another day and another stab at getting this going. At this point I'm convinced that there's something wrong with locally building and not in fact with the patch since it does not touches dump_symbols.cc and since today's Linux symbols are up and fine (ted checked) which means that make buildsymbols does in fact work somewhere...just not for me (or Dave).

Moving pdbstr.exe into the mozilla environment

Working on 0.3 release of source server is all about getting the pdb files gathered up when symbolstore.py is made and then feeding them to pdbstr.exe. I've copied the binary into the mozilla/toolkit/crashreporter/tools folder where symbolstore.py is called and I've spent the last 4 hours or so playing around with python.

So far I have learned this:

1. THIS IS WRONG, PLEASE IGNORE That I can call pdbstr.exe on an unindexed pdb file and it will in fact read and generate a data block that has all the information needed to set up the cvs cmd calls - see this output from calling pdbstr -r -p:thepdbfile -s:srcsrv > thepdbfile.stream ** for corrections see next post **

2. The next part would be to write this block back to the pdb file i think - so that it's available to the source server

This is what calling pdbstr -r gets me:

SRCSRV: ini ------------------------------------------------
VERSION=2
INDEXVERSION=2
VERCTRL=http
DATETIME=Tue Nov 27 03:56:53 2007
SRCSRV: variables ------------------------------------------
CVSDATE=11/25/07
CVSDATETARG=11-25-07
CVS_WORKINGDIR=%targ%\%var2%\%fnbksl%(%var3%)
SRCSRVVERCTRL=http
MYSERVER=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
HTTP_ALIAS=http://avnerd.tv/source
HTTP_ALIAS=http://avnerd.tv/source
HTTP_EXTRACT_TARGET=%HTTP_ALIAS%/%var2%/%var3%/%var4%/%fnfile%(%var1%)
SRCSRVTRG=%http_extract_target%
SRCSRVCMD=
SRCSRV: source files ---------------------------------------
c:\ffDebug\mozilla\accessible\public\msaa\AccessibleMarshal.def*MYSERVER*mozilla/accessible/public/msaa/AccessibleMarshal.def*1.3
SRCSRV: end ------------------------------------------------


So the next goal is to successfully keep a list in symbolstore.py of all the pdb so that pdbstr can be called on them.

Source Server 0.2 release - HOWTO and Contribution Opportunity

Here are the goals for my 0.2 release

• release build of ff with debugger-info enabled to create pdbs without being a full debug build


• using lib xul - make an installer for people to download


• take the symbols and index them and distribute the source


• set up the symbol server but indexed so that someone can pull the source code


• which involves running the cv2http.cmd on the indexed code

And the results:

How to set up the Source and Symbol Server

1. add a path to my symbolServer http://avnerd.tv/symbols/2007110200 as per MDC docs in Tools->Debugging->Symbols
   
   
   


2. check the option for turning on the source server in Tools->Debugging->General
   
   
   


3. Download the installer from here Firefox Installer **(FYI you have to use my installer because it is the one that I indexed code and uploaded symbols for - if you do not use this installer you will not get the proper results)



4. install the firefox you just downloaded (it will install as Minefield in Program Files\Minefield)



5. make sure you run it with --no-remote and -profilemanager so you don't mess up your normal Firefox experience



6. once it's running, go to Tools->attach to process... and attach VStudio to the firefox instance



7. after it's attached, you can go to Debug->break all and then poke around at the firefox functions code (You will get a warning about downloading code from an unknown source, click on "run" because you trust me)

This is what it might look like:

Contribution Opportunity

You need to be in the Windows platform and have a copy of Visual Studio to run the above instructions. If this applies to you, please take a minute to download the installer and follow the instructions as I've outlined them.

Please leave a comment answering the following questions:

1. Did it work for you?


2. Were the instructions clear?


3. Any constructive feedback?

Contrib Opportunity! Help with resolving a path with spaces and a perl script

I have a perl script called cv2http.pl that is called upon by cv2http.cmd

Both are in c:\Program Files\Debugging Tools for Windows\sdk\srcsrv directory, which is where I am making this command line call from.

I am trying to do this:

walk.cmd c:\symbols\*.pdb cv2http.cmd HTTP_ALIAS http://avnerd.tv/source

What this will do is swap out the cvs information and replace it with a path to my server where I have the source code.

When I run it I get this:

"Can't open perl script "C:\Program": No such file or directory

for every .pdb that is found.

It looks like it is breaking on the space between Program Files in the path.

So - does anyone know how to either: make this script be forgiving of blank spaces in a path or change the directory this points to so that I can move it and try from there.

**** UPDATE *****
Turns out it was as easy as adding "" around the ~%d0\cv2http.pl call in the cs2http.cmd file - thanks to all who helped on #seneca

Test Day X 2

Well today I got to run Full Functional Tests on Firefox 3 beta 1 and then this evening as I procrastinated from dealing with my Hamiltonian circuit assignment, I ran tests on Miro's 1.0rc release.

The two testing systems are very different even though they both use Litmus-based testing. Miro's tests are fewer, some are not well written and in some ways I find them to be very superficial. The Firefox tests have been around a lot longer so I'm sure that has allowed for quite a bit of fine tuning.

I'm not giving up on Miro but so far it's been challenging to get involved with that project to the degree with which I have been able to immerse myself in Mozilla projects. The lab the other day threw me into Thunderbird in a quick 'n dirty way and that was great - the documentation is all there. With Miro - well...when I went to look for docs on how to build with Windows...this and this. The gist of it is: "We can't really tell you how to build on windows because it's hard".

Well, now that I'm working on the Mac Book and I have VMWare Fusion with an Ubuntu vm, perhaps I can try building it in a Unix/Linux environment? There's a doc for Mac OS X and for Linux. We'll soon see.

Now, back to Mozilla stuff. My 0.2. release is due soon, I have already done a non-debug build of FF with the --enable-debugger-info-modules in the mozconfig and I've made the buildsymbols for it. Now I just have to index the pdb files and get the symbols and the source up on my server. Immediately after that is done (in the next day or two) I will look into a) how to make an installer and b) how to work with the CVStoHTTP script that comes with the srctools.

When I have an installer up, I'll be putting out the contrib call for anyone who can to grab the installer and try to hook it up with my symbol and source servers.

Stay tuned.

0.1 Release

Here's what we're looking at.

I built a version of ff that was checked out from mozilla's cvs. From this version I called "make buildsymbols" and created the folder 2007092823 which contained all the pdb and symbol files for that build.

For information on how to use the symbol server - check out Benjamin Smedbergs blog. Instead of pointing VStudio to the mozilla symbol server, I wanted to point to a local server of symbols.

In order to test whether these symbols worked, I set up IIS on my laptop and served up the pdbs locally to make sure that Visual Studio was able to pull them properly. After a quick detour to set up the MIME types on my server, this worked just fine.

Then I used the indexing tools that come with the microsoft.com/whdc/devtools/debugging/default.mspx Debugging for Windows package.

First - using cvsindex.cmd and pointing out the symbol path and the source path, the pdb files are then indexed with the location of the source code for that build.

Second - using srctools.exe I pulled a sample list of the source files that are contained in a pdb file

All of this is available in a zip file - PDB files and sample source list and I look forward to feedback from anyone who wants to look at the sample source list and discuss how the next step of connecting the cvs to the local pdb files might work. Also you can go to my wiki page to sign up as a contributor or to read more.

At the moment this will not work for someone who is debugging unless they have my build of FF. That's what 0.2 is all about.

IIS is serving up PDB files locally

Well, thanks to Peter McIntyre (Resident Windows Genius) at Seneca College, I have now managed to set up my local symbol server.

Peter actually sent me a link to information about configuring IIS 6.0 but it turned out I had 5.1 - no problem he had given me enough so that a quick Google was all it took to find this script to which I merely added ".pdb", "application/octet-stream" and voila - my MIME map is updated. Now my localhost symbol server can serve up .pdb files without a hitch.

Started up Minefield, attached to VStudio and the symbols loaded up as smooth as can be.

Small success leads to much dancing around with the hound dog.

In other news, finished part 1 of the IBM "Master the Mainframe" contest (if anyone wants to do it, register using Peter as your Faculty person) and am slowly working through part 2...with breaks for reading up on bugs. I still find the whole bugzilla environment quite daunting, it's like that feeling that everyone else knows what they're doing except me. This is me, peering around the corner and listening in...

Setting up a Symbol Server Sandbox

After today's IRC chat with luser, I now have a list of things to do in order to acheive 0.1:

• set up a localhost server


• make buildsymbols from my own build


• load those symbols onto the local server


• connect them up to my debugger to make sure it all works


• get the microsoft scripts to work adding source code to my local pdb files


• so that the debugger can access the source code from being pointed to my server

Starting off, I set up IIS on my computer so that I can have a localhost webserver. This was a bit tricky with some unexpected authentication issues but I think that I have it working now. If I point my browser to http://localhost/symbolServer/ I have a directory of the pdb files that I created by calling make buildsymbols in my objdir.

The next move was to point my debugger (Visual Studio 2005) to the localhost symbol server. However, this is where I got stuck. I could load up the microsoft symbols but it would skip right over my firefox ones.

So I'm going back to square 1. I'm rebuilding with debug disabled because this may be a part of the issue. If this doesn't work I need to look into either a) symbol server directory structure because maybe I'm missing something about the hierarchy or b) perhaps my IIS set-up authentication issues are preventing VStudio from accessing the symbols.

Of course, in the back of my head I know it could also be c) something else entirely.

Back to the building.