Build failing using MsBuild command line

Topics: Bugs, General Questions
Aug 24, 2007 at 8:23 AM
We are using CruiseControl.NET on our build server to perform CI builds. CC.NET uses MsBuild to build our C# solutions. We have removed our Sandcastle build scripts in favour of using DocProject but the build is failing from the command-line. The documentation project runs fine inside the VS2005 IDE but when I run the build script from the command-line either on my machine or the build server the paths seem to be all wrong.

The first error I get is that the XML files cannot be found in the comments directory of the docproject project. By modifying the entry in projectpaths - comments in DaveSexton.DocProject.dll.config directory to our release directory (relatively) as ..\..\bin\release this gets past this error. However the next error that occurs is that MRefBuilder fails with an error that it cannot find the dll files from the solution as they are not in projectname/bin/release but are all held in the bin/release directory at a higher level eg ..\..\bin\release as before.

Furthermore, when I compare the MRefBuilder command line that is generated from the DocProject.exe during build and the one created using the command line itself, I see that all the extra dependancy declarations are not there.

It seems to me that the behaviour of running DocProject from the command line is considerably different to what you get from running it in the IDE or DocProject.exe contrary to what is stated in the help pages. Why is this so? While this behaviour is so different I think I will have to go back to our build scripts to manipulate Sandcastle as I cannot make head nor tail why DocProject is performing this way.
Aug 24, 2007 at 10:31 AM
Edited Aug 24, 2007 at 10:32 AM
Hi,

First, thanks for the feedback.

The reason for the problems that you're seeing, I believe, is a bug that I've identified in DocProject where it does not work with Release builds. If you try building with the Debug configuration selected it may work. Also, if the relative value for OutputPath properties in each of your source projects are different than your DocProject's OutputPath it will fail as well.

There is a thread on the MSDN forums that I've started, which contains the solution and some other information as well:

Thread: DocumentationFile Property
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2006773&SiteID=1&mode=1

I will fix this in the next release. If you want to try rebuilding the source code yourself and let me know if it works I'd appreciate it.

See my response in that thread at 17 Aug 2007, 10:56 PM UTC for instructions and retrieve the patched files in the Team Build work item.
See How To Use The Source Code and feel free to ask me for further assistance if you need it.

There are two other relative discussions on this problem in case you're interested:

Thanks,
Dave
Aug 24, 2007 at 12:06 PM
Thanks for the quick response Dave.

Firstly I would like to say that I think the aim project is a good one and will hopefully demystify Sandcastle once and for all. However, it is a shame that this doesn't work out of the box just yet as I have been fighting Sandcastle and MsBuild for several months now and hoped your project would remove my woes.

I am up against it at the moment but will try and get the source running and try to step through to help resolve this if I can. Relative paths have been the bane of my existance with Sandcaastle for a while (and variable definitions in MsBuild also) so perhaps I can help in some way. Clearly though this seems to be a problem that does not just affect TeamBuild but other CI tools as well.

I will post more of my progress when I get some more time on this next week.

Regards,

Kurt
Aug 24, 2007 at 12:33 PM
Hi Kurt,

Thanks for the reply.


Firstly I would like to say that I think the aim project is a good one and will hopefully demystify Sandcastle once and for all. However, it is a shame that this doesn't work out of the box just yet as I have been fighting Sandcastle and MsBuild for several months now and hoped your project would remove my woes.

I can certainly understand your frustration. My experience with paths in Sandcastle and MSBuild have not been so pleasant either, but I do believe that this will be worked out in the next release.


I am up against it at the moment but will try and get the source running and try to step through to help resolve this if I can. Relative paths have been the bane of my existance with Sandcaastle for a while (and variable definitions in MsBuild also) so perhaps I can help in some way. Clearly though this seems to be a problem that does not just affect TeamBuild but other CI tools as well.

I will post more of my progress when I get some more time on this next week.

I expect that these path issues will affect any tool that automates builds using MSBuild. The problem, as I stated in the other threads, is that the current configuration is not passed into DocProject's build task. Also, MSBuild does not provide a way for the task to know whether OutDir was set globally or if it was simply evaluated as the value of the DocProject's OutputPath property, which means that project references cannot override their own OutputPath property.

The former problem was fixed in those patched files that I linked to in my earlier response (the Team Build work item) but they haven't been tested yet. The latter problem does not have a complete solution yet but the work-around in the forums thread seems like it might work for most real-world scenarios (thanks Aaron!).

I appreciate your offer to help. Please let me know if you need any further assistance.

Thanks,
Dave
Aug 29, 2007 at 4:31 AM
Hi Kurt,

I've received a positive response for the new updates that I made to support configuration and platform settings as well as custom output paths in project references. I've attached a .zip file with the modified files to the Team Build work item (the .zip replaces the previous .zip file that was attached).

If you can find the time please try to patch your installation of DocProject 1.7.0 with the files found in the new .zip and let me know if it solves your problems too.

Information about the files can be found in my last post in the forums thread that I cited previously. If you need help please ask.

Thanks,
Dave
Aug 30, 2007 at 8:03 PM
Kurt,
I'm pretty sure you're running into the same issues I did (see my post about Using MSBuild - incorrect paths for MRefBuilder). Apply the latest patch and things should work much better.
Aug 30, 2007 at 9:56 PM
Thanks Dave and Gambit,

Sorry for the delay in posting back but I thought I had but must never have hit submit!
Anyway. I spent some time on this straight after Dave's last post and updated my source with the updates. The first small issue I had was that the AboutForm.cs was missing. I commented this out. Next, I wanted to debug the source to watch what was happening and ran into a load of conflicts between the installed version and the version under debug. Therefore I uninstalled the release version that was already installed. I also changed the bin location to the same place for all projects and copied the target file there too. This allowed me to debug the code just fine and intercept the code when the MsBuild scripts were running.

Next problem was that without the full version installed the registry setting that is read in the InstallPath propoerty was not set. I changed the code to default to the location I had the bin files. I now could debug the code and the MsBuild script completed correctly and the chm was created so great news.

At this point though I was running out of time to spend on this at the moment so when I found that the installer would not build correctly for me with a errror returned from the batch file I had to shelve this for the time being. But I must agree that the patch does seem to fix our woes. Any chance of a new release soon Dave? It would be much appreciated.

Finally, a question re using the GAC. Is there a reason that most of the dlls are added to the GAC? Is it because addins need to be in the GAC? From a build server and continuous integration point of view it would be nice to be able to run the dlls locally (with xcopy gets from source control). I tried removing the files from the GAC in the build but I again had to give up due to time constraints with errors occuring when the build engine configurations were being loaded. Any thoughts, ways forward?

Thanks again,

Kurt
Aug 30, 2007 at 10:45 PM
Hi Kurt,

I appreciate the response and I'm happy to hear that the patched files worked for you. They'll be in 1.8.0.

You can monitor the next release here: 1.8.0 Release Candidate.


The first small issue I had was that the AboutForm.cs was missing

By any chance did you download the source code from CodePlex? I highly recommend against that because I've been using TFS since the original beta of DocProject and it always seems to exclude files or keep older copies after checking in changesets. The reason why I continue to use it is merely a convenience for those that want to browse the code online. When I started a thread in the TFS forums about the problems I was having I was basically told that it's my fault since I'm not using the product correctly :p

I checked for that file in the installler and it's there, so use that instead. You'll find the installed copy of the complete source code in DocProject's Source folder, which comes with each version that I release and I test it so it should be up-to-date.


Next, I wanted to debug the source to watch what was happening and ran into a load of conflicts between the installed version and the version under debug
<snip>

I suggest uninstalling DocProject completely and removing any GAC assemblies or changes that you've made. Then reinstall DocProject from scratch and read the How To Use The Source Code article more thoroughly, which details how I use the source code myself to build DocProject.

When I test the source code on VMs I simply install DocProject, copy the Source folder to My Documents\Visual Studio 2005\Projects and rename it to DaveSexton.DocProject. From there it's usually only a few seconds to make some mods to the project settings and to create a new debug .AddIn file, and then everything's up and running exactly how it is on my dev machine.


Next problem was that without the full version installed the registry setting that is read in the InstallPath propoerty was not set
<snip>

You are correct that DocProject must be installed to use the source code. That's the way it is on my dev machine as well and I haven't actually found this to be a problem at all. If anything, it's actually made debugging easier.


Finally, a question re using the GAC. Is there a reason that most of the dlls are added to the GAC? Is it because addins need to be in the GAC?

The assemblies are added to the GAC for versioning support (which I haven't actually taken advantage of yet, but I will when I release DocProject 2008) and also because they need to be referenced from the installed templates.

But there's actually a limitation in the way the add-in assembly is loaded that prevents me from referencing the GAC assembly in the .AddIn file, so the installer actually patches the file before it gets installed onto your system. Obviously, if you choose a different installation path for DocProject then the installer has to be smart enough to use the path you've specified and this was a major source of grief for me before I had written the installer's custom action, which actually handles this now. Note however that the GAC assembly is actually used at runtime even though the path points to DocProject's assembly in the bin folder.

Fortunately, the templates allow a GAC reference so I don't need to manually patch each of them with the user's chosen path, which would probably be impossible anyway since they are zipped ;)


From a build server and continuous integration point of view it would be nice to be able to run the dlls locally (with xcopy gets from source control). I tried removing the files from the GAC in the build but I again had to give up due to time constraints with errors occuring when the build engine configurations were being loaded. Any thoughts, ways forward?

Take my advice and start over, following the How To... directions. It shouldn't be nearly as difficult to set up as you've experienced :)

Debugging should actually be exteremely simple (and basically transparent). For example, each time you build a project it replaces the assembly in the GAC automatically using a post-build event. Also, each time you run VS the Add-In (if you've followed the instructions and created a debug version of the .AddIn file) uses the latest assemblies from the build and the GAC so it runs as if it's installed already. But if you have to make changes to the DaveSexton.DocProject.targets file that's the only file that must appear in the installation's bin folder, and is one of the reasons why DocProject must be installed. As you've discovered already, the reason is because of the DocProjectBuildPath environment variable (which is actually DocProjectPath in the 2008 release).

The DSZip utility is used automatically to package the templates so I only need to run the .vsi file to test template changes without having to rebuild and use DocProject's installer.

I could go on... If you have any other questions just ask :)

Thanks,
Dave
Aug 31, 2007 at 7:51 AM
Dave,

Thanks for the detailed response. You are correct, I was using source files from CodePlex! I must have missed that part of your instructions on how to use the source. I think also a lot of my trouble came about fighting the GAC which in hindsight I should have just let be.

I look forward to the 1.8.0 RC as I think the app is great and so far I haven't even started looking at the DocSite stuff.!

If I get a chance I will start afresh from the installed source files and patch again.

Regards,

Kurt
Aug 31, 2007 at 2:20 PM
Thanks Kurt. Hopefully I'll be able to get 1.8.0 out soon...