The more there are developers committing changes to the project source code and the more distributed they are, the more challenging it becomes to keep track of all the changes. Without having enough test coverage this is a recipe for new bugs being introduced by less experienced developers not being aware of all the project intricacies. Introducing a source control monitor notifying code owners of changes can be a good way to reduce this risk. Although there are dedicated tools available for this task, this post introduces a simple way to use your existing CruiseControl.NET setup for this task. A similar approach should be possible with other continuous integration servers as well.

This is a basic project configuration which to use for this purpose:

<project name="My Project Monitor" category="SVN Monitor">
    <sourcecontrol type="filtered">
        <sourceControlProvider type="svn">
            <autoGetSource>false</autoGetSource>
            <trunkUrl>$(SvnUrl)My Project/trunk</trunkUrl>
            <executable>$(SvnExe)</executable>
            <username>$(SvnUsername)</username>
            <password>$(SvnPassword)</password>
        </sourceControlProvider>
        <inclusionFilters>
            <pathFilter>
                <pattern>/trunk/**/*.cs</pattern>
            </pathFilter>
            <pathFilter>
                <pattern>/trunk/**/*.vb</pattern>
            </pathFilter>
        </inclusionFilters>
    </sourcecontrol>
    <triggers>
        <intervalTrigger seconds="60" />
    </triggers>
    <publishers>
        <xmllogger />
        <email from="$(MailFrom)" 
               mailhost="$(MailServer)" 
               mailhostUsername="$(MailUsername)" 
               mailhostPassword="$(MailPassword)" 
               includeDetails="true">
            <subjectSettings>
                <subject buildResult="Success" 
                         value="SVN Monitor: My Project" />
            </subjectSettings>
            <users>
                <user name="Developer 1" 
                      group="CodeOwner" 
                      address="developer.1@mycompany.com" />
            </users>
            <groups>
                <group name="CodeOwner">
                    <notifications>
                        <notificationType>Success</notificationType>
                    </notifications>
                </group>
            </groups>
        </email>
    </publishers>
</project>

A couple of things worth noticing about it:

• To simplify maintenance of a large configuration file I’m using preprocessor text constants, i.e. values for all $(ConstName) placeholders are defined at the beginning of the configuration file.
• In the sample above Subversion is used as the source control system, though CruiseControl.NET comes with built-in support most if not all source control systems available today, meaning that unlike dedicated desktop based tools you can use it no matter which and have many different source control systems you are using.
• Filtered source control block allows additional filtering of detected changes, by both including and excluding certain changes. In my case I’m simply filtering changes by file extension but there are other types of filters available as well. By additionally using a multi source control block you could also monitor multiple repositories in different source control systems with a single project.
• Setting the autoGetSource property to false, prevents the sources being retrieved from the source control system. This avoids unnecessary delays and network traffic.
• Triggers give great flexibility over how you want the notification to happen. In my case the project triggers every minute making the notification immediate but you could also configure the notification to happen less often or even only once a day at a specific time (e.g. at the beginning of the work day).
• Sending out emails to notify developers works great in enterprise scenarios when you need to notify a selected group of people. By setting the includeDetails property to true the list of changes is included in the emails being sent out. A custom email subject makes it easier for the recipients to filter the incoming emails.
• Including the default XML log publisher insures that the build logs containing all the changes are also generated on the server making it possible for everyone interested to see all past changes at a later time.

Since version 1.5 CruiseControl.NET includes an FTP Task which can be used for uploading build results to a remote FTP server. Its most typical use case is for publishing websites to a remote server. Configuring this correctly requires some thought and at the moment there are very few helpful resources available. Below are some suggestions for the most simple case when the files to be uploaded only have to be retrieved from the source control and there is no additional build process involved.

The core of the configuration is of course the FTP task. Its configuration is pretty straightforward. You only need to supply it with login details and the path to the local and the remote folder. This also means that there are almost no configuration options available – you can only choose between uploading the files recursively or not.

The problem with the lack of options is that most source control systems put additional files in the local folder which you don’t want to upload. In the case of Subversion this is a .svn folder in every local folder. Because the FTP task can’t be configured to skip or ignore certain files, these files have to be deleted before the FTP task is executed. The best way to do that is by executing a batch file:

FOR /F "tokens=*" %%G IN ('DIR /B /AD /S *.svn*') DO RMDIR /S /Q "%%G"

If you do this directly in the working directory where the files are retrieved from Subversion, then the next retrieval from source control will fail unless you set the cleanCopy configuration element to true and retrieve all of the files from Subversion every time. Doing this has two downsides:

• If the project is large, this will significantly decrease performance. Downloading lots of files from Subversion takes time.
• The file timestamps will be reset every time. This information could potentially be useful for the FTP task (but isn’t yet as I explain at the end of the post).

To avoid this you should first copy the retrieved files to a different folder. The best way to do this is by using the build publisher. Just don’t forget to set the alwaysPublish configuration element to true. Otherwise it won’t copy anything because you are using it in the tasks section where the build is not yet successful.

To sum it all up, your tasks section should something look like this (I’ve used preprocessor constants for most of the values to increase readability):

<tasks>
    <buildpublisher>
        <sourceDir>$(WorkingDir)</sourceDir>
        <publishDir>$(PublishDir)</publishDir>
        <cleanPublishDirPriorToCopy>true</cleanPublishDirPriorToCopy>
        <useLabelSubDirectory>false</useLabelSubDirectory>
        <alwaysPublish>true</alwaysPublish>
    </buildpublisher>
    <exec>
        <executable>$(ClearSvnFilesBatch)</executable>
        <baseDirectory>$(PublishDir)</baseDirectory>
    </exec>
    <ftp>
        <serverName>$(FtpServerName)</serverName>
        <userName>$(FtpUserName)</userName>
        <password>$(FtpPassword)</password>
        <action>UploadFolder</action>
        <ftpFolderName>$(FtpRemoteFolder)</ftpFolderName>
        <localFolderName>$(PublishDir)</localFolderName>
        <recursiveCopy>true</recursiveCopy>
    </ftp>
</tasks>

I’ll conclude this post with a bad news. Unfortunately there is no support in the FTP task to only upload the files that have changed since the previous build. For a small website this might not be an issue but for larger ones this is quite a problem. You really don’t want to upload tens or even hundreds of megabytes every time. Let’s just hope that the issue will be fixed soon.

Recently I’ve been solving some issues with queue starvation in CruiseControl.NET. Since I haven’t found much information about this problem I’ve decided to round up my findings in this blog post.

So, what is queue starvation? It’s a problem that can occur when scheduling tasks in priority queues if there are too many high priority tasks so that lower priority tasks never get scheduled and are just waiting indefinitely. For example, let’s say we have three queues: high, medium and low priority. Tasks in the high priority one always get processed first. Once this queue is empty the tasks from the medium priority queue get processed. Of course the tasks in the low priority queue get processed only if the other two queues are empty. If there are too many high and medium priority tasks incoming all the time so that the queues never get empty, then the low priority tasks are not processed at all.

And how is this related to CruiseControl.NET? As you might be aware, projects can be assigned to different queues. Only projects in the same queue (as the name implies) are built sequentially while projects in different queues are built in parallel to one another. Queues themselves therefore cannot cause starvation because different queues don’t depend on one another and projects are always put at the end of its queue. This means that sooner or later each project will be built.

Now, queue priorities enter the stage. Each project can be given a priority defining where the project will be put into the queue: before all of the lower priority projects already in the queue instead of at the end. Effectively this partitions a single sequential queue into multiple queues with different priorities as described at the beginning of the post. We have all the necessary prerequisites for queue starvation. If too many projects with higher priorities have to be built (typically caused by checking in code or triggering a forced build), the projects with lower priorities will never get processed.

What I have observed is that this can happen even if higher priority projects don’t have to be built at all. The reason for that is in the way CruiseControl.NET is processing the projects to determine whether there were any source code changes. This is taken care of with an interval trigger which puts a project in the queue, but it only gets built if a modification exists. Nevertheless, even checking if such a modification does exists, takes some time. In extreme cases this can be enough to cause queue starvation and this is also what happened in my case. Lower priority projects never got built, not even during the night and no matter how often a build was forced for them.

Fortunately there is a solution for this problem: the seconds attribute which specifies how often the project is put into the queue for checking if a modification exists. By default it is set to 60 seconds. Increasing the value can solve the queue starvation issue. Instead of simply defining the trigger:

<intervalTrigger />

A longer period can be specified:

<intervalTrigger seconds="300" />

Of course this has its downside: the latency before the project gets built after a modification increases. Therefore you should use this solution with caution. Only increase the value as much as you have to. And don’t do it at all if you can apply any of the other solutions:

• Increase the performance of the build server to speed up the processing and building of the projects.
• Parallelize the build process by using more queues. Put the projects in the same queue only if you have to.
• Don’t use queue priorities if it’s not necessary. In most cases you can get by without them.

Subversion and CruiseControl.NET can be invaluable tools in your .NET development process. There are many resources available to help you get started which I’ll try to gather in this post along with some of my personal experiences.

Let me start with the list of recommended software:

• VisualSVN Server is the ultimate Windows version of Subversion including a simple setup and powerful management tools. If you are planning to install a Subversion server on Windows it should be your first choice.
• AnkhSVN is a Subversion Source Control Provider (SCC) for Visual Studio. As long as you’re not using Express editions of Visual Studio, this is the suggested way of working with SVN directly from Visual Studio IDE.
• TortoiseSVN is a Windows shell extension for working with Subversion from within Windows Explorer. When you're not working with Visual Studio solutions this is the best choice for using SVN.
• CruiseControl.NET is a continuous integration server including a web dashboard and CCTray - a system tray client application for monitoring and controlling builds.

If you’re not already familiar with the above mentioned products, you should consult their documentation or search for tutorials. I will rather focus on setting up your development and release process. If you haven’t done so already I suggest you first read the following articles by Ariejan de Vroom:

• How to structure your repository
• How to release software properly
• How to fix bugs properly

I mostly based my configuration on the ideas in these articles. I have projects configured in CC.NET to build all copies of the project: trunk (ProjectName-Trunk), all branches (ProjectName-REL-#.#) and all tags (ProjectName-v#.#.#). To identify individual builds I am using CC.NET’s Assembly Version Labeller together with AssemblyInfo MsBuild Community Task.

Assembly Version Labeller is really simple to configure. You only need to add a short snippet to each project:

<labeller type="assemblyVersionLabeller">
    <major>1</major>
    <minor>0</minor>
    <build>0</build>
</labeller>

I’m using the following versioning policy:

• I start each project with version 1.0.0.
• Once it’s ready for release I make a copy of the trunk in the branches directory, named REL-#.# containing the major and the minor version number. Immediately afterwards I bump the version of the trunk (only minor or major and minor, depending on the nature of the new features planned).
• In the release branch I make the necessary changes before release (e.g. I change the AssemblyProduct name to distinguish between development and release quality builds) and make another copy in the tags directory, named v#.#.# containing the major, minor and build version numbers. Immediately afterwards I increase the build version number in the release branch.
• I make no changes to the copies in the tags directory. All bug fixes go to the release branch. Once I’m ready for a new release I repeat the previous step.

Since I don’t specify the revision number directly, the SVN Revision number gets used automatically. This makes it possible to match each build to the revision of the code in SVN.

To put the generated assembly version in the build I am using the AssemblyInfo MsBuild task. There are two steps involved in doing this.

First you need to move the AssemblyProduct, AssemblyInfo and AssemblyFileVersion attributes from the auto generated AssemblyInfo.cs file into a new file. In my case the AssemblyVersion.cs has the following contents:

using System.Reflection;

[assembly: AssemblyProduct("ProjectName DEV")]
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]

Next you have to modify your project file (*.csproj) by importing the community tasks and adding a call to the AssemblyInfo MsBuild task:

<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
<Target Name="BeforeBuild">
    <AssemblyInfo Condition="'$(CCNetLabel)' != ''"
        CodeLanguage="CS" 
        OutputFile="Properties\AssemblyVersion.cs" 
        AssemblyProduct="ProjectName TRUNK" 
        AssemblyVersion="$(CCNetLabel)" 
        AssemblyFileVersion="$(CCNetLabel)" />
</Target>

If you have never edited a project file before, you might want to read these first:

• How to: Edit Project Files
• How to: Extend the Visual Studio Build Process

One more thing to note which might not be all that obvious. The Condition in the AssemblyInfo task is met only when building from CC.NET. For builds in Visual Studio the task doesn’t regenerate the AssemblyVersion.cs file therefore the revision number is always 0 and the AssemblyProduct has a DEV suffix as defined in the original file. Also I remove the TRUNK suffix from the AssemblyProduct attribute of the AssemblyInfo task when moving code from trunk to release branches to separate between the two.

Once you start putting CruiseControl.NET to production use you'll sooner or later encounter the need for custom build tasks. There's only a limited set of them in the package and Executable Task can only do so much. Unfortunately there is not much information available on development of custom tasks. Your best sources will be:

• Online documentation contains an article on the subject describing the first steps to get you going. It should be your first stop.
• There's a high level architectural overview of a simple task available at Josh's Blog.
• Lutz Roeder's Reflector for .NET will be your best friend. The source of the tasks in ThoughtWorks.CruiseControl.Core will soon become your best resource.

Apart from that I feel obliged to mention a few of the most important points I've come across during the development of a few custom tasks:

• ThoughtWorks.CruiseControl.Core.Util.ProcessExecutor is a nice little wrapper around System.Diagnostics.Process class you'll end up using quite a lot.
• You can add your own information to the build log by calling AddTaskResult on the IIntegrationResult instance passed to your ITask.Run method. There are two overloads available: one accepting a System.String and another one accepting ITaskResult to which you can pass a new FileTaskResult instance to quickly include a complete file.
• If you're doing any checkins to your source control system as a part of the build you should call the MarkStartTime method of your IIntegrationResult instance afterwards to prevent triggering another build of the same project by setting the last build start time after the last checkin time.
• Make sure you use a unique ReflectorType name for your task. The service will just silently fail to start in case of a duplicate value.

This information should make your first attempts at making your own custom task a little easier.

The 1.3.0.2918 build of CruiseControl.NET has an error in msbuild.xsl file which causes an XslLoadException to be thrown when trying to view the MSBuild output in the web dashboard. One of the users was nice enough to describe the changes to the file necessary to fix the problem. Unfortunately even the latest version of the file on the CruiseControl.NET Live site doesn't include the changes therefore I'm attaching the file to this post as convenience.