Integrating Fitnesse with CCNet

Just have to laugh because the CURRENT CruiseControl.Net documentation on integrating Fitnesse and CCNet says, "Use the TestRunner which comes as part of the standard fitnesse distribution to run all the fitnesse tests and generate the results. You will need to use the task," and dates from 2006. This is somewhat akin to teaching your grandmother how to send email by saying, "Use Outlook."

Anyway, I have a test running automatically with the build and passing. The problem is with my report. I want to have a link from the CCNet dashboard for the project build to a nice, neat fitnesse report. CCNet 1.6 has something in the packages called Fitnesse Results package, but there is, of course, no documentation on using it.

CruiseControl.Net, or Is There A Limit to my Patience?

Continuing with my trials and tribulations in regards to CCNet

Building my projects. I needed to update the svn settings file we have with the path to the svn executable. Then, we are using Rodemeyer.MsBuildToCCNet.dll for the MSBuild tasks and the server needed that library. So I had to copy that to the CCnet server directory.

I ended up just installing VisualStudio 2010 on the build server though I wanted to avoid it. We are using utilities like resgen.exe as well as our unit tests using MSTest. I'm aware I can jump through hoops to install these separate of VisualStudio, but it ended up being a lot easier to just install VS.

I am setting up CCNet to run an NCover task to run our unit tests in MSTest (Apparently I was on drugs when I was thinking earlier that our tests were done in NUnit; I'm sort of wishing now that we had stuck with NUnit rather than using MSTest). The path to MSTest is in it's regular place after installing VisualStudio: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe. NCover is at its default installation place: C:\Program Files (x86)\NCover\ncover.console.exe. Hopefully this is as simple as putting the NCover profiler task into my ccNet project definition (http://www.cruisecontrolnet.org/projects/ccnet/wiki/NCover_Profiler_Task).

<ncoverprofile>
<executable>C:\Program Files (x86)\NCover\NCover.Console.exe</executable>
<program>C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe</program> <testproject>myproject.test.dll</testproject>
<workingdir>build\unittests</workingdir>
<includedassemblies>myproject.*.dll</includedassemblies>
</ncoverProfile>

Yeah... definitely not that easy. First off, the ncoverProfile tag has a bug and doesn't handle spaces in the paths. At least in the 3.3 version of NCover I was using to begin with. I got around that by using the DOS shortnames in the paths. CCNet just passes the testProject value as the first argument to MSTest, which also causes a problem. You get the error from MSTest: 'Invalid switch "c"'. The assembly name in the testProject node needs to have the /testcontainer switch appended. Then I had set the includedAssemblies to *.dll since I figured that would include all the dlls in my working directory. I got an error from NCover that the includedAssemblies included an invalid wildcard '*'. Really?!? Ok, so I just lef the includedAssemblies value blank figuring NCover would just be smart enough to include the dlls in the working directory, and it seems that maybe it was close anyway, because I finally got the MSTest command to at least run my unit tests. My task at this point looked like this:

<ncoverprofile>
<executable>C:\PROGRA~2\NCover\NCover.Console.exe</executable>
<program>C:\PROGRA~2\MICROS~2.0\Common7\IDE\MSTest.exe</program> <testproject>/testcontainer:MembershipTests.dll</testproject>
<workingdir>build\unittests</workingdir> <includedassemblies></includedassemblies>
</ncoverProfile>

MSTest was running but NCover was erroring saying that there was no coverage data: "NCover.Console is returning exit code #20000". Supposedly this can happen when you use NCover 64 bit to cover a 32 bit assembly or vice-versa. My assembly was built for AnyCPU, so I thought I should be ok. I tried running the tests through NCoverExplorer and was getting a similar error, so since only the 32-bit version of NCover was installed on this build machine (since that's the installer we had locally available before), I figured it wouldn't hurt to install the 64-bit version and see if that was indeed the problem.

Once I have the 64-bit version installed, my tests ran fine through NCoverExplorer even though I was still referencing the 32-bit installation of NCover on the machine! So, I was like... ok, let's try this out through CCNet now. After working through an issue with an old NCover task being in the MSBuild project (It suddenly reared its head since it HAD been looking for NCover in C:/Program Files/NCover instead of C:/Program Files (x86)/NCover and was simply failing to even load until I installed the 64-bit version on the new machine *sheepish*), I am getting 42 passing tests, 1 failure with 72% coverage. Not horrible considering it has been a while since we've gotten these tests to run at all. Just gotta go fix that one test, get my build to go green (it fails automatically on the failed unit test) and see if I can't get the coverage up at least to our 85% target. Then the more daunting task: to integrate this into the other VisualStudio projects that comprise the system. While the Membership providers were written in a TDD manner, most of the rest of the system was not. Last I looked, our unit test coverage was somewhere around 15%. Ugh. Finally our real goal here is to get some decent coverage with our acceptance testing. I still have to get Fitnesse running with CCNet on the new machine.

I frankly am wondering what sort of testing ThoughtWorks does before releasing some of their updates. I have to guess they aren't doing much testing on a Windows environment with Microsoft tools. I have to agree that it seems more and more that CCNet and MSBuild with MSTest simply don't mix well and that, though I have gotten this far, Continuous Integration shouldn't be this hard.

Getting CruiseControl.net Running

I have CCNet installed on the build server and a couple packages installed as well, the same ones I added to Jenkins, essentially: NCover, NUnit, MSBuild, and Fitnesse reporting along with StyleCop reporting. I need to make a note to myself to look for that with Jenkins. Alas, there is no ChuckNorris package built-in with CCNet. I restarted the web server but still there is no evidence that those packages are installed other than they say "[Installed]" in the package list. I don't have any build projects created, yet, though. I like that I can administer packages for CCNet like this now. Not thrilled that I need to restart the web server through IIS Mgr to see the configuration changes in general. Not a huge deal though.

I need to install Subversion on here first of all, and then I can check out the ccnet and msbuild scripts from our version control. I'll try this installer for CollabNet client 1.6.6.4 to install to "~Programs\CollabNet\Subversion Client". I tried 'svn' on the command line to confirm and it told me to "Type 'svn help' for usage.", so that's good.

Now I need to create the directory structure to checkout my code. Also, this is going to go easier if I have TortoseSVN installed as well, so I add that and reboot.
When I get back on there is a CollabNet settings manager in my start bar. Let's lookee and see what we can configure. Ooo! Turn OFF the automatic updates. We fear change, particularly on servers that we barely otherwise pay attention to except when they tell us our client application is outdated and all work must halt until they are brought back up to speed.

My TortoiseSvn commands aren't showing up in the context menu. Maybe the version I have doesn't work in Windows Server 2008. I check the MSBuild scripts and CCNet scripts out manually with svn on the command line.

Now to change the ccnet config files to start building my projects.
Issue: CCNet server starts and stops apparently because of the notification attribute in the email group node of the email settings file we have. "notification" has been changed to "notifications". The error in the event lgo was : 'ERROR CruiseControl.NET [(null)] - Exception: Unused node detected: notification="always"' for those who are interested.

Ok! CCNet is running now. I have a couple of our basic projects that are almost set to build. I need to go modify the build scripts for them to reflect the directory structure on this new server and we should be set to go.

*Grumble* CCNet

I have my new build server. So in the interest of time, at least I thought it was, I am installing CCNet on the new server. I'm even taking advantage of the opportunity to upgrade to 1.6. We were running 1.3. But really, CCNet, I need to give permissions to the AppPool user to update the files that you just installed? So far this is not going smoothly. I'm coming in this weekend to install Jenkins here too and have everything being built and tested by Jenkins come Monday. (♪♫ It'll be all right. ♪♫)

Automating With Jenkins

I'm re-configuring our project build to use Jenkins instead of CruiseControl. I'm sure the entire world is bound to find this extremely fascinating, so I am blogging about it. There are a number of little reasons for this, but mostly it's simply because I find CCNet to bed tired and old and Jenkins to ben the new hotness. Actually, I am just up for learning something new, and since we need to create a new build environment anyway, I figured I'd take a shot at using Jenkins.

A bit about the project

It is a .Net project that uses the .Net 4.0 framework with a WPF client that communicates to a WCF Web Service that communicates with a WCF Windows Service. It's not a bad architecture, though probably not the BEST architecture. It is definitely not an architecture that is easily testable. It uses SQL Server 2008 as the data store and our version control is SVN.

Our old build environment, like I said, uses CruiseControl.Net. It was what we were using for previous projects, and was adequate for simply doing our build. Now that we are getting into automating our unit and acceptance tests, however, its age is showing. We are generally building for x64 processors and the build environment is an x86 processor. SQL Server 2005 is installed, but not 2008. The version of NCover and NUnit that are installed are outdated and NUnit 2.4.8 will not handle assemblies written for .Net 4.0.

Onward!

So while our IT department is building me a virtual machine with the build environment that I want, I am going to try to and get an automated build running on my laptop with Jenkins. Wish me luck.