Friday, April 11, 2008

Sample F# Test Runner (and C# too...)

While I was fooling around with F# and testing my functions, I got really annoyed with switching between applications to run my tests, Visual Studio and NUnit Gui. The source of my fustration is during the day I get to use a fantastic tool TestDriven.Net to run my unit tests in C# and Visual Studio. With F#, however, TestDriven.Net does not recognize the different syntax and no tests run. So I decided to write my own test runner, by taking advantage of .Net's FileSystemWatcher class. (Note, I used the FileSystemWatcher.Changed event.1 ) The first iteration I wrote in C# so I could have an excuse to use xUnit. I probably over engineered the C# code, but old habits die hard. In the C# implementation, there are three files. The program class sets everything up, the HarnessRunner class sets up the filesystem watcher and listens for the file changed event, and finally the ProcessStarter to run the process. The application is configurable to use the app.config or to pass any necessary commands to the program. ( Hence a little of the bloat.)
using System;
using System.IO;

namespace HarnessRunner
{
    public class program
    {
        [STAThread]
        public static void Main(string[] args)
        {
            Settings settings = Settings.Default;

            if (ValidateInputValue(settings.TestRunnerCommand))
            {
                Console.WriteLine("Enter the fully qualified command to run:\r\n");
                settings.TestRunnerCommand = Console.ReadLine();
            }

            if (ValidateInputValue(settings.TestAssembly))
            {
                Console.WriteLine("Enter the file to test:\r\n");
                settings.TestAssembly = Console.ReadLine();
            }

            if (ValidateInputValue(settings.TestRunnerSwitches))
            {
                Console.WriteLine("Enter any swtiches to the test runner:\r\n");
                settings.TestRunnerSwitches = Console.ReadLine();
            }
                     
            Console.WriteLine("Setting up the watcher to run: \r\n{0} {1} {2}", Path.GetFileName(settings.TestRunnerCommand),
                              Path.GetFileName(settings.TestAssembly),settings.TestRunnerSwitches);
            try
            {
                HarnessRunner runner = new HarnessRunner(new FileSystemWatcher(), new ProcessStarter());
                runner.InitializeFileSystemWatcher(settings.TestAssembly);
            }
            catch (Exception e)
            {
                Console.WriteLine("There was an exception during the run: {0}{1}{2}", e.Message, Environment.NewLine,
                                  e.StackTrace);
            }
            Console.ReadLine();
        }

        private static bool ValidateInputValue(string command)
        {
            return string.Compare(command, string.Empty) == 0;
        }
    }
}
using System;
using System.Diagnostics;
using System.IO;

namespace HarnessRunner
{
    public interface IStartTestHarnesses
    {
        string Start();
    }

    public interface IFileSystemWatcher
    {
        bool EnableRaisingOfEvents { get; set; }
        string Path { get; set; }
        NotifyFilters NotifyFilter { get; set; }
        event EventHandler Changed;
    }

    public class ProcessStarter : IStartTestHarnesses
    {
        public string Start()
        {
            Settings settings = Settings.Default;
            Process process = new Process();
            process.StartInfo.FileName = settings.TestRunnerCommand;
            process.StartInfo.Arguments = settings.TestAssembly + " " + settings.TestRunnerSwitches;
            process.StartInfo.UseShellExecute = false;
            process.StartInfo.RedirectStandardOutput = true;
            process.Start();

            return process.StandardOutput.ReadToEnd();
        }
    }
}
using System;
using System.IO;

namespace HarnessRunner
{
    public class HarnessRunner
    {
        private FileSystemWatcher fileSystemWatcher;
        private IStartTestHarnesses startTestHarnesses;

        public HarnessRunner(FileSystemWatcher fileSystemWatcher, IStartTestHarnesses startTestHarnesses)
        {
            this.fileSystemWatcher = fileSystemWatcher;
            this.startTestHarnesses = startTestHarnesses;
            fileSystemWatcher.Changed += fileSystemWatcher_Changed;
        }

        private void fileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
        {
            Console.WriteLine(startTestHarnesses.Start());
        }

        public void InitializeFileSystemWatcher(string filetowatch)
        {
            fileSystemWatcher.Path = Path.GetDirectoryName(filetowatch);
            fileSystemWatcher.Filter = Path.GetFileName(filetowatch);
            fileSystemWatcher.NotifyFilter = NotifyFilters.LastWrite;
            fileSystemWatcher.EnableRaisingEvents = true;
        }
    }
}

So, once I had that working, I decided to write the F# equivalent. There are two functions, one that starts the process and one to setup the filesystemwatcher. Other than that, there are just some config options.
#light

open System
open System.IO
open System.Diagnostics
open System.Configuration

let mutable command = ConfigurationManager.AppSettings.Item("TestRunnerCommand")
let mutable testAssembly = ConfigurationManager.AppSettings.Item("TestAssembly")
let mutable commandSwitches = ConfigurationManager.AppSettings.Item("TestRunnerSwitches")

let startProcess f =
     let p = new Process()
     p.StartInfo.FileName <- command
     p.StartInfo.Arguments <- f ^" "^commandSwitches
     p.StartInfo.UseShellExecute <- false
     p.StartInfo.RedirectStandardOutput <- true
     let started = p.Start()
     printfn "%O" (p.StandardOutput.ReadToEnd())

let SetupFileSystemWatcher f =
     let fileSystemWatcher = new FileSystemWatcher()
     fileSystemWatcher.Path <- System.Environment.CurrentDirectory
     fileSystemWatcher.Filter <- f
     fileSystemWatcher.NotifyFilter <- NotifyFilters.LastWrite
     fileSystemWatcher.EnableRaisingEvents <- true
     fileSystemWatcher.Changed.Add(fun _ -> startProcess f)
   
if command = ""  command = null then
     printfn "Enter the fully qualified command to run:\r\n"
     command <- Console.ReadLine()       
if testAssembly = ""  testAssembly = null then      
     printfn "Enter the file to test:\r\n"
     testAssembly <- Console.ReadLine()
if commandSwitches = ""  commandSwitches = null then
      printfn "Enter any swtiches to the test runner:\r\n"
      commandSwitches <- Console.ReadLine()

printfn "Setting up watcher for %A" testAssembly  
SetupFileSystemWatcher testAssembly 
read_line() 
So, now that the code is posted, I'm left with some sort of follow up point to wrap this post up. The only problem is, I can't seem to come up with any points! I put together this article to show a program that performs the same function, implemented in two separate languages. Comments, feedback and discussions are welcome! 1 For some reason, the FileSystemWatcher.Changed event fires three times when my assembly is compiled.

Wednesday, March 19, 2008

Visual Studio 2008 launch in Detriot

Wow - I just came back from Microsoft's Hero's Happen Here event in Detroit to launch Visual Studio 2008, Windows Server 2008 and SQL Server 2008. It was a well organized event, but more important to me was to catch up with everyone I met at this years CodeMash event. I always leave these events with a tremendous sense of community. Special thanks to Keith Elder, who hosted the geek dinner afterwards. (An extra special thanks to Randy Pagels!) At this event I was able to meet fellow F# blogger Dustin Campbell and share a few thoughts on F#. I wish I had more time to talk shop......... I was also able to meet Jeff McWherter, Joe Wirtley and spend six quality hours in the car with Corey Haines and Michael Letterle. Well, Corey was only awake for three.... Another reason for the feeling of community was the presence and dedication of Microsoft's regional representatives, Josh Holmes, Brian Prince, Darryl Hogan and Jeff Blankenburg. Hope to see you all again soon!

Saturday, March 8, 2008

Records and Discriminated Unions

Like most other programming languages, it is possible with F# to create your own types when needed. Because F# is a blend of both functional languages and imperative languages, you get support for types from a functional perspecive as well as types when you have to deal with OO programming. Records Records are concrete type definitions, if you have an OO background, records will look very similar to classes. To create a record, you need to tell F# the type name, the labels and the label type. Here’s an example, using FSI
> type FamilyMember = {Name: string; relation: string};;
type FamilyMember = {Name: string; relation: string;}
There are two different ways to create record. The first way:
> {Name = "Nate";relation = "Father"};;
val it : FamilyMember = {Name = "Nate"; relation = "Father";}
Or a more explicit syntax:
> {new FamilyMember with Name = "Kelly" and relation = "Mother"};;
val it : FamilyMember = {Name = "Kelly"; relation = "Mother";}
F# makes it easy for you to access the record labels:
> let f = {new FamilyMember with Name = "Nate" and relation = "father"};;
val f : FamilyMember
> f.Name;;
val it : string = "Nate"
Records can be cloned, and like everything else with F#, it’s amazingly simple!
> let d1 = {new FamilyMember with Name = "reilly" and relation = "daughter"};;
val d1 : FamilyMember
> let d2 = {d1 with Name = "Cora"};;
val d2 : FamilyMember
> d2;;
val it : FamilyMember = {Name = "Cora";
relation = "daughter";}
Records can be results of functions as well. Take the following type, which has a count of files and a count of directories:
type FilesAndDirectorys = {Files: int; Directories: int;}
We can use it for a function like so:
let GetFilesAndDirectories loc =
let dirs = Directory.GetDirectories(loc)
let files = Directory.GetFiles(loc)
let results = {new FilesAndDirectorys with Files = files.Length and Directories = dirs.Length}
results
Next call the function with the path you want to search
let counts = GetFilesAndDirectories @"C:\" The identifier counts now contains the following data:

val it : FilesAndDirectorys = {Files = 9; Directories = 18;}

There is another type, discriminated union types, but I'll save those for another blog entry.

Friday, February 15, 2008

Basic Type conversions with F#

As I work my way through the ProjectEuler problems they forced me to look into working with the different types in F#. F# is a very strongly typed language, but it has the same basic types as the types in the other .Net languages. Here is a list of the basic types and how to tell the compiler you want to use them:
let int = 42
let string = "This is a string"
let char = 'c'
let bool = true
let bytearray = "This is a byte string"B
let hexint = 0x34
let octalint = 0o42
let binaryinteger = 0b101010
let signedbyte = 68y
let unsignedbyte = 102uy
let smallint = 16s
let smalluint = 16us
let integer = 345l
let usignedint = 345ul
let nativeint = 765n
let unsignednativeint = 765un
let long = 12345678912345L
let unsignedlong = 12345678912345UL
let float32 = 42.8F
let float = 42.8
(Definitions for the types are listed here.) F# also has BigInt and BigNum types, they stand for arbitrary large integer and arbitrary large number respectively. (I don't know how big they are yet.)
let bigInt = 9876543219876I
let bigNum = 123456789987654N
The F# compiler will determine the types you are working with, a feature called Type Inference. To see what types are inferred, compile your fs files using the –i switch to create an FSI (F# Interface file) or use the mouse in Visual Studio. Most of the F# programming you will do, inference will work. Now, I know some of you might be thinking; “Yeehaw! I don’t have to worry about declaring types! I’m free!!” Well…some of you might have, I did. If you want force a type and not let inference handle it for you, you have to use the conventions above like so:
> 3423456573476N;;
val it : bignum = 3423456573476N
> "This will be a string of bytes"B;;
val it : byte []
= [84uy; 104uy; 105uy; 115uy; 32uy; 119uy; 105uy; 108uy; 108uy; 32uy; 98uy;
101uy; 32uy; 97uy; 32uy; 115uy; 116uy; 114uy; 105uy; 110uy; 103uy; 32uy;
111uy; 102uy; 32uy; 98uy;121uy; 116uy; 101uy; 115uy]
> 0x06D;;
val it : int = 109
But what happens if you have to convert between types? Well, F# has conversion methods like so:
> let x = 42;;
val x : int
> let bigx = Int64.of_int x;;
val bigx : int64
> bigx;;
val it : int64 = 42L
The first statement let x = 42 and the resulting line val x : int is an example of type inference. The F# compiler infers that 42 is of type Int32. OK, not really too much here to write home about. The second statement Int64.of_int actually converts x to a type of Int64, as demonstrated by the output “42L”. Again, not too much here to write home about. There are methods to convert the types between each other. I just didn't write them all. Type inference is great, but you have to be careful when you try things like this:
> let reallybignum = 123456789456123789;;
let reallybignum = 123456789456123789;;
-------------------^^^^^^^^^^^^^^^^^^^
stdin(4,19): error: error: This number is outside the allowable range for 32-bit signed integers
Oops, I tried to stuff a number larger than what a 32 bit number can hold. To fix this, we need to specify a 64 bit integer:
> let reallybignum = 123456789456123789L;;
val reallybignum : int64
OK, as exciting as writing about types and type inference are there is another part to this post. I was poking around through the source code and came across the conversion code F# uses. Here’s the method for converting an int to other data types:
let inline int32 (x: ^a) = (^a : (static member ToInt32: ^a -> int32)(x))
when ^a : string = (System.Int32.Parse(castToStringx,System.Globalization.CultureInfo.InvariantCulture))
when ^a : float = (# "conv.i4" x : int32 #)
when ^a : float32 = (# "conv.i4" x : int32 #)
when ^a : int64 = (# "conv.i4" x : int32 #)
when ^a : int32 = (# "conv.i4" x : int32 #)
when ^a : int16 = (# "conv.i4" x : int32 #)
when ^a : nativeint = (# "conv.i4" x : int32 #)
when ^a : sbyte = (# "conv.i4" x : int32 #)
when ^a : uint64 = (# "conv.i4" x : int32 #)
when ^a : uint32 = (# "conv.i4" x : int32 #)
when ^a : uint16 = (# "conv.i4" x : int32 #)
when ^a : unativeint = (# "conv.i4" x : int32 #)
when ^a : byte = (# "conv.i4" x : int32 #)
Wow, there’s a lot going on here, but overall it should look familiar; it's a function. The inline keyword is a pseudo-function marker for code expansion. Which means the compiler will copy the function inline to the call site. The ^a parameter designates a static head-type, which means the type must be known at compile time. The : type parameter in this case is a type constraint on the value. The (# “conv.i4” : int32 #) is a special syntax for a feature of the F# language, inline il. I know I went through that fast, but at this point a lot of this stuff is specific to the compiler. More detail than I can explain.

You could read this line: when ^a : int64 = (# "conv.i4" x : int32 #) as "when ^a is a type of int64 use the il instruction conv.i4 passing in the value (x) to convert and tell the compiler the type is an int32".

A little about the IL part of the line; (# “conv.i4” x : int32 #). The (# #) block tells the compiler, here comes an IL instruction. The conv.i4 is the il opt code for convert to an int32, x is value to convert and the : int32 completes the IL instruction to enforce the int32 type.

Even though this example if fairly implicit, I ran and got my copy of Expert .Net 2.0 IL Assembler book by Serge Linden and found conv is indeed the IL code for convert operations and i4 is the int32 type. Conv takes the value from the stack, converts it and puts it back. Type conversions are tricky, if you reduce the size of a value, i.e. – int64 -> int32, the most significant bytes are throws away. Likewise if you increase the size of the value int32 -> int64 the value is zero extended.

let reallybignum = 123456789456123789L;;
val reallybignum : int64
> let truncated = Int32.of_int64 reallybignum;;
val truncated : int32
> truncated;;
val it : int32 = -1062963315
If we look at how F# handles these conversions, we find the code:
when ^a : int64 = (# "conv.ovf.i4" x : int32 #)
The optcode conv.ovf.i4 is the IL overflow conversion operator. If the conversion truncates, an Overflow exception is thrown. That’s all for now, comments, questions and corrections are welcome!

Sunday, February 10, 2008

Project Euler

Wow - If you are looking for a great web site for exercising you skills in F#, or your programming language of choice, then sign up for ProjectEuler. I've made it through a few of the problems, and they are mind stretchers. I’ve had to try to remember stuff I haven't had to think about for over 15 years! Another advantage of this site is once you solve the problem, you can see how others have solved them in a variety of languages.

Sunday, January 27, 2008

Walking the F# List namespace - Post #3

This will probably be my last series on examining the source behind the List namespace. Only because I was running out of interesting functions that have interesting implementations, or functions that I feel I could adequately explain. And frankly, writing that List.filter actually imlements List.filter from another namespace is not too interesting. So, I took a look inside list.fs to find some interesting implementations and found a few more functions to walk through. List.assoc: List.assoc is roughly equivalent to a hashtable or dictionary lookup in c#, except the list is a list of tuples. (Also, I’m not sure if there are any index optimizations either.) Throws a not_found() exception if the key is not found. The signature for List.assoc is ‘a -> (‘a * ‘b) list -> ‘b
‘a – the key value to find
(‘a * ‘b) list – the source tuple list
‘b – the value for the key.
Here’s an example, using FSI:
> let list = [for x in 1 to 50000 -> (x,x*x)];;
val list : (int * int) list
>let findSquare s = List.assoc s list;;
val findSquare : int -> int
> findSquare 14513;;
val it : int = 210627169
If we look at the source code, List.assoc is implemented as:
let rec assoc x l =
match l with
|[] -> not_found()
|((h,r)::t) -> if x = h then r else assoc x t
The interesting code block with this function is the ((h,r)::t) syntax. The syntax of (h,r) is breaking up the tuple in the first element on the list, with h getting the key and r getting the value. If the key matches the passed in value, then return r. Also, if the list is empty ([]), then raise the not_found() exception. (I didn’t go into any detail about the rec or match, I describe them in further detail in my previous posts.) The documentation for List.assoc suggests we use List.try_assoc. So, lets try. List.try_assoc: essentially the same as List.assoc, with two main differences. First, try_assoc, will not throw a not_found() exception if the key is not found. Second, try_assoc returns an option type. When using F# code from other .Net languages, the empty option type (None) is equalivant to the null value. To create a value of the option type, you need to use Some or None. The signature for List.try_assoc is: ‘a -> (‘a * ‘b) list -> ‘b option
‘a – the first tuple element to find
(‘a * ‘b) list – the soure list
‘b option – the returned value
Here’s an example, from FSI.
> let list = [for x in 1 to 50000 -> (x,x*x)];;
val list : (int * int) list
> findSquare 50001;;
val it : int option = None
>findSquare 23;;
val it : int option = Some 529
If we look at the source code, List.try_assoc is implemented as
let rec try_assoc x l =
match l with
|[] -> None
|((h,r)::t) -> if x = h then Some(r) else try_assoc x t
Wow, this looks surprisingly similar to List.assoc, with the exception of the return types of None and Some(r). List.assq: List.assq is almost like List.assoc, except is used the PhysicalEquality operator. PhysicalEquality is defined in as “Reference/physical equality. True if boxed versions of the inputs are reference-equal, OR if both are value types and the implementation of Object.Equals for the type of the first argument returns true on the boxed versions of the inputs.” as defined in: http://research.microsoft.com/fsharp/manual/FSharp.Core/Microsoft.FSharp.Core.LanguagePrimitives.html So, generally speaking, List.assq does reference equality on values on the stack, whereas value equality compares values on the heap. I posted a question on an implementation of PhysicalEquality here (Kudos to zakaluka!) Anyway here’s how List.assq is implemented:
let rec assq x l =
match l with
|[] -> not_found()
|((h,r)::t) -> if PhysicalEquality x h then r else assq x t
List.try_assq is exactly the same as List.try_assoc, except the return type is an option type. Here’s the implementation:
let rec try_assq x l =
match l with
|[] -> None
|((h,r)::t) -> if PhysicalEquality x h then Some(r) else try_assq x t
List.for_all: Returns true if all the elements in the list satisfy the given predicate, false if one fails. The list elements are anded together, visualized as p iO && p i1 && .. p iN. The signature for list.for_all is (‘a -> bool) -> ‘a list -> bool
(‘a -> bool) – A function that evaluates the list element and returns a bool
‘a list -> the list to process
Bool -> return value
Here’s an example:
[<Test>]
member t.For_allExample () =
let list1 = [0 .. +2 .. 20]
let list2 = [1 .. 10]
let evens = List.for_all (fun x -> x % 2 = 0) list1
let notevens = List.for_all (fun x -> x % 2 = 0) list2
Assert.IsTrue(evens)
Assert.IsFalse(notevens)
List.for_all implements
let rec for_all f l1 = Microsoft.FSharp.Primitives.Basics.List.for_all f l1
Hmm, not so much fun to describe.....so on we go. List.for_all2: Returns true if all the elements in two lists satisfy the given predicate, false if one fails. The lists must have the same length. The signature for list.for_all2 signature is (‘a -> ‘b -> bool) -> ‘a list -> ‘b list -> bool.
(‘a -> ‘b -> bool) – A function that takes an element from the first list and the second list and returns a bool
‘a list – the first list
‘b list – the second list
Bool -> the return value
Here’s an example:
[<Test>]
member t.For_all2Example () =
let list1 = [0 .. +2 .. 20]
let list2 = [20 .. +2 .. 40]
let list3 = [1 .. 10]
let evens = List.for_all2 (fun x y -> x % 2 = 0 && y % 2 = 0 ) list1 list2
let notevens = List.for_all2 (fun x y -> x % 2 = 0 && y % 2 = 0) list2 list3
Assert.IsTrue(evens)
Assert.IsFalse(notevens)
List.for_all2 implements:
let rec for_all2 f l1 l2 =
match l1,l2 with
|[],[] -> true
|(h1::t1),(h2::t2) -> f h1 h2 && for_all2 f t1 t2
|_ -> invalid_arg "for_all2"
This recursive function is interesting, in particular the line (h1::t1),(h2::t2) -> f h1 h2 && for_all2 f t1 t2. The function aggregates the results of the function and the list elements!

Friday, January 18, 2008

Dustin Campbell on F#

I caught Dustin Campbell's presentation @ Code Mash. It was a great experience for me to just watch someone talk and type F# code for 40 minutes. Dustin has started blogging about F# also. Here's his podcast with Scott Hanselman.