Wow – What a trip. F#, C#, TDD, COBOL and enlightenment

Life truly has a fantastic humbling ability. A few months ago I was in the midst of a deep dive, swimming in the goodness of F#. I had all sorts of wonderful ideas on future blog posts. I was treading in a veritable pool of functions, records, pattern matching, currying; all while not getting bogged down in semi-colons and missing curly brackets! I was on top of the world.

Then life reminds you what is truly important, my wife and I were blessed with our 3rd child at the end of May. What a wonderful addition to our family. So after one month of anxiety waiting for the baby to arrive and then another two months of interrupted sleep, burping, changing diapers and worrying about poop, (those of you with kids understand!) my thoughts never strayed too far the pool. So here I go, with a great big cannonball, I splash back into the blog waters. It was during those few months out of the pool, when my only concentration on coding was in C#-ville from 9 to 5, several statements of a profound nature crept into my head. I have been long since spoiled on the benefits of Test Driven Development (TDD); write a test, write some code, make the test pass, refactor, run tests - beautiful simplicity.

As simple as that is, one of my co-workers would always write extra code; code that was not needed to make the tests pass. He could not bring himself to just write the simple code that made the test pass. I would always ask the question, why did you write that? The test would still pass if that block was not there.

His reply was always, "We'll need it soon." Rather than argue, because he would be right, and delete the code he had written, we would go on our way and complete the task at hand. (By the way, when teaching TDD to someone new, deleting code that has no coverage is a great way to enforce the rule that you shouldn't write any code that is not a result of a test!)

So one day, our group was having a small discussion while taking a break from coding and talking about past jobs and different languages we knew. This guy mentioned he programmed COBOL for several years prior to migrating to java and C#. While I snickered and mentioned my own experience with COBOL; a mere two semesters in college and a professor that couldn't code, he laughed and said, "you had to be careful about your code, if you had a bug, you could crash the entire system for everyone". So after my casual chuckling, I repeated his statement and added something to the effect of, "wow, I would have surly pissed of a lot of other developers, my code is always buggy, I rely on tests to find them, not the final run." As soon as I said that, the little light switch in my head clicked and I then realized why our coding styles were so different, why he just couldn't write the simple code and trust the tests.

Being faced with the situation of crashing an entire mainframe, COBOL developers, at least this one, had to make sure the code was sound before submitting your code to the queue. Comparatively speaking, if I crash my system, 1 person is down for a little while, but if you take down an entire mainframe region, there's a little more impact. (I am not picking on COBOL developers here, it's just part of the story.)

Compare that to TDD, where my mindset is, "hey, if I refactor this chunk of code into a method, these two other block are the same, and could be made a delegate. Let's try that!" Or, things like "I don't like that switch statement, lets make it a dictionary". Compile, Run Tests…. oops, the entire test fixture fails, oh well, ctl-z.

This tantalizing story leads me up to these few points:

Profound statement #1: TDD is an incredible gift. Having a series of unit tests gives you the confidence and comfort to try really crazy things with your code. It also gives you the ability to enforce some of the higher level design patterns to make your code truly awesome; open closed principle, single responsibility, Liskov substitution principle, to name a few.

Profound statement #2: TDD has made me lazy. I let my tests find my bugs.

Profound statement #3: TDD has made me confident. I realize that this statement might conflict with statement #2, but mixing of these traits just makes me smile. If my QA tester finds a bug, I can write a test to re-produce the issue and can guarantee that bug will not surface again.

Profound statement #4: You need productivity tools like ReSharper and TestDriven .Net. Not to sound like a commercial for ReSharper, but man, that stuff is like crack! When you combine those tool suites, with your IDE, you can do really fun things when pairing like "mouse free Thursday's". Once you learn all the keyboard shortcuts within an IDE, you realize the mouse is a distraction. The speed, fluency and pace you can write, and refactor code triples when you are not fumbling around with that little pointy arrow. You find that your thoughts are focused on your end goal and not scrolling through your code looking for those pesky syntax problems like missing curly bracket or missing semi-colon. And yes, it is a bad habit to rely on the compiler to tell you were that missing semi-colon is!

So now here's my disclaimers.

1. I have respect for COBOL programmers, my intent here was to not poke fun at them.
2. I am not pushing ReSharper or TestDriven.Net, those are the tools we use. There are productivity tools out there. ( If Dustin Campbell reads this, I'll have already heard the groans when I mention ReSharper!)
3. Mouse-Free Thursdays is an awesome experience. While pairing, each developer unplugs their mouse, only for Thursday and only while coding. After a few MFT's, you find you are a much more efficient coder.

F# Pattern Matching

After my last post on the sample F# test runner, I realized I missed a post on one of the critical constructs F# has to offer; pattern matching. Pattern matching does exactly what it sounds like, matches patterns.

let action x = 
     match x with
     | "Red" -> "Stop"
     | "Green" -> "Go"
     | _ -> "I don't know"

In this example, when I pass the colors Red or Green to the action function, it returns the matching value. If I were to pass blue to the function, I would get back "I don't know".

> action "Red";; val it : string = "Stop" > action "Blue";; val it : string = "I don't know"

Notice the "_" at the end, it is a wildcard and it makes this pattern match exhaustive. If I didn't include the _, F# would complain I had an incomplete match. You do have to be careful of where you put your wildcard character. The following function would create a scenario where the "Green" match statement would not be evaluated:

let action x = 
     match x with
     | "Red" -> "Stop"
     | _ -> "I don't know"
     | "Green"  -> "Go"

This control structure should seem very familiar to anyone coming from the C language family. Indeed, the action function could be written as an if..else statement or a switch statement in C#:

   public static string Action(string color)
        {
            if (color == "Red")
            {
                return "Stop";
            }
            else if( color == "Green")
            {
                return "Go";
            }
            else
            {
                return "I don't know";
            }
        }

You can include conditions within your matches, take the following function:

let isEven x =
     match x with
     | _ when x % 2 = 0 -> true
     | _ -> false

With conditions however, you need to have a catch all. If I were to change the above function to read like this:

let isEven x =
     match x with
     | _ when x % 2 = 0 -> true
     | _ when x % 3 = 0 -> false

the complier would complain of an incomplete pattern match. With pattern matching you can decompose structured values. This example breaks down the tuples for the match.

let isCubeEven x =
     match x with
     | _ when snd x % 2 = 0 -> (fst x,true)
     | _ -> (fst x,false)

This function returns the following in fsi:

> isCubeEven (3,9);; val it : int * bool = (3, false)

The example "isCubeEven" highlights two features of what makes F# exciting. First, notice that I didn't need to explicitly annotate any of my variables with types? The F# complier inferrs that x is a tuple because of the way I use it within my function. And second, the fst and snd functions that let me easily work with tuples to get the first and second values. Pattern matching becomes really powerful when you combine it with rec, the keyword for recursive functions. Take this function for adding up the numbers in a number:

let rec sumDigits x y =
    match x with
          | x when x = 0 ->  y
          | x -> sumDigits (x / 10) (x % 10) + y

This particular function I wrote to solve some of the Project Euler problems. With this function y accumulates the running summed value and x gets chopped up from right to left.

> sumDigits 123456789 0 val it : int = 45

When you combine the rec keyword and pattern matching, the possibilities are infinite. Take the fold_left function from the F# list namespace:

let rec fold_left f s l = 
     match l with 
     | [] -> s 
     | (h::t) -> fold_left f (f s h) t

The fold_left function takes a function f, a seed value s, and a list l. The match uses the list processing keys to get the first element h and remainder of the list t. Then calls itself with the same function, the result of the function applied to the seed value and the first element and the rest of the list. Pattern matching also works great on records, here's a great post on that by Mike Gold. Question and comments are welcome!

Sample F# Test Runner (and C# too...)

While I was fooling around with F# and testing my functions, I got really annoyed with switching between applications to run my tests, Visual Studio and NUnit Gui. The source of my fustration is during the day I get to use a fantastic tool TestDriven.Net to run my unit tests in C# and Visual Studio. With F#, however, TestDriven.Net does not recognize the different syntax and no tests run. So I decided to write my own test runner, by taking advantage of .Net's FileSystemWatcher class. (Note, I used the FileSystemWatcher.Changed event.¹ ) The first iteration I wrote in C# so I could have an excuse to use xUnit. I probably over engineered the C# code, but old habits die hard. In the C# implementation, there are three files. The program class sets everything up, the HarnessRunner class sets up the filesystem watcher and listens for the file changed event, and finally the ProcessStarter to run the process. The application is configurable to use the app.config or to pass any necessary commands to the program. ( Hence a little of the bloat.)

using System;
using System.IO;

namespace HarnessRunner
{
    public class program
    {
        [STAThread]
        public static void Main(string[] args)
        {
            Settings settings = Settings.Default;

            if (ValidateInputValue(settings.TestRunnerCommand))
            {
                Console.WriteLine("Enter the fully qualified command to run:\r\n");
                settings.TestRunnerCommand = Console.ReadLine();
            }

            if (ValidateInputValue(settings.TestAssembly))
            {
                Console.WriteLine("Enter the file to test:\r\n");
                settings.TestAssembly = Console.ReadLine();
            }

            if (ValidateInputValue(settings.TestRunnerSwitches))
            {
                Console.WriteLine("Enter any swtiches to the test runner:\r\n");
                settings.TestRunnerSwitches = Console.ReadLine();
            }
                     
            Console.WriteLine("Setting up the watcher to run: \r\n{0} {1} {2}", Path.GetFileName(settings.TestRunnerCommand),
                              Path.GetFileName(settings.TestAssembly),settings.TestRunnerSwitches);
            try
            {
                HarnessRunner runner = new HarnessRunner(new FileSystemWatcher(), new ProcessStarter());
                runner.InitializeFileSystemWatcher(settings.TestAssembly);
            }
            catch (Exception e)
            {
                Console.WriteLine("There was an exception during the run: {0}{1}{2}", e.Message, Environment.NewLine,
                                  e.StackTrace);
            }
            Console.ReadLine();
        }

        private static bool ValidateInputValue(string command)
        {
            return string.Compare(command, string.Empty) == 0;
        }
    }
}
using System;
using System.Diagnostics;
using System.IO;

namespace HarnessRunner
{
    public interface IStartTestHarnesses
    {
        string Start();
    }

    public interface IFileSystemWatcher
    {
        bool EnableRaisingOfEvents { get; set; }
        string Path { get; set; }
        NotifyFilters NotifyFilter { get; set; }
        event EventHandler Changed;
    }

    public class ProcessStarter : IStartTestHarnesses
    {
        public string Start()
        {
            Settings settings = Settings.Default;
            Process process = new Process();
            process.StartInfo.FileName = settings.TestRunnerCommand;
            process.StartInfo.Arguments = settings.TestAssembly + " " + settings.TestRunnerSwitches;
            process.StartInfo.UseShellExecute = false;
            process.StartInfo.RedirectStandardOutput = true;
            process.Start();

            return process.StandardOutput.ReadToEnd();
        }
    }
}
using System;
using System.IO;

namespace HarnessRunner
{
    public class HarnessRunner
    {
        private FileSystemWatcher fileSystemWatcher;
        private IStartTestHarnesses startTestHarnesses;

        public HarnessRunner(FileSystemWatcher fileSystemWatcher, IStartTestHarnesses startTestHarnesses)
        {
            this.fileSystemWatcher = fileSystemWatcher;
            this.startTestHarnesses = startTestHarnesses;
            fileSystemWatcher.Changed += fileSystemWatcher_Changed;
        }

        private void fileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
        {
            Console.WriteLine(startTestHarnesses.Start());
        }

        public void InitializeFileSystemWatcher(string filetowatch)
        {
            fileSystemWatcher.Path = Path.GetDirectoryName(filetowatch);
            fileSystemWatcher.Filter = Path.GetFileName(filetowatch);
            fileSystemWatcher.NotifyFilter = NotifyFilters.LastWrite;
            fileSystemWatcher.EnableRaisingEvents = true;
        }
    }
}

So, once I had that working, I decided to write the F# equivalent. There are two functions, one that starts the process and one to setup the filesystemwatcher. Other than that, there are just some config options.

#light

open System
open System.IO
open System.Diagnostics
open System.Configuration

let mutable command = ConfigurationManager.AppSettings.Item("TestRunnerCommand")
let mutable testAssembly = ConfigurationManager.AppSettings.Item("TestAssembly")
let mutable commandSwitches = ConfigurationManager.AppSettings.Item("TestRunnerSwitches")

let startProcess f =
     let p = new Process()
     p.StartInfo.FileName <- command
     p.StartInfo.Arguments <- f ^" "^commandSwitches
     p.StartInfo.UseShellExecute <- false
     p.StartInfo.RedirectStandardOutput <- true
     let started = p.Start()
     printfn "%O" (p.StandardOutput.ReadToEnd())

let SetupFileSystemWatcher f =
     let fileSystemWatcher = new FileSystemWatcher()
     fileSystemWatcher.Path <- System.Environment.CurrentDirectory
     fileSystemWatcher.Filter <- f
     fileSystemWatcher.NotifyFilter <- NotifyFilters.LastWrite
     fileSystemWatcher.EnableRaisingEvents <- true
     fileSystemWatcher.Changed.Add(fun _ -> startProcess f)
   
if command = ""  command = null then
     printfn "Enter the fully qualified command to run:\r\n"
     command <- Console.ReadLine()       
if testAssembly = ""  testAssembly = null then      
     printfn "Enter the file to test:\r\n"
     testAssembly <- Console.ReadLine()
if commandSwitches = ""  commandSwitches = null then
      printfn "Enter any swtiches to the test runner:\r\n"
      commandSwitches <- Console.ReadLine()

printfn "Setting up watcher for %A" testAssembly  
SetupFileSystemWatcher testAssembly 
read_line() 

So, now that the code is posted, I'm left with some sort of follow up point to wrap this post up. The only problem is, I can't seem to come up with any points! I put together this article to show a program that performs the same function, implemented in two separate languages. Comments, feedback and discussions are welcome! ¹ For some reason, the FileSystemWatcher.Changed event fires three times when my assembly is compiled.

Visual Studio 2008 launch in Detriot

Wow - I just came back from Microsoft's Hero's Happen Here event in Detroit to launch Visual Studio 2008, Windows Server 2008 and SQL Server 2008. It was a well organized event, but more important to me was to catch up with everyone I met at this years CodeMash event. I always leave these events with a tremendous sense of community. Special thanks to Keith Elder, who hosted the geek dinner afterwards. (An extra special thanks to Randy Pagels!) At this event I was able to meet fellow F# blogger Dustin Campbell and share a few thoughts on F#. I wish I had more time to talk shop......... I was also able to meet Jeff McWherter, Joe Wirtley and spend six quality hours in the car with Corey Haines and Michael Letterle. Well, Corey was only awake for three.... Another reason for the feeling of community was the presence and dedication of Microsoft's regional representatives, Josh Holmes, Brian Prince, Darryl Hogan and Jeff Blankenburg. Hope to see you all again soon!

Records and Discriminated Unions

Like most other programming languages, it is possible with F# to create your own types when needed. Because F# is a blend of both functional languages and imperative languages, you get support for types from a functional perspecive as well as types when you have to deal with OO programming. Records Records are concrete type definitions, if you have an OO background, records will look very similar to classes. To create a record, you need to tell F# the type name, the labels and the label type. Here’s an example, using FSI

> type FamilyMember = {Name: string; relation: string};;
type FamilyMember = {Name: string; relation: string;}

There are two different ways to create record. The first way:

> {Name = "Nate";relation = "Father"};;
val it : FamilyMember = {Name = "Nate"; relation = "Father";}

Or a more explicit syntax:

> {new FamilyMember with Name = "Kelly" and relation = "Mother"};;
val it : FamilyMember = {Name = "Kelly"; relation = "Mother";}

F# makes it easy for you to access the record labels:

> let f = {new FamilyMember with Name = "Nate" and relation = "father"};;
val f : FamilyMember
> f.Name;;
val it : string = "Nate"

Records can be cloned, and like everything else with F#, it’s amazingly simple!

> let d1 = {new FamilyMember with Name = "reilly" and relation = "daughter"};;
val d1 : FamilyMember
> let d2 = {d1 with Name = "Cora"};;
val d2 : FamilyMember
> d2;;
val it : FamilyMember = {Name = "Cora";
relation = "daughter";}

Records can be results of functions as well. Take the following type, which has a count of files and a count of directories:

type FilesAndDirectorys = {Files: int; Directories: int;}

We can use it for a function like so:

let GetFilesAndDirectories loc =
let dirs = Directory.GetDirectories(loc)
let files = Directory.GetFiles(loc)
let results = {new FilesAndDirectorys with Files = files.Length and Directories = dirs.Length}
results

Next call the function with the path you want to search

let counts = GetFilesAndDirectories @"C:\" The identifier counts now contains the following data:

val it : FilesAndDirectorys = {Files = 9; Directories = 18;}

There is another type, discriminated union types, but I'll save those for another blog entry.

Basic Type conversions with F#

As I work my way through the ProjectEuler problems they forced me to look into working with the different types in F#. F# is a very strongly typed language, but it has the same basic types as the types in the other .Net languages. Here is a list of the basic types and how to tell the compiler you want to use them:

let int = 42
let string = "This is a string"
let char = 'c'
let bool = true
let bytearray = "This is a byte string"B
let hexint = 0x34
let octalint = 0o42
let binaryinteger = 0b101010
let signedbyte = 68y
let unsignedbyte = 102uy
let smallint = 16s
let smalluint = 16us
let integer = 345l
let usignedint = 345ul
let nativeint = 765n
let unsignednativeint = 765un
let long = 12345678912345L
let unsignedlong = 12345678912345UL
let float32 = 42.8F
let float = 42.8

(Definitions for the types are listed here.) F# also has BigInt and BigNum types, they stand for arbitrary large integer and arbitrary large number respectively. (I don't know how big they are yet.)

let bigInt = 9876543219876I
let bigNum = 123456789987654N

The F# compiler will determine the types you are working with, a feature called Type Inference. To see what types are inferred, compile your fs files using the –i switch to create an FSI (F# Interface file) or use the mouse in Visual Studio. Most of the F# programming you will do, inference will work. Now, I know some of you might be thinking; “Yeehaw! I don’t have to worry about declaring types! I’m free!!” Well…some of you might have, I did. If you want force a type and not let inference handle it for you, you have to use the conventions above like so:

> 3423456573476N;;
val it : bignum = 3423456573476N
> "This will be a string of bytes"B;;
val it : byte []
= [84uy; 104uy; 105uy; 115uy; 32uy; 119uy; 105uy; 108uy; 108uy; 32uy; 98uy;
101uy; 32uy; 97uy; 32uy; 115uy; 116uy; 114uy; 105uy; 110uy; 103uy; 32uy;
111uy; 102uy; 32uy; 98uy;121uy; 116uy; 101uy; 115uy]
> 0x06D;;
val it : int = 109

But what happens if you have to convert between types? Well, F# has conversion methods like so:

> let x = 42;;
val x : int
> let bigx = Int64.of_int x;;
val bigx : int64
> bigx;;
val it : int64 = 42L

The first statement let x = 42 and the resulting line val x : int is an example of type inference. The F# compiler infers that 42 is of type Int32. OK, not really too much here to write home about. The second statement Int64.of_int actually converts x to a type of Int64, as demonstrated by the output “42L”. Again, not too much here to write home about. There are methods to convert the types between each other. I just didn't write them all. Type inference is great, but you have to be careful when you try things like this:

> let reallybignum = 123456789456123789;;
let reallybignum = 123456789456123789;;
-------------------^^^^^^^^^^^^^^^^^^^
stdin(4,19): error: error: This number is outside the allowable range for 32-bit signed integers

Oops, I tried to stuff a number larger than what a 32 bit number can hold. To fix this, we need to specify a 64 bit integer:

> let reallybignum = 123456789456123789L;;
val reallybignum : int64

OK, as exciting as writing about types and type inference are there is another part to this post. I was poking around through the source code and came across the conversion code F# uses. Here’s the method for converting an int to other data types:

let inline int32 (x: ^a) = (^a : (static member ToInt32: ^a -> int32)(x))
when ^a : string = (System.Int32.Parse(castToStringx,System.Globalization.CultureInfo.InvariantCulture))
when ^a : float = (# "conv.i4" x : int32 #)
when ^a : float32 = (# "conv.i4" x : int32 #)
when ^a : int64 = (# "conv.i4" x : int32 #)
when ^a : int32 = (# "conv.i4" x : int32 #)
when ^a : int16 = (# "conv.i4" x : int32 #)
when ^a : nativeint = (# "conv.i4" x : int32 #)
when ^a : sbyte = (# "conv.i4" x : int32 #)
when ^a : uint64 = (# "conv.i4" x : int32 #)
when ^a : uint32 = (# "conv.i4" x : int32 #)
when ^a : uint16 = (# "conv.i4" x : int32 #)
when ^a : unativeint = (# "conv.i4" x : int32 #)
when ^a : byte = (# "conv.i4" x : int32 #)

Wow, there’s a lot going on here, but overall it should look familiar; it's a function. The inline keyword is a pseudo-function marker for code expansion. Which means the compiler will copy the function inline to the call site. The ^a parameter designates a static head-type, which means the type must be known at compile time. The : type parameter in this case is a type constraint on the value. The (# “conv.i4” : int32 #) is a special syntax for a feature of the F# language, inline il. I know I went through that fast, but at this point a lot of this stuff is specific to the compiler. More detail than I can explain.

You could read this line: when ^a : int64 = (# "conv.i4" x : int32 #) as "when ^a is a type of int64 use the il instruction conv.i4 passing in the value (x) to convert and tell the compiler the type is an int32".

A little about the IL part of the line; (# “conv.i4” x : int32 #). The (# #) block tells the compiler, here comes an IL instruction. The conv.i4 is the il opt code for convert to an int32, x is value to convert and the : int32 completes the IL instruction to enforce the int32 type.

Even though this example if fairly implicit, I ran and got my copy of Expert .Net 2.0 IL Assembler book by Serge Linden and found conv is indeed the IL code for convert operations and i4 is the int32 type. Conv takes the value from the stack, converts it and puts it back. Type conversions are tricky, if you reduce the size of a value, i.e. – int64 -> int32, the most significant bytes are throws away. Likewise if you increase the size of the value int32 -> int64 the value is zero extended.

let reallybignum = 123456789456123789L;;
val reallybignum : int64
> let truncated = Int32.of_int64 reallybignum;;
val truncated : int32
> truncated;;
val it : int32 = -1062963315

If we look at how F# handles these conversions, we find the code:

when ^a : int64 = (# "conv.ovf.i4" x : int32 #)

The optcode conv.ovf.i4 is the IL overflow conversion operator. If the conversion truncates, an Overflow exception is thrown. That’s all for now, comments, questions and corrections are welcome!

Project Euler

Wow - If you are looking for a great web site for exercising you skills in F#, or your programming language of choice, then sign up for ProjectEuler. I've made it through a few of the problems, and they are mind stretchers. I’ve had to try to remember stuff I haven't had to think about for over 15 years! Another advantage of this site is once you solve the problem, you can see how others have solved them in a variety of languages.