Archives for : Programming Standards

Live Filter Design Pattern in XAML

GetTheCode

I decided I should start titling my posts with XAML instead of WPF as they are applicable regardless of whether you are developing in Windows 7 with WPF or Windows 8. I’m going to take the BreadCrumb design pattern sample and extend it to include the Live Filter design pattern.

The premise behind the Live Filter design pattern is pretty simple, show the user everything and then allow them to filter the results, immediately showing how the filter affects the results. As noted in the BreadCrumb post dense trees can easily become overwhelming to users. Providing a simple, easy to use live filter makes the experience significantly better.LiveFilter

In the above image you can see there we have a search text box with two filter parameters. As the user types into the search box the results are immediately filtered.

<Grid Grid.Row="0" Grid.ColumnSpan="3" Margin="5">
    <xctk:WatermarkTextBox Margin="2" Watermark="Search" Text="{Binding SearchValue, Mode=TwoWay, UpdateSourceTrigger=PropertyChanged}" />
    <Button Margin="5 0 5 0" Width="16" Height="16" HorizontalAlignment="Right" FontFamily="Segoe UI Symbol" Opacity=".75" ToolTip="Clear" Command="{Binding ClearCommand}">
        <TextBlock Margin="0 -3.5 0 0" Foreground="#FF63A3B2">👽</TextBlock>
    </Button>
</Grid>
<StackPanel Grid.Row="1" Grid.ColumnSpan="3" Margin="5" Orientation="Horizontal">
    <CheckBox Margin="5" IsChecked="{Binding SearchProperty2, Mode=TwoWay}" Content="Include Property 2 In Search" />
    <CheckBox Margin="5" IsChecked="{Binding SearchProperty3, Mode=TwoWay}" Content="Include Property 3 In Search" />
</StackPanel>

Above is the XAML for the search controls. Most of it is standard MVVM binding. The problem with standard MVVM binding is that for textbox it only updates the underlying view model property on lost focus. That doesn’t help us as we want to update the results in real-time. To fix this the UpdateSourceTrigger is set to PropertyChanged. This way as soon as the user types a letter the property is changed and we can perform a search. (I’m actually using the nuget package for the Extended WPF Toolkit to provide a watermark textbox but it works like a regular textbox).

<TreeView Grid.Row="2" Grid.RowSpan="2" Margin="5" 
        ItemsSource="{Binding ChildNodes}"
        VirtualizingStackPanel.IsVirtualizing="True"
        VirtualizingStackPanel.VirtualizationMode="Recycling"
        SelectedItemChanged="TreeView_SelectedItemChanged">
    <TreeView.ItemContainerStyle>
        <Style TargetType="{x:Type TreeViewItem}">
            <Setter Property="IsExpanded" Value="{Binding IsExpanded, Mode=TwoWay}" />
            <Setter Property="Visibility" Value="{Binding Visibility, Mode=TwoWay}" />
        </Style>
    </TreeView.ItemContainerStyle>
    <TreeView.ItemTemplate>
        <HierarchicalDataTemplate ItemsSource="{Binding Children}">
            <StackPanel>
                <TextBlock Text="{Binding Property1}" FontSize="18" />
            </StackPanel>
        </HierarchicalDataTemplate>
    </TreeView.ItemTemplate>
</TreeView>

The first important part is the TreeViewItemContainerStyle. The nodes implement the properties IsExpanded and Visibility. This is extremely important. We don’t want to have to deal with the ItemContainerGenerator. This can be a major pain the rear and will make the code extremely sluggish. If we’re going to go with binding the nodes in the tree let’s take full advantage of it. The other important part is setting up the VirtualizingStackPanel values. The search is really rather trivial and usually works extremely fast. The slow part is updating the UI. We leave that to the masters at Microsoft by binding everything and letting them figure out how to render it. But we can help things out a bit. By setting up the VirtualizingStackPanel the control doesn’t have to render all the children in the tree. Now use this with some caution. Because the nodes will only be updated as the user drags the scroll bar you can get some choppy and sluggish operations as the control updates.

string searchValue;
public string SearchValue
{
    get { return searchValue; }
    set
    {
        if (SetProperty(ref searchValue, value))
        {
            PerformSearch();
        }
    }
}

bool searchProperty2 = false;
public bool SearchProperty2
{
    get { return searchProperty2; }
    set 
    {
        if (SetProperty(ref searchProperty2, value))
        {
            PerformSearch();
        }
    }
}

bool searchProperty3 = false;
public bool SearchProperty3
{
    get { return searchProperty3; }
    set 
    { 
        if(SetProperty(ref searchProperty3, value))
        {
            PerformSearch();
        }
    }
}

These are the properties in the view model. We’re using BindableBase from Prism for the view model. Thus SetProperty returns a true if there was a change. We need to minimize extraneous calls to PerformSearch as much as possible.

private static object lockObj = new object();
private CancellationTokenSource CancellationTokenSource { get; set; }
private void PerformSearch()
{
    //if we're doing a search then we probably have some new search term
    clearCommand.RaiseCanExecuteChanged();

    lock (lockObj)
    {
        if (CancellationTokenSource != null)
            CancellationTokenSource.Cancel();

        CancellationTokenSource = new CancellationTokenSource();

        resultCount = 0;
    }

    Task.Run(() =>
    {
        DateTime now = DateTime.Now;
        try
        {
            if (string.IsNullOrEmpty(SearchValue))
            {
                ClearAllNodes(ChildNodes, Visibility.Visible, CancellationTokenSource.Token);
                return;
            }
            else
            {
                ClearAllNodes(ChildNodes, Visibility.Collapsed, CancellationTokenSource.Token);
            }

            var options = new ParallelOptions { CancellationToken = CancellationTokenSource.Token };
            try
            {
                Parallel.ForEach(ChildNodes, options, (childNode) =>
                {
                    PerformSearch(childNode, options);
                });
            }
            catch (OperationCanceledException)
            {
                //Noop
            }
        }
        finally
        {
            LastSearchTookInMilliseconds = (DateTime.Now - now).Milliseconds;
            OnPropertyChanged(() => ResultCount);
        }
    }, CancellationTokenSource.Token);
}

private void PerformSearch(RandomValueNode childNode, ParallelOptions options)
{
    if (options.CancellationToken.IsCancellationRequested)
        return;

    if (childNode.Property1.StartsWith(SearchValue) ||
        (SearchProperty2 && childNode.Property2.StartsWith(SearchValue)) ||
        (SearchProperty3 && childNode.Property3.StartsWith(SearchValue)))
    {
        Interlocked.Increment(ref resultCount);
        childNode.IsExpanded = true;
        childNode.Visibility = Visibility.Visible;
    }
    foreach (RandomValueNode node in childNode.Children)
    {
        if (options.CancellationToken.IsCancellationRequested)
            break;

        PerformSearch(node, options);
    }
}

And finally we do the search. To make this as responsive as possible we need to cancel the search each time as quickly as possible. Thus I’m using a CancellationToken in the Task.Run and the Parallel.ForEach to do the search. Remember that it is only effective to spin off threads from the thread pool if you can give them enough work. In my case there will only be ten nodes but I give each of those plenty to do. I’m passing the options into the PerformSearch so that we kill the search as quick as possible. As the code recurses I want to kill everything. It may seem like overkill to check for options.CancellationToken.IsCancellationRequested at the top of the method and in the foreach, and it may very well be.

So this pattern is pretty straightforward, simply update live results as the user changes the filter parameters. The key here is taking into account the result set size and to stop any existing search quickly, really try and take advantage of all that the TPL provides. As you can see in the image at the top I can iterate over 700,000 nodes pretty quickly and the UI is still pretty responsive. Of course that is really more because of the Task.Run and using VirtualizingStackPanel.

There is, of course, another way to do this. If you find that things are moving much too sluggish the other alternative is to show a minimal amount of results (like when doing a Google search and it begins to suggest terms). Then in the background have a thread that starts to update the results.

I would encourage you to get the code from the “GET THE CODE” button at the top.

Thanks for reading,
Brian

This continues my series on ways you’ve probably used design patterns in real-life and may not have even known it. The previous post was on the Adapter Design Pattern.
This is a kind of “catch-all” post where I want to talk not only about the Iterator Design Pattern but also custom enumerators for Parallel.ForEach and ensuring you give your threads enough work.

The iterator pattern is a way to move through a group of objects without having to understand the internals of the container of those objects. Anything in .NET that implements IEnumerable or IEnumerable<T> provides an iterator to move over the values. List<T> and Dictionary<TKey, TValue> are good examples.

If we look at my TPL sampler in my GreyScaleParallelSample we have the following code:

System.Drawing.Imaging.BitmapData bmData = bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
int stride = bmData.Stride;
unsafe
{
	byte* start = (byte*)(void*)bmData.Scan0;

	int height = bmp.Height;
	int width = bmp.Width;

	Parallel.For(0, height, y =>
	{
		byte* p = start + (y * stride);
		for (int x = 0; x < width; ++x)
		{
			byte blue = p[0];
			byte green = p[1];
			byte red = p[2];

			p[0] = p[1] = p[2] = (byte)(.299 * red
				+ .587 * green
				+ .114 * blue);

			p += 3;
		}
	});
}
bmp.UnlockBits(bmData);

This code is very similar to code I used in some image manipulation I had to implement. Here, however, all we’re doing is setting each pixel to grey scale (I’m not sure why but for some reason I use the British spelling of grey). If we look at it we’re iterating over the height and then by the width. But an image is really just a byte array where every three places identifies the blue, green and red bytes for a given pixel. We don’t need to treat it like a map with height and width.

Now to do this we’ll need a custom iterator (see? I brought it back to the purpose of this post 🙂 Fortunately Parallel.ForEach allows you to define an IEnumerable so that you can customize how it iterates over the values. We can just set up a simple for loop and yield on each value.

public static IEnumerable<int> ByVariable(int max, int increment)
{
	for (int i = 0; i < max; i+= increment)
		yield return i;
}

What this does is allow you to iteratate over a Parallel.ForEach by some amount up to some supplied maximum. I’ve added a new sample to my TPLSampler called GreyScaleBySingleParallelSample that uses this.

System.Drawing.Imaging.BitmapData bmData = bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
int stride = bmData.Stride;
System.IntPtr Scan0 = bmData.Scan0;
unsafe
{
	byte* start = (byte*)(void*)Scan0;

	Parallel.ForEach(ByVariable(bmp.Height * bmp.Width * 3, 3), i =>
	{
		byte* p = (start + i);
		byte blue = p[0];
		byte green = p[1];
		byte red = p[2];

		p[0] = p[1] = p[2] = (byte)(.299 * red
					+ .587 * green
					+ .114 * blue);
	});
}
bmp.UnlockBits(bmData);

The max value of ByVariable is the height of the image by the width times 3 (since each byte represents one color of the three that make up a pixel) and the amount to increment is by 3. This way we can move through the byte array 3 bytes (or 1 pixel) at a time.

So this is awesome, right? We’ll spin off a bunch of threads and this will crank through a big image in no time. So let’s run this against an 8 MB image and compare it to the first method.

Reseting Image
Starting Grey Scale Parallel Sample
Completed Grey Scale Parallel Sample
Grey Scale Parallel Sample ran in 00:00:00.1700515

Reseting Image
Starting Grey Scale By Single Parallel Sample
Completed Grey Scale By Single Parallel Sample
Grey Scale By Single Parallel Sample ran in 00:00:01.5654025

Wait, what? This second method runs significantly slower (and “Resetting” is spelled wrong). As I’ve mentioned in the past, when you can’t give your threads enough work such that you overcome the cost of having to spin up and/or set up the thread you just end up wasting time. If you’ve read my past posts on this, I know I may seem like I keep harping on this but it is important. I’ve seen quite a few cases where people think that the solution to a problem with a long running process is just to throw more threads at it. It may very well be that is a solution but you need to understand what your code is doing. It doesn’t make sense when optimizing code to just throw everything against a wall and see what sticks.

That being said, there are times where using the “ByVariable” enumerable is helpful. There is an interface I interact with that returns a string array where the values are grouped by (value, unit, error). I have to do a bunch of handling and work on the values that returned in the array. In this use it makes sense.

So what have we covered?

  1. What the Iterator Design Pattern is.
  2. It’s implementation in .NET.
  3. How to use a custom iterator in a Parallel.ForEach.
  4. Making sure to give each thread in a Parallel.For/Each enough work.

Thanks,
Brian

So you don’t need to know software design patterns. But, as I hope got across in my post, knowing and understanding patterns can only benefit you. I wanted to put together a real-life example of some of the instances where I’ve used patterns, even if the use of the patterns was unintentional. Hopefully you will get some use out of them.

I had a requirement where I had to track a list of files and needed to be able to save this list. But the list needed to be able to not only have a list of files but needed to support a list of lists. The use case was that the user could define a list of files say, “My files from client A.” The user could then put together a list of lists defined as, “My clients from the east coast” which would be comprised of lists from any clients on the east coast.

Of course this defines the composite pattern. The composite pattern is just an object that contains zero or more of that object. The two most obvious classes in .NET that use this are TreeViewItem and MenuItem. A MenuItem contains it’s own content as a MenuItem but also contains children MenuItems. In my case my class, “FileGroupModel” has a list of files (think of that list as the content of the class) as well as a list of FileGroupModels, which is exactly what the composite design pattern is.

Now to facilitate this I obviously need to save out the FileGroupModel, which is the memento pattern. As you’ll see in the code I went with a DataContract to save out the data.

using CompositeMementoSample.Infrastructure;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.IO;
using System.Linq;
using System.Runtime.Serialization;
using System.Text;
using System.Threading.Tasks;

namespace CompositeMementoSample.Models
{
    [DataContract(IsReference = true)]
    public class FileGroupModel : DomainObject
    {
        public virtual string OpenFileDialogFilter 
        {
            get { return "Files & Groups (*.*)|*.*|File Groups (*.fxml)|*.fxml"; }
        }
        public virtual string SerializedExtension 
        {
            get { return ".fxml"; }
        }

        [DataMember]
        private string name;
        public string Name
        {
            get { return name; }
            set
            {
                if (this.name != value)
                {
                    this.ValidateProperty("Name", value);
                    this.name = value;
                    this.RaisePropertyChanged("Name");
                }
            }
        }

        [DataMember]
        private ObservableCollection<FileInfo> files;
        public ObservableCollection<FileInfo> Files
        {
            get
            {
                return files;
            }
        }

        [DataMember]
        private ObservableCollection<FileGroupModel> groups;
        public ObservableCollection<FileGroupModel> Groups
        {
            get
            {
                return groups;
            }
        }

        public ReadOnlyCollection<object> CompositeList
        {
            get
            {
                List<object> allItems = new List<object>();
                allItems.AddRange(files);
                allItems.AddRange(groups);
                return new ReadOnlyCollection<object>(allItems);
            }
        }

        public FileGroupModel()
        {
            files = new ObservableCollection<FileInfo>();
            files.CollectionChanged += child_CollectionChanged;
            groups = new ObservableCollection<FileGroupModel>();
            groups.CollectionChanged += child_CollectionChanged;
        }

        void child_CollectionChanged(object sender, System.Collections.Specialized.NotifyCollectionChangedEventArgs e)
        {
            this.RaisePropertyChanged("CompositeList");
        }

        public void WriteToFile(string Path)
        {
            using (MemoryStream memStm = new MemoryStream())
            using (StreamWriter outfile = new StreamWriter(Path))
            {
                DataContractSerializer ser = new DataContractSerializer(typeof(FileGroupModel));
                ser.WriteObject(memStm, this);
                memStm.Seek(0, SeekOrigin.Begin);
                string result = new StreamReader(memStm).ReadToEnd();
                outfile.Write(result);
            }
        }

        public static FileGroupModel ReadFromFile(string Path)
        {
            string contents = System.IO.File.ReadAllText(Path);
            using (Stream stream = new MemoryStream())
            {
                byte[] data = System.Text.Encoding.UTF8.GetBytes(contents);
                stream.Write(data, 0, data.Length);
                stream.Position = 0;
                DataContractSerializer deserializer = new DataContractSerializer(typeof(FileGroupModel));
                object o = deserializer.ReadObject(stream);
                return o as FileGroupModel;
            }
        }

        public override string ToString()
        {
            return Name;
        }
    }
}

To start with, FileGroupModel extends DomainObject which in turn implements INotifyPropertyChanged and INotifyDataErrorInfo. This is done so that the model can integrate into MVVM. DomainObject can be found in my series on MVVM. So the MVVM stuff out of the way, we can get on to the composite design pattern.

Implementing Composite

Looking at line 42 you can see the observable collection that contains our FileInfos. At line 52 you can see the observable collection that contains our FileGroupModels, which makes this the composite pattern.  That’s it.  The composite pattern is just about implementing tree structure data.

CompositeMementoBut we need to use this in MVVM and we can only to bind an ItemsSource to one list. That is the purpose of the CompositeList. By hooking into the CollectionChanged event of the two observable collections, anytime files or file group models are added to either list we can raise a property changed event so the view way down the link that gets hooked up to the model via the view model, can get notified of the change to the collection.  My use of naming the combined list “CompositeList” is a bit unfortunate as I obviously mean that to indicate a combined list and not anything that really has to do with the composite pattern.

Implementing Memento

The last requirement is to be able to save out the state of the object (i.e. the memento design pattern). In .NET, arguably, the two easiest ways to write out an object is BinaryFormatter and DataContract. I tend to only use the binary formatter if there is proprietary data that needs to be stored. When that’s the case I use not only the BinaryFormater but I use it with a CryptoStream to ensure the security of the data. Most of the time, however, I try to use a DataContractSerializer. It’s a bit more in the set-up, having to define the DataContract and DataMember attributes but on the whole everything seems a bit cleaner. That way, if I need to, I can read the XML directly and see what’s going on with the data.

Now the tough part is that if you are going to use multiple classes that extend from a base class then you have to violate the Open-Closed principle. In my use of the above class I actually extend FileGroupModel (which is an abstract in my production code) to limit the types of files that are embedded. The problem with this approach is that you have to define a KnownType attribute in the base class so when you deserialize the object the DataContractSerializer knows what to do with it. This means that every time you add a class that extends the base class you have to add a KnownType for that class in the base. See? An obvious violation of the OCP but definitely a situation where we can ignore the rules on the paint can.

Next week I’ll follow up this post with an MVVM sample that is a bit closer to how I actually use it. I’ll show a sample extending FileGroupModel so you can get a better idea of using DataContractSerializer, but this also leads into using the command design pattern.

Thanks,
Brian

Abstract_factory.svg

Abstract Factory

If you follow good design principles and good dev practices and utilize the APIs and SDKS Microsoft has made available within the .NET Framework, you don’t need to know anything about patterns.

There, I said it. You don’t need to know anything about patterns.

Now why would I think this? Well, a few years ago I interviewed for company here in Tucson and had one of the best interviews I’ve ever had. But there was some confusion and the position was in Mountain View, and I’m in Tucson. And I had to decline the position. I was incredibly disappointed. The company was great, but most of all the people I had met were great. It was the best of all worlds.

So a couple of months later another position opened up at the same company with another development group. I applied and was asked to come in for an interview. Unlike my previous interview, which was just an hour or so and had felt more like a bunch of engineers hanging out shooting the breeze, this interview as far more formal.

In the first of the five interviews for the day, all three of the software engineers has sheets of papers with questions. After introducing themselves they started on the questions read right from the sheet of paper. Eventually they got to a question on patterns which was pretty simple, “What do you know about design patterns?” I said, “Well, I know about them cursorily as I read about them in a few of the blogs I follow, and I know that are established solutions to common problems, but I can’t think of any patterns off the top of my head.” The person that asked the question looked flummoxed. I’d like to think it was because up until this question I had been answering pretty well.

But I couldn’t answer a question on patterns.

Template_Method_pattern_in_LePUS3

The second and third interviews proceeded in the same manner. They would show up, ask me questions randomly off of the sheets of paper, things seemed to go pretty well until they got to the patterns question. They all asked the patterns question, I always answered the same way and I always got the same flummoxed look. I finished up the last two interviews, with the department head and hiring manager and went home. I was contacted a couple weeks later saying they weren’t interested in hiring me. The only thing that I could point to as a problem was the questions about patterns.

Fine, I need to know patterns. And so I get the ultimate design patterns book, “Design Patterns: Elements of Reusable Object-Oriented Software” by a group of software engineers know colloquially as the Gang of Four. In fact, in Uncle Bob’s papers on SOLID he quotes from the gang of four several times calling them the GoF. Wonderful! I’ve been a success software engineer up until now and knowing all about these patterns is going to make me even better. Right?

The design patterns in this book are descriptions of communicating objects and classes that are customized to solve a general design problem in a particular context.
Design Patterns: Elements of Reusable Object-Oriented Software, Section 1.1, What is a design pattern?,
Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides

Okay, so the first chapter is all about what a design pattern is, and how you would select the patterns and when to use them. There’s also a description on MVC in SmallTalk. I mean, I use MVVM regularly as a part of my every day development cycle in UI development, which is an extension of MVC, so I guess it’s kind of cool (for archival purposes?). There is also a brief description of a each of the design patterns in this first chapter.

And there is where it hits me.  You don’t need to know design patterns.  Chances are if you don’t know anything about them and you’re developing in .NET you are already using design patterns and don’t even know it.

So let’s see what the hell I’m talking about:

Creational:

Structural:

Behavioral:

Well, I’ll be a monkey’s uncle, the majority of patterns presented by the gang of four I already use in some form. Additionally, both composite and facade patterns, as well as the state pattern, I’ve used as solutions when architecting applications without understanding them to be design patterns, as such. They simply seemed like the obvious answer.

So I’ve decided after reading all about design patterns I don’t need to know about them. Read them yourself. Buy Design Patterns: Elements of Reusable Object-Oriented Software and you’ll see you don’t need to know them. Read the links above and once you understand them you’ll see you too don’t need to know them.

I mean, come on. If I was a horrible software engineer I wouldn’t have accomplished what I have and wouldn’t hold the position I do now. Knowing design patterns won’t make a bad software engineer good. So conversely, not knowing the patterns won’t make a good software engineer bad, right? Didn’t Uncle Bob say something like this?

Following the rules on the paint can won’t teach you how to paint.

This is an important point. Principles will not turn a bad programmer into a good programmer. Principles have to be applied with judgement. If they are applied by rote it is just as bad as if they are not applied at all.

Having said that, if you want to paint well, I suggest you learn the rules on the paint can. You may not agree with them all. You may not always apply the ones you do agree with. But you’d better know them. Knowledge of the principles and patterns gives you the justification to decide when and where to apply them. If you don’t know them, your decisions are much more arbitrary.

Getting a SOLID start, Robert C. Martin

So I guess Uncle Bob didn’t quite say what I said. He said you should know the principles and patterns. They’re another tool in your toolbox. So why would I think I don’t need to know anything about patterns? How have I gotten to where I am without knowing patterns?

It’s Andy’s fault

I blame Andy. See, I worked under Andy at my last company. Not in .NET but in Java. Andy grokked patterns. He could instantly recognize a scenario and apply the right pattern. It was fluid. Code flowed together. Problems solved. With Andy it was about the application of theory. He was one of the most prolific software engineers I’ve ever known. But here is what will really blow your mind, it was quality, clean, well-designed code. He was a truly incredible engineer that I learned an incredible amount from.

500px-FactoryMethod.svg

Factory Method

I learned design patterns but without understanding them as design patterns. Under Andy I learned that given problem X do solution Y but understand there will be constraint Z. I learned this by working in Andy’s code, extending it, fixing bugs (yes, even the best occasionally have bugs), being a dwarf standing on the shoulders of a giant. And then writing my own code under Andy’s review, having him point out problems and solutions.  Okay, so maybe it’s not really his fault.

I took what I had learned under Andy and starting applying it to doing development in .NET, recognizing parts of the framework that fit to a given pattern and applying it as the correct solution. When I use events in WinForms dev do I need to know that it is the Observer pattern? When I use a DataContractSerializer to write out files for serialized objects do I need to know that this is the Memento pattern? When I use a command in MVVM do I need to know it’s part of the Command pattern?

But ultimately this is the flaw to my argument.

  • It presupposes that we all work in .NET environment.
  • That we’re all developers who work in such a limited scope.
  • That we don’t need to think outside of the tray of ideas Microsoft has provided.
  • That Microsoft has provided for all problems.

Microsoft has applied design patterns to arrive at solutions that try and make our jobs easier but it doesn’t solve all my problems. As a software engineer I should be expected to be able to apply those ideas to any language and environment. Limiting myself to just the .NET environment does a disservice to myself. I am not merely a code monkey. I do more than just write code. I engineer systems. To do so I must understand design principles and patterns and I must be able to apply design principles and patterns in such a manner that makes the use and extension of that code easy to work with and maintain and that strives for a minimum of deficits.

So to bring this full circle, I had a bad interview, boo hoo. In the first interview it was more about how I would solve different scenarios and I gave solutions that I would later understand to be patterns. In the second interview it was a bad format with bad questions. I was blessed with a mentor who guided my early career in such a manner that I was taught sound design principles and patterns. Unfortunately we are not all blessed. And that is why you need to know design patterns.

Good design principles and patterns aren’t nearly as rampant as one would hope.

Start being part of the solution, understand, use and apply design principles and patterns.  Make your life easier.  Which by extension makes my life easier.

Thanks,
Brian

Image credits:
Abstract Factory
Template Method
Factory Method

Programming for Maintenance

Code Complete defines Maintainability as “The ease with which you can modify a software system to change or add capabilities, improve performance, or correct defects.”

I’ve been thinking for awhile now about this post and I’ve been unsure of exactly what needs to be here.  I know I’ve done a lot of maintenance work.  It’s inevitable.  You name the field and chances are we’ve developed something for it.  A lot of the software we develop we do so because there just isn’t commercial off the shelf (COTS) software for the business process.  COTS is for the broad market.  So customers come to us when COTS doesn’t meet their needs.

Which gets us to maintenance.  We try the best we can but in any software application there will be bugs.  It is inevitable, it is a fact of life.  Unless you are just writing a “Hello World” program there will be people who use your product in ways you didn’t expect.  We have had clients who take the application we’ve written for them and walk away.  Not because they’re unhappy but because the cost to fix every single bug and add in every feature they want can often be prohibitive.  They’re happy where the app is and that’s good enough.

Most clients stay with us for a maintenance tail.  They prioritize the bugs they want fixed and features they want added and budget accordingly.  We want happy clients, it’s that simple.  The happier they are with us the more work they bring to us.  This is common across all businesses, whether you’re a dry-cleaner or doing software services, a happy client is good.

So in software services how do you make a client happy?  In my experience it’s been minimizing maintenance tail costs.  But how do you do this?  I’m going to try and sum up where I have noticed problems.  I could reprint Code Complete here and say, “There, that’s how you do it” but that would be meaningless.  The point of this post is to focus on a few key points.

1.  The obvious:  Requirements, requirements, requirements.
As much as possible try to impress upon your clients the importance of requirements.  The more requirements analysis and design that is done up front before even beginning development the cheaper the maintenance tail.  It’s fact the earlier you find a problem the cheaper it is to fix. (Code Complete, Table 3-1, Average Cost of Fixing Defects Based on When They’re Introduced and Detected)

The biggest problem with this?  Most of the time clients don’t know what they want.  That’s right, they come to us and say, “Well, I want an application that does this.”  But what does that mean?  Does this need to mirror an existing business process?  Is it something completely new?  Do you have an idea of how the process should flow?  Do you know where you get your data and where it should go?  What kind of reporting are you going to need? (Customers never know that last one)

A lot of time we end up building up an existing business process in order to understand what needs to be developed for the application.  That’s okay.  I can accept that business analysis may be a part of the job.  So this is key:  (most of the time)  Clients don’t know what they want.  They will usually have a generally vague idea but you will have to help them.  Try to come up with a basic set of requirements.  Next, at a minimum, whip out Visio (there are better products out there but Visio is probably the most common) and throw together a few pages of the app.  Put together some sort of flow.  Help them bring their vision to reality.  Only then will they really begin to understand their own requirements and process.  Only then will they see what’s there and what’s missing.  Most clients can’t envision software.  They’re paying money for something that is completely virtual and just can’t see the final product.

Now, this is all well and good.  The problem is that a lot of clients don’t want to spend several weeks (or longer) designing the app, they want to get moving on it.  This leads to iterative development.  It’s more expensive.  But done properly I’ve still had happy clients.  Prioritize parts of the application.  Model a couple of those pages, build it, move on.  You will have problems.  I had one client who missed telling us about a key part of data for the app.  The ERD we had didn’t support this new data so we had to build out some new tables and fix everything.  This was after we’d done requirements and modeled the pages.  It wasn’t until the app was in beta that he went, “Oh, and you need to put this in there.”

2.  Naming Conventions: Use it, love it
This is going to sound stupid but one of the biggest problems I’ve had coming in to maintain an app I didn’t design or build is naming conventions.  Even if the names are consistent and meaningful, when there is no standard across an application where multiple developers have been working on it, it takes way too long to understand everything.  I’m not saying that your company should enact a strict naming convention company wide.  That would be nice but it’s not reality.  Your project lead, however, should.  Even if you’re working under different project leads you need to adapt.  This is especially true in this polyglot world of development.  At one time I was working on a PHP project, a java project and a C# project.  Each has their own naming conventions.  Yes, it took longer when I had to switch projects to re-orient myself but that in and of itself is the nature of polyglot programming.  I know that when someone comes behind me to maintain the app (and in a lot of cases it’s even me) it is a hell of a lot easier to get going on maintenance when there is some sort of naming convention across the app.

So what do I mean by naming conventions?  Method names, event and delegate names, variable names.  If it takes a name there needs to be a standard.  Whether it’s public, private, static, constant, an enum or class, there needs to be a standard.  The only place I haven’t seen a need for naming conventions is internal to methods.  If the app is coded right and adhering to basic OOP methodology variables internal to a method shouldn’t be around that long.  They still should be meaningful but I don’t care if you want to put an underscore in front of your internal variables or not.  I don’t care if you want to capitalize them or not.  Anytime I have to walk through from class to method to class to method to delegate to method there better be some sort of convention.  Otherwise it costs the client money.  There will always be some sort of spin-up when entering maintenance.  Just try and minimize it.

So you’ve got your requirements and naming conventions, what’s next?

3.  OOP, if you use it, use it
The biggest trouble I’ve had, excepting those mentioned above, are projects where developers seemed to take a procedural approach to OOP.  God classes and methods (functions?), poor use of inheritance and virtually no use of interfaces.  This makes maintenance tough.  I have to admit I myself was bit lacking in this area out of college.  Sure I took an OOP class, sure I had the theory but applying the theory is wholly different then having the theory.  And I’m not the only one.  It seems one of the universal constants having new engineers on my team fresh out of college is that they can’t apply OOP.  This often also applies to self-taught programmers who many times don’t even have the theory.

I want a black box (Code Complete, Encapsulate Implementation Details, Information Hiding).  When I step into a method during troubleshooting I want to only go into that method once.  There after I want to trust that the method works as intended.  I want a method to only do one thing and not affect parts outside of itself.  If I have a login control it should only be doing login.  I want a class to default all properties (getters/setters) to private.  I don’t want those variables being changed outside the class unless they need to be.  I want interfaces(Code Complete, Form Consistent Abstractions).  It seems like this is the least understood by programmers straight out of college.  So often new engineers like to inherit and override.  Obviously this has it’s time and place.  But think if this really is needed or if you just need to create an interface (Code Complete, Inherit-When Inheritance Simplifies the Design).

I digress, I’m starting to write out Code Complete.  If nothing else, use and apply Code Complete, 5.3 Design Building Blocks: Heuristics.

So how to fix this?  If you have an engineer on your team that you’re not familiar with do a code review.  Assign him a task and take a look at his code.  I don’t think I carry the title Senior Software Engineer because I such a great programmer.  Really I barely consider myself adequate.  I have seen a lot of code, written a lot myself, I lead teams pretty well as a project lead and clients like to work with me.  If you’re leading a project remember the code of the engineers underneath you is your responsibility.  Yeah, you might ruffle some feathers.  We programmers take our code personally.  We’re proud of it.  No different than an architect whose proud of a house he’s built or a chef whose proud of his food.

4.  Stay up-to-date
I won’t harp on this long because I’ve said this before.  Programmers use what’s available to them.  I love LINQ, but I know LINQ because I read about it before it was even released publicly.  I can do a lot of stuff easier with LINQ.  I love WPF, but I know WPF because I read about it before it was even released publicly.  I can do a lot of stuff easier with WPF then I could with WinForms.  If you don’t stay current there’s a good chance you’ll jump into a project where it’s using something you don’t understand.

5.  Code to maintain
I suppose this is really the heart of the issue.  What the hell does this mean?  Any adequate programmer should be able to jump into a piece of code and start working on bugs and adding features.  I’m not saying there won’t be a spin-up.  Of course there is going to be some spin-up.  But try and minimize that.  When most people are working on code they’re not thinking that in a year or more someone is going to have to come behind them and read the same code trying to figure out what they were doing.  But they should be.

Finally, I want happy clients.  That means minimizing maintenance.  There will always be maintenance, it’s inevitable.  But when I can jump into the maintenance tail of an project faster, fix bugs faster, implement new features faster than clients are happier and my job is easier.  And I like it when my job is easier.  It will always be challenging but I prefer challenging and fun to challenging and a pain in my ass.

I could keep going.  Someone could write a book on the subject (yeah, I know, Code Complete).  This post is just intended to hit a few top issues I’ve seen when maintaining code. YMMV.  You may have seen other issues that top this list.  Tell me about them.  Post a comment.

Later,
Brian

Book Review: Code Complete

I suppose for any serious software engineer one must read Code Complete. Last week I finished it and have been trying to think of how to write a review of it. I’ve decided to do so briefly.

Let me start off by saying I’m excited that I finished it. There is a lot of good information and there is no doubt that it is the standard for sound software developement principles. That being said, remember when you try to tackle this bood that it is a text book. Sure there are some interesting anecdotes about this and that. I suppose I should have been more thrilled about reading it but couldn’t drag myself to pour through it like I have some of my other software books. It took me a year to read Code Complete. During that time I re-read The Belgariad series, The Malloreon series, most of the Song of Ice and Fire series, and read for the first time Pro WPF in C# 2008, Effective C# and I’m almost done with More Effective C#.

So what am I saying? I know there are those of you out there that love Code Complete and read it voraciously like a good fiction novel. I couldn’t even read it like a good code book. Pro WPF in C# 2008 I read like a good fiction novel. I drank it in and loved it. Code Complete I just couldn’t do that with. As I said, it’s sound, solid developement principles I try to apply every day on the job. I learned a lot and will continue to try and apply the princibles of Code Complete. Simply because of that it is a reason to read the book.

If you’re like me and are struggling to get through Code Complete, spread it out, read a chapter and then take a break and read something else. There are 35 chapters in Code Complete. Read a chapter every two weeks, in between other books and it will take you just a bit over a year to read it.

What else can I say? 99% of the book makes perfect sense. Some of the code formatting seemed a bit off but I just let visual studio format my code for me. Ctrl-K + Ctrl-D, Baby!

Read it, grok it, follow it and your code will be better, cleaner and easier to maintain.
That’s all there it to it.

Brian

Top 25 Most Dangerous Programming Errors

Follow the below link, read it, consume it, grok it.

2009 CWE/SANS Top 25 Most Dangerous Programming Errors (If it doesn’t come up be patient. It looks like the server is getting hit pretty hard with bloggers like myself linking to it.)

Or read a summary of it and then go to the link over at Coding Horror.

On a side note, now that the holidays have calmed down I’ve been working on a series of “Often Unused Operators” like ~ and |=. Credit goes to Jeff Clark for turning me onto this. Hopefully I’ll have the first one up on Friday.

Brian

Making Classes and Properties Immutable

Hopefully we all know string is immutable. When you type:

string x = "";
x = "asdf";

what you are really doing is creating two different strings, not changing the value, per se, of x but actually creating a new x.
That is why in heavy string operations it is recommended you use StringBuilder like:

StringBuilder sbName = new StringBuilder();
sbName.Append("asdf");

since appending to sbName doesn’t create a new string but continues to add to the string value of sbName.

So why do you care? Well, maybe you don’t. I’ve started to move into a more multi-threaded world where the state of objects and their properties could potientially be invalid if an external thread changes the value of a property. I’ve been reading Effective C# and More Effective C# and enjoying them. They are books that target specific development issues and paradigms that help you become a better C# developer. The books are written by Bill Wagner who is a regular blogger I follow.

Even though Effective C# came out in the .NET 2.0 days a lot of the book is still relevant, however, quite a few of the code samples could be updated.

Item 7 in Effective C# says “Prefer Immutable Atomic Value Types”. Now to paraphrase the chapter to an extreme it basically says, “Immutable code is easier to maintain”. I would add an addendum that whether you like it or not there is a good chance that code you write will be used in a multi-threaded environment and immutable will matter.

Now obviously to blindly say all classes must be like this is absurd. As Bill’s Item 7 says, however, “Prefer Immutable Atomic Value Types”.

In .NET 3.0 properties made things a bit easier in general. Prior to 3.0 for a property you would have to do:

private int _myInt;
public int MyInt
{
    get { return _myInt; }
    set { _myInt = value; }
}

In .NET 3.0 you don’t have to define the private. When the code is compiled it will take care of that for you. So the above code becomes:

public int MyInt { get; set; }

Now say you have to class:

public class Customer
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public Address Address { get; set; }
    public Customer(){}
}

If you were working in a multi-threaded environment you might have a problem if this happens:

public void HandleCustomerData(Customer MyCustomer)
{
    MyCustomer.FirstName = "George";

    //use myCustomer to do a bunch of stuff

    MyCustomer.FirstName = "Joe";

    //use myCustomer to do a bunch of stuff
}

This could cause a big problem if MyCustomer is getting passed around a lot in different threads. Now I know this is rather contrived but it is still a real issue.

Properties an additional feature here in 3.0 that makes things easier for creating an immutable. Imagine you have to class:

public class Customer
{
    public string FirstName { get; private set; }
    public string LastName { get; private set; }
    public Address Address { get; private set; }
    public Customer(string FirstName, string LastName, Address Address)
    {
        this.FirstName = FirstName;
        this.LastName = LastName;
        this.Address = Address;
    }
}

You can see here that I’m using the private keyword on the set method. As it implies this forces the set methods of the properties to private so they can only be changed internally in the class. I would recommend reading “Effective C#” for a better explanation of why to use immutable values but by using the private in a property this becomes easier.

So there it is, prefer immutable atomic values in your classes and use the private keyword to help you.

Later,
Brian

Update:
The more I read this the more I think I over-simplified Bill’s reason for preferring immutable atomic value types. Just get the book and read it. It’s a fairly small book but every nugget has value.

Assert Yourself!

Java folks, please don’t get scared off by the C# examples, the contents of the post apply regardless if it is java or C#.

None of us use Asserts enough. It’s that simple, we don’t. Asserts are essentially a way to ensure we don’t get a value we shouldn’t have. The great thing about using asserts is that they aren’t included in final code (compiled in “Release”), only code complied with “Debug”. They make testing easier by verifying that when code is being tested in Debug that it doesn’t go beyond some condition. They can also be used to ensure when troubleshooting that you have the values you should.

Take a look at the code below:

private void pass_Click(object sender, RoutedEventArgs e)
{
    int temp = 1;
    Debug.Assert(temp > 0);
    MessageBox.Show(“Pass”);
}

private void fail_Click(object sender, RoutedEventArgs e)
{
    int temp = -1;
    Debug.Assert(temp > 0);
    MessageBox.Show(“Fail”);
} 

This is a sample of using an assert. I know the example is trivial but I wanted to show a simple use of it. These are simply a Click event on a “Pass” button and a Click event on a “Fail” button. I would recommend you create your own solution and put these on a pass and fail button so you can see and understand what happens when Debug.Assert is called.

Now lately I’ve been coming across a lot of code like (anonymous-ized for your protection):

else
{
    MessageBox.Show(“ERROR: YOU SHOULD NEVER SEE THIS MESSAGE.  Invalid TransformString!”, “SERIOUS ERROR occurred in MyClass:MyMethod”, MessageBoxButton.OK, MessageBoxImage.Error);
}

I’m sorry but this is simply wrong.

Code Complete 2nd edition has a great chapter on Assertions. The above example is exactly where asserts should be used.

Per the guidelines for “Using Assertions” (McConnnel, Code Complete 2, 2004) the following guidelines should apply for using assertions:

“Use error-handling code for conditions you expect to occur; use assertions for conditions that should never occur.

Avoid putting executable code into assertions.

Use assertions to document and verify preconditions and post conditions.

For highly robust code, assert and then handle the error anyway.”

That’s right. The very first usage guideline says to use asserts for conditions that should never occur.

Take a look at a sample from Code Complete (originally VB but I ported it to C#):

private double Velocity(double Latitude, double Longitude, double Elevation)
{
    //Verify preconditions
    Debug.Assert(Latitude >= -90 && Latitude <= 90, “Invalid Latitude”);
    
Debug.Assert(Longitude >= 0 && Longitude <= 360, “Invalid Longitude”);
    
Debug.Assert(Elevation >= -500 && Elevation <= 75000, “Invalid Elevation”);
    
    double velocity = 0;
    
/* DO THE WORK */    //Verify postconditions
    Debug.Assert(velocity >= 0 && velocity <= 600, 
                 
“Invalid Velocity”
                 
“Can’t seem to properly work the velocity formula”);
    
return velocity;
}

On this McConnell has to say:
“If the variables Latitude, Longitude, and Elevation were coming from an external source, invalid values should be checked and handled by error-handling code rather then by assertions. If the variables are coming from a trusted, internal source, however, and the routine’s design is based on the assumption that these values will be withn their valid ranges, then assertions are appropriate.” (McConnell, 2004)

Now assertions aren’t the end-all, be-all of validating values. Chances are that in most cases you are still going to want to validate and do error checking. But they’re a good start.

And just for the java folks that hung around, an assert in java:

//Check to make sure we’re not going to do a divide by zero
assert denominator != 0: “denominator is 0”;

Refs:

Code Complete (Second Edition), Microsoft Press, Steve McConnell, 2004

C# Online.NET – Assert

Debug.Assert Method

And just for the java folks:
Programming With Assertions