Implementing Done, In Process, and Ready Queues in LeanKit Kanban

Posted by indomitablehef | Filed under LeanKit, Lean, Kanban, Tools

In two blog posts, “A Variation on Queues - Pipelines for WIP and Done” from December 2008, and WIP Queues: Done vs Ready from September 2009, Derick Bailey, (with comments from Scott Bellware) examined ways to break down WIP queues into Ready, In Process, and Done queues. Ideas from these blog posts made it into our electronic Kanban implementation in LeanKit Kanban, and I’d like to show you how they can be implemented and analyzed using those features in LeanKit Kanban.

In “A Variation on Queues - Pipelines for WIP and Done” Derick describes how:

we can use the concept of a pipeline and split our existing queues into a WIP and Done step. For example, we want developers to pull work from the Analysis queue into the Development pipeline. We can show which cards are ready to move by splitting Analysis into sub-columns of WIP and Done.

I’m replacing his illustrations with mine, from LeanKit Kanban:

Then in September of 2009, in WIP Queues: Done vs Ready he took it a step further describing how the pipeline can be modeled to include a “Ready” state just before an “In Process”, _or_ a “Done” state just after the “In Process” state.

Like this, showing the LeanKit Kanban board in “edit mode”. (we’re proud of our “inline”, wysiwyg board editing features, too)

Notice the drop-down lists in each column/subcolumn header - these allow you to specify that a particular column is a “Ready” Queue, “In Process” queue, or “Completed” queue.

Or, in some cases, you might want both a “Ready” Queue and a “Completed/Done” Queue in your pipeline:

Notice the little Queue Type icons in each sublane header - shown in the call-out bubble. These allow you to see at a glance what kind of queue you’re looking at.

As Derick and Scott Bellware discussed in the WIP Queues: Done vs Ready post and comments, having a “Ready” queue may indicate a “Push” situation in your pipeline:

An important distinction of “Ready” is that it receives items via push rather than pull. It’s a thing that’s usually found and the seam between two process stages or two parts of an organization that aren’t easily reconciled by pull.

Before we move on to how you can use these distinctions in your analysis of process efficiency, here is one more illustration from LeanKit Kanban, showing a Ready/WIP pipeline modeled as a horizontal sublane, instead of a vertical one:

Efficiency Analysis Based on Queue Type

Now let’s look at how making these distinctions on the types of Queues in your process, along with the automatic statistics capture your get from an electronic tool, and the Efficiency Analysis chart in LeanKit Kanban can show you valuable information about the efficiency of your process, and how much “dead” time there might be in waiting for handoffs.

Let’s take, as an example, a more complex value stream, shown below (click to enlarge):

Notice how the parent “In Development” and “Testing” lanes are broken down horizontally into “Feature Groups”, each with a Ready Queue, In Process Queue and a Completed Queue. The Bugs section of the “In Process” lane is modeled with an In Process and a Completed queue, whereas the Bugs section of the Testing Lane is modeled using a Ready queue and an In Process Queue.

Using the Efficiency Diagram in LeanKit Kanban, and all the flexibility it provides to define the parameters used to analyze the data, we can take a closer look at

1. Just the “In Process” and “Testing” lanes and their children/sublanes
2. For the first 3 weeks in January
3. Taking Card Size into account

… in order to see

What percentage of the Cards on the board were “In Process”, as opposed to “Ready” or “Completed” at any given time over the course of those three weeks?

Check out the diagram below in detail, with annotations: (click to enlarge)

So, for this completely made-up data set I used to generate this chart, it looks like we’re running a fairly efficient process. What do those occasional spikes in “Completed” queues mean? Maybe those are Fridays, when people push to finish the work they currently have on their plate. Or maybe it’s no big deal - but noticing patterns like that, and asking the questions that they inevitably bring to mind, can be an important part of the “continuous improvement” of your Kanban system.

Thanks to Derick and Scott, for their contributions to the Lean Software Development and Kanban community and to my own learning process over the past couple of years. Thank You for reading this far down, and please do go and give LeanKit Kanban a spin. You can sign up for a fully functional free account, which includes all the advanced features and analytics described in this post, and allows you one Kanban board and up to 5 users for free.

Build Master

Posted by indomitablehef | Filed under Continuous Integration

I remember, with alarming clarity, when someone first explained Continuous Integration to me. I was in a rented car, between Atlanta and Nashville, with Stephen Franklin and Craig Israel. Craig was telling us how his company used NAnt and CruiseControl.Net to manage their automated builds and continuous integration. I’d heard of CI before, but never really understood it. Hearing it described that day, however, it immediately resonated with me.

I remember thinking:

Wow…this is how things ought to be done…this is the “right way”…

and then

…but I wouldn’t want to be the Build Master….that would really suck

Anyone care to guess what I spend a large percentage of my time on now?

Yep. Never one to leave an unclaimed responsibility on the table, it immediately became my mission to bring Continuous Integration back to my company. We even do database continuous integration now. All the databases can be dropped and recreated from source code with a single command. Yep, we do CI here. All the way to Eleven.

And me, well, I’m the Build Master.

“Honey Do” Kanban

Posted by indomitablehef | Filed under Kanban

My “Honey Do” Kanban (click to enlarge)




http://LeanKitKanban.com

Indefatigable

Posted by indomitablehef | Filed under LeanKit, Kanban, Personal

I have this little tagline at the top of my blog: “Inimitable. Indefatigable. Indomitable.” Now, those are mostly there because I’m a lover of words, and I just like those words. But I also see them as something to aspire to. I think I can lay some claim to being “Inimitable”, and at times even “Indomitable”. But until recently, I don’t think I ever understood how hard “Indefatigable” can be. (Indefatigable = tireless). You see, I’ve started my own business, with a couple of partners. I still have my regular job, too. My wife says that I’m now working three full time jobs (she counts being a Dad and Husband as one of those jobs, which is nice).

My third job (the new business) starts every night at about 10 or 11PM, after my daughter is asleep, and my wife and I have had time to sit together and drink a glass of wine and watch a little TV. At midnight, we have our daily standup call with my partners here in Tennessee and our European contractors, who are waking up early (at 7 or 8AM) to join the call. This time pretty much guarantees that everyone is equally sleepy.

From there, I work until 2 or 3AM, or even later, and finally collapse into bed when I start falling asleep at the keyboard. Our daughter likes to wake up at 5 or 6 in the morning, so my wife and I take turns being the one to wake up first, and whoever gets that first duty usually gets a few more minutes of sleep right before having to get up and start the day. My wonderful wife is very supportive and works out ways to give me blocks of time on weekends where I can work as well.

It’s been a lot of hard work for the past few months, but we’re finally getting very close to a full product launch. We’ve been in beta for awhile, and we may still call it “beta” for a little while after we launch, but it will be out there and available for use and purchase by early November. I should mention, too, that my partners, Stephen Franklin and Daniel Norton, have also been burning the candle at both ends - and I send out a special “thanks” to Leslie and Mary (their wives) for also being so supportive during this time. And we couldn’t have done it without the help of our intrepid young team of contractors - those guys are rock stars, and I feel lucky to know them.

I don’t expect I’ll be back to regular blogging anytime soon. There’s still much work to be done, and my personal blog is definitely going to take a back seat for awhile. But someday, when “we own the company we want to work for”, I’ll be back to writing on my own a bit.

Oh, yes…the new company! We’re called Bandit Software, LLC. (named after Stephen’s dog, Bandit). Our first product is LeanKit Kanban, and you can sign up for the beta, and get a free account, at http://LeanKitKanban.com

Server side Paging and Sorting in .NET with jqGrid

Posted by scott.walters | Filed under Uncategorized

One of the tasks on our radar has been replacing the comparatively heavy, slower server side grids on our pages with a lighter weight client side solution. Many of our users only have cell modem connectivity so we need to be careful about response time and bandwidth issues.We tried jqGrid as a potential candidate for this, and it worked out well on single page grids where paging and sorting weren’t required. Before we could commit to using jqGrid as our permanent grid solution though, we had to figure out how to do paging and sorting with it.

Since many grids require fields for the user to enter filtering criteria, we also had to verify that the data from these fields would be included in the posts to the server and figure out how to access them.

To demonstrate these techniques, I coded a small, self contained sample which can be downloaded here. Here’s a screenshot of the final page. Click to enlarge.

The sample form has a select for the color filter and a div to put the jqGrid placeholder objects for the grid body and pager areas.

<form id="form1" runat="server">
    Choose Color: 
    <select id = "colorSelect">
        <option value="All">All</option>
        <option value="Red">Red</option>
        <option value="Green">Green</option>
        <option value="Blue">Blue</option>
    </select>
    <br/>
    <br />
    <div>
        <table id="grid" class="scroll" cellpadding="0" cellspacing="0"></table>
        <div id="gridpager"></div> 
    </div>
</form>

This is the easy part. The rest of the markup page will take more explaining. All the script code to create the grid and set the property values for it is contained in a single call to jqGrid. The grid initialization data is passed as a JSON data structure. In the sample, this call is made inside the document ready function like this

 
$("#grid").jqGrid({
    datatype: function(postdata) {
        jQuery.ajax({
            url: '<%= ResolveUrl("~/GridSearchService.asmx/GetGridData") %>',
            data: $.toJSON(postdata),
            type: 'POST',
            dataType: "json",
            contentType: "application/json; charset=utf-8",
            complete: function(servicedata, stat) {
                if (stat == "success") {
                    var serviceJson = $.parseJSON(servicedata.responseText, true);
                    var thegrid = jQuery("#grid")[0];
                    thegrid.addJSONData(serviceJson.d);
                }
            }
        });
    },
    jsonReader: { repeatitems: false },
    mtype: 'POST',
    rowNum: 10,
    postData: { "colorToSearch": "All" },
    rowList: [10, 20, 30],
    height: 250,
    width: 500,
    colNames: ['id', 'Name', 'Color', 'Amount'],
    colModel: [
        { name: 'id', index: 'id', width: 60, sorttype: "int", hidden: true },
        { name: 'name', index: 'name', width: 100 },
        { name: 'color', index: 'color', width: 80, align: "left", sorttype: "float" },
        { name: 'amount', index: 'amount', width: 80, align: "right", sorttype: "float" }
    ],
    pager: '#gridpager',
    viewrecords: true,
    caption: "JqGridSearchAndPage Sample"
});

The jqGrid method is called with a jQuery selector for the div that will contain the grid. The first field is datatype. In the sample, this lets you specify the code that gets executed when the grid calls the server for data, which it does as soon as the grid is initialized, and for every page and sort request thereafter. There are also baked-in values for datatype like “json” and “xml”, but I couldn’t figure out how to get them to do what I needed, so the function won out.

When this function is called, it will get an object as a parameter, which will have properties containing the current values for grid state items like current page, sort field and page size. The sample function contains an ajax call to invoke a web service method, passing the grid state values as parameters. If this succeeds, the JSON it returns is parsed and added to the grid. For those of you in a hurry, the signature of the method on the server that receives this call is

public GridJsonData GetGridData(bool _search, long nd, int rows, int page,
string sidx, string sord, string colorToSearch)

The next field, jsonReader, is used to set a the repeatItems property, which has a drastic effect on how the grid interprets the returned JSON data structure. This is documented exhaustively in the jqGrid docs.

The only other field that isn’t obvious is postData. This contains a JSON data structure that gets appended to the postdata parameter passed to the data fetching function defined in datatype. This is where the values for the color filter go in the sample. It’s value when the grid is initialized is “All”, but we will change it when we select a different color.

The only other interesting part in the client code is the change event handler on the color select. This sets the new value for postData and triggers the grid reload.

$('#colorSelect').change(function() {
    var color = $('#colorSelect').val();
    $('#grid').setGridParam({ postData: { "colorToSearch": color} }).trigger("reloadGrid");
});

Now on to the server. There’s no code behind for the markup page. All the action is in the web service. For simplicity, all the code for the web service is in one file.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Web;
using System.Web.Script.Services;
using System.Web.Services;
 
namespace JqGridSearchAndPage
{
    [WebService(Namespace = "http://tempuri.org/")]
    [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
    [System.ComponentModel.ToolboxItem(false)]
    [ScriptService]
    public class GridSearchService : System.Web.Services.WebService
    {
        private static List<GridRow> GridData;
        private static Random rnd = new Random();
        static GridSearchService()
        {
            GridData = new List<GridRow>();
 
            //add 100 red, 200 green, 300 blue
            //set id=rownum, name=rownum, amount=random (1-1000) / 100, tax = amount * .05
            for (int ndx = 0; ndx < 600; ndx++)
            {
                string clr;
                if (ndx < 100)
                    clr = "red";
                else if (ndx < 300)
                    clr = "green";
                else
                    clr = "blue";
 
                GridData.Add(new GridRow()
                                 {
                                     id = ndx,
                                     name = ndx,
                                     amount = ((double)rnd.Next(1, 1000))/10,
                                     color = clr
                                 });
            }
        }
 
        [WebMethod]
        [ScriptMethod(UseHttpGet = false, ResponseFormat = ResponseFormat.Json)]
        public GridJsonData GetGridData(bool _search, long nd, int rows, int page,
            string sidx, string sord, string colorToSearch)
        {
            var gdq = GridData.AsQueryable();
 
            //apply color filter
            if (colorToSearch != "All")
                gdq = gdq.Where(gd => gd.color.ToLower() == colorToSearch.ToLower());
 
            //get count
            int totalcnt = gdq.Select(gr => gr.id).Count();
 
            //apply sort
            gdq = ApplySort(gdq, sidx, sord);
 
            //get data
            var dataList = gdq.Select(gr => gr).Skip((page - 1) * rows).Take(rows).ToList();
 
            var tgd = new GridJsonData
                          {
                              page = page,
                              records = totalcnt,
                              total = totalcnt / rows,
                              rows = dataList.ToArray()
                          };
            return tgd;
        }
 
        private IQueryable<GridRow> ApplySort(IQueryable<GridRow> gdq, string sidx, string sord)
        {
            if (string.IsNullOrEmpty(sidx))
                return gdq;
 
            IQueryable<GridRow> gdqout = gdq;
 
            if (sord == "asc")
            {
                if (sidx == "name")
                    gdqout = gdqout.OrderBy(gr => gr.name);
                if (sidx == "color")
                    gdqout = gdqout.OrderBy(gr => gr.color);
                if (sidx == "amount")
                    gdqout = gdqout.OrderBy(gr => gr.amount);
            }
            else
            {
                if (sidx == "name")
                    gdqout = gdqout.OrderByDescending(gr => gr.name);
                if (sidx == "color")
                    gdqout = gdqout.OrderByDescending(gr => gr.color);
                if (sidx == "amount")
                    gdqout = gdqout.OrderByDescending(gr => gr.amount);
            }
 
            return gdqout;
        }
    }
 
    public class GridJsonData
    {
        public int page { get; set; }
        public int total { get; set; }
        public int records { get; set; }
        public GridRow[] rows { get; set; }
    }
    public class GridRow
    {
        public int id { get; set; }
        public int name;
        public string color;
        public double amount;
    }
 
}

The GridSearchService class has a static initializer that generates some grid data to simulate data returned from a database. GetGridData uses the parameter values passed to it to return the correct page from the simulated database data as a JSON array. For the JSON/C# mapping to work properly, the field names must match the parameter names on the method. This is case sensitive so look carefully. The same thing is true for the output JSON. It’s fields names have to match the names the client is expecting. For the row names, these are the values specified in the colModel field in the jqGrid call.

Once I understood the jqGrid approach to this, I had to admit that it’s pretty clean, although I did have to spend some time in Firebug and the jqGrid docs to figure it all out. Eventually, I’ll also have to figure out how to do hierarchical grids and inline editing, and will possibly do a post on that too.

Guest Blogger: Scott Walters (@cpound)

Posted by indomitablehef | Filed under Uncategorized

Please join me in welcoming my friend and colleague, Scott Walters (@cpound on twitter), as a guest blogger. His first post will be on jqGrid, and I know he’s got some cool LINQ stuff he’s been working on….looking forward to reading more about that, too.

LeanKit Kanban is now in public beta

Posted by indomitablehef | Filed under LeanKit, Lean, Kanban, Asp.Net MVC, S#arp Architecture

There’s still much work to be done, but we’re ready for the masses to come and take a look. Thank You, to all of you who helped us to beta test it during our private beta phase, and those of you at IMIHealth and BBCWorldWide for participating in a full pilot. Your feedback and encouragement have been invaluable.

LeanKit Kanban is a visual process management application, based on Lean principles, and optimized for distributed teams. I invite everyone to sign up for the free 1 board 5 user account at http://LeanKitKanban.com

Demonstrating Software Quality: Cut the Volvo in Half

Posted by indomitablehef | Filed under Craftsmanship, SOLID

In this excellent post: Quality Sounds Good…I Think Justin Etheredge compares the experience of buying a car and hearing about all the safety and quality features of the car to the experience of “selling” the quality features we, as developers, care about when building software. Essentially, even though I don’t really have a deep understanding of the quality features of my car, I know I want a safe, quality car, and the dealer does a good job of selling me on those features. Justin proposes that we can and should do the same for the quality features we build into our software, such as Unit testing, Dependency Injection/IOC, BDD, DDD, Single Responsibility Principle, etc., etc., etc. Even though the customers and product owners we are selling to may not fully understand the details, we can do a better job of selling them on those features than we have in the past.

Last summer, I was shopping for a new family car. Our daughter Cecilia had just been born, and wrangling the car seat in and out of my wife’s two-door Alero was proving to be quite a chore. Safety and quality were some of our primary concerns. Obviously, we were carrying around more precious cargo than we ever had before, and my wife and I both tend to keep cars for a long time. She had been driving the Alero for 7 years, and I’ve been driving my Jeep for 10.

Early in our search, I visited the local Volvo dealership to look at some used models. Predictably, the focus of their sales tactics was safety and quality. While I was there, they took me into a very interesting showroom, and gave me the rundown on the Volvo safety features. In the showroom, there were two cars sliced completely in half, right down the middle. From that cutaway view, you could see the heavy steel bars that re-enforced the car body, the crumple zones, the anti-roll stabilizers, and much more. They also had a couple of cars in there that had actually been in horrible accidents, complete with descriptions of the accident and pictures of the wreckage from the scene. At first glance, the damage to these vehicles looked horrific, but upon closer inspection, one could see that the passengers inside the vehicle had been completely protected.

Reading Justin’s post on how we can better “sell” the quality features of our software, I was reminded of my own car-shopping experience with Volvo. And I began to wonder, why can’t we as software developers do something similar? It obviously won’t be quite as easy. As Justin wrote in his post:

…if these qualities are the same in software as they are in many physical goods, then why is it so hard to sell people on quality software? Well, with the car you can put your hands on it. You can pop the hood, test drive it, kick the tires… you can feel the quality. Or at least you think you can perceive the quality. The important thing is that you cannot underestimate the emotional attachment to a physical item, and being able to see and feel the item gives a person confidence in it. The seller can throw around brand name parts, features that we probably don’t even know what they do, construction processes that make no sense to use, but in the end, it all builds up a picture in our head of quality.

So, at the very least, we can change the way we talk about our software, putting emphasis on things like Continuous Integration, Lean methods, and SOLID principles. Even though our clients and product owners won’t completely understand any more than I fully understand the safety features of my car, they can be made to understand enough to realize that these things are “good” and worth having.

Beyond that, though, can we do something more like the Volvo showroom? Can we, as a community, create the kind of examples that can take that extra step toward demonstrating the importance of adding quality features to our software? (I’m mostly wondering aloud here, hoping someone will chime in and continue the conversation.)

They’d have to be simple enough examples to be understood, but realistic enough to be relevant, and not dismissed as purely theoretical. In some cases, maybe short screencasts would be appropriate. Maybe just a community site, where we could begin to share our “brochures” (as described in Justin’s post) for the quality features in our software, and where we could piece together shared content for inclusion in our own specs/proposals/marketing materials.

What do YOU think? Is it do-able? Worth doing?

Slides and Source Code from Devlink Posted: “Multi-Tenancy: Beyond the Whiteboard”

Posted by indomitablehef | Filed under .Net

I’ve posted the slides here. I created a google code repository for Sok Munkae here. I’d love to have some contributors get involved to help me take Sok Munkae to the next level, and make it really useful as application guidance.

Devlink Presentation Posted: “Asp.Net MVC: Patterns and Anti-Patterns”

Posted by indomitablehef | Filed under DevLink, Asp.Net MVC

I’ve posted the slides for my Asp.Net MVC presentation at Devlink to slidehshare here. Enjoy.