Call the Baby Ugly

Posted by indomitablehef | Filed under Productivity

In the mail: Reclaim Your Life: A Two Week Challenge to Help You Regain Time - Stuart R. Levine [hat tip: Jon Terry]

Mind Mapping

Posted by indomitablehef | Filed under Tools, Productivity

On Sunday, I spent some time talking to my buddy Edward about a graduate class he is taking at TSU. He pointed me toward the Institue for Human and Machine Cognition. There’s a lot of interesting stuff there, that I’m only beginning to scratch the surface of. Check out the research papers in theresearch section, and the videos in the Evening Lecture Series.

This got us talking about “Concept Mapping”, which I had heard of before as “Mind Mapping”. IHMC has a link to their CMap Tools on their homepage. I ran into bubbl.us awhile back [via lifehacker] In this context, you can use bubbl.us as a visual way to brainstorm. I used it successfully awhile back when I was first attempting to come up with an education program for my new job. I had lot’s of “stuff” floating around in my head that I knew I wanted to communicate, but it was hard to get it all wrangled into a structure that I could present without rambling, or creating some Castro-esque all night rant. Using bubbl.us, I was able to ramble in a structured way, connecting ideas to each other visually, until the whole thing took on a much clearer structure.

I get the impression that the Concept Mapping ideas at IHMC are a bit more sophisticated than bubbl.us, but for what I needed, I found bubbl.us quite serviceable. The comments to this post on lifehacker.com list a whole slew of other “mind mapping” applications, some free some not, some simple some way more robust.

I think I’ll try this one next.

UPDATE: Well, it looks really cool, but it’s so slow tonight that I can’t seem to make it do anything. I’ll try again tomorrow.

Refactoring

Posted by indomitablehef | Filed under Agile

Some people will tell you that “Refactoring” is just another word for “rework”.

Almost, but not quite.

Properly understood, “refactoring” implies an important distinction: it doesn’t change the end result. Refactoring is changing software code in order to improve its internal structure, without changing its external behavior. [Fowler] So, when you decide to change how the system works, fix something that doesn’t work now, or add something that wasn’t there before, you aren’t exactly “refactoring”. Those are all activities you will engage in during your Iterative Development cycle, to be sure, but Refactoring is something else.

When you refactor, you improve the design of existing code. You redesign the internals, but what the code actually does doesn’t change. Usually this is done to make future changes to the code easier. It can be understood to improve the application overall, but only if the application is intended to grow and change after the refactoring. If you are looking at an application you have no intention of changing and that works as it is, there’s no reason to refactor.

There are a couple of situations you may find yourself in that will drive you to refactor. If you look at the code and realize that it doesn’t effectively communicate the proper meaning and structure you intended, refactor. This can be as simple as changing the names of classes, properties, and methods to make them more descriptive of what they do. It can be as complex as taking an application built on Table Module and Transaction Script patterns and converting it to a Domain Model with Data Mappers.

You also refactor in response to gaining new understanding. If some part of your application seems clumsy, as you proceed through your Iterative cycle, it may be a clue that you need to refactor. When you suddenly gain new insight into “how the business works” based on your collaboration with project stakeholders, you will often want to refactor that new understanding into your domain model. Refactoring in the domain model is a high-value target during your development cycle. Sometimes it takes some commitment to take the time to do all the work required to embed your new knowledge into the domain layer, but it is usually worth it.

Refactoring is a much discussed topic in Computer Science. I can hardly do it justice, considering the volume of resources available on the subject. For now, I’m going to direct you to some of those resources, and bid you Godspeed. Next time, we’ll talk about the safety net you’ll want to employ while refactoring: Test Driven Development.

Refactoring Resources:

• Refactoring with Martin Fowler: A Conversation with Martin Fowler, Part I
• Refactoring at Agile Alliance.org
• Martin Fowler’s refactoring.com
• Xp’s “Refactor Mercilessly”
• Database Refactoring
• A Story about refactoring
• Resharper

Books:

• Refactoring: Improving the Design of Existing Code
• Refactoring to Patterns
• Refactoring Databases: Evolutionary Database Design

Active Stakeholder Participation

Posted by indomitablehef | Filed under Agile

As mentioned in the previous post, one of the key success factors for doing iterative and incremental development successfully is Active Stakeholder Participation.

…Active Stakeholder Participation is an expansion of eXtreme Programming (XP)’s On-Site Customer that describes the need to have on-site access to people, typically users or their representatives, who have the authority and ability to provide information pertaining to the system being built and to make pertinent and timely decisions regarding the requirements, and prioritization thereof. While this level of participation is required to make your software development efforts effective it often isn’t sufficient in many organizations, particularly those where politics and not reason are the order of the day. Project success often requires a greater level of involvement by project stakeholders – senior management needs to publicly and private support your project, operations and support staff must actively work with your project team towards making your production environment ready to accept your system, other system teams must work with yours to support integration efforts, and maintenance developers must work to become adept at the technologies and techniques used by your system.(agilemodeling.com)

Whether you are attempting to put Iterative Development into practice or not, having this kind of cooperation from the customer/client/user/stakeholder is key - obstacles to good communication and cooperation with stakeholders are obstacles to writing good software, plain and simple. The realities of time, distance, politics, access, priorities, and personalities can get in the way of the kind of cooperation we want. Users and stakeholders themselves can be an obstacle:

Customer buy-in on the development process, particularly end-user involvement, is crucial. Insufficient end-user involvement is the number one reason why projects fail. Unfortunately, some customers resist getting involved. “Why should I spend time thinking about what I want?” they complain. “I’m paying you a lot — you go figure it out!” In fact, in such cases, the customer is delegating the definition of the problem rather than the task of solving it. That is why it is important to get their buy-in on the development process during project start-up.(Cardozo, 2002)

First and foremost, the responsibility for creating the kind of collaborative environment required to develop useful software falls upon you, the software developer. You must become evangelist, negotiator, counselor, beggar, cat-herder, and knee-capping thug in order to get the kind of cooperation you need. You’ll need to bargain, seduce, cajole, empathize, threaten, feint, jab, pick-and-roll. No one told me I would need those kinds of skills when I was studying Computer Science in school. It’s not enough to be a really smart guy, or to write really “good” code (as opposed to really “useful” code). Unless you are content with having someone to blame for your failures, be prepared to gird up your loins and wade into the fray, in order to snatch victory from the jaws of office politics.

Some tips on recruiting your customer into the active participation you need from them:

1. Learn about their business, and let them see you do it.
   Show a sincere interest in learning everything you can about how they do business. Never take the attitude that you have “some great ideas for how you can do things better”, or demand “business process re-engineering”. Instead, be humble, and be hungry and thirsty for knowing everything about their business. When they are confident that you really understand them, they will trust you more and be more willing to work closely with you.
2. Explain the idea of “Ubiquitous Language”
   When you express to the customer that “I want to learn to speak your language”, they will teach you. Again, stay humble. Don’t start trying to teach them new words, or change the names that they call things. If they need help hashing out the meanings of things among themselves, you should be a neutral facilitator whenever possible, or just stay out of it entirely.
3. Use “Inclusive Modeling Techniques”
   Don’t try to teach them UML. Most people, however, can learn to interpret boxes, lines, stick figures, and arrows on a whiteboard. Use simplified diagrams, index cards, post-it notes, string, modeling clay, legos, lincoln logs and large mounds of mashed potatoes if you have to. Whatever it takes to get clear understanding between you and the domain experts/users/customer/stakeholders.

Worth reading again: The seven habits of effective iterative development

Iterative and Incremental Development

Posted by indomitablehef | Filed under Agile

There are many aspects of software development that are often given “lip service”, but (relatively) seldom implemented successfully. Near the top of this list, you’ll find the next three topics we are going to cover: Iterative and Incremental Development, Refactoring, and Test Driven Development.

We’ll start with Iterative Development:

The basic idea behind iterative enhancement is to
develop a software system incrementally, allowing
the developer to take advantage of what was being
learned during the development of earlier, incremental,
deliverable versions of the system.
Learning comes from both the development and
use of the system, where possible. Key steps in the
process were to start with a simple implementation
of a subset of the software requirements and
iteratively enhance the evolving sequence of versions
until the full system is implemented. At each
iteration, design modifications are made along
with adding new functional capabilities. [1975, Vic Basili and Joe Turner, see reference below]

It is hard for thee to kick against the pricks

As we said before, “you never know enough.” Rather than treat this as something that went wrong, such that you have to retreat to a previous stage of development and re-consider your design, we will embrace it. We’ll not struggle against it, but instead use it to shape our new modus operandi. Rather than a sequential, phased approach of analyze, design, build, deploy, we “iterate” over the problem, beginning with the simplest part, and aggressively refactoring to make the application better and more complete with each iteration.

At the beginning, don’t even try to include all the features you do know about. Take a simplistic view of what appears to be the core of the functionality you intend to deliver. Begin with the core. Do the simplest part in your first iteration. In your second iteration do the hardest part. Or part of the hardest part. Let the software grow and get better and better.

Sounds simple enough, but in practice the entropy of the waterfall model is often hard to shake. Without Active Stakeholder Participation (which we’ll discuss next), it’s sometimes not even feasible. If you can’t get cooperation from the customers, you’ll have no choice but to compromise in your attempt to employ this model. That doesn’t mean you have to abandon it completely. You can do internal iterations, among the development team, and still get some benefit from that. Whenever possible, though, you should demand access to and cooperation from the users/stakeholders of the system.

Incremental Development, then, is an approach in which features of the application are delivered to the users in increments, in whatever order is most beneficial. So, you carve up the application’s eventual feature set into manageable chunks, and deliver them one chunk at a time. In many cases, the final product cannot be delivered piecemeal, however, so sometimes your “increments” are internal to the team. It’s still a useful approach, however. At the end of each increment, the team will evaluate their approach, and use what they learn from that in the next increment. Each increment will contain several iterations, where the team iterates over the feature set intended for that increment, until all the planned features for that increment have been completed. (clear as mud?)

Recommended Reading:
Iterative and Incremental Development: A Brief History (pdf)
Another Look at Incremental and Iterative Development(page13) (pdf)

Lest you think think me an expert, (or for those of you that know me, lest you think me full of shit), let me say this: Every project I have worked on so far has had at best an imperfect, flawed implementation of Iterative and Incremental development. Some projects have been closer to the ideal than others, but none has yet matched “how I think it should be.” I’m out to change that, however. In Drive.

With that in mind, check out: The seven habits of effective iterative development

Reference: Vic Basili and Joe Turner, 1975, from
R. Zultner, “The Deming Approach to Quality Software
Engineering,” Quality Progress, vol. 21, no. 11,
1988, pp. 58-64.cited from Iterative and Incremental Development:A Brief History

The Waterfall Must Die

Posted by indomitablehef | Filed under Agile

Up to now, we’ve been talking about design goals, giving special attention to Domain Modeling and Domain Driven Design. We talked about doing our design with a focus on creating a “ubiquitous language”, such that the software people and the business people call the same things by the same names, and understand them to work in the same way. We talked about layering our software such that we strive to create a distinct domain layer that reflects the things we have learned about the domain - the deep knowledge about the business - the real “intellectual property” part of our software.

Now let’s start talking about processes and tools that will help us to reach those design goals in particular, and just help us create good software in general. In this post, we shift the focus to “what developers do”, but it has implications for business analysts and project managers, too.

He’s Only Mostly Dead

Most organizations today still use some variation of the old waterfall method of software development, whether they like to admit it or not. The waterfall method describes a process where application development proceeds through distinct stages of analysis, design, construction, testing and deployment. When it was originally taught it included “feedback loops”, where you go back to a previous stage to get more information, and then proceed through the stages again. In practice, most people chose to simplify the charts that described the process, removing the feedback loops, and though they gave lip service to “iteration”, it usually got “simplified out” in actual practice as well. [see Going Round and Round and Getting Nowhere eXtremely Fast? Another Look at Incremental and Iterative Development]

The problem is that you never really know enough. In the waterfall paradigm, you would do analysis until you knew enough, then design until you knew enough, and then build it, test it, and deploy it. The “feedback loops”, (which were rarely done), were supposed to be done “as necessary”, whenever you realized that you didn’t know enough at the current stage, and needed to go back to the previous stage.

In contrast to that, one of the core principles of the family of methodologies known as “agile development” is that, indeed, you never know enough…so rather than treating that an an exceptional event, as in the waterfall method, accept it, embrace it, expect it, and tailor your process to work within this reality, rather than struggling against it. [see Agile Software Development, by Alistair Cockburn]

Rather than struggling to know as much as possible up front, change your approach completely: learn a little, build it, get feedback, refactor and build a little more.

The core development activities/methods that enable this kind of approach are
Iterative Development, Refactoring, and Test Driven Development. Doing it successfully demands Active Stakeholder Participation throughout the development process.

We’ll look in detail at each of these activities in the next post.

Recommended Reading: Agile Software Development, by Alistair Cockburn

Design Patterns: Now What?

Posted by indomitablehef | Filed under Agile

In our last post, we introduced Design Patterns. Today, we’re going to talk about how this changes things, and how it doesn’t.

When I discovered Design Patterns, I was initially skeptical, but by the time I finished PoEAA, I had design pattern stars in my eyes. Some of the patterns were difficult to understand (some I still don’t completely “get”), and some seemed so obvious that I wondered why they were mentioned at all. But in the middle I found some powerful ideas. Some, like Singleton and Factory Method, were so universally useful that I find myself including them in almost every application I write. Some, like Decorator and Strategy, provided elegant ways to express ideas that were sometimes clumsy before.

When I started to design my first application after having discovered these patterns, I was determined to use them in my new design. I spent days, weeks, studying the requirements, reworking my design, and “looking for” patterns. I took the wrong approach entirely, I think. I spent a great deal of time searching for places I could use the patterns I had learned in my design. I barely understood most of the patterns, and I was far too concerned about creating the “right” design. I would’ve been embarrassed if someone reviewed it and asked “Why didn’t you use an Abstract Factory here?…It’s so obvious, any fool could see…”. Before, I had been a confident, even cocky, software developer. I’d delivered some good applications, been praised for my work, and felt like I was pretty good at my job. Somehow, though, I lost some of that confidence, and started trying to do things like the textbooks (design pattern books) said I should.

Don’t get stuck on stupid

The reality is, there is a learning curve involved in understanding and making good use of design patterns. If you are learning this on your own, get used to the idea that you are going to stumble over them from time to time. When I look back at my earliest uses of design patterns, I feel a little embarrassed. I recently went through a section of application code I wrote in that first application I mentioned before, and realized that all the classes that I had called Entities were really Transfer Objects, and most of the classes I had called Transfer Objects would really have been better as Entities. (I think that’s actually worse than the average beginner mistakes with design patterns)

If you have someone around who has more experience with design patterns, get them to help you. It can be a delicate situation for the experienced person, critiquing someone else’s design…especially when that person is as excited about the cool new design patterns they had “found” as I was. The best situation is to design the system together with an experienced person, so you can work through the thought process together collaboratively.

The one thing you don’t want to do is to get stuck. Design your application just as you would have before. If you see an obvious use for a design pattern, use it. Look up the definition, understand the purpose of the thing, and maybe try out some example code. If you find yourself going back over the design again and again, and flipping through PoEAAand GOF to see if you missed anything, Stop! Go build it. If you run into a problem, consider researching design pattern solutions before building your own solution, but if you don’t find one don’t get stuck.

Don’t worry, I speak Jive

Among a team of developers experienced in using design patterns, the names of commonly used design patterns can become part of the lingo. If I tell a team member that the Profile Utility Service is a Singleton, he knows that there is only one instance of the service running for the entire application. He has some idea of how the application makes calls to the service, though not necessarily anything about what the service does. If someone asks, “What does this “Exportable” class do?” I can say “It’s a Decorator for the SearchQuery object, which is just a Query Object. It gets passed to the Data Mapper, and we get back an Aggregate of Transfer Objects (that represent matches to the query) from the Identity Map.” And voila! You’ve said a mouthfull. It’ll take time for this kind of communication to develop.

In the context of a domain driven design, design patterns play an important role:

Developing a good domain model is an art. But the practical design and implementation of a model’s individual elements can be relatively systematic. Isolating the domain design from the mass of other concerns in the software system will greatly clarify the design’s connection to the model. Defining model elements according to certain distinctions sharpens their meanings. Following proven patterns for individual elements helps produce a model that is practical to implement. - Eric Evans, Domain-Driven Design

Using certain design patterns can help us not only make the model practical to implement, but also help us segregate application functionality from domain functionality. The use of Factories, for example, allow us to separate the details about what an object is and what it does from the details about how that object is created (or re-created from the database).

Check out the title page of Domain Driven Design Quickly (pdf) for a look at some of the patterns described by Evans as enablers for domain driven design.

“There’s no exit ramp, and you wouldn’t have time to stop, even if there was.”

It’s no good calling a halt to everything to take the time to learn a bunch of design patterns from a book. That’s not a good idea, even if you could pull it off, in my opinion. Don’t worry too much about design patterns up front, in your initial design. Most of your use of them will come during your refactoring process. You’ll recognize that you only want one instance of the Profile Utility Service to run in memory at a given time when you see that this object takes a while to build, and application performance suffers under load. Sounds like a Singleton to me, boss.

For more discussion on refactoring to design patterns, check out Refactoring to Patterns

Design Patterns

Posted by indomitablehef | Filed under Agile

As I mentioned before, there was a point in time where I became aware that there was a significant body of work out there on object-oriented design and processes. Of course, I knew the major OO principles already (Inheritance, Modularity, Encapsulation, Polymorphism), but as far as deep thinking philosophical views on design in an object oriented world, I was definitely a noob. My first glimpse into that world came, as I suspect it did for many of us, when I began to learn about “design patterns.” A design pattern is a “general repeatable solution to a commonly occurring problem in software design.” (Wikipedia) It’s not code that you can just copy into your application, but rather a template, a well thought out way to solve a particular common problem. At the simplest levels, things like loops and if-then-else conditional statements could be considered “design patterns”. The design patterns I’m talking about, however, are a level or two above those in terms of the complexity of the problems they solve. These include commonly used patterns like Factory Method and Singleton, and even more interesting ones such as Strategy and Decorator. Domain Model, which we’ve been talking about so much here already, is itself a design pattern. My first exposure to these patterns came from a couple of books. The first was Design Patterns: Elements of Reusable Object-Oriented Software, by Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides, i.e., “The Gang of Four”. This book is well known as “the Gang of Four patterns book”, and when referenced, is usually noted as GOF. The second one was Patterns of Enterprise Application Architecture, (PoEAA) by Martin Fowler, which I’ve mentioned previously. Both books are great to read, and definitely desirable to have around as reference books. My favorite “at your fingertips” resource for patterns (which is also perfect for the patterns beginner) is dofactory.com. They have a short list of some of the most common creational , structural, and behavioral design patterns, along with illustrative code samples.

More great web resources for learning about design patterns:

• Martin Fowler’s online catalog of the patterns described in (PoEAA)
• Martin Fowler’s links to other catalogs of enterprise software patterns
• Ward Cunningham’s Portland Pattern Repository
• Microsoft’s Enterprise Solution Patterns Using Microsoft .NET
• Another list of links to pattern sites from hillside.net

Another interesting type of pattern is the “analysis pattern”. An analysis pattern is basically an example of a domain model for a particular domain. It works particularly well when working in domains that are already well-defined, such as accounting or physics. (see Analysis Patterns: Reusable Object Models , also by Martin Fowler). Then there are AntiPatterns.

Next post: Putting patterns in perspective

How should we then live?

Posted by indomitablehef | Filed under Agile, DDD

If we are convinced of the value of domain driven design, what do we do next?

1. Don’t try to model everything

Let me ’splain.
[pause]
No, there is too much. Let me sum up. - Inigo Montoya

You could create a complete set of UML structural diagrams for your entire application (package diagrams, class diagrams) and some detailed interaction diagrams (sequence diagrams, state machine diagrams) and use them to generate source code.

You could.

I did it once.

But as Inigo Montoya says, “there is too much”. Instead, carve out that distinct domain layer. This is the core of your application. This is what you should spend time thinking deep thoughts about. This is what you should refactor over and over again. This is what you should model.

That is not to say that the only diagrams you do are the ones that are specific to the domain model. It does mean, however, that you shouldn’t bother trying to maintain the most of the other diagrams you do while designing and building an application. When you are done, the artifacts that will remain are probably only the high-level use case diagrams, the deployment diagram, and the diagrams that are part of the domain model. You may decide to keep some of your activity diagrams, etc., but pretty soon the application will be updated, and those diagrams will be out of sync, and it won’t be worth it to keep up with them. If you want to keep it, keep it. But don’t get caught up in maintaining diagrams that aren’t valuable anymore. We have better things to do.

2. Do maintain the domain model

If you have really carved out the domain model into its own distinct layer, this should turn out to be easier than it sounds. The day-to-day requests of users for little tweaks to application functionality should not change the way the business operates, should it? Neither should those requests require changes to the domain layer. If you’ve really captured deep knowledge about the domain, it will be obvious both to you and to the business users when they have a request that affects this core. They will recognize it as a change to their business practices or structure. You will recognize it as a change to the domain model. Keep the code, the diagrams, and the language/glossary up-to-date and in sync.

3. Your software experts must become domain experts

The tendency among very talented software developers is sometimes to take the position that “I am a software expert, not a(fill in the blank)expert”. (Not a healthcare expert, not a manufacturing expert, etc. ) If we want to do better than we have done in the past, we have to take a different attitude. The best developers/architects (I’m starting not to like the term ‘architect’, but I don’t have a good replacement for it) will have to begin to become domain experts. They will need to have a close relationship with the business domain experts. Their domain expertise becomes an asset of significant value to the company. Of course, they still have to be software experts, too. It’s becoming clearer and clearer that developers are architects/designers/modelers. These roles cannot be so distinct that a designer writes the spec and a developer goes off and writes the code according to the spec. (except in narrowly defined situations)

(a topic I intend to address further in the future is how a company can ramp up development, making use of competent mid-level developers, without having to search for developers that already have domain expertise, who often come at a premium)

4. Do it first, and put your best man on it

Domain-driven design flows from the premise that the heart of software development is knowledge of the subject matter and finding useful ways of understanding that subject matter. The complexity that we should be tackling is the complexity of the domain itself — not the technical architecture, not the user interface, not even specific features. This means designing everything around our understanding and conception of the most essential concepts of the business and justifying any other development by how it supports that core. - Eric Evans

Do it first. Let the evolution of the domain model dictate your other decisions whenever possible. Your best developers should be tasked with developing the most important parts of your software, right? Put them on the domain model first.

What is a domain model?

Posted by indomitablehef | Filed under DDD

Maybe I’ve convinced you that a domain model is useful. Maybe I haven’t. Either way, let’s talk more about what a domain model is.

It is not a UML diagram. It is not a bunch of Objects in an object oriented programming language. (go ahead, cry “heresy!” if it will make you feel better)

A domain model is a conceptual model of key business concepts, entities, processes, and their relationships.
It has three important components:

1. a visual representation of the business concepts and their relationships
2. a shared language between developers/modelers and the business experts (domain experts)
3. a software representation of the domain model

Some or all of the domain model will eventually be turned into software. When it is, that portion of the software that represents the domain model becomes part of the domain model. A change to the software becomes a change to the model.

The domain model may be supplemented by specification of a set of business rules. It may be expounded upon by detailed representations of the interactions between key business entities. The domain model often expresses the state of the business - the state of the business entities and relationships - at a point in time. In such cases, the model may be expounded upon by detailed representations of the possible states of the business entities and relationships.

Ubiquitous Language

Throughout the creation of a rich domain model, developers/modelers must communicate effectively with domain experts. These developers and business experts must learn to speak the same language. Since we are writing software to support the business, instead of the other way around, this means that the developers must learn to speak the language of the business (for the most part). As the model begins to develop, a ubiquitous language begins to develop, and is shared by the developers and domain experts. This shared language is an essential part of the domain model itself. It may even take the form of a glossary document that becomes a model artifact. Even if their is no glossary document, however, the ubiquitous language is an essential part of the model.

It is the language that tells you when the model is probably wrong. If developers and domain experts have trouble expressing themselves to each other, trouble understanding the meaning of language that is important to the business, then something is wrong with the model. If the language is clumsy and unwieldy, and you have to continually translate back and forth from developer understanding of the domain to business people’s understanding of the domain, then something is wrong with (or missing from) the model.

Further reading about Domain Models and Domain Driven Design:

• Patterns of Enterprise Application Architecture: Domain Model
• Why Domain Driven Design Matters
• Domain Driven Design Quickly (free download-able book in pdf)
• 
  Anemic Domain Model (an Anti-Pattern)
• Domain Logic and SQL
• 
  Yahoo! group on Domain Driven Design