By Don Worthley on 8/22/2007 8:18 PM

When Fred Brooks wrote No Silver Bullet in 1986, he was convinced that the majority of complexity left in the software development process was not a result of what he called accidental complexity, the complexity that results from the way we go about solving a problem; rather, it was a result of essential complexity, complexity intrinsic to the problem domain. 

I'm not as interested in Brook's speculation regarding whether we would see a 10 fold increase in productivity over a 10 year period of time.  I honestly think we're getting close to this through the use of code generation, better design patterns and advancements in object-relational mapping (ORM).  I'm fascinated by the bifurcation of the software development process into these two discrete segments: accidental and essential complexity.

It's an exciting time to be working in the software development industry, and I think the excitement builds as we are able to more clearly separate the complexity of any project into that which is related to the business problem and that which is related to the way we go about solving the business problem.  

I heard someone say on a training video years ago about component development that you wanted to "keep things separate that should be separate and keep things together that should be together."  At first, I thought, "How simple. who wouldn't follow this principle?"  But after years of working as a developer and trainer, I realize now that this rule of separation is extremely important to the development process, and in a much larger sense to the ability to create working solutions in the enterprise.

As I think through the architecture of a solution now, I realize that my mind is sifting through the problem domain trying to identify orthogonality.  In some cases, this can be hard to identify.  One of the questions that I can ask myself is "What are the areas intrinsic to this specific problem that are intrinsic to the last 10 problems I've had to solve?" 

It is this attempt to identify orthogonality which has lead to the growing interest in things like attribute-oriented programming (@OP), ORM frameworks and software factories. 

In spite of all of the advancements in areas related to the accidental complexity of software development, we still find ourselves slogging through a jungle of process and technically oriented difficulty before we ever reach the complexity of the actual business problem.

Why is this?  In part, I think it's due to the relative age of our profession.  Yes we've been around for some 50 or 60 years, but as Steve McConnel notes in many of his books and more recently in Construx, his blog and a discussion forum of his for software engineers, we still have our work cut out for us when it comes to solving problems like our engineering counterparts in other industries. 

While wonderful advancements have been made in our industry, we still fall short--in part, because of the overabundance of competing approaches to solving this problem.  I was having lunch with another software developer in the area who was describing the frustration the developers felt in their shop when they had to support a solution with a middle tier generated by LLBLGenPro.  Although I haven't used this particular ORM tool, I've heard great things about it.  The issue was that most of the developers on the team didn't have experience with this ORM either.  They've probably had experience with other ORM tools, but unfortunately the infrastructure of each ORM tools is different.  Take .netTiers, for example, which--although, like LLBLGenPro, generates the classes based on your data model-- does not use the same templating engine.  .netTiers uses CodeSmith, which is a great code generation tool, but has its own learning curve.  Or take NHibernate, or Hibernate if you're a Java developer (I realize Hibernate came first, but I'm a hopeless .NET developer).  With NHibernate, you develop the description of the domain model and the tool generates the data model to support your domain objects, kind of the reverse of .netTiers and LLBLGenPro.

Even Microsoft has jumped on board the ORM wagon with LINQ and LINQ to SQL.  As you may have seen in previous posts, I have grown fond of SubSonic, a very simple, easy-to-use and open source ORM.

But the issue of orthogonality isn't just about ORM tools.  As I mentioned above, attribute-oriented programming techniques have helped tremendously in the effort of dividing the technical and business related domains.  In fact, attribute-oriented programming probably should have been called orthogonality-oriented programming, but OOP was already taken.  The whole goal with attribute-oriented programming is to identify those aspects of business solutions that are orthogonal to the business logic, aspects that occur in almost every business application written, aspects such as logging, wrapping operations in a transaction or enforcing some form of authorization.  You can see this in action with the Enterprise Services features built into the .NET framework. 

Additionally, the recent interest in Service-Oriented Architecture (SOA) has lead to wonderful improvements, on the technical side, in how business process are properly organized and made available to disparate solutions within an organization.  At the heart of these improvements is the desire to identify the proper kind of separate between business logic and the technical aspects of how that business logic is organized and made accessible.  Over the years, we have found that further dividing our business logic into orthogonal services that are accessed with clearly defined and immutable contracts makes it easier to reduce the duplication of effort that had become so common in enterprise development.

So how is all of this important to me on the business side?  It's critical in today's market to be identifying solutions that have implemented a high degree of orthogonality between the technical and business domains as well as between different technical areas that need to be properly separated.  Hire an independent consultant or have your in-house architects take a look at the infrastructure of any proposed solutions within your organization in light of this issue of orthogonality and ask questions about how the proposed solution makes use of current best practices to keep things separate the should be separate and keep things together that should be together.  The solution doesn't have to use an ORM or to even be programmed in an environment that makes use of attribute-oriented programming or SOA, but it does need to be extensible and maintainable, two areas that suffer the most when an application is not designed well. 

Here are some issues I would consider when thinking about the orthogonality of a proposed solution:

• What tools, frameworks or factories have been used in the development of the solution? 
• Are they proprietary or open source? 
• If they are proprietary, how widely has this platform been adopted in the industry?  What kind of documentation is available and how active are the support forums?
• If they are open source, how widely is the open source project adopted and how active is the community?
• Has the vendor re-invented the wheel in solving common technical infrastructure issues?  For example, have they written their own workflow engine when they could have used an existing workflow framework?
• How has the business logic been organized internally and what are the APIs for accessing that logic.  Are they using industry standards or principles such as SOA? 
• If the solution provides access through web services, how have the web services been developed?  How is authentication handled?
• What design patterns were used in the development of the software?  For example, does the solution use MVP or MVC to keep business logic separate from the UI?
• How have the extensibility layers been developed? 
• Have best practices been used to help ensure that extending the product won't cause issues with upgrades to the baseline product?

Recently, I've spent some time looking at Microsoft Dynamics CRM 3.0, and, although I haven't had the chance to write an application on top of this framework, I've been very impressed with what I've seen, especially related to the orthogonality of the business and technical domains.  While there are some limitations such as the inability to natively handle many-to-many relationships between entities, a limitation that I hear will be addressed in the next release of CRM code-named Titan, the framework seems to provide a way to begin focusing on the business domain without spending an inordinate amount of time writing your own infrastructure.  The extensibility points are there, and they have been designed so that upgrades to the baseline product should have minimal to no effect on customizations.

How about you?  What software have you used in your organization that has maintained a high degree of orthogonality between business and technology? I’m interested to hear from you!

By Don Worthley on 8/8/2007 8:15 PM

WARNING:  This is a technical entry with no business related information.  It relates to Microsoft SQL Server, SubSonic and .NET with code written in C#.

I was responding to an issue in the SubSonic forums a few weeks back when I discovered that the INFORMATION_SCHEMA.PARAMETERS view didn't contain information on the nullability of a stored procedure's parameters.  No problem, I did a little searching and found the following two stored procedures: sp_sproc_columns and sp_procedure_params_rowset.  Excited at the promise of valuable information on the nullability of my parameters, I ran the following script to create a quick stored procedure that contained both a parameter with a default and one without.

CREATE PROC TestProc (@intDefault int = 0, @intNoDefault int)
AS
SELECT @intDefault as 'ValueEntered'

I ran both of the stored procedures above only to find that the nullability for both parameters was the same:  nullable.  Hmm.  Ok.  I must be misunderstanding what it means for a parameter to be nullable.  At this point, I began to see that I had confused having a default value, which determines if having at least some value (maybe NULL) is required, and being nullable.  So I dug into SQL Books Online and found that parameters are nullable by default.  So in SQL Server you can have a parameter that's required, but nullable.  That is, the parameter is required, but you can pass in NULL as a value for the parameter.

That said, let's jump over to SubSonic, my favorite ORM of late, which, by the way, I'm really hoping continues to maintain a critical mass of developer interest even as Microsoft is working hard to release LINQ to SQL.  I think Rob, Eric, et. al. have done a great job balancing simplicity and ease of use with critical features.  There's a lot behind SubSonic, but it's also easy to use,  an extremely important balance that's needed for any technology to survive.

One of the things I absolutely love about SubSonic is the fact that it supports the strongly typed use of all of my stored procedures.  I wish there was a named indexor for the parameters collection, but that may be a good topic for another post.  The issue at hand with SubSonic's implementation of the stored procedure parameters is that it uses nullable types for all of the parameters regardless of whether there is a default value assigned.  Phil Haack mentions in this issue on CodePlex that it would be great if SubSonic could distinguish between parameters that have default values and those that don't and are therefore required parameters.

Having searched high and low through the meta data available to mere mortals like me in SQL Server, and having seen a post in the newsgroups from someone who said that the only way to determine if a stored procedure parameter had a default value was to parse the TSQL, I decided to write a routine to do just this using regular expressions. 

The routine is available inside the patch download in the SubSonic Issues log on CodePlex.  I've also included code you can use to test retrieving the default value status of each of a stored procedure's parameters.  To use the following code, create a web form with two textboxes, two labels and a button as follows: txtDatabase, txtSproc, lblSproc, lblArguments and btnProcessSproc.  In the click event of the button, place the following code (Written in C#):

        string cnString = @"Data Source=Server;Initial Catalog=" + txtDatabase.Text +

                          ";User ID=sa;Password=password;";

        string SQL = "EXEC sp_helptext " + txtSproc.Text;

        SqlDataReader sprocReader;

        StringBuilder sprocBuilder = new StringBuilder();

        string sproc = string.Empty;

        string arguments = string.Empty;

        string[] argumentArray = new string[0];

        Hashtable parameterNullibility = new Hashtable();

        using (SqlConnection cn = new SqlConnection(cnString))

        {

            cn.Open();

            SqlCommand cmd = new SqlCommand(SQL, cn);