Agile Business Intelligence

I was reading through the blogosphere his morning and came across some interesting observations on Agile BI. Basically, the author threw out the question, "what would our children turn out like if we raised them like we create BI systems?" The authors contention is that most people treat BI like a protected entity until it is absolutely ready for consumption, and even then rolls it out in a very limited fashion.

This got me to thinking about what Agile really means in the context of BI systems. The way I see it, Agile and BI are so naturally complimentary that you almost have to embrace agile in order to build and deploy a successful BI system. BI systems are all about condensing data into usable information then helping the consumer to develop that information into workable knowledge. Everyone who's ever built a BI system knows that the best measure of success in BI is how well that knowledge is used to close the loop and affect change on the processes that the BI system measures. If this doesn't scream "agile, iterative development" I really don't know what does. Consider the following tenets of agile programming:

• Individual Interactions over Processes and Tools
• Working Software over comprehensive documentation
• Customer Collaboration over Contract Negotiation
• Responding to change over following a plan

Now consider how they apply to a BI project:

• It doesn't matter what you use to build the product, it's all about understanding the requirements and working towards a system that fulfills those requirements.
• The system is going to change many times throughout the development lifecycle. Building a system that actually works and provides the output that the consumer needs is much, much more important that providing a manual and weeks of training.
• Understanding what the customers need is more important that telling them what you're going to build.
• See the second bullet. The system is going to change. The information needed to form knowledge will change. The desired output will change. The loop that you're trying to close will change. Everything changes.

 

I always tell my team that it is OK to be uncomfortable during the development cycle. As a matter of fact, I try and make sure that they ARE uncomfortable because that tells me that they're pushing the envelope. Waterfall methodologies just don't work in a BI world, and while agile can lead to discomfort, I really believe that the end result is well worth it.

Configuration Intelligence Analytics - aka Really Cool Stuff!

I don't spend a lot of time talking about the products and services that my company, Configuresoft, offers in this blog, mainly because I wanted it to be more of a repository for technical information and things that might help others. However, sometimes things are just too cool not to mention.

In my role as Product Strategy Architect, I'm responsible for our Configuration Intelligence Analytics product. We recently went "Gold" on version 1.0 of the product. This is a product that could very easily revolutionize the systems management world (yes, I'm biased so it's easy for me to say that). We've take a bunch of leading and bleeding-edge products from Microsoft, applied time-tested Business Intelligence methodologies and delivered a product that marries an organizations Configuration Management data with operational data from their Service Desk. The result is in my not- very-humble-at-all opinion is one kick-ass product. I was onsite with a customer this week to help deploy the tool into their environment, and it was really awe-inspiring for me. Have I mentioned lately how much I truly love my job?

SANs and SQL Server Performance

Those that know me know that I spend a fair amount of time in the world of SQL Server internals and performance tuning. Over the last year I haven't been able to spend much time in that space due to other project priorities. It's good to see though that there's more and more information being published/blogged/etc. About SQL Perf-tuning.

Here's a really great article by Linchi Shea over on SQLBlog that caught my eye (Yes Chris, I'm aware you've blogged about it first!).

The best thing about this article is that it really isn't a "SANs are bad because...." type article, but rather gives you examples of things to test in your particular environment.

Exception Handling in WCF

I have recently had the pleasure (yes, it really is a pleasure!) to work a bit closer with the new .NET technologies such as Windows Communications Foundation (WCF). One of the things that I've found myself struggling with is some of the more esoteric aspects of precedence in exception handling. (Yeah, I know, this is basic stuff, but I was surprised at how I got caught with this and figure maybe there's one or two more out there that can benefit from my bumbling)

Consider the following code snippet (from MSDN):

using System;
using System.ServiceModel;
using System.ServiceModel.Channels;
using Microsoft.WCF.Documentation;

public class Client
{
public static void Main()
{
// Picks up configuration from the configuration file.
SampleServiceClient wcfClient = new SampleServiceClient();
try
{
// Making calls.
Console.WriteLine("Enter the greeting to send: ");
string greeting = Console.ReadLine();
Console.WriteLine("The service responded: " + wcfClient.SampleMethod(greeting));
Console.WriteLine("Press ENTER to exit:");
Console.ReadLine();
}
catch (TimeoutException timeProblem)
{
Console.WriteLine("The service operation timed out. " + timeProblem.Message);
wcfClient.Abort();
Console.ReadLine();
}
// Catch the contractually specified SOAP fault raised here as an exception.
catch (FaultException<GreetingFault> greetingFault)
{
Console.WriteLine(greetingFault.Detail.Message);
Console.Read();
wcfClient.Abort();
}
// Catch unrecognized faults. This handler receives exceptions thrown by WCF
// services when ServiceDebugBehavior.IncludeExceptionDetailInFaults
// is set to true.
catch (FaultException faultEx)
{
Console.WriteLine("An unknown exception was received. "
+ faultEx.Message
+ faultEx.StackTrace
);
Console.Read();
wcfClient.Abort();
}
// Standard communication fault handler.
catch (CommunicationException commProblem)
{
Console.WriteLine("There was a communication problem. " + commProblem.Message + commProblem.StackTrace);
Console.Read();
wcfClient.Abort();
}
}
}

Notice that the generic CommunicationException is handled as the last item in the catch block. The reason for this is that FaultException is actually derived from CommunicationException. This causes any FaultExceptions to be caught by the CommunicationException handler...

What this means practically is that you must take care and order your exception handlers properly when dealing with these types of Exceptions. Generally speaking, when writing WCF-based code, you'll want to structure your exception handlers similar to the sample above.

The moment of "Duh"!

During my GeekSpeak session on Wednesday, I was asked to provide a high-level overview of Business Intelligence. I provided what I thought was a decent answer, had a couple of follow-on statements, and Glen chimed in with some info as well.. Sounds good, right???? Well, here it is Friday now, and i was just going through the MSDN SharePoint Forums on BI when it hit me, "Duh!" Why didn't you tout Lynn's book on the subject????

So, better late than never. Lynn Langit has written a fantastic business intelligence overview book.

Amazon.com: Foundations of SQL Server 2005 Business Intelligence: Books: Lynn Langit ISBN: 1590598342 ISBN-13: 9781590598344

Yes, he really did beat me!

It was pointed out to me by my friend and colleague Chris Randall (Who by the way is an excellent SQL trainer, if you ever need to take a SQL Class, you should look him up at Ameriteach in Denver) that he blogged about the SQL Internals Viewer in this blog post well before I blogged about it...

So yes, it has been proven that there is at least person who's more geeky than I when it comes to SQL Server guts!

SharePoint 2007 "Farm" Sizing

Yesterday, during my GeekSpeak webcast (Which was fun by the way) the question came up as to how many SSRS instances could be supported per SharePoint farm. I kind of stumbled through the question and didn't have a very good answer, so I spent a little bit of time yesterday researching the topic. Unfortunately I don't have a much better answer to post here today, but one thing that comes to mind is that there is a clear statement from Microsoft that there should be 1 Database Server for every 8 Web Servers in a farm. (See the Microsoft SharePoint sizing recommendations here: http://technet2.microsoft.com/Office/en-us/library/6a13cd9f-4b44-40d6-85aa-c70a8e5c34fe1033.mspx?mfr=true )

So, using the ideas in the document above, and understanding that there can only be 1 SSRS instance for each Web Server, and "taking a bit off the top" for the overhead on the database server that an SSRS instance will generate, I'd say you should modify the recommendations slightly and say that you should limit SSRS integration to 5 instances of SSRS per Database Server in the farm. (This is in no way an official limitation, it's simply conservative extrapolation on my part)

SQL Server Storage Engine Internals

I'm a geek. Yes, it's true.. No question about it.. As mentioned in earlier posts, I've written articles and book chapters dedicated to the internal workings of various SQL Server components.. Well, it appears that I'm not the only geek out there with an interest towards SQL Server internals.. Turns out there's a pretty good community of us actually.. :)

Anyway, check this site out: http://www.sqlinternalsviewer.com/ basically it's a tool that let's you see exactly how the SQL Server storage engine is actually storing your data.. Way cool!