Using the PIVOT operator on SQL Server 2005+

The PIVOT operator is used to transform rows into column by an aggregate function. One big limitation is that you have to know the columns, but still, it's useful on some use cases, like when you want to aggregate something by hours (for example).

About having to know the column names limitation: This is probably a design decision, because someone could write a query that has too many columns and cause all sort of problems, like exceeding the limit of columns or return column that the application is not prepared to receive.

To keep the example easy to understand we will create a simple Authentication table (which holds authentications):

IF OBJECT_ID('Authentication') IS NOT NULL
 DROP TABLE Authentication

CREATE TABLE Authentication
(
 AuthenticationId INT,
 Date SMALLDATETIME,
 DocumentId UNIQUEIDENTIFIER
)

INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (1, DATEADD(hh, +1, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (2, DATEADD(hh, +1, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (3, DATEADD(hh, +2, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (4, DATEADD(hh, +3, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (5, DATEADD(hh, +4, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (6, DATEADD(hh, +4, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (7, DATEADD(hh, +5, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (8, DATEADD(hh, +6, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (9, DATEADD(hh, +7, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (10, DATEADD(hh, +7, GETDATE()), NEWID())
INSERT INTO Authentication (AuthenticationId, Date, DocumentId) VALUES (11, DATEADD(hh, +8, GETDATE()), NEWID())

Now we have some authentications. Let's say some manager wants a report of authentications by hour to know how many people comes to pay bills on the company stores (assuming that some kind of authentication is done when receiving payments). The information can be obtained by doing a query like this:

SELECT [0], [1], [2], [3], [4], [5], [6], [7], [8], [9], [10],
  [11], [12], [13], [14], [15], [16], [17], [18], [19],
  [20], [21], [22], [23]
FROM (SELECT AuthenticationId, DATEPART(HOUR, Date) AS Hour FROM Authentication) AS TBL
PIVOT (
 COUNT(AuthenticationId)
 FOR Hour IN
 ([0], [1], [2], [3], [4], [5], [6], [7], [8], [9], [10],
  [11], [12], [13], [14], [15], [16], [17], [18], [19],
  [20], [21], [22], [23])
) AS PVT

The result will show the authentication count for each hour. Now you can copy it and paste on Excel. Maybe generate some charts (assuming your use case has more data and more rows).

Tracking UPDATE changes on SQL Server 2005+ with the OUTPUT clause

It is common to have to execute a mass update on some table, for example, to update a customer credit classification. When such updates are executed, you may want to know what has changed and maybe log it. There are several ways to do it, but i recently found one that makes it easier: the OUTPUT clause.

First, we create a customer table for the example and populate it with some data:

IF OBJECT_ID('Customers') IS NOT NULL
 DROP TABLE Customers
GO

CREATE TABLE Customers (
 CustomerId INT,
 [Name] VARCHAR(30),
 Income MONEY,
 Classification CHAR(1)
)

INSERT INTO Customers (CustomerId, [Name], Income, Classification) VALUES (1, 'JOHN DOE', 5000, 'C')
INSERT INTO Customers (CustomerId, [Name], Income, Classification) VALUES (1, 'JANE DOE', 6000, 'B')
INSERT INTO Customers (CustomerId, [Name], Income, Classification) VALUES (1, 'JOHN SMITH', 6500, 'B')
INSERT INTO Customers (CustomerId, [Name], Income, Classification) VALUES (1, 'HOMELESS JOHN', 100, 'Z')

Now, let's update the data (imagine that there are thousands of rows):

UPDATE Customers
SET Classification = 'C'
OUTPUT DELETED.CustomerId,
    DELETED.[Name],
    DELETED.Classification AS FromClassification,
    INSERTED.Classification AS ToClassification
WHERE Income BETWEEN 5000 AND 7000

The OUTPUT part will cause the following output:

CustomerId  Name                           FromClassification ToClassification
----------- ------------------------------ ------------------ ----------------
1           JOHN DOE                       C                  C
1           JANE DOE                       B                  C
1           JOHN SMITH                     B                  C

If you want to persist the values:

IF OBJECT_ID('CreditChangeLog') IS NOT NULL
 DROP TABLE CreditChangeLog
GO

CREATE TABLE CreditChangeLog (
 CustomerId INT,
 FromClassification CHAR(1),
 ToClassification CHAR(1)
)

UPDATE Customers
SET Classification = 'C'
OUTPUT DELETED.CustomerId,
    DELETED.Classification AS FromClassification,
    INSERTED.Classification AS ToClassification
    INTO CreditChangeLog
WHERE Income BETWEEN 5000 AND 7000

SELECT * FROM CreditChangeLog

This will save the changed values on the CreditChangeLog table.

Avoiding code execution when debugging or testing an ASP.net application

It is common to have some code that is part of some workflow and you don’t want it to execute when you are testing. For example, if your code sends an e-mail to some users, you don’t want to crud your users mail box with e-mails originated from your tests. Unless you are testing if sending the e-mail works, you can do this to test the rest of your code:

        static void SendMail(string to, string subject, string body)
       {

#if !DEBUG
           using(var message = new MailMessage())
           {
               message.From = new MailAddress(SiteSettings.Settings.AdminMail);
               message.To.Add(new MailAddress(to));
               message.Subject = subject;
               message.Body = body;
               message.BodyEncoding = Encoding.UTF8;
               message.SubjectEncoding = Encoding.UTF8;

               var mailClient = new SmtpClient();
               mailClient.Send(message);
           }
#endif
       }

The #if !DEBUG pre-processor will ensure that the code is not compiled into the assembly so it does not gets executed. Just remember to set your build configuration to “Debug” on Visual Studio.

Caching HTML helpers on ASP.net MVC with optional SQL Cache Dependency

When thinking about performance, one of the top things that comes in mind is caching. Caching means that you will save some server resources by saving the result on the “first” time it’s processed. With ASP.net MVC, it’s common the use of HTML Helpers, that are just methods that return a resulting HTML. We can combine both and cache the resulting HTML on the first time the method is called, round house kicking the performance of the page.

To archive this, we first create a CacheExtensions class to hold our caching methods:

public static class CacheExtensions
{
   public static bool TryGet (this Cache cache, string key, out T value)
   {
       var obj = cache.Get (key);

       if (obj != null)
       {
           value = (T)obj;
           return true;
       }

       value = default(T);

       return false;
   }

   public static void CacheData (this Cache cache, string key, string profileKey, T data)
   {
       var dc = DataCacheSettings.GetDataCacheProfile (profileKey);

       if (dc == null)
           throw new ArgumentException ("Data cache profile {0} not found".FormatWith (profileKey));

       if (data.Equals(default(T)))
           return;

       if (dc.Enabled)
       {
           if (dc.UseDependency)
           {

               if (string.IsNullOrEmpty(dc.SqlDependency))
                   throw new InvalidOperationException("sqlDependency must be set when dependency is enabled.");

               var sqlDepInfo = dc.SqlDependency.Split(new char[] {':'}, StringSplitOptions.RemoveEmptyEntries);

               if (sqlDepInfo.Length != 2)
                   throw new InvalidOperationException("sqlDependency option must obey the Database:Table format");

               cache.Insert (key,
                             data,
                             new SqlCacheDependency(sqlDepInfo[0], sqlDepInfo[1]),
                             Cache.NoAbsoluteExpiration,
                             TimeSpan.FromMinutes(dc.Duration),
                             dc.Priority,
                             null);
           }
           else
           {
               cache.Insert (key,
                             data,
                             null,
                             Cache.NoAbsoluteExpiration,
                             TimeSpan.FromMinutes (dc.Duration),
                             dc.Priority,
                             null);
           }
       }
   }
}

The TryGet method will check if the cache key exists and assign it to the out value, returning true if there was a matching key and false if the cache didn’t exist.

The CacheData method caches the data by the provided key. This method supports cache profiles and SQL Server Cache Dependency which i will explain later on this same article.

To use it:

public static class HtmlHelperExtensionsWithCache
{
   public static string Hello(this HtmlHelper helper, string name)
   {
       string key = String.Concat("hello-", name);
       string ret;

       if (Cache.TryGet(key, out ret))
           return ret;

       //if you have to format some html, use StringBuilder
       ret = String.Format("Hello {0}, have a nice day!", name);

       Cache.CacheData(key, "Html", ret);

       return ret;
   }
}

This extension method will act as a HTML helper and cache the message depending on the user name. The “Html” parameter of the CacheData method will read the “Html” cache profile from web.config, which is configured this way (for example):

<datacachesettings>
   <profiles>
   <!-- duration in minutes -->
       <add priority="High" duration="30" name="Html" />
       <add priority="Normal" duration="10" name="User" />
       <add priority="AboveNormal" duration="15" name="StoryList" usedependency="true" sqldependency="DotNetBurner:Story" />
       <add priority="BelowNormal" duration="15" name="VoteButton" />
       <add priority="Normal" duration="60" name="Planet" />
       <add priority="Low" duration="1440" name="StaticResource" />
   </profiles>
</datacachesettings>

Configuration Properties:

name: The profile name. On this case we used “Html”; duration: The duration, in minutes; priority: The cache priority (the same used on ASP.net output cache configuration). Lower priorities will likely be invalidated first. usedependency: If SQL dependency is enabled; sqldependency: Folows the format <dependency configuration name>:<table name> where dependency configuration name is also configured on web.config, as follows:

<sqlcachedependency enabled="true" polltime="5000">
   <databases>
       <add name="DotNetBurner" connectionstringname="DotNetBurnerConnectionString" />
   </databases>
</sqlcachedependency>

This will check the database every 5 seconds. If the table changed, the profile is invalidated. This means that every cache that depends on this profile will be invalidated.

This is used on DotNetBurner and it’s working very well so far. I used this caching technique where i could (except on user dependent data) and the site is very fast, even for Chuck Norris.