In my last post on this subject I talked about how where you store your images in Kentico can have a large impact on your site performance. Today I am going to talk about a little problem I mentioned in the last post that can also have a large impact if your storing your images in the CMS tree. It is important to note that this only affects items stored in the CMS (Files, Images, and CSS) and not files that are stored in media libraries (unless you are using permenant URLs to retrieve them), or files stored on the filesystem.
The 304 Not Modified Problem
While I was working one of my recent projects I noticed that even though I had setup caching in Kentico and enabled the CMSFullClientCache key in the web.config that my images were still being requested from the server. The server would dutifuly respond with a "304 Not Modfied" response and the browser would accept this and move on to the next image and the cycle repeated itself. This was puzzling to me because my understanding was that since I had enabled full client caching the browser shouldn't be downloading images (or CSS, etc...) unless they had expired in the browser cache. Since enabling CMSFullClientCache tells Kentico to use expires headers properly in the response, the browser should have had a cached image most of the time. After some sleuthing on the matter I found the culprit; Kentico was not always setting the expires header. This was causing the browser to get into a state where it had the item in cache, but the cache had expired so it was forced to retrieve the item from the server every time. Since it recieved a 304 response it did not refresh the item from the server and thus the cache was neither purged, nor refreshed. Essentially the client wasn't REALLY using it's cache; sure the actual data came from cache but the browser still had to make the request and get the 304 before it could use the image. This was causing slow page builds, "missing" images, and other problems.
Here is the technical walkthrough of what happens in a default Kentico install that causes this problem. This walkthrough was performed with Kentico 5.5 R2 and one of images contained in that install. I used Firebug to capture the response headers for the images below. I chose a 2 minute cache time for demonstration purposes, in practice this would be much higher; example an hour long cache time would allow images that are going to be re-used on the site to be cache by the browser for the duration of most web browsing sessions, but would require that the browser re-download those images for future visits. This is a great way of both maximixing user experience and content freshness.
Notice how the "Expires" and the "Date" values are the same. This tells the browser not to cache this file, it will always go back to the server. This behaviour is expected since the cache settings are at 0 min.
Again you can see that the Expires and Date values are the same. Without CMSFullClientCache turned on the cache settings only control if Kentico will try and re-retrieve the image from the database, it has no effect on the browser cache. This is the expected and desired behaviour.
For this screen capture I changed the view in Firebug so that you can see the raw headers for this request. You can now see here that the Date and Expires headers are not the same, in fact you can see that the expires header is set to exactly 2 minutes in the future. This allows the browser to cache the image for 2 minutes before requesting it from the server again; this is a good thing when this image will be shown more than once per visit to a site.
For this request I turned on a feature in Firebug that lets you see cache responses. Here you can see that the file is greyed out, meaning that it was a cache response, and you can see that the headers match exactly was sent the last time. This is because the browser didn't actually request it at all, it saw that it had a valid item in cache and stopped. If you turn off the Firebug feature that allows you to see this response you actually see no request at all, no matter how many times you refresh the image.
About 2 minutes later I requested the image again. As you can see after the cache expired the browser asked for the file again. You will also notice that in this 304 response there is no "expires" header. Since the server is not sending an expires header with the 304 response the browser now has an expired cache item that it will try and use everytime. No matter how many times you request the image it will now always request it from the server, and will get the 304 response. This issue in this situation is that the browser cache never gets refreshed, and will not get refreshed until either Kentico thinks that the item has changed, or the browsers cache is cleared. Even though it is getting the 304 response the browser still has to make the request, wait on the server, and in many cases wait on Kentico to go check and see if the file has been modified. This can causs poor perceived performance since the users "sees" the image load everytime the browser requests it. If you are using the CMS tree to serve files that make up the structure of your site; such as navigation, avatar images, product images, images stored as attachments, or something similar then your users going to see those items load as the page builds. This can cause builds to look slow, rather than the silky smooth page near instant builds that users have come to expect when browsers are caching items properly.
Some might argue that you should could just set the cache value to be large value and that could mostly eliminate this problem. I would argue against that approach since setting a large value into the cache means that the browser will not download that file again until it has expired, if this file is updated in the mean time then you will have users who are not seeing the latest content on your site. The largest value I set for the caching time is the longest time I am willing too accept that users might have out-dated images (or CSS) on your site. For most of my projects this is about 1 hour, for some 1 day, but rarely does it go longer than that.
I do not feel that leaving a browser with an expired cache item in-definatly is the desired behavour of the "CMSFullClientCache" key so I came up with a solution that I feel more accurately reflects the intended behaviour of this feature.
DISCLAIMER: I am about to show you edits that I made to a built-in Kentico file. I do not warranty what will happen to your site(s) if you make these changes, and I accept no liability for anything that might happen from you doing so. If you are not comfortable making these changes then please do not attempt them. Making these changes requires some level of comfort with .NET code and Visual Studio. In addition since your changing a built-in Kentico file there is a possibility that your changes could be overwritten when you upgrade Kentico. I also believe there is probably a reason why Kentico coded it this way and since I don't understand that reason I could be disabling or tampering with functionality in Kentico that I am just not using. That being said please know that I have submitted this as a bug report in the Kentico forums, and explained this situaiton to Kentico when I attended the Kentico Developers Conference in Prague this year so I imagine that they will improve this in a future version of Kentico.
Since I knew the problem was in the way that Kentico was sending files back to the browser I figured that to fix this problem I needed to edit the code in the CMSPages\GetFile.aspx.cs file as shown below.
This is the file is responsible for processing and sending the vast majority of binary files from Kentico to the browser. After reading through the file and performing some debugging, I found the following lines of code in the file:
Based on my debugging, I determined that this code is responsible for sending the "304 Not Modified" response to the browser. Futher inspection also revealed the following code elsewhere in the file:
Again I used debugging to verify that this code is repsonsible for sending the 200 response with the proper expires header when the CMSFullClientCache key, and the cache settings are configured properly.
I decided that in order to achieve my goal of sending expired headers for 304 responses the easiest way was to modify the part of the file that sends the 304 response to include the expires header to include the code above. The result is below:
The lines I added are noted in the comments starting on line 412 of the file. You can see that all I really did was copy the block of code that checks the cache settings from the 200 response and paste them into the block that handles the 304 response prior to the "return;" statement. Here is the code that I added:
//Added to fix 304 cache handling
DateTime expires = DateTime.Now;
// Send last modified header to allow client caching
Response.Cache.SetLastModified(file.LastModified);
if (!file.IsSecured)
{
Response.Cache.SetCacheability(HttpCacheability.Public);
if (DocumentBase.AllowClientCache() && DocumentBase.UseFullClientCache)
{
expires = DateTime.Now.AddMinutes(CacheHelper.CacheImageMinutes(CurrentSiteName));
}
}
Response.Cache.SetExpires(expires);
//END Added to fix 304 Cache handling
After making this change the resulting response from the server now looks like this:
As you can see here the 304 response now includes an expires header which reset the expire time in the browsers cache and allow it to continue using the cached image without forcing it to be redownloaded. This had a dramatic effect in the percieved performance on the project I was working on. Since during my investigation I found the same issue was happening for CSS files, I went on to make similar changes in the "GetCSS.aspx.cs" file.
I am sure that Kentico will improve this soon, but in the mean time this fix has allowed me to increase the performance of my sites for the end user, hopefully the information here helps you do the same.
