Triple hop issue with ASP.NET delegation Part II: Fixing our remote users.

Where did I left the previous part?... ah yes, our Terminal Server users. Well the XP users were okay and happy but the remote users were not... what could possibly be different for both?

First, the OS, our remote users connect to windows servers (Windows 2000 Standard servers ) so we checked the kerberos.dll version and nope, everything was up to date...

We also checked the kerberos tickets for the logged user and there they were...We again checked our SPN on the domain (setspn), both the SQL Server service and our web service URL were registered in AD with the proper ports.

With the help of a tech support call we landed on this KB article:
Unable to negotiate Kerberos authentication after upgrading to Internet Explorer 6

Windows Server 2000 ships with Internet Explorer 5, it turns out that when IE5 is upgraded to IE6 the advanced option Enabled Integrated Security Option (requires restart) is checked off by default. This option is normally checked on on Windows Server 2003, Windows XP, Vista and 2008.





It turns out that this setting indicates to IE to use NTML as the authentication protocol when the option is unchecked.

This option is equivalent to the following registry key:

HKEY_CURRENT_USER\Software\Microsoft\Windows
\CurrentVersion\Internet Settings

and administrators can enable Integrated Windows Authentication by setting the EnableNegotiate DWORD value to 1

The name of the option is very misleading. You can find more information on this setting here, unfortunately I didn't find much on the official Microsoft curriculum.

On our web server, we do not allow protocol transition which means the authenticated user should use Kerberos in order to enable the credentials to be delegated to a second hop:







The PROBLEM:

With the client set up to disable Kerberos authentication and with the IIS box set up to disable protocol transition, the credentials passed to the back end were not the end user credentials. Challenge protocols such as NTML do not allow delegation of credentials to a second hop.
A great article regarding the difference between Kerberos and the NTLM, WWW-Authentication such as Basic, Digest etc can be found on Larry Osterman "the Ping of Death"'s amazing blog post.


The SOLUTION:

Enabling Protocol Transition on our web box at the domain controller would have done the trick:



Or setting up a Group Policy for our end users to have the IE setting checked up. See how to create a Group Policy here.

Cheers!

Triple hop issue with ASP.NET delegation Part I: Our Windows XP Pro desktops

Last Friday we had an issue in production: we have a very simple web application with one single page on our intranet that consumes an array of web services. These web services talk to a back end SQL Server.

All in all this is a very typical scenario and like most companies with .NET technology we have web applications using ASP.Delegation in the intranet, the only particular point regarding this web page is that it is called inside an old legacy windows application (not a .NET app). For remote users, this old legacy application is used via Terminal Services.

For our remote users also, the application didn't work and our DBA was registering a bunch of anonymous requests coming from the web server box...

On the other hand we set up our web services tracing to debug and were able to see the end user credentials on each HTTP request, so the end user had managed to authenticate using Integrated Windows Security on our web box and the web service trying to open a SQL connection to the back end.


We used impersonation and Integrated Windows Authentication on our web application and web services (this is an intranet after all). ASP.NET impersonation gave us the chance to restrict the access on the back end based on AD groups and at the same time gave us the ability to audit the user's actions to a very fine grained degree (user name).

The PROBLEM with our Windows XP Pro desktop users

The application worked for our desktop users if and only if they had logged off and on their desktops in the past 48 hours. If the desktops users hadn't logged on for a while, like me, that I lock my computer instead of logging myself off, the application didn't work either and the sql box passed an anonymous login attempt back to our web tier. The web services then passed a SOAP Exception with the NT Service/Anounymous user error message to our web app...

System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> System.Data.SqlClient.SqlException: Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'.


At first we thought it was the same problem, but it turns out the TS users couldn't use the application even when they logged of and back on, not even when the TS server was restarted, hrm....

By dividing and conquering we applied the kerbtray.exe tool on our web server and one of the desktops and enabled Kerberos logging on both boxes. We noticed that when the application worked the user logged in the web server box used Kerberos, but after a few days the logging defaulted to NTML.

SOLUTION for the Windows XP Pro Desktops

It turns out this was a bug in the kerberos.dll running on Windows XP SP2, SP3 has this problem solved. More information can be found on this MSDN thread. Also the hotfix for Windows XP Professional SP2 can be found on this Microsoft Knowledge Base article. Although this article describes a different problem the hotfix provided here contains the fixed kerberos dll.


There are quite a bit of articles regarding ASP.NET delegation

And quite a few MSDN forum threads, like this one I initiated and has a heated discussion with the moderator, my fault most of it.

The best resources I have found so far, and I hope this digested summary will help you if you have the same double/triple hop issue, are:

Ken Schaefer's blog post regarding IIS and Kerberos Part 5 - Protocol Transition, Constrained Delegation, S4U2S and S4U2P.

Keith Brown's article on MSDN: Credentials and Delegation
and
nunos's Blog: Concerning the credentials double hop issue

and the best of all is a webcast by Yung Chou *all kudos to his explanation of Protocol Transition*

MSDN Webcast: Getting Delegation to Work with IIS and ASP.NET:
The Ins and Outs of Protocol Transition (Part 1 of 2) (Level 300)


This webcast specifically helped us troubleshooting and fixing the second part of our problem, our failed connection when the end users connected remotely via terminal servers.

I'll post more of the problem and the resolution on Part II...

...stay tuned.