Forefront TMG 2010, Schannel and the SSL Renegotiation DoS

by adrian 2. December 2011 21:58

As you may be aware some time ago a tool[1] to exploit a known SSL Renegotiation DoS issue[2] was released. More technical details about the DoS can be found on Vincent Bernat’s blog[3].

What has this to do with Forefront TMG 2010 ?
It has if you published a secure server with TMG using the web server publishing rules because:

  • SSL client side initiated renegotiation is enabled by default(even when you don’t needed); I’ve notified about this Microsoft Security Response Center on 25.10.2011.
  • during the DoS attack hundreds of SSL handshakes are triggered within the same TCP connection nullifying TMG’s flood mitigation features[4].
  • TMG makes use of the (limited) Schannel so you cannot disable the client side initiated renegotiation on TMG if you use client certificate authentication[5][6].
  • TMG’s NIS currently does not include a signature to mitigate against the SSL Renegotiation DoS.

Schannel behavior
By client side initiated renegotiation we understand that the client(e.g. browser) is allowed to initiate SSL renegotiation to the server(TMG). This is not needed for secure web server publishing rules even with client certificate authentication. What is needed is the server side initiated renegotiation(for client certificate authentication), meaning allow TMG to initiate SSL renegotiation to client.

From what I have seen, on TMG when you use client certificate authentication:

  1. between the client and TMG a SSL connection is established without the server asking for a client cert.
  2. the client issues its HTTP request.
  3. TMG renegotiates asking for a client cert.
  4. if the client presents the correct cert will get access; if not an error (HTTP) message will be presented by TMG.

With the DisableRenegoOnServer registry entry[7] on the TMG(server) we can control two separate functions:

  • client side initiated renegotiation –> what we want to disable as we don’t want the server to respond to renegotiation requests from the client due to the SSL Reneg DoS issue.
  • server side initiated renegotiation –> what we need enabled as we want the server(TMG) to be able to initiate renegotiation requests to clients in secure web server publishing scenarios using client certificate authentication.

The Windows Schannel currently(to my knowledge) does not provide separate registry entries for the above functions.

Bottom of the line:

  • TMG, in its default configuration, is vulnerable to the SSL DoS renegotiation attack.
  • TMG in secure web server publishing scenarios using client certificate authentication is vulnerable to the SSL DoS renegotiation attack; no current work around on TMG itself exist. If you have another device(e.g. IPS) in front of TMG you may create(if possible) a rule to mitigate against the SSL DoS renegotiation attack.
  • Setting the DisableRenegoOnServer registry entry to 1 on TMG mitigates against the SSL DoS renegotiation attack but TMG will not be able to handle the above mentioned scenario.

Note: TMG can also be configured to request a client certificate[8] but this does not seem to need SSL renegotiation. There are further implications regarding to SSL renegotiation and TMG not discussed here.
Extra note: Sites published with TMG can be potentially found using Google. Set the DisableRenegoOnServer registry entry to 1 on TMG if you do not need the SSL renegotiation feature.

References:

[1] THC-SSL-DOS tool to verify the performance of SSL
http://www.thc.org/thc-ssl-dos/

[2] IETF mailing lists [TLS] SSL Renegotiation DOS
http://www.ietf.org/mail-archive/web/tls/current/msg07553.html

[3] SSL computational DoS mitigation
http://vincent.bernat.im/en/blog/2011-ssl-dos-mitigation.html

[4] Overview of flood mitigation
http://technet.microsoft.com/en-us/library/cc995196.aspx

[5] Using client certificate authentication for publishing over HTTPS
http://technet.microsoft.com/en-us/library/cc995097.aspx

[6] About Kerberos constrained delegation
http://technet.microsoft.com/en-us/library/cc995228.aspx

[7] Microsoft Security Advisory: Vulnerability in TLS/SSL could allow spoofing
http://support.microsoft.com/kb/977377

[8] Web listener advanced authentication options
http://technet.microsoft.com/en-us/library/cc995192.aspx

Tags: ,

Forefront TMG | SSL

Comments (2) -

Jason Jones United Kingdom
12/5/2011 11:23:27 AM #

Hi Adrian,

There are now two registry key available for both client and server renegotiation control:

http://support.microsoft.com/kb/977377

I assume this is the level of control you wanted?

Cheers

JJ

adimcev
12/6/2011 11:53:27 AM #

Ho Jason,

Not really. The DisableRenegoOnClient registry entry controls client side behavior and is of not much use for TMG in web server publishing scenarios.

The DisableRenegoOnServer registry entry is useful in web server publishing scenarios; the problem is that if you use cert auth with web server publishing scenarios you can't set it to 1 otherwise authentication won't work anymore.

So:
- if you have web server publishing scenarios not using cert auth you can set DisableRenegoOnServer registry entry to 1 and prevent the SSL Reneg DoS.
- if you have web server publishing scenarios with cert auth you cannot set DisableRenegoOnServer registry entry to 1 and thus TMG will remain vulnerable the SSL Reneg DoS. You can't do nothing about this on TMG.

This was confirmed by MSRC, but it seems they do not have any plans to fix it. I assume the same thing will apply to IIS as well or any service that uses Schannel with cert auth. Probably a registry value DisableRenegoFromClient would help.

Thanks,
Adrian

Comments are closed

Home | Hire me

Support this blog

Adds

Book Shelf

 

Month List