It is up to the application to do a proper federated sign-out and Sharepoint 2010 OOB is not doing this in the correct way. If you take a look at the HTTP conversation with fiddler Sharepoint will not call the wa=wsignout1.0 action on ADFS, it will simply clean the current authentication cookie.

How to configure Sharepoint to call the wa=wsignout1.0 action?

1. You need to modify the Welcome.ascx page on C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\CONTROLTEMPLATES\ on every front-end server that hosts this web application.
2. Find the section
   id="ID_Logout2"
3. Modify that section to id=”ID_Logout2” and then add the ClientOnClickNavigateUrl attribute to point to your ADFS
   
4. Now when you click the Sign-Out button on Sharepoint 2010 you will be redirected to ADFS,
   

Identified Caveats

1. Kerberos ticket will not expire: Although you logged-out from ADFS you will not be able to login as a different domain user unless you close and re-open your browser to expire the Kerberos ticket.
2. Sign-out will not redirect back to Sharepoint: According the WS-Federation protocol specification (http://msdn.microsoft.com/en-us/library/bb608217.aspx) appending the &wreply=encoded_URL to the query string will do the trick to redirect back to the current Sharepoint page. However with simple tests I was not able to do the trick and we need to go deeper on this.
3. Impact of the change: Replacing the entire Welcome.ascx file will need to be included as a post-configuration for the SharePoint deployment.

Reference : http://www.shailen.sukul.org/2010/05/adfs-2-sharepoint-2010-signout.html, here everything is figured out, however I will need to go deeper to tackle the identified caveats.