Thursday, January 28, 2010

A deadlock occurs within a COM component that uses CComCritSecLock when an asynchronous exception occurs and the component was called from .Net

Problem: A deadlock occurs within a COM component that uses CComCritSecLock when an asynchronous exception occurs and the component was called from a .Net client (CLR 3.5 and earlier).

Solution: Fix the source of the asynchronous exception.

Workaround: Compile the COM component with the /EHa compiler switch (Configuration Properties\ C/C++ \ Code Generation \ Enable C++ Exceptions)

Cause:
The CLR (which also uses SEH) is then being “helpful” and handling the exception by passing it up to the client as a managed exception (AccessViolationException) and therefore does not allow the CRT destructor tear down to run. The correct solution is to identify and fix the source of the exception. Any CLR AccessViolationException should be immediately investigated. This same issue can just as easily leak other resources such as handles or memory which can be a beast to track down after the fact. Note that this code is throwing an exception across an interface when the /EHa switch is not set which is also a no-no. Enabling SEH and C++ exceptions and using littering catch (…) all over your code will make finding many issues very, very hard. But you can still break on the first chance exception in a debugger. To enable breaking on first chance exception in VS 2008 select Debug \ Exceptions … \ Win32 exceptions. Access violations triggers a break in windbg by default.
Fortunately .Net 4.0 may help out with this: http://msdn.microsoft.com/en-us/magazine/dd419661.aspx

Sample Code:
A COM object that uses CComCritSecLock

#pragma once
#include "resource.h" // main symbols

#include "SimpleObject_i.h"

// CSimpleton

class ATL_NO_VTABLE CSimpleton :
public CComObjectRootEx,
public CComCoClass,
public IDispatchImpl
{
public:
CSimpleton()
{
}

DECLARE_REGISTRY_RESOURCEID(IDR_SIMPLETON)

BEGIN_COM_MAP(CSimpleton)
COM_INTERFACE_ENTRY(ISimpleton)
COM_INTERFACE_ENTRY(IDispatch)
END_COM_MAP()

DECLARE_PROTECT_FINAL_CONSTRUCT()

HRESULT FinalConstruct()
{
return S_OK;
}

void FinalRelease()
{
}

public:

STDMETHODIMP DoSomething()
{

try
{
//This will catch depending upon the compiler setting for EH
// no /EH -> deadlock
// /EHsc -> deadlock
// /EHa -> no deadlock

//If we walk all the way up to the CLR then the destructor will not be called.
//Also, we're throwing across the interface boundary which is a no-no
//See http://msdn.microsoft.com/en-us/library/1deeycx5(VS.71).aspx
//and http://msdn.microsoft.com/en-us/library/d42ws1f6(VS.71).aspx
//for the details of the compiler switches

//use RAII for critical section
if (m_cs.m_sec.OwningThread != 0)
{
ATLTRACE(L"Trouble!\n");
}

ATLTRACE(L"Critical section owning thread %p count %d\n", m_cs.m_sec.OwningThread, m_cs.m_sec.LockCount);

CComCritSecLock lock(m_cs, true);

//This triggers SEH
int *p = NULL;
*p = 0;
}
catch(...)
{
ATLTRACE(L"Caught it.\n");
}

ATLTRACE(L"Exiting\n");
return S_OK;
}


private:

CComAutoCriticalSection m_cs;
};

OBJECT_ENTRY_AUTO(__uuidof(Simpleton), CSimpleton)


A C# client using an instance of the COM object:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace LockUp
{
class Program
{
[MTAThread()]
static void Main(string[] args)
{
//changed to x86 target for W7:
//http://blogs.msdn.com/karthick/archive/2006/02/28/540780.aspx
SimpleObjectLib.SimpletonClass testObj = new SimpleObjectLib.SimpletonClass();

//Call on first thread
Thread thread1 = new Thread(LockIt);
thread1.Start(testObj);
thread1.Join();

//No call from a separate thread
Thread thread2 = new Thread(LockIt);
thread2.Start(testObj);
thread2.Join();
}

//Use the object normally
static void LockIt(object state)
{
try
{

System.Diagnostics.Trace.WriteLine(string.Format("Preparing to lock with thread {0:x}", AppDomain.GetCurrentThreadId()));
SimpleObjectLib.ISimpleton testObj = (SimpleObjectLib.ISimpleton)state;
testObj.DoSomething();
}
catch (AccessViolationException)
{
//CLR is “helpful” and wraps the SHE instead of just crashing
}
System.Diagnostics.Trace.WriteLine("Lock Released?");
}
}
}


The output is:
With the /EHa switch (note the output will be the same with or without the try / catch in DoSomething)

Preparing to lock with thread 5e0
Critical section owning thread 00000000 count -1
First-chance exception at 0x5e759ade (SimpleObject.dll) in LockUp.exe: 0xC0000005: Access violation writing location 0x00000000.
Caught it.
Exiting
Lock Released?
The thread 0x5e0 has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x5e0) has exited with code 0 (0x0).
Preparing to lock with thread 15ac
Critical section owning thread 00000000 count -1
First-chance exception at 0x5e759ade (SimpleObject.dll) in LockUp.exe: 0xC0000005: Access violation writing location 0x00000000.
Caught it.
Exiting
Lock Released?

With no /EH switch or /EHsc
Preparing to lock with thread b48
Critical section owning thread 00000000 count -1
First-chance exception at 0x5e8294a2 (SimpleObject.dll) in LockUp.exe: 0xC0000005: Access violation writing location 0x00000000.
A first chance exception of type 'System.AccessViolationException' occurred in LockUp.exe
Lock Released?
The thread 0xb48 has exited with code 0 (0x0).
The thread 'Win32 Thread' (0xb48) has exited with code 0 (0x0).
Preparing to lock with thread 570
Trouble!
Critical section owning thread 00000B48 count -2