Friday, February 12, 2010

Problem: Unable to register an in-house generated PIA for a core component in Vista or Windows 7

Scenario:
One may create a PIA for a third party COM component when the vendor does not supply a PIA. However, if the component is a core Windows component such as MSScript.ocx or MSXMLx.dll the generated PIA cannot be registered.

Symptoms:
Regasm produces no error messages but the PIA is not registered for the component and the values are placed under HKLM\Software\Volatile.

And/Or

An installer runs but the PIA is not registered for the component and the values are placed under HKLM\Software\Volatile.

And/Or

You receive the following error when running tlbimp: TlbImp : error TI0000 : System.ApplicationException - Referenced type library ' does not have a primary interop assembly registered

Cause:
All registry keys for core windows components are owned by TrustedInstaller (Windows Resource Protection) and all others, including local service and administrators have Read Only access.

Resolutions:
Factor out the third party component by wrapping the exposed types or moving the dependencies into a module where a local interop assembly can be used or eliminate the use of components where PIA are not supplied.


It is possible to take ownership of the CLSID key from TrustedUser, change the permissions, register the component, and then revert the permissions and ownership. However it defeats the purpose of Windows Resource Protection and will break other components if valid PIA information is overwritten or a PIA is made available in the future.

Thursday, January 28, 2010

A deadlock occurs within a COM component that uses CComCritSecLock when an asynchronous exception occurs and the component was called from .Net

Problem: A deadlock occurs within a COM component that uses CComCritSecLock when an asynchronous exception occurs and the component was called from a .Net client (CLR 3.5 and earlier).

Solution: Fix the source of the asynchronous exception.

Workaround: Compile the COM component with the /EHa compiler switch (Configuration Properties\ C/C++ \ Code Generation \ Enable C++ Exceptions)

Cause:
The CLR (which also uses SEH) is then being “helpful” and handling the exception by passing it up to the client as a managed exception (AccessViolationException) and therefore does not allow the CRT destructor tear down to run. The correct solution is to identify and fix the source of the exception. Any CLR AccessViolationException should be immediately investigated. This same issue can just as easily leak other resources such as handles or memory which can be a beast to track down after the fact. Note that this code is throwing an exception across an interface when the /EHa switch is not set which is also a no-no. Enabling SEH and C++ exceptions and using littering catch (…) all over your code will make finding many issues very, very hard. But you can still break on the first chance exception in a debugger. To enable breaking on first chance exception in VS 2008 select Debug \ Exceptions … \ Win32 exceptions. Access violations triggers a break in windbg by default.
Fortunately .Net 4.0 may help out with this: http://msdn.microsoft.com/en-us/magazine/dd419661.aspx

Sample Code:
A COM object that uses CComCritSecLock

#pragma once
#include "resource.h" // main symbols

#include "SimpleObject_i.h"

// CSimpleton

class ATL_NO_VTABLE CSimpleton :
public CComObjectRootEx,
public CComCoClass,
public IDispatchImpl
{
public:
CSimpleton()
{
}

DECLARE_REGISTRY_RESOURCEID(IDR_SIMPLETON)

BEGIN_COM_MAP(CSimpleton)
COM_INTERFACE_ENTRY(ISimpleton)
COM_INTERFACE_ENTRY(IDispatch)
END_COM_MAP()

DECLARE_PROTECT_FINAL_CONSTRUCT()

HRESULT FinalConstruct()
{
return S_OK;
}

void FinalRelease()
{
}

public:

STDMETHODIMP DoSomething()
{

try
{
//This will catch depending upon the compiler setting for EH
// no /EH -> deadlock
// /EHsc -> deadlock
// /EHa -> no deadlock

//If we walk all the way up to the CLR then the destructor will not be called.
//Also, we're throwing across the interface boundary which is a no-no
//See http://msdn.microsoft.com/en-us/library/1deeycx5(VS.71).aspx
//and http://msdn.microsoft.com/en-us/library/d42ws1f6(VS.71).aspx
//for the details of the compiler switches

//use RAII for critical section
if (m_cs.m_sec.OwningThread != 0)
{
ATLTRACE(L"Trouble!\n");
}

ATLTRACE(L"Critical section owning thread %p count %d\n", m_cs.m_sec.OwningThread, m_cs.m_sec.LockCount);

CComCritSecLock lock(m_cs, true);

//This triggers SEH
int *p = NULL;
*p = 0;
}
catch(...)
{
ATLTRACE(L"Caught it.\n");
}

ATLTRACE(L"Exiting\n");
return S_OK;
}


private:

CComAutoCriticalSection m_cs;
};

OBJECT_ENTRY_AUTO(__uuidof(Simpleton), CSimpleton)


A C# client using an instance of the COM object:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace LockUp
{
class Program
{
[MTAThread()]
static void Main(string[] args)
{
//changed to x86 target for W7:
//http://blogs.msdn.com/karthick/archive/2006/02/28/540780.aspx
SimpleObjectLib.SimpletonClass testObj = new SimpleObjectLib.SimpletonClass();

//Call on first thread
Thread thread1 = new Thread(LockIt);
thread1.Start(testObj);
thread1.Join();

//No call from a separate thread
Thread thread2 = new Thread(LockIt);
thread2.Start(testObj);
thread2.Join();
}

//Use the object normally
static void LockIt(object state)
{
try
{

System.Diagnostics.Trace.WriteLine(string.Format("Preparing to lock with thread {0:x}", AppDomain.GetCurrentThreadId()));
SimpleObjectLib.ISimpleton testObj = (SimpleObjectLib.ISimpleton)state;
testObj.DoSomething();
}
catch (AccessViolationException)
{
//CLR is “helpful” and wraps the SHE instead of just crashing
}
System.Diagnostics.Trace.WriteLine("Lock Released?");
}
}
}


The output is:
With the /EHa switch (note the output will be the same with or without the try / catch in DoSomething)

Preparing to lock with thread 5e0
Critical section owning thread 00000000 count -1
First-chance exception at 0x5e759ade (SimpleObject.dll) in LockUp.exe: 0xC0000005: Access violation writing location 0x00000000.
Caught it.
Exiting
Lock Released?
The thread 0x5e0 has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x5e0) has exited with code 0 (0x0).
Preparing to lock with thread 15ac
Critical section owning thread 00000000 count -1
First-chance exception at 0x5e759ade (SimpleObject.dll) in LockUp.exe: 0xC0000005: Access violation writing location 0x00000000.
Caught it.
Exiting
Lock Released?

With no /EH switch or /EHsc
Preparing to lock with thread b48
Critical section owning thread 00000000 count -1
First-chance exception at 0x5e8294a2 (SimpleObject.dll) in LockUp.exe: 0xC0000005: Access violation writing location 0x00000000.
A first chance exception of type 'System.AccessViolationException' occurred in LockUp.exe
Lock Released?
The thread 0xb48 has exited with code 0 (0x0).
The thread 'Win32 Thread' (0xb48) has exited with code 0 (0x0).
Preparing to lock with thread 570
Trouble!
Critical section owning thread 00000B48 count -2