Wednesday, April 08, 2020

Handling SendMessage from DLLs to WPF C# Application

This is my take on the code flow of handling an WPF C# app which handles messages sent to it from a DLL. I started searching on this due to an personal requirement to send messages from the DLL loaded by the WPF App.

First, let me setup the DLL. SendMessage is a Win32 API, and hence, it needs to be imported like this
[DllImport("User32.dll")]
public static extern int SendMessage(IntPtr hWnd, int wMsg, string wParam, IntPtr lParam);
While there are many ways (better ways too) to get mainWindow handles, I just passed through the first function I use to store them for later use. Here's the full DLL code
    public class mainDLL
    {
        IntPtr hwnd = IntPtr.Zero;

        [DllImport("User32.dll")]
        public static extern int SendMessage(IntPtr hWnd, int wMsg, string wParam, IntPtr lParam);

        public void InitFunction(IntPtr pval)
        {
            hwnd = pval;            
        }

        public void msgthread()
        {
            for (int i = 0; i < 5; i++)
            {
                string sz = "Counter :" + i.ToString();
                Thread.Sleep(1000);
                SendMessage(hwnd, 0x112, sz, IntPtr.Zero);
            }
        }

        public void MsgFunction()
        {
            Thread mythread = new Thread(new ThreadStart(msgthread));
            mythread.Start();
        }
    }
For the calling application, declare the relevant variables.
        // DLL relevant variables
        private Assembly DllAssembly;
        Type mydlltype;
        object InstanceType;

        MethodInfo miInitFn;
        MethodInfo miMsgFn;
The main DLL setting up
    if (File.Exists(@"sendmsgDLL.dll"))
            {
                DllAssembly = Assembly.LoadFrom(@"sendmsgDLL.dll");
                
                mydlltype = DllAssembly.GetType("sendmsgDLL.mainDLL");
                InstanceType = Activator.CreateInstance(mydlltype);
                miInitFn = mydlltype.GetMethod("InitFunction");
                miMsgFn = mydlltype.GetMethod("MsgFunction");


                var myHandle = new WindowInteropHelper(this).EnsureHandle();
                HwndSource source = HwndSource.FromHwnd(myHandle);
                source.AddHook(WndProc);
            }
And the corresponding WndProc method to capture the message sent to the application. Here, we are using the msg value of 0x112. do not values lower than 0x100, as they will be used by the system. The captured message is dumped into a RichTextBox's AppendText
    private IntPtr WndProc(IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled)
        {            
            if (msg == 0x112)
            {
                string szt = Marshal.PtrToStringAnsi(wParam);
                rtbMainDisplay.AppendText (szt + "\n");
            }

            return IntPtr.Zero;
        }
And on a button click, launch the DLL functions
   private void bnStartTests_Click(object sender, RoutedEventArgs e)
        {
            // Get the handle of this window

            var newHandle = new WindowInteropHelper(this).Handle;

            miInitFn.Invoke(InstanceType, new object[] { newHandle });
            miMsgFn.Invoke(InstanceType, null);
        }