Monitoring Keyboard and Mouse Activity on a touch screen Monitor.

[JohnatanBravo]

Hi guys,

My program communicates with a PLC, and one of my outputs to the PLC is the keyboard and mouse activity.

Long story short, when the PC is not being actively used for a period of time, the automation guy uses my output to automatically log out the user.

I am using the 'initialize Keyboard.vi', 'initialize Mouse.vi' and 'Acquire Input Data.vi' from Input Device Control palette.

This works very well when we are using a keyboard and mouse.

But, on the deployment machine, we have a Windows PC with a touch screen monitor, and here's where things start to get sketchy...

The 'Acquire Input Data.vi' will detect some of the longer presses, and a dragging motion, but it's not detecting tapping (going between windows/tabs and inputting data on the on-screen-keyboard)- which is how the user is going to interface with this machine.

Would any of you guys have any ideas on how to overcome this issue?

[Bob_Schor]

About a decade ago, I helped a student port a LabVIEW program to a Dell All-in-One that used a touch-screen monitor.  What a mess!  I think this was still Windows 7, but it involved a number of "deep dives" into figuring out how to get around exactly the issues you are facing now, when mouse-down left-right button presses need to get "trapped" and re-interpreted (because there are no real "left" and "right" buttons).  I don't remember (and probably can't easily locate the code) how we "solved" it, but I do remember my recommendation to get a PC + monitor + mouse ...  

Bob Schor

[tst]

A couple of thoughts:

1. I'm assuming your monitoring VI runs in a loop. Make the wait time in that loop shorter, so it can react more quickly.
2. Try calling Windows API functions directly to detect keyboard or mouse activity. As far as I remember this doesn't work differently to the mouse and keyboard VIs, but that's completely anecdotal and I may be misremembering.
3. If your code is all in LV, you can try monitoring for Key Down and Mouse Down events for the relevant VIs and use that instead. There is a VI Activation event which should give you the VI with focus, but I believe the event is private (might be available if you enable VI scripting) and I don't know if it works in an EXE (I would guess yes, but I don't remember checking). If that isn't available, you can probably just statically monitor all of the VIs you care about.

[Walker34 · Accepted solution]

---

@JohnatanBravo wrote:

Hi guys,

I am using the 'initialize Keyboard.vi', 'initialize Mouse.vi' and 'Acquire Input Data.vi' from Input Device Control palette.

This works very well when we are using a keyboard and mouse.

---

I did it using mouse pointer position, not the click. It does the job. But it was with a special touchscreen, not sure if it will do the job in your case. You should test.

[JohnatanBravo]

Thanks for your suggestions guys.

I just came back today to check for any new answers and to post the solution.

@ Bob_Schor

I laughed hard at this, thanks 🙂 Unfortunately I could not accept that as a solution 🙂

@ tst

1. My loop is at 50ms, I did try faster, but it still didn't work.

2. I was looking into that, then I came up with a solution.

3. As far as I know the Key Down and Mouse Down works only within the Labview, my Automation guy is using some 3rd party program to run his HMI.

Thanks for the suggestions!

@ Walker34

I tried using this originally, it seemed to be detecting the acceleration of the mouse, and returning to 0 when mouse was stopped again.

Only now I'm seeing that there is a 'key mode' setting which can be changed between Relative and Absolute... 🤦

I actually ended up using 'Call Library Function Node', with 'user32.dll' and the function 'GetCursorPos'.

It does exactly the same thing 'Acquire Input Data.vi' in Absolute mode...

As I'm already using 'Acquire Input Data.vi', I will revert into using it, thanks!

This is another method, using the Call Library Function Node:

So basically, the trick was to use the position of the cursor, and checking if it has changed.

'Acquire Input Data.vi' in Absolute mode can do this very well, but I was using it in relative mode...