Assist human unobtrusively in a 3rd party Windows GUI

Marshall has asked for the wisdom of the Perl Monks concerning the following question:

I have a complex app that I run on a Windows platform. This app has multiple communication links open to both the internet and also to local (and remote) serial port devices.

There many GUI windows in this application. There are two windows where a "RESET" button can appear. When that happens, I have to move my mouse over there, click on it and then have to click on the window where I was typing in order to bring the typing and F key focus back.

This sounds crazy because it is! The program authors want me to explicitly click on this "RESET" button. I asked, quite sensibly, "If there is no decision for me to make and I always click on this RESET button, and you guys know that a reset is required, why can't you guys just do the reset automatically for me?". Their answer is that they want to force me to be aware that a reset was needed because of possible side-effect X. Geez! The probability of side-effect X happening is directly related to how fast I can click on the RESET button! The only possible corrective action I have is to reset the comm link ASAP.

The RESET button means that a serial port connection has been lost. Side-effect X can happen because current information is not available from the connection that failed. This involves 2 hardware and at least 2 software vendors. Finger pointing is rampant! This error can happen every couple minutes and is a real PITA.

What I want is an application that can as unobtrusively as possible from a user point of view, click on this RESET button. That would mean that I'm entering keyboard text into some window, and "poof" the reset button gets clicked so fast that I am not even aware of it!

I have asked a question similar to this before, but I've been unable to find that thread. The application was different and it was a long time ago. This is not a "test the GUI" application. I want this thing to run in the background and in parallel with normal human GUI interactions.

Update:
I really appreciate the comments and ideas for solutions to my problem!

I had heard about Win32::GuiTest before. I didn't know about the associated program spy.pl.Thanks Grandfather and Corion!

roboticus suggested "AutoHotkey" and that suggestion looks like a very promising avenue to explore. In particular, the "WinWait" function sounds like what I need if I am able to implement that. harangzsolt33 also suggested "Autoit".

When I am using this application, I typically have 15+ distinct windows open on my dual monitor desktop. I actually have to do quite a bit of moving and adjusting of windows in order to see all the stuff that I need to see within the monitor space that I have. The application involves multiple hardware and software vendors. I am "ticked off" that the vendor that presents the "reset button" can't just do the reset themselves! Unfortunately I am in the less than 1% of users who have a setup where this is a problem. So they are not inclined to help me. Sometimes software support is like that.

I hope that with some hacking, I can get this thing to work "like it should work".

Comment on Assist human unobtrusively in a 3rd party Windows GUI Download Code

Replies are listed 'Best First'.
Re: Assist human unobtrusively in a 3rd party Windows GUI by GrandFather (Saint) on Dec 13, 2020 at 00:14 UTC
While it's not a "test the GUI" application, Win32::GuiTest probably holds the answer. Using Win32::GuiTest you can dig through windows and dialogues, see controls (including buttons) and poke stuff. Seems like exactly what you want. Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond	[reply]
Re^2: Assist human unobtrusively in a 3rd party Windows GUI by Corion (Patriarch) on Dec 13, 2020 at 07:04 UTC
To expand a bit, Win32::GuiTest includes the `spy.pl` and `spy--.pl` scripts, which dump the structure of available windows. If you can find the "RESET" button through this, you can automatically click it and then restore the focus to the original window. My advice is to not send mouse clicks but to use keyboard navigation/hotkeys or to send window messages with the ID of the target control to activate it. Be sure to log all the actions of your assisting program to a file so you can find what situation caused the program to act unexpectedly, later. If the application does paint its own controls, such as to have "fancy" buttons (for no good reason), you will have to take screenshots and compare the screenshots, which also can work but is fragile if the font or screen scale changes. Another low-effort thing to try is AutoHotkey, which has its own quirky programming language. I wouldn't want to write any form of logic in it, but if AutoHotkey can do it then you can later replicate the mechanism AutoHotkey uses with Perl.	[reply] [d/l] [select]
Re^3: Assist human unobtrusively in a 3rd party Windows GUI by Marshall (Canon) on Dec 15, 2020 at 22:46 UTC
My advice is to not send mouse clicks but to use keyboard navigation/hotkeys or to send window messages with the ID of the target control to activate it. I am not sure what you mean by this? We (my fellow users who are affected by this) have not been able to find any kind of keyboard sequence that would have the effect of clicking upon this button. If we had found such a thing, then I could push a keyboard button to send a keyboard macro. Pushing a single keyboard button would be less terrible than it is now (move hand to mouse, move cursor to "reset", click it, move cursor back to original window, click, put hand back on keyboard). I suspect that you are telling me something about the GUI_Test UI that I don't understand. send window messages with the ID of the target control to activate it sounds like what I need to do.	[reply]
Re^4: Assist human unobtrusively in a 3rd party Windows GUI by Corion (Patriarch) on Dec 16, 2020 at 07:09 UTC
Re^4: Assist human unobtrusively in a 3rd party Windows GUI by Anonymous Monk on Dec 16, 2020 at 13:23 UTC
Re: Assist human unobtrusively in a 3rd party Windows GUI by roboticus (Chancellor) on Dec 13, 2020 at 18:03 UTC
Marshall: A windows utility 'AutoHotkey' might have the capability to do what you want, I've used it in the past and it was relatively easy and nice to use. Looking at the docs, there's a "WinWait" function that waits for a window to exist, so you might be able to use it and have it automatically press the RESET button for you when that window pops up. (Not affiliated, just a satisfied user of the program a while ago.) ...roboticus When your only tool is a hammer, all problems look like your thumb.	[reply]
Re: Assist human unobtrusively in a 3rd party Windows GUI by jmlynesjr (Deacon) on Dec 13, 2020 at 00:22 UTC
It's a shame that your vendors didn't implement a serial protocol that would automatically restart and resync. A serial implementation of IBM's Bisync protocol works nicely. James There's never enough time to do it right, but always enough time to do it over...	[reply]
Re^2: Assist human unobtrusively in a 3rd party Windows GUI by Marshall (Canon) on Dec 13, 2020 at 00:51 UTC
The protocols run at 38400 baud which is as fast as RS-232 serial can go. There is no idea of a "connection" or a "packet of information". A multi-byte command is sent and then a multi-byte response is expected. There are many ways for this to go wrong! When the response is "nonsense" (including some time out values), a RESET is required. The typical serial device does not implement any kind of idea a "connection".	[reply]
Re^3: Assist human unobtrusively in a 3rd party Windows GUI by GrandFather (Saint) on Dec 13, 2020 at 02:28 UTC
Actually the RS-232 standard doesn't specify bit rates at all, although it does suggest bit rates above 20,000 aren't supported. I routinely use a baud rate just below 1e6 Baud for a debug port on embedded systems I'm working on. That connects using a FTDI serial to USB cable. It would commonly be called RS-232, even though no part of the system bears any relationship to the RS-232 standard which is all about plugs and signalling levels. To my way of thinking a serial cable between two devices is essentially the definition of a "connection", although that has become a much abused word in this context, rather like RS-232 actually. Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond	[reply]
Re^3: Assist human unobtrusively in a 3rd party Windows GUI by jmlynesjr (Deacon) on Dec 14, 2020 at 00:29 UTC
Are you receiving a checksum or CRC-16? If not, how do you know that any message is received correctly? Are you connecting via modems? At higher bit rates cable lengths become a problem. It's very difficult to make these open-ended systems operate reliably. Do you have any capability to run any loop-back tests on your circuits? James There's never enough time to do it right, but always enough time to do it over...	[reply]
Re: Assist human unobtrusively in a 3rd party Windows GUI by harangzsolt33 (Chaplain) on Dec 15, 2020 at 00:13 UTC
This is a task that AutoIt might handle pretty well. I'd look into that. AutoIt is a scripting language for Windows designed specifically to solve problems like this!	[reply]


more useful options
	PerlMonks