4

I'd like to set-up X11 Forwarding to run remote X applications on the server and the X11 client can handle the UX interaction.

However, I would like to configure the X11 client or fake it to actual do the user interaction programmatically (from the script).

For example, I want to run an app which requires some mouse clicks or keyboard interaction (such as installers), so I could programmatically send these clicks or keyboard key press signals to the app from the script until it finishes.

Is it something possible? How this can be achieved or where to start? Or how I can re-use/hijack X11 protocol to inject my own non-user interaction?

Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
kenorb
  • 20,988

1 Answers1

3

So far I've found xdotool tool which can fake input from the mouse and keyboard quite easily.

Some examples with simple keyboard interaction:

xdotool key a
xdotool key Down
xdotool key Tab
xdotool key "Return"
xdotool key "Control_L+t"
xdotool key Alt+1
xdotool key ctrl+v
xdotool key Super
xdotool key KP_Enter
xdotool key ctrl+Page_Up
xdotool key ctrl+U005C
xdotool key ctrl+shift+e
xdotool key --delay 1000  shift+minus # for underscore
xdotool key --clearmodifiers shift+Insert
xdotool key --clearmodifiers --window 0x2600006 alt+1 0 9 8 7 6 5 4 3

Example script with mouse interaction:

WINDOWID=$(xdotool selectwindow)

xdotool set_window --overrideredirect 1 $WINDOWID windowunmap $WINDOWID windowmap $WINDOWID
xdotool windowsize $WINDOWID 10 100%


Set behaviors


xdotool behave $WINDOWID mouse-enter windowfocus windowsize --usehints 80 100% &
xdotool behave $WINDOWID mouse-leave windowsize 4 100% &

Another example: How to send F5 keystroke to the first Chrome window.

More script examples please find them at GitHub or on the official page.

You can install it either from apt repository like: sudo apt-get install xdotool or compile from the sources.

Wine

In case the X11 app is run under Wine, you can also use Winetricks. Check the source file (it's a shell script) for the guidance how to control the apps using the AutoHotkey tool.

More advance method can include using winedbg debugger and attaching to the process:

$ winedbg
Wine-dbg>info process
 00000008 3        'terminal.exe'
Wine-dbg>attach 8
0xf7709c0e __kernel_vsyscall+0xe in [vdso].so: int  $0x80

then you can interact directly by using the debugger (see: man winedbg for help).

Scout07
  • 5
kenorb
  • 20,988