Customize option dired-guess-shell-alist-user so that it runs the shell command (i.e., program) that you want (e.g. vlc) for files of the particular type (based on their file-name extensions).
M-x customize-option RET dired-guess-shell-alist-user RET
C-h v dired-guess-shell-alist-user tells you:
dired-guess-shell-alist-user is a variable defined in dired-x.el.
Its value is ()
Documentation:
User-defined alist of rules for suggested commands.
These rules take precedence over the predefined rules in the variable
dired-guess-shell-alist-default (to which they are prepended).
Each element of this list looks like
(REGEXP COMMAND...)
where each COMMAND can either be a string or a Lisp expression that evaluates
to a string. This expression can access the file name as the variable file.
If several COMMANDs are given, the first one will be the default
and the rest will be added temporarily to the history and can be retrieved
with M-x previous-history-element (M-p) .
The variable dired-guess-shell-case-fold-search controls whether
REGEXP is matched case-sensitively.
You can set this variable in your ~/.emacs. For example, to add rules for
.foo and .bar files, write
(setq dired-guess-shell-alist-user
'(("\\.foo\\'" "FOO-COMMAND")
("\\.bar\\'"
(if condition
"BAR-COMMAND-1"
"BAR-COMMAND-2"))))
You can customize this variable.
You need to load standard library dired-x.el to be able to use this option; that's where it is defined and used. Just do this in your init file:
(require 'dired-x)