I've found the solution, (intern (ivy-state-current ivy-last)) will return the symbol of the current ivy result, which I can then pass to describe-function or describe-variable.
(defun counsel-describe-function-or-variable ()
"Display help about the currently selected ivy result.
Assumes the symbol is a function and tries with a variable describe-function fails."
(interactive)
(let ((inhibit-message t)
(current-symbol (intern (ivy-state-current ivy-last))))
(condition-case nil
(describe-function current-symbol)
('error
(describe-variable current-symbol)))))
(define-key counsel-describe-map (kbd "TAB") 'counsel-describe-function-or-variable)
I didn't find a way to properly distinguish between counsel-describe-variable and the rest of the describe commands, so I catch the error and try describe-variable when describe-function fails.