Bringing a function into a lexical environment

Question

I'm using a closure to keep track of some stuff to do with state; I want to be able to ‘include’ a function into that closure, so it can access the state. The only way this is possible is by creating that function inside the closure.

My idea is to have a function include, itself created inside that lexical environment, which takes a quoted lambda form, and then returns the actual function (which will have been created inside the closure).

Given that lambda is actually a macro which expands to another macro call

(function (lambda ...))

my best attempt has been

;;; -*- lexical-binding: t; -*-

(let ((x 0)) ;; to represent some state variable
  (defun include (quoted-lambda)
    (eval
     (macroexpand quoted-lambda)
     t))) ;; last `t' is to eval with lexical binding

and then

(defalias 'incx (include '(lambda ()
                            (incf x))))

(incx) ;; should get 1

but this just results in (void-variable x)!

How can this be fixed?

Answer 1

You're trying here to access the lexical variable x from outside its lexical scope, which obviously can't work. What you can do is pass a lexical environment to eval:

LEXICAL can also be an actual lexical environment, in the form of an alist mapping symbols to their value.

;;; -*- lexical-binding: t; -*-

(defun include (quoted-lambda)
  (eval quoted-lambda
        ;; Using a quoted constant means all expressions returned by
        ;; `include' will share the same environment.  Use `copy-tree'
        ;; to get indenpendent copies.
        '((x . 0))))

;and then

(require 'cl-lib)
(defalias 'incx (include '(lambda ()
                            (cl-incf x))))

(incx) ;; should get 1

Answer 2

What you seem to want is access to the lexical-environment of a function, which would allow something like this

(let ...
  (defun include (lambda-form)
    (eval lambda-form
          (this-lexical-environment))))

But such a function does not exist and while there is a variable internal-interpreter-environment, it's not accessible via Lisp. But let's remember how closures are constructed:

(let ((x 0))
  (lambda () (* x x)))
;; => (closure ((x . 0) t) nil (* x x))

So it seems that we can access the environment of a closure.

(butlast (cadr (let ((x 0))
                 (lambda () (* x x)))))
;; => ((x . 0))

Armed with this knowledge, we can write a function, which injects the environment of one closure into a second one.

(defun lexenv-grant-access (from to)
  "Grant access of FROM's lexenv to TO."
  (let (name env)
    (when (symbolp from)
      (setq from (symbol-function from)))
    (when (symbolp to)
      (setq name to
            to (symbol-function to)))
    (pcase from
      (`(closure ,env-form ,args . ,body)
       (setq env (butlast env-form)))
      (_ (error "FROM is not a closure")))
    (pcase to
      (`(closure ,env-form ,args . ,body)
       (eval
        (if name
            `(defalias ',name (lambda ,args ,@body))
          `(lambda ,args ,@body))
        (append (butlast env-form)
                env)))
      (_ (error "TO is not a closure")))))

(let ((x 0))
  (defun incx ()
    (cl-incf x)))

(defun decx ()
  (cl-decf x))

(decx)
;; => (wrong-type-argument number-or-marker-p nil)

(lexenv-grant-access 'incx 'decx)
(list (incx) (decx))
;; => (1 0)

So what are the problems ? First, this uses an internal implementation detail, i.e. how closure lists are constructed. But more importantly, it won't work with compiled functions. Such a function has a completely different structure and it may not even store the names of the variables, only their values.

;;Byte compiling the function's file...
(symbol-function 'incx)
;; => #[0 "\300\211\242T\240\207" [(0)] 2]
(disassemble 'incx) ;; => ...

Can you spot the value of x ? Anyway, I don't think you want to do this.

Answer 3

To "give access" to a variable, you simply have to pass it to the closure:

(let ((x 0))
  (defun include (fun)
    (funcall fun x)))

Of course, in your case, you really want to pass the state itself rather than the state's value. You can do that in the following way, using gv-ref

(let* ((x 0)
       (xref (gv-ref x)))
  (defun include (fun)
    (funcall fun xref)))

(defun incx (xref)
  (cl-incf (gv-deref xref)))

Where gv-ref is like C's & and gv-deref is like C's *.

Of course, you can also do it without gv, as in:

(let ((xref (cons 0 nil)))
  (defun include (fun)
    (funcall fun xref)))

(defun incx (xref)
  (cl-incf (car xref)))