9

I'm looking at emacs helm specifically, which has the following properties:

  • it has thousands of commits
  • it is largely maintained by one user
  • the maintainer has no other profiles (social media, etc.) I was able to find on a few searches
  • it is actively maintained (today)

Since I'm about to install arbitrary code on my computer to use in my text editor, I wanted to verify if this has undergone any review process. I'd like to say "well it's open source" but I really am far from the elisp ability to audit all the code myself. I would like to assume others in the community have reviewed it but one that's probably false, and two there are up-to-the-minute commits. Are there other strategies I am missing?

For the record the vector is simple: "open source" doesn't matter all that much if the contributor is working under a throwaway account, or if there's no review process.

djechlin
  • 923
  • 8
  • 21
  • 6
    "*Verify*" is a big word. In the regular sense of the word, I think the answer is, "You don't (you can't)." You can follow the development activity for a library, and you can check digital signatures and such, but there is nothing that is sure and certain, I believe. – Drew Apr 01 '15 at 18:29
  • 3
    Good question. FWIW, code that comes with Emacs is reviewed (via the emacs-diffs mailing-list), but if some contributor were to add a security hole, I'm sure he'd be able to do that without being noticed. Of course, I'd be surprised if there aren't already lots of large accidental security holes in Emacs anyway, so I'm not sure how much of a problem this might be. So I think in order to be able to get a more re-assuring answer, one needs to focus on specific a simple threats (e.g. it should be fairly easy to check if the code performs some kind of non-hidden surveillance). – Stefan Apr 02 '15 at 01:00
  • @Stefan, maybe Emacs needs a small tool that can grep a given package for calls to `start-process` and related functions. Outside of those, isn't Emacs just sandboxed to itself and the local filesystem? – abo-abo Apr 02 '15 at 07:10
  • 2
    The maintainer is active on emacs devel for many years, which can be verified with a search, so this is some assurance: http://lists.gnu.org/archive/cgi-bin/namazu.cgi?query=Thierry+Volpiatto&submit=Search%21&idxname=emacs-devel&max=20&result=normal&sort=score – Tom Apr 02 '15 at 08:28
  • @abo-abo: depends if you want a guarantee or just a spot check (you can call start-process without mentioning its name in the source, e.g. by calling mapatoms and selecting the start-process symbol from this by hashing the name and comparing it to the hash of "start-process"). And of course, access to the file-system can be dangerous as well. – Stefan Apr 02 '15 at 12:38
  • 1
    @Stefan, I was thinking of only a spot check. From my point of view, `mapatoms` could be put into the "dangerous" group together with `start-process`, `eval` and `funcall`. Of course there would be some false positives, but if the package doesn't use any of those functions, it can be marked as harmless with large certainty. – abo-abo Apr 02 '15 at 13:29
  • 1
    You need to add the new `make-process`, as well as `call-process`, `dbus-`, `make-network-stream`, and then `vc-do-command`, `vc-git-command`, .... And if you put `eval` and `funcall` in the "dangerous" category, then most/all packages are dangerous. – Stefan Apr 02 '15 at 14:13

1 Answers1

7

The short answer is unless you read through the code yourself you are taking a lot on trust. Having said that trusting a project sourced from an upstream SCM is a little safer than one which has been pulled directly from the Emacs Wiki for example. However fundamentally you are trusting the author of the package not to turn evil and abuse the ability to run arbitrary code under your Emacs session.

There are some things that may make you feel safer:

  • Stick to popular packages

While popular packages don't automatically get more review the fact they have more users mean the chances of malicious behaviour being detected before it affects you increases.

  • Look at the code metrics

On github you can get a fairly good view of the projects contribution history. Even if the principle author makes most of the commits having a wide range of authors shows there are other people with an active interest in the stability and effectiveness of the code.

  • Prefer GNU ELPA and MELPA Stable to repositories tracking HEADs (MELPA) or allowing un-vetted submissions (Marmalade)

While living on the bleeding edge has a certain appeal for those who want the latest shiny features it does mean you might be the first to accept the latest commit into your environment. Even if not malicious you may well get a package in a state of flux while an "official" release will of least had some sort of cursory review. For example GNU ELPA packages will only get committed by those emacs-devel users with commit rights.

Ultimately I'm sure there will at some point be a security breach via the package system even if it is only a proof of concept experiment. That will be the point you may have to rely on your regular backups.

stsquad
  • 4,626
  • 28
  • 45
  • 1
    Marmalade is a poor archive when it comes it “trust”. It allows anyone to upload any packages under any name, with zero verification or review. And it's not exactly well-maintained (just look at the number of open issues). –  May 28 '15 at 16:10
  • @lunaryorn: good point, I'll amend the answer. – stsquad May 28 '15 at 18:06
  • @lunaryorn MELPA has even more open issues than the elmarmalade repository, so I doubt this is an usable metric. – wasamasa May 28 '15 at 18:28
  • @stsquad The only thing MELPA Stable gives you is tagged releases of things that would have ended up in MELPA anyways, so I don't see how that's safer than using Marmalade. – wasamasa May 28 '15 at 18:29
  • 1
    @wasamasa Sorry, my comment was phrased poorly. The number doesn't tell much, for sure, but the open issues *are* telling: There is zero reaction on new issues, no comments, not even labels or assignments. Marmalade is effectively dead. –  May 28 '15 at 20:30
  • MELPA at least guarantees where packages me from and does some cursory code reviews. Marmalade has no reviews or verifications at all. It doesn't even check whether package owners on Marmalade are actually upstream maintainers/contributors, and it has no process for ownership disputes. –  May 28 '15 at 20:32