how to speed up org-babel-tangle?

Question

I have an org file of 470 lines, 20 headings and 15 noweb references.

From it I tangle 2 files that are 250 lines and differ very little. Tangling takes over a minute and a lot of painful redisplay:

the modeline shows clauses being tangled
the buffer jumps to the different references

How can I speed this up? I tried the following but it has no effect:

;; tangle is too slow, try to speed it up
(add-hook 'org-babel-pre-tangle-hook 
          (defun my-org-babel-pre-tangle-hook () (setq inhibit-redisplay t)))
(add-hook 'org-babel-post-tangle-hook 
          (defun my-org-babel-pre-tangle-hook () (setq inhibit-redisplay nil)))

Also, I'm afraid that if it had effect, an exception (or user-quit) may leave inhibit-display on and lock me out of emacs.

Setting `inhibit-message t` has effect. But I was afraid, it leaves emacs in a state where it doesn't update the screen. Luckily the mini-buffer works so I can use `M-: (setq inhibit-display nil)` — Vladimir Alexiev, Oct 28 '19 at 15:09
Have you tried tangling using the `--batch` command line, e.g. `emacs --batch -f package-initialize -eval "(setq org-confirm-babel-evaluate nil)" your-file.org -f org-babel-tangle`? — Melioratus, Oct 28 '19 at 17:05
No but I want to tangle it many times during interactive dev... — Vladimir Alexiev, Oct 28 '19 at 17:55
Thanks! I wasn’t sure if interaction was a requirement. I’ve configured tangle into CI/CD pipelines using batch flag so others did not have to use emacs as editor but could still take advantage of literate programming workflow. — Melioratus, Oct 28 '19 at 18:05
Would you mind posting link to an example org file so I can reproduce issue(s)? It would also be helpful if you would provide your emacs+org versions & operating system for comparison with future solutions. I’m very interested tuning tangling as I work with org files several orders of magnitude larger than what you described on a regular basis. Thank you. — Melioratus, Nov 10 '19 at 19:38

score 2 · Answer 1 · edited May 20 '20 at 11:06

I have a growing org file of 5000 lines, 200 headings and 150 noweb references.

I tangle a 3000 lines file.

Using the built-in org-babel-tangle takes over 10 minutes.

I wrote ob-tangle.pl to do - quickly, under a second - the limited feature tangle I need:

#!/usr/bin/perl
# ob-tangle.pl -- simplified noweb tangle of a named org-mode SRC block
# ob-tangle.pl  file.org  main
#     Named block must start with (just) 2 lines: #+NAME: and #+BEGIN_SRC:
#         #+NAME: main
#         #+BEGIN_SRC ...
#         ...
#         #+END_SRC
#     Tangled code is sent to STDOUT.
#     All BEGIN_SRC parameters (like ":comments noweb") are ignored.

die "Usage: ob-tangle.pl orgFilename srcBlockName"
    if @ARGV != 2;
my ($orgFile, $blockName, @rest) = @ARGV;
my %name_src = readNamedSrcBlocks($orgFile);
tangle($blockName, 0);  # 0 means no added indent

sub tangle {
    my ($blockName, $indent) = @_;
    my $spaces = ' ' x $indent;
    my $src = $name_src{$blockName};
    my @src = split /^/, $src;
    while (my $srcline = shift @src) {
        chomp $srcline;  # remove trailing linefeed
        my ($spaces2, $nowebName) = $srcline =~ /^(\s*)<<(.+?)>>\s*$/;
        if ($nowebName) {
            my $indent2 = length($spaces2);
            tangle($nowebName, $indent2+$indent);
        } 
        else {
            print $spaces, $srcline, "\n";
        }
    }
}

sub readNamedSrcBlocks {
    my ($orgFile) = @_;
    open(my $F, '<', $orgFile) || die "Can't open $orgFile: $!";
    my $text = join('', <$F>);
    close $F;
    (%name_src) = $text =~ /^\s*#\+NAME:\s*(\S+)\s*?\n\s*#\+BEGIN_SRC.*?\n(.*?)\n\s*#\+END_SRC/msig;
}

Works for me -- YMMV.

Vladimir Alexiev · Answer 2 · 2019-10-28T17:26:42.740

Partial answer: this does it and reduces tangling time by about 40% (subjective). But I'm looking for more ways to speed it up. Compared to eg the C preprocessor cpp, org-tangle is very slow.

(defadvice org-babel-tangle-single-block (around inhibit-redisplay activate protect compile)
  "inhibit-redisplay and inhibit-message to avoid flicker."
  (let ((inhibit-redisplay t)
        (inhibit-message t))
    ad-do-it))

(defadvice org-babel-tangle (around time-it activate compile)
  "Display the execution time"
  (let ((tim (current-time)))
    ad-do-it
    (message "org-tangle took %f sec" (float-time (time-subtract (current-time) tim)))))

how to speed up org-babel-tangle?

2 Answers2