1

I have near 400 git repos on my machine.

And this is a script that I use to find their status collectively:

function Check() 
{
    gitFolder="$1"
    parent=$(dirname $gitFolder);
    Status=$(git -C $parent status)
    if [[ $Status == *Changes* ]] || [[ $Status == *Untracked* ]]; then
        Info $parent;
        git -C $parent status --porcelain
        if [ -d /Policies ]; then
            /Policies/Run.sh $parent
            /Policies/Git/Run.py $parent
        fi
        Divide
    elif [[ $Status == *ahead* ]]; then
        Warning "Push $parent";
        Divide
    elif [[ $Status == *diverged* ]]; then
        Warning "Sync $parent";
        Divide
    fi
}
export -f Check
FindGits | parallel -j0 Check

FindGits and Info and other functions are imported via . ScriptPath notation.

However, when I run it, I see this message:

parallel: Warning: Only enough file handles to run 252 jobs in parallel.
parallel: Warning: Try running 'parallel -j0 -N 252 --pipe parallel -j0'
parallel: Warning: or increasing 'ulimit -n' (try: ulimit -n ulimit -Hn)
parallel: Warning: or increasing 'nofile' in /etc/security/limits.conf
parallel: Warning: or increasing /proc/sys/fs/file-max

If parallel checks all of those 400 git repos, then I don't care about this message.

But if this means that it only checks 252 repos, and leaves the rest, then I should change the limit (though I rather don't change the limit and let parallel check repos in chunks of 252 repos).

I'm unsure about which scenario happens. Any clarification of this message's implications would be appreciated.

ctrl-alt-delor
  • 27,993
Saeed Neamati
  • 681

1 Answers1

1

It is a warning. Not an error.

It checks all 400 repos, but it will only run 252 checks in parallel at any given time.

So you can safely ignore the warning.

The reason is that the default limit of file handles is ridiculously low by modern standards (1024), and there seems to be no reason not to increase it. You can do that with:

ulimit -n `ulimit -Hn`

Please help by explaining how the message should have been phrased so that you would have understood the implications by simply reading it.

Ole Tange
  • 35,514
  • Thank you so much for your help. If the message had this line I would have understood it simply: Don't worry, we will process every item. But we do it in batches of 252 item batches. – Saeed Neamati Jan 02 '23 at 13:01