2

I wrote a bash script to simplify some boring task in my codebase. As most of my coworkers are using MacOS, I aimed at POSIX complience and tried to stay away from bashisms.

When it was time to distribute it to my colleges, I found out the hard way that, POSIX or not, sed and grep do not behave the same on linux and on mac. Fair enough.

Now to my question : how can I test my script with bsd tooling on linux ?

I didn't find any BSD grep package for linux (similar to the coreutils package on MacOS / BSD). Why is that ? Do they interact too much with the BSD kernel to be ported on linux ? How can I test my script to modify it ? I really don't want to install a BSD like in VirtualBox for something so simple.

RaK
  • 21
  • May this help you ? https://unix.stackexchange.com/questions/48786/how-can-i-test-for-posix-compliance-of-shell-scripts. You should especially have a look at ShellCheck, as explained in the first answer ;-). – ramius Oct 16 '23 at 06:52
  • 1
    @ramius Shellcheck doesn't give me anything on my use of grep / sed ... – RaK Oct 16 '23 at 09:26
  • 1
    The tools are easy to port. The reason why there are no such packages is probably that there are not enough people like you caring about compatibility. Fetch the tools, do a ./configure && make && make install and configure them as alternative. I did that for sed at least without finding it a big deal. – Philippos Oct 16 '23 at 09:54
  • 1
    I found something else about portability, I hope it will help https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/html_node/Limitations-of-Usual-Tools.html – ramius Oct 16 '23 at 10:12
  • @ramius by the way, many Linux distros do package bsd utilities (rarely all of them). But without knowing which Linux distro you're on, it's impossible to look up what you'd need to install, and what executable names you'd need to use. – Marcus Müller Oct 16 '23 at 10:24
  • 1
    You could also use the minimal sbase collection. Its man pages document all deviations from the POSIX standard. – Devon Oct 16 '23 at 11:29
  • This is interesting... I don't know much about busybox but it seems closer to POSIX... – ramius Oct 16 '23 at 11:39
  • "POSIX or not, sed and grep do not behave the same on linux and on mac." - Did you limit yourself to POSIX-defined functionality in sed and grep? Or did you use GNU extensions in those tools not covered by POSIX? – marcelm Oct 16 '23 at 12:10
  • @marcelm I hope I used POSIX options. I used sed's -E POSIX flag for exemple. But to figure out the source of the problem, it would be way easier to have access to the plain bsd tools on linux. I'm also aware of grep's POSIXLY_CORRECT environment variable. I'll give this a try in case I can't install those tools – RaK Oct 16 '23 at 12:36
  • @MarcusMüller I'm on archlinux, and I couldn't find anything – RaK Oct 16 '23 at 12:41
  • 1
    It's a bit of a jump, but you could run a little Docker/kvm/whatever container for this. – Panki Oct 16 '23 at 13:58
  • 2
    @Philippos as someone who's actually ported BSD tools to Linux: "easy" might be an overstatement, at least for FreeBSD's grep; aside from a few file open modes that you don't get on Linux (but which only affect performance, not functionality, as far as I can tell), you need to port libregex, and that tends to be a bit iffy in terms of libc functionality. Nothing fundamental, but also definitely not "run ./configure && make (or, because this is BSD Makefiles, bmake -m /path/to/src/freebsd-src/share/mk) – Marcus Müller Oct 16 '23 at 15:21
  • @Panki : You can not use dockerized-images of BSD OSs on linux. I could use kvm, but then again it's pretty similar to VirtualBox, and I'd prefer to have something smaller – RaK Oct 16 '23 at 15:59
  • 1
    Everyone seems to be in favor of adding BSD tools to Linux to make this script work. What about the other way around - adding GNU tools to MacOS? For example, with Homebrew's grep and gnu-sed packages. – Sotto Voce Oct 16 '23 at 22:11
  • @SottoVoce That's exactly what made me ask this question. Everybody suggests installing GNU tools on BSD, never the other way around. Why ? Anyway, I'm not requiring my coworkers to install some tools they already have. Their distaste for scripting is sufficiently high as is. – RaK Oct 17 '23 at 10:25
  • @RaK In your situation it seems like making either MacOS have GNU tools or Linux have BSD tools is not easy. So the path forward is to either: (a) support Linux only (or MacOS only) in your script, or (b) write your script to detect which sed/grep is present and use the appropriate invocations to get the results you need. I used to see this a lot in SunOS/Solaris scripts, where both SVR3/SVR4 and BSD4.x commands were available in $PATH. It's more work to write and maintain, but your question and comments point in this direction. – Sotto Voce Oct 17 '23 at 22:27

0 Answers0