cat
is the command to concatenate files, however it seems it doesn't work with UTF-8 files with BOM (byte order mark).
Those files have some bytes at the beginning which define the byte order (big / little endian). They are invisible in text editors like vi
.
If I concatenate such files, I don't expect to have the BOM multiple times in the concatenation result. Unfortunately cat
doesn't care about that. It took me hours to find the problem, because I didn't knew my files have BOMs and Notepad++ doesn't even show them if they are in the middle of the file.
So my question is: How can I concatenate multiple UTF-8 files with BOM? At the moment I'm trying to remove the BOM with sed
and then add it again to the resulting file, but it still doesn't work and I wondered if there is a more simple solution or if I didn't use cat
correctly.
cat
to remove a BOM from the second and subsequent files that you are concatenating together. – fpmurphy Oct 07 '20 at 09:38