0

I was reading an answer regarding sending large files from one Linux system to another by quick methods, and I was trying this solution
The answer is saying that if you don't want to use the tar command, you can use the cat command instead [Like if you for example already have a tar file and you don't want to create another one, you can use the cat command instead directly]

So I have this command that I want to execute:

sam@sam:~/Documents$ cat test.mp4 | mbuffer -s 1K -m 512 | ssh 192.168.1.33 "cat > ~/Documents/here/test.mp4"

What I am trying to do using the above command is to send a large file (test.mp4) from the Linux machine (A) to the Linux machine (B). The file (test.mp4) is available on the Linux machine (A) at ~/Documents, what I want to do is to send this file from the Linux machine (A) to the Linux machine (B) on ~/Documents/here

However, running the above command does not send the file, but instead, the Linux machine (B) is creating a plain-text file called test.mp4.

So the problem is that the cat command needs a way to read the input from the previous command, Got it :D?

[Edit]: After running the command above, I always get the following warning message

mbuffer: fatal: Number of blocks must be at least 5.

Normal
  • 349
  • If your goal is to redirect standard input to a file, you could use the tee command instead of cat. But cat reads standard input by default if you have not given it anything, so your command should work – tbrugere Dec 14 '21 at 22:46
  • 1
    cat does read from standard input, and ssh does connect its stdin to that of the remote command, so yeah, the pipeline does work. What's wrong there is with the options to mbuffer, -s 1K -m 512 asks for 1 kB blocks for a total buffer of 512 bytes, which obviously doesn't work. You probably want -m 512k, or -m 512M, depending on what size buffer you'd like, though a block size of 1 kB also seems quite small. Of course you could also do without mbuffer. – ilkkachu Dec 14 '21 at 22:48
  • @Nephanth, now I tried the tee command the following way: sam@sam:~/Documents$ cat test.mp4 | mbuffer -s 1K -m 512 | ssh 192.168.1.33 "tee > ~/Documents/here/test.mp4" but it didn't make any change – Normal Dec 14 '21 at 22:48
  • cat reads stdin by default. You can give it the filename arg - to represent stdin, but it makes no difference. How big is the output test.mp4, and what does the start of the plain-text look like? I suspect mbuffer is failing (or not even installed), so ssh has nothing to pass to cat. Maybe try mbuffer on a smaller local file without using the remote system yet. – Paul_Pedant Dec 14 '21 at 22:49
  • the mp4 file is 17 GigaByte size – Normal Dec 14 '21 at 22:50
  • @Nephanth, not much use for tee there if the data is going to just one place (it's called tee since it acts like a T-shape connection). Doing something like tee filename > /dev/null would work, but the writes to stdout would be useless... – ilkkachu Dec 14 '21 at 22:50
  • I tried to omit the mbuffer part from the command and it worked, now what would be the suitable way to inject the mbuffer tool? – Normal Dec 14 '21 at 22:51
  • @Normal, are you sure you're not getting any errors from mbuffer...? – ilkkachu Dec 14 '21 at 22:52
  • @ilkkachu, sorry for that, it wasn't included in my question, yes I was getting the following error the whole time "mbuffer: fatal: Number of blocks must be at least 5. " – Normal Dec 14 '21 at 22:53
  • 2
    @Normal, well, given how a pipeline foo | bar | blah works so that the output of foo goes to the input of bar, and the output of bar goes to the input of blah, it seems rather reasonable that if bar fails due do invalid options, the whole thing breaks down. – ilkkachu Dec 14 '21 at 22:54
  • OK, thank yous, now I'm figuring out what would be the correct parameters of the buffer – Normal Dec 14 '21 at 23:01
  • What are you trying to achieve by using mbuffer? – Bravo Dec 14 '21 at 23:02
  • 1
    I don't know wnat mbuffer does, but why are you not using scp? – glenn jackman Dec 15 '21 at 00:05
  • @Bravo, I don't know, I'm trying the solution provided in the answer, new to me – Normal Dec 15 '21 at 05:31
  • @glennjackman, because it's slow, I've tried FTP, SCP, SFTP. There was also an option to do it through HTTP, but I didn't try it though, thought it would be much slower. So I thought I could manipulate something related to buffer or some nitty-gritty stuff. Because it seems like the default transfer speed on LAN is 2MBps (over WI-FI and on Linux Mint 20 --> Linux Mint 20) – Normal Dec 15 '21 at 05:36
  • 1
    provided in the answer read the comments, and you'll see that the answer you copied verbatim causes errors :p – Bravo Dec 15 '21 at 07:01

0 Answers0