1

I am using zcat to output the contents of a rather large .gz compressed text file. I am piping the output to grep and searching for a specific string. Below is the command I am running.

dylan@xaelah:/media/dylan/ExtHD$ zcat wpaPasswords2.gz | grep baconisdelicious

The command runs for a while and then exits with the following error.

gzip: wpaPasswords2.gz: unexpected end of file

I have also tried piping the output to the tail command, which with default options it completes with output, but not with the last 10 lines of the compressed files, it outputs 10 lines but not the last 10 lines. If I pipe the output to 'tail -n 1' such as:

dylan@xaelah:/media/dylan/ExtHD$ zcat wpaPasswords2.gz | tail -n 1

I get the same error message of

gzip: wpaPasswords2.gz: unexpected end of file

While the command runs I watch the output of htop and the bottle neck is I/O, the computer is not running out of ram so I am unclear of how exactly to determine what the cause of this issue is. I have checked the logs and have not found anything of value.

The reason I am trying to do all of this is because I am going to combine several very large .gz files while sorting them and removing duplicates however I want to make sure my process will work before running my scripts over gigs and gigs of data. Any advice would be appreciated, thanks!

Gilles 'SO- stop being evil'
  • 829,060
Dylan
  • 1,038
  • 1
    What does gzip -t wpaPasswords2.gz say? – Stephen Kitt Mar 10 '15 at 05:33
  • same error message of unexpected EOF... Guess it is corrupted, dont know why i didnt think of that :/ is there a way to repair the file or just remove the corrupted block? i'll research as well... – Dylan Mar 10 '15 at 06:12
  • The end of the file is missing, so the corrupted blocks are at the end of the file; the easiest way to rebuild a good .gz containing what's left is to run gunzip -c wpaPasswords2.gz > wpaPasswords2 (make sure wpaPasswords2 doesn't exist first) followed by gzip wpaPasswords2 (confirming that you want to overwrite wpaPasswords2.gz). You might want to copy wpaPasswords2.gz somewhere safe before you do this... – Stephen Kitt Mar 10 '15 at 06:23
  • ok the command is running so i will let you know the results tomorrow, thanks!!! – Dylan Mar 10 '15 at 07:04
  • The command just finished running and it appears to have worked. I am now running 'gzip wpapasswords2' but don't expect any errors, I will update when this command completes. Thanks again! – Dylan Mar 10 '15 at 17:32
  • duplicate of http://unix.stackexchange.com/questions/17452/how-to-recover-a-corrupted-tar-gz-file – steve Dec 24 '16 at 17:36

1 Answers1

0

The file might be corrupt. You can check with gzip -t. You might also want to look at zgrep.

alephnull
  • 9