3

I would like to delete everything from a folder except csv files

I am trying with a bash script and I am getting this error:

syntax error near unexpected token `('

This is my script :

 PATH=/tmp/

 run_spark_local
 rm -v !($PATH*.csv)

 cp -r $PATH /data/logs/

I have also tried 

rm -v !("$PATH*.csv")
Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
MobZSPARK
  • 33
  • 3

3 Answers3

6

Please do not set $PATH it is a environment variable.

For BASH, so long as the extglob shell option is enabled (that is the default for a lot of linux distributions), it is just: 

rm !(*.csv)

With a folder path it will be something like...

rm yourfolder/!(*.csv)

If you think extglob is not enabled in your environment, just do this:

shopt -s extglob
Luciano Andress Martini
  • 6,628
5

You should avoid setting the PATH variable. This is used by your shell to find valid commands, setting it to /tmp/ is going to prevent the script from being able to find the rm and cp commands altogether.

You can accomplish what you want with the following find command:

find /tmp -not -name '*.csv' -not -path /tmp -exec rm -vr {} \;

Note: this will delete any subdirectories under /tmp as well. If you do not want this you must change to:

find /tmp -not -name '*.csv' -type f -exec rm -v {} \;

Another note: This will still recurse into the subdirectories and delete the files in them. If you do not want this you can use the maxdepth argument:

find /tmp -not -name '*.csv' -maxdepth 1 -type f -exec rm -v {} \;

Extra note: I would never run a find ... -exec command that you find online without verifying it will do what you need it to do first. You should run:

find /tmp -not -name '*.csv' -not -path /tmp

And verify it is finding only the files you want before adding the -exec rm -vr {} \; bit.

jesse_b
  • 37,005
  • find /tmp -not -name '*.csv' -not -path /tmp -exec rm -vr {} ;

    This Worked well for me Thank you

    What if i want to do the same thing but in the hdfs . does this work ? hdfs dfs find /tmp -not -name '*.csv' -not -path /tmp -exec rm -vr {} ;

     – MobZSPARK May 10 '19 at 15:47
4

Instead of deleting everything else in /tmp, I’d recommend only copying the files you’re interested in:

cp /tmp/*.csv /data/logs/

or even

mv /tmp/*.csv /data/logs/

if you don’t need to keep them in /tmp.

This assumes that all the files you’re interested in are directly in /tmp; your use of rm suggests that they are.

Since /tmp is a shared temporary directory, it could contain other files which other running processes expect to find there, and deleting everything in /tmp apart from the CSV files could have adverse consequences. As others have mentioned, you shouldn’t change PATH either since your shell uses that to find the commands you’re using.

Stephen Kitt
  • 434,908