1

I am trying to replace these numbers "A00002 X53307 BB145968 CAA42669 V00181 AH002406 HQ844023" in the following for loop with a new list of numbers. But my new list is a .CSV file and there are hundreds of numbers in it. My question is, can I read the .CSV file directly and make it work as the list in the for loop? 

for ACC in A00002 X53307 BB145968 CAA42669 V00181  AH002406  HQ844023
do
   echo -n -e "$ACC\t"
   curl -s "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&id=${ACC}&rettype=fasta&retmode=xml" |\
   grep TSeq_taxid |\
   cut -d '>' -f 2 |\
   cut -d '<' -f 1 |\
   tr -d "\n"
   echo
 done

The .csv file looks like this:

WP_004064712.1
WP_023555236.1
WP_051593235.1
KAJ52037.1
WP_012103448.1
WP_049740904.1
WP_003346264.1
WP_026134014.1
WP_051870539.1
AKF93952.1
XP_008397367.1
XP_014896959.1
XP_007567109.1
XP_014847432.1
EHG27035.1
EGX75147.1
WP_033630878.1
Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
Anna
  • 35

4 Answers4

2

If as @Mark asks the CSV file contains one value per line, you can do this trivially by replacing your initial list with a command substitution:

for ACC in `cat csvfile`
do 
  ...
done
John
  • 17,011
  • I just tried and it works! My another question is, I want to save the result in a output file, instead of printing them to the monitor, is there a command that can do that? – Anna Mar 29 '16 at 19:07
1

If you know to what values you'll be replacing "A00002 X53307 BB145968 CAA42669 V00181 AH002406 HQ844023"for you can do this:

CSV=`cat csvfile`
for LINE in $CSV
do 
  sed -i "s/A00002/NewValue/g" $CSV
  sed -i "s/X53307/NewValue/g" $CSV
  ...
done

Sed command explanation:

sed -i "s/X53307/NewValue/g" $CSV

What this command is doing is: Replace X53307 with NewValue directly in the $CSV file.

Rhyuk
  • 1,115
1

You're forgetting 2 things here:

  1. String expansion in your Curl statement does produce output.
  2. You can use the CSV file for the input control, as suggested by @John.

As such, you don't need to replace the string values, you need to just overwrite them.

Old:

<?xml version="1.0"?>
 <!DOCTYPE TSeqSet PUBLIC "-//NCBI//NCBI TSeq/EN" "http://www.ncbi.nlm.nih.gov/dtd/NCBI_TSeq.dtd">
 <TSeqSet>
<TSeq>
  <TSeq_seqtype value="nucleotide"/>
  <TSeq_gi>39899</TSeq_gi>
  <TSeq_accver>X53307.1</TSeq_accver>
  <TSeq_taxid>1423</TSeq_taxid>
  <TSeq_orgname>Bacillus subtilis</TSeq_orgname>
  <TSeq_defline>Bacillus subtilis epr gene for a novel serine protease</TSeq_defline>
  <TSeq_length>2521</TSeq_length>
  <TSeq_sequence>GTTAACAGGATATCCGAGCTTATCGGCCCACTCGTTCCCAAACACACTCGCCATGAAATCAGCATACCCCGGAATCGGCAAGCTCGTTAAAATCAAGAAGACAGACCCGATAATAATCAGCGGCATGGACTGGATAATTCCGTCACGCAAAGCGCTGAGATGCCGCTGCCCGGCAATTTTCCCGGCGACAGGCATTATTTTTTCCTCCATCACCCGAGTGAATGTGCTCATCTTAAAAACCCCCTTTTCTCATTGCTTTGTGAACAACAACCTCCGCAATGTTTTCTTTATCTTATTTTGAAAACGCTTAGAAATTCATTTGGAAAATTTCCTCTTCATGCGGAAAAAATCTGCATTTTGCTAAACAACCCTGCCCATGAAAATTTTTTCCTTCTTACTATTAATCTCTCTTTTTTTCTCCGATATATATATCAAACATCATAGAAAAAGGAGATGAATCATGAAAAACATGTCTTGCAAACTTGTTGTATCAGTCACTCTGTTTTTCAGTTTTCTCACCATAGGCCCTCTCGCTCATGCGCAAAACAGCAGCGAGAAAGAGGTTATTGTGGTTTATAAAAACAAGGCCGGAAAGGAAACCATCCTGGACAGTGATGCTGATGTTGAACAGCAGTATAAGCATCTTCCCGCGGTAGCGGTCACAGCAGACCAGGAGACAGTAAAAGAATTAAAGCAGGATCCTGATATTTTGTATGTAGAAAACAACGTATCATTTACCGCAGCAGACAGCACGGATTTCAAAGTGCTGTCAGACGGCACTGACACCTCTGACAACTTTGAGCAATGGAACCTTGAGCCCATTCAGGTGAAACAGGCTTGGAAGGCAGGACTGACAGGAAAAAATATCAAAATTGCCGTCATTGACAGCGGGATCTCCCCCCACGATGACCTGTCGATTGCCGGCGGGTATTCAGCTGTCAGTTATACCTCTTCTTACAAAGATGATAACGGCCACGGAACACATGTCGCAGGGATTATCGGAGCCA
AGCATAACGGCTACGGAATTGACGGCATCGCACCGGAAGCACAAATATACGCGGTTAAAGCGCTTGATCAGAACGGCTCGGGGGATCTTCAAAGTCTTCTCCAAGGAATTGACTGGTCGATCGCAAACAGGATGGACATCGTCAATATGAGCCTTGGCACGACGTCAGACAGCAAAATCCTTCATGACGCCGTGAACAAAGCATATGAACAAGGTGTTCTGCTTGTTGCCGCAAGCGGTAACGACGGAAACGGCAAGCCAGTGAATTATCCGGCGGCATACAGCAGTGTCGTTGCGGTTTCAGCAACAAACGAAAAGAATCAGCTTGCCTCCTTTTCAACAACTGGAGATGAAGTTGAATTTTCAGCACCGGGGACAAACATCACAAGCACTTACTTAAACCAGTATTATGCAACGGGAAGCGGAACATCCCAAGCGACACCGCACGCCGCTGCCATGTTTGCCTTGTTAAAACAGCGTGATCCTGCCGAGACAAACGTCCAGCTTCGCGAGGAAATGCGGAAAAACATCGTTGATCTTGGTACCGCAGGCCGCGATCAGCAATTTGGCTACGGCTTAATCCAGTATAAAGCACAGGCAACAGATTCAGCGTACGCGGCAGCAGAGCAAGCGGTGAAAAAAGCGGAACAAACAAAAGCACAAATCGATATCAACAAAGCGCGAGAACTCATCAGCCAGCTGCCGAACTCCGACGCCAAAACTGCCCTGCACAAAAGACTGGATAAAGTACAGTCATACAGAAATGTAAAAGATGCGAAAGACAAAGTCGCAAAGGCAGAAAAATATAAAACACAGCAAACCGTTGACACAGCACAAACTGCCATCAACAAGCTGCCAAACGGAACAGACAAAAAGAACCTTCAAAAACGCTTAGACCAAGTAAAACGATACATCGCGTCAAAGCAAGCGAAAGACAAAGTTGCGAAAGCGGAAAAAAGCAAAAAGAAAACAGATGTGGACAGCGCACAATCAGCAATTGGCAAGCTGCCTGCAAGTTCAGAAAA
AACGTCCCTGCAGAAACGCCTTAACAAAGTGAAGAGCACCAATTTGAAGACGGCACAGCAATCCGTATCTGCGGCTGAAAAGAAATCAACTGATGCAAATGCGGCAAAAGCACAATCAGCCGTCAATCAGCTTCAAGCAGGCAAGGACAAAACGGCATTGCAAAAACGGTTAGACAAAGTGAAGAAAAAGGTGGCGGCGGCTGAAGCAAAAAAAGTGGAAACTGCAAAGGCAAAAGTGAAGAAAGCGGAAAAAGACAAAACAAAGAAATCAAAGACATCCGCTCAGTCTGCAGTGAATCAATTAAAAGCATCCAATGAAAAAACAAAGCTGCAAAAACGGCTGAACGCCGTCAAACCGAAAAAGTAACCAAAAACCTTTAAGATTTGCATTCCAAGTCTTAAAGGTTTTTTTCATTCTAAGAACACCACACACAACCTTTTTCCCATCCATTGTACAGGCTTTTCATACTATTGCTATACAGCCATGAAC</TSeq_sequence>
</TSeq>
</TSeqSet>

New:

<?xml version="1.0"?>
 <!DOCTYPE TSeqSet PUBLIC "-//NCBI//NCBI TSeq/EN" "http://www.ncbi.nlm.nih.gov/dtd/NCBI_TSeq.dtd">
 <TSeqSet>
<TSeq>
  <TSeq_seqtype value="protein"/>
  <TSeq_gi>490166065</TSeq_gi>
  <TSeq_accver>WP_004064712.1</TSeq_accver>
  <TSeq_taxid>97253</TSeq_taxid>
  <TSeq_orgname>Eubacterium plexicaudatum</TSeq_orgname>
  <TSeq_defline>hypothetical protein [Eubacterium plexicaudatum]</TSeq_defline>
  <TSeq_length>1508</TSeq_length>
  <TSeq_sequence>MKKSFMTRVLAVSLSAAMAFSMSSASNLVTASAASTVNLKTTFKTLKVGQTYKLTLKKNTLNWKITKVQTTNKKICTVYGKTASSVMLKGKGVGRAKISVKVKTTKRKYPKNIKIMKCTANVKAADGSGTTDEFKVTSATASSNTEVRVMFSKAIDAAEMTNFTVSDSVTVSKAELSEDKKSVLLTIAGAEYGKNYELTVNGIKVAGKEQAAQKVTFTTPSASEKYPTTLEAKDPVLASDGHSQTLVTFTIKDANGNPITDKGVEVAFATSLGKFAEQRVSIQNGVATVMYTSEALMETQTSAITATVVESTDNQELMGLSATSSITLTPNPDEFNIVPIITSITAPTADRVIAYFNEKVSASDFKTASGKLDHSKFTANVAWGFDNGFDELGNRLVGRSNVVGILDVPGSDNALQLLVDRPMTDNTNISVTFENKTKASSLVSASNTVYTKLTDAHQPSVLTAKGDGLRTVVVNFSEAVLPTAYCDNVETDKKNANQTLFAADNIENYLIDGKPLSYWGVTEVKTPDSETPDDTSSNLKKESSKNDATKTGSEKPGEIQVGSYKDGEDNRHVVTIKLSRERFLEPGTHSMTISNVGDWAAKTDRERNIVNTQTFDFVVENNDVIPTFEVEEQSPEQWLLKFNSDIEPVSETLTTPNSQYSDQASILKLQELVGSTWVDISDSDAAGKNPIRVSQVDDTRNYVVEVRKDWTEVYNTSSTKQNYFNKQLRLHIDAGKIVNIANNKQNGTIDIPLDGTIMRTPDVVSPEIGEVTPAEDTSGNVLDSYNVKLSEPVKLSDGTGGAGGANGEGLTPSQIQSANGSNSNNQGVPMPSAQFIRVDNGQTVEGIITSNVFVDAYDTTINIAPESALSAGKWRLVISSISDDYGNTASTVAHEIDVTQESVTTDFKIVWAAVSDQQTYAEDHIGVERGRYIFVKFSKPVTMTGNSVNAGVTGNYTVNGATLPTGTQIRANIVGYDDHDAVTDSVTIMLPTGNVNAGWGATGDYTV
SGKNAMLNVSRAITATTGENLSNGGLIRIPFQYGSATEDTGYNDYNDSLTALTDAVWGNYRSETRAGYDNLRDYYKALKSALENDKYRRVVLTAPLDLSNPDDNPNEDQKDAVAVFGRSHTLTIKRAVDFDLNGNNITGNVVISTTDAVNRIKLHSSKERAHIYGYANNKDNVATLTVNAGSAKEFLLDNVEVHETDKGNALNINDTWKASFVNNGVIDGKIRITDTNGCGFKNENTTDGFTNRTRFIIDSTGDVNLKGDLSALRNLTDEFGITVNQAAKLSFGVDSKDETTPCDISGVKIVVRGPGARVIFTPVATTTADTALTAEADNVRVQLSQANSGSGKIQFFTDRGGKIVAVDKDNKEVTSDSKDAVKISSDDIKVTGIQKALENLDVQTGVITDGKVDSTVTISCGAISGGSYNIEELAKNIKKAEFEYKGKPDTTGIVANYSLLSTNLLKKDSTHIWPKDNWTDQKDDVSDTIRVTLAYDGYTMVKYIKVTRV</TSeq_sequence>
</TSeq>
</TSeqSet>
eyoung100
  • 6,252
  • 23
  • 53
  • 1
    @anna Apart from accepting the recognised answer, now that you have 15 points you can, if you so wish, upvote an answer that you consider useful (even if you settle to accept another one). You can also accept and upvote the answer most to your liking. This gives the answerer reputation points and although they are not the main objective of most people here, it is nice way of recognising/appreciating effort, and with reputation comes additional features/responsibilities on the site. – Anthon Mar 29 '16 at 19:41
  • @Anna Shmeless Plug: I think what he's trying to tell you is to upvote my answer. The reason is: Since the Q&A is separate each time, sometimes part of one answer could belong in another, i.e. my 2 points could fit nicely in John's answer. – eyoung100 Mar 29 '16 at 19:53
  • Yeah you are right @eyoung100 I am a newbie and thanks for letting me know how to appreciate on this site! – Anna Mar 29 '16 at 20:17
0

Here's a refactoring which avoids reading the entire CSV file into memory, and simplifies the postprocessing slightly.

# Use lower case for private variables
# and https://mywiki.wooledge.org/DontReadLinesWithFor
while read -r acc; do 
   curl -s "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&id=${acc}&rettype=fasta&retmode=xml" |
   # Run a single awk script for extraction and formatting
   awk -v acc="$acc" '/TSeq_taxid/ {
      sub(/>.*/, ""); sub(/.*</, ""); print acc "\t" $0 }'
done <csvfile
tripleee
  • 7,699