4

I have a CSV file that contains usernames and passwords. The file looks something like this:

user1,password1
user2,password2
user3,password3

I need to loop through each line to grab the username and password, use those variables, and then grab the next set of usernames and passwords, replace the content of the variables with the new ones, etc.

I've been searching for the best way to do this, but I'm not very familiar scripting and I'm getting lost. I've used awk to grab both individually, but I'm struggling to figure out how to use awk within a while loop. And I'm reading that might not be a great approach.

user420837
  • 67
  • 2
    You need to be very explicit about your requirements. What dows "replace the content of the variables" mean? Do you need to modify the file, or is the new value only relevant for the duration of the running program? – glenn jackman Jul 02 '20 at 20:22
  • 1
    Can the passwords contain commas? If yes, how would they be encoded if at all? – Stéphane Chazelas Jul 03 '20 at 15:06
  • The right answer all depends on what you mean by use those variables - if you post some concise, testable sample input and expected output then we can help you. – Ed Morton Jul 04 '20 at 13:11

3 Answers3

8

Assuming you have some pretty good passwords:

user1,",3 ""e`$^~´"
user2,""")& Eu`id`"
user3,ThisIsAlsoAGoodPasswordBecauseItIsLong

Then you need something that can parse CSV ("" inside " is a ").

cat user+password.csv | parallel --csv do_stuff {1} {2}
Ole Tange
  • 35,514
  • One of these days, we'll find something that parallel can't do... ;P More seriously though, does parallel really need you to cat the file? That feels like a UUoC, is there no other way? – terdon Jul 02 '20 at 20:08
  • @terdon You know there are plenty of other ways :) – Ole Tange Jul 02 '20 at 20:12
  • Funnily enough, those passwords are no longer secure because they are available online. also, is the do_stuff a function? A command/alias? Or something else? – Ismael Miguel Jul 03 '20 at 09:02
  • @IsmaelMiguel True. It works directly if it is a command. If it a function or an alias you use env_parallel instead of parallel. – Ole Tange Jul 03 '20 at 14:59
  • @IsmaelMiguel, the password also ends up being passed as argument to a shell and to do_stuff, which at least in the case of the shell, means it will be exposed in the output of ps. – Stéphane Chazelas Jul 03 '20 at 15:22
  • @OleTange Do you think it is a good fit for this question to add the env_parallel information? – Ismael Miguel Jul 03 '20 at 15:26
  • @StéphaneChazelas Interesting ... But it's not as bad as using a password from a website accessible to everybody. But I didn't knew it would should in ps. Though it would just show the cat :/ – Ismael Miguel Jul 03 '20 at 15:28
7

I don't see any need for awk here:

#!/bin/bash

while IFS=, read -r user pass; do


something with "$user" "$pass"


done < path/to/file.csv
steeldriver
  • 81,074
1

If the file is a proper CSV file, you can use ksh93 instead of bash which has support for parsing CSVs:

#! /usr/bin/env ksh93
while IFS=, read -rS user password; do
  something with "$user" and "$password"
done

On an input like

user,"a""b
c,d"

That would correctly set $user to user and $password to a"b<LF>c,d.

Stéphane Chazelas
  • 544,893
  • How do you enter a password containing newline? – Ole Tange Jul 23 '20 at 06:18
  • @OleTange, that very much depends on the interface that inputs those passwords if any. For instance, if the password is retrieved as an environment variable, that can be PASSWORD=$'a"b\nc,d' some-command. With basic HTTP authentication, newline would be base64 encoded like any other character. In any case, that's hardly relevant to this Q&A. – Stéphane Chazelas Jul 23 '20 at 06:38