4

I need to run a local script on remote servers. Running script to run in parallel is important.

[ec2-user@ip-172-31-43-140 ~]$ cat hosts.txt
             ec2-user@18.218.191.143
             ec2-user@18.220.183.27
             ec2-user@18.222.199.72
             ec2-user@13.58.207.76
             ec2-user@18.191.231.120
[ec2-user@ip-172-31-43-140 ~]$ cat hosts.txt  | xargs -I {} ssh {} -T 'bash -s' < ./file.sh
ssh: Could not resolve hostname #!/bin/sh: Name or service not known
xargs: ssh: exited with status 255; aborting
[ec2-user@ip-172-31-43-140 ~]$
Rui F Ribeiro
  • 56,709
  • 26
  • 150
  • 232
vishnu
  • 41

3 Answers3

5

I like this form when doing this:

$ cat hosts.txt | xargs -n1 -P8 sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh

Or with {} if you need them:

$ cat hosts.txt | xargs -n1 -P8 -I{} sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh {}

Example

hello.bash

$ cat hello.bash
#!/bin/bash

echo "hi from server: $(hostname)"

ssh xargs

$ cat hosts.txt | xargs -n1 -P8 sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh
hi from server: mulder.mydom.com
hi from server: skinner.mydom.com
hi from server: manny.mydom.com

Details:

  • -n1 -P8 - tells xargs to take 1 argument as input and run 8 instances of ssh
  • sh -c 'ssh -T "$1"
    • this invokes a shell and then runs the command after -c "..."
    • notice we're passing $1 in here, this is to avoid injection attacks.
    • -T disables pseudo terminal. $1 is the contents of the file being cat'ed in.
  • bash -s < ./hello.bash' - the commands that will get passed to ssh
  • sh - the trailing sh is what gets passed to xargs as the shell to invoke, argument #0 ($0).

References

slm
  • 369,824
  • 1
    Yeah, I think it's safer to not invoke word splitting and filename generation. If not for any other reason than that error messages may be confusing in some situations when filename generation picks up random files. – Kusalananda Jul 23 '18 at 14:38
1

hello.bash:

doit() {
  echo Define
  echo what you want done in a
  echo function
}
export -f doit

Then do:

. hello.bash
parallel --env doit --slf hosts.txt --tag --nonall doit
Ole Tange
  • 35,514
1

Yes, you can do this with AWS Systems manager. AWS Systems Manager Run Command allows you to remotely and securely run set of commands on EC2 as well on-premise server. Below are high-level steps to achieve this.

Attach Instance IAM role: The ec2 instance must have IAM role with policy AmazonSSMFullAccess. This role enables the instance to communicate with the Systems Manager API.

Install SSM Agent: The EC2 instance must have SSM agent installed on it. The SSM Agent process the run command requests & configure the instance as per command.

Execute command : Example usage via AWS CLI: In --instance-ids give the list of ec2 instance ids. Execute the following command to retrieve the services running on the instance. Replace Instance-ID with ec2 instance id.

aws ssm send-command --document-name "AWS-RunShellScript" --comment "listing services" --instance-ids "Instance-ID" --parameters commands="service --status-all" --region us-west-2 --output text

More information here

ExploringApple
  • 111