Bash and shellscript page index
Bash the for loop

Simple usage:

Example:
for i in $(cmd); do cmd $i; done Example 2:
TEST="1 2 3" for i in $TEST;do echo $i;done 1 2 3 Then with double quotes:
for i in "$TEST";do echo $i;done 1 2 3

A for loop used with a docker example:

docker images REPOSITORY TAG IMAGE ID CREATED SIZE php fpm 0a757334c1f6 12 days ago 368MB mariadb latest 4828ff028cad 12 days ago 401MB phpmyadmin/phpmyadmin latest 4bdc31ab2ded 13 days ago 164MB owncloud/server 10.0 06205d21154b 2 weeks ago 782MB redis latest bfcb1f6df2db 5 weeks ago 107MB nginx latest ae513a47849c 5 weeks ago 109MB debian latest 8626492fecd3 5 weeks ago 101MB ubuntu latest 452a96d81c30 5 weeks ago 79.6MB webhippie/mariadb latest ca4fdebdc099 8 weeks ago 202MB webhippie/redis latest 906a93578062 2 months ago 55MB Now with a for loop:
for i in $(docker images | awk '{print $3}' | grep -v IMAGE); do echo $i; done Shows:
0a757334c1f6 4828ff028cad 4bdc31ab2ded 06205d21154b bfcb1f6df2db ae513a47849c 8626492fecd3 452a96d81c30 ca4fdebdc099 906a93578062 In a text file, that we'll remove:
for i in $(docker images | awk '{print $3}' | grep -v IMAGE); do echo $i > inTextFile.txt \ && cat inTextFile.txt && rm -f inTextFile.txt; done

Manage Servers with a for loop

Below we'll update remote host files with name servers mapped to LAN IP addresses and ping them:
for s in $(cat servers_list | grep servers_codefix);do ssh $s 'cat /root/servers_list >> \ /etc/hosts && export servers=$(cat /etc/hosts | grep 192 | cut -c1-15) && ping -c 2 $servers';done Below we'll manage Web services with a for loop:
services='apache2 nginx'(<= First the back end server and then the front end server) for services in $services; do /etc/init.d/$services start|stop|status|reload; done Below we'll look into a servers list and add to each of their root account a file named env_vars containing their IP addresses:
for s in $(cat servers_list); do ssh $s 'echo "export eth=$(ifconfig | \ grep -i "inet addr:" | cut -c21-31 | grep -v "127.0.0.1")" >> /root/env_vars';done

Manage data with a for loop

Display all IP addresses resulted by an ifconfig command:
for i in $(ifconfig | grep -i "inet addr:" | awk '{print $2}' | cut -d: -f2); do echo $i;done Manage specific files copy with a loop:
for f in $(ls filename[0-9]); do cp $f /dir; done (or) do cp $(f)_$DATE; other_command; another_one; etc; done

My loops

A loop to install a Docker Compose configuration file hosted in an AWS S3 bucket on every host in the cluster:

for s in $(cat servers-cluster);do ssh -i PRIVATE_KEY_FILE USER_NAME@$s \ 'sudo aws s3 cp s3://S3_BUCKET_NAME/SUB_DIR/DOCKER_FILE.conf /root/SUB_DIR/DOCKER_FILE.conf/';done

This loops lanches docker-compose on every host in the cluster:

for s in $(cat servers-cluster);do ssh -i PRIVATE_KEY_FILE USER_NAME@$s \ "sudo su -c 'cd /root/SUB_DIR && docker-compose down && docker-compose up -d'";done

The two loops below add a fail2ban rule based on the HTTP logs:

for s in $(cat /logs/main_access.log |grep -i 401 |grep -v Pingdom | cut -f 1 -d ''-''); \ do fail2ban-client set wordcodess banip $s;done for s in $(cat /logs/main_access.log |grep POST |grep -v Pingdom | grep -v ciro | cut -f 1 -d ''-''); \ do fail2ban-client set wordcodess banip $s;done

HTTP logs manipulation:

cat /logs/main_access.log |grep -i 401 | grep -v pingdom | cut -f 1 -d ''-'' cat main_access.log |grep POST |grep -v ciro | grep -v pingdom | cut -f 1 -d ''-''
Colors in Bash

The sequence below trigger color usage in Bash:
\033[XXm Where XX is the code for a chosen color. The color code 0 reset to default values.
Font code with background:
echo -e "\033[45mHello world on pink background\033[0m" Stacking color codes:
echo -e "\033[34m\033[45mBlue Hello world on pink background\033[0m" Stacking color codes does not require to use \033[ sequence for each color.
Codes can be added to the sequence using ; in a single sequence. Bold, underlined, red, blinking text can be written like this:
echo -e "\033[31m\033[1m\033[4m\033[5m\033[7mThe Text\033[0m" Or:
echo -e "\033[31;1;4;5;7mThe Text\033[0m" Bold code 1, underlined 4, blinking 5 overlined 7: echo -e "\033[34mBlue, \033[1mBold, \033[4mUnderlined, \033[5mBlinking \033[7mAnd striked through\033[0m"

Summary of formating codes: Thirties (30, 31, 32, etc) are for text color. Values in the fourties are for background color (40, 41, 42, etc).
the main availables colors are:
30, 40 : black ; 31, 41 : red ; 32, 42 : green ; 33, 43 : yellow ; 34, 44 : blue ; 35, 45 : pynk ; 36, 46 : cyan ; 37, 47 : grey. The first tenth codes are reserved for text formating:
0 : reset to default values ; 1 : bold ; 4 : underlined ; 5 : blinking ; 7 : overlined.

Shell Standard Outputs in Bash

The three standards streams

STDIN = 0 STDOUT = 1 STDERR = 2

2>&1 = redirect STDERR to STDOUT
2>/dev/null = redirect STDERR to void
1>/dev/null = redirect STDOUT to void

Any process has three standard streams: one IN (like the keyboard) and two OUT (standard OUT and error OUT being commonly the screen or a file).
These three streams are represented by these three constants: STDIN, STDOUT and STDERR of which values are respectively 0, 1 and 2.

Redirect STDOUT to /dev/null with > will only apply to STDOUT, not STDERR.
Messages sent to STDERR will show on screen, or in a log file depending on the standard output used in the script).

With 2>/dev/null we'll redirect only STDERR.
With 2>&1 we'll say "redirect STDERR where STDOUT" is redirected (the "1").
If STDOUT is previously redirected to /dev/null, then STDERR will be too.
If NOT, STDERR will be displayed on screen or written in the file which the stream is redirected to.

Translated and adapted from: lanterne-rouge.info (Fr)


Passing arguments to a shell script in Bash

debug script -x Equals -v but -x is more verbose, useful for debuging.

Shell script ARGS

$0 Name of the script itself, used for the "usage" declaration or the script help.
$(basename $0) Used to display only the name and not the full absolute path, or `basename $0`.
$1 Positionnal argument, it's the first argument given to the script after its name in the prompt.
${10} If two or more digits are necessary to declare the argument. {argument's value}.
Two digits with a declared value are expected.
$# Argument counter. Useful if we are expecting a precise number of options passed to the script in order for it to execute properly.
Argument counter is used to define this number of required mandatory options.
$* Refers to all the arguments added after the script name without a limit number.
Simple quotes protects the whole expression from being interpreted by the BASH interpreter including the variable declaration sign $ and its content.
echo '$1' = $1 whereas: echo "$1" = echo the content of $1 \ is used to escape from a protection, I.E.:
"$USER text \$4" = Content of the variable $USER text $4 exit 0 Mandatory in the end of a script, used to check on the script exit.
0 = Exit without error 1 = Exit with error


Miscellaneous shell Tips & Infos

Word Count

wc -l, --lines ==> print the newline counts wc -L, --max-line-length ==> print the maximum display width wc -w ==> prints the words count

Commands operators

&& Launch the second command if the first exit successfully || Launch the second command if the first exit on error

Date(s) in Bash

To use date in a shell script:
`date +$Y$m$d$H$M` Defining Date as a variable to use it in a shell script:
myDateVar=$(date +%Y%m%d%H%M) echo $myDateVar returns 201703251313

PS - Process Management Commands

ps -ef | grep -i "pid" | grep -i "une date type Feb10" | awk `{print $2}` | xargs (example xargs kill -9) ps -aef or -ef ps aux

-e or -A lists all active processes.
-au or -aux list all processes in the BSD format.
-ef or -eF list all processes in wide format lines.
-x list running processes.
More on processes management in command line: tecmint.com.

WSL Bash for Windows Backup

## Backup LXSS #! /bin/bash # tarlxss.sh # script to back up lxss local directories export myDate=`exec date '+%Y%m%d'` # change this line!!! Probably should be someplace in DrvFS (/mnt/...) export backdir=/mnt/appropriate/backup/path # this next line is quite long, make sure it doesn't "wrap" export TARCMD="/bin/tar --verbose --create --one-file-system --xz --exclude=./var/run --index-file=${backdir}/local-${myDate}.log --file \ ${backdir}/local-${myDate}.tar.xz " # export XZCMD="/usr/bin/xz" # /etc contains some things worth keeping, like modified config files and # sshd certificates, so it might be good to add ./etc/ssh (or just ./etc) to the backup # the first thing after $TARCMD (below) must be the list of dirs to back up # start in the root of the filesystem cd / $TARCMD ./home ./root ./usr/local tarreturn=$? echo $0: finished, returned $tarreturn >> ${backdir}/local-${myDate}.log $XZCMD ${backdir}/local-${myDate}.log xzreturn=$? echo $0: finished normally, tar with $tarreturn, xz with $xzreturn #=================================================================================== # You can run it daily (for example) from Task Scheduler. Under "Actions" put: # Action: Start a program # Program/Script: c:\windows\system32\bash.exe # Add arguments (optional): -c 'sudo /path/to/tarlxss.sh' #===================================================================================

VMware Batch

VMware Autolaunch VMs .bat Put in a BATCH file in start menu folder:
cd "\Program Files\VMware\VMware Workstation" vmware -x "C:\My virtual machine\winxppro.vmx" Or:"C:\Program Files (x86)\VMware\VMware Workstation\vmware.exe" -X "Z:\Virtual Machines\XP_Dev\winXPPro.vmx"