Archive | GNU/Linux İpuçları

Process Types

Before we start talking about Linux process management, we should review process types. There are four common types of processes:

• Parent process
• Child process
• Orphan Process
• Daemon Process
• Zombie Process

Parent process is a process which runs the fork() system call. All processes except process 0 have one parent process.

Child process is created by a parent process.

Orphan Process it continues running while its parent process has terminated or finished.

Daemon Process is always created from a child process and then exit.

Zombie Process exists in the process table although it is terminated.

The orphan process is a process that still executing and its parent process has died while orphan processes do not become zombie processes.

Memory Management

In server administration, memory management is one of your responsibility that you should care about as a system administrator.

One of most used commands in Linux process management is the free command:

$ free –m

The -m option to show values in megabytes.

Our main concern in buff/cache.

The output of free command here means 536 megabytes is used while 1221 megabytes is available.

The second line is the swap. Swapping occurs when memory becomes to be crowded.

The first value is the total swap size which is 3070 megabytes.

The second value is the used swap which is 0.

The third value is the available swap for usage which is 3070.

From the above results, you can say that memory status is good since no swap is used, so while we are talking about the swap, let’s discover what proc directory provides us about the swap.

$ cat /proc/swaps

This command shows the swap size and how much is used:

$ cat /proc/sys/vm/swappiness

This command shows a value from 0 to 100, this value means the system will use the swap if the memory becomes 70% used.

Notice: the default value for most distros for this value is between 30 and 60, you can modify it like this:

$ echo 50 > /proc/sys/vm/swappiness

Or using sysctl command like this:

$ sudo sysctl -w vm.swappiness=50

Changing the swappiness value using the above commands is not permanent, you have to write it on /etc/sysctl.conf file like this:

$ nano /etc/sysctl.conf

vm.swappiness=50

Cool!!

The swap level measures the chance to transfer a process from the memory to the swap.

Choosing the accurate swappiness value for your system requires some experimentation to choose the best value for your server.

Managing virtual memory with vmstat

Another important command used in Linux process management which is vmstat. vmstat command gives a summary reporting about memory, processes, and paging.

$ vmstat -a

-a option is used to get all active and inactive processes.

And this is the important column outputs from this command:

si:                           How much swapped in from disk.

so:                          How much swapped out to disk.

bi:                           How much sent to block devices.

bo:                         How much obtained from block devices.

us:                          The user time.

sy:                          The system time.

id:                           The idle time.

Our main concern is the (si) and (so) columns, where (si) column shows page-ins while (so) column provides page-outs.

A better way to look at these values is by viewing the output with a delay option like this:

$ vmstat 2 5

Where 2 is the delay in seconds and 5 is the number of times vmstat is called. It shows five updates of the command and all data is presented in kilobytes.

Page-in (si) happens when you start an application and the information is paged-in. Page out (so) happens when the kernel is freeing up memory.

System Load & top Command

In Linux process management, the top command gives you a list of the running processes and how they are using CPU and memory ; the output is a real-time data.

If you have a dual core system may have the first core at 40 percent and the second core at 70 percent, in this case, the top command may show a combined result of 110 percent, but you will not know the individual values for each core.

$ top -c

We use -c option to show the command line or the executable path behind that process.

You can press 1 key while you watch the top command statistics to show individual CPU statuses.

Keep in mind that certain processes are spawned like the child processes, you will see multiple processes for the same program like httpd and PHP-fpm.

You shouldn’t rely on top command only, you should review other resources before making a final action.

Monitoring Disk I/O with iotop

The system starts to be slow as a result of high disk activities, so it is important to monitor disk activities. That means figuring out which processes or users cause this disk activity.

The iotop command in Linux process management helps us to monitor disk I/O in real-time. You can install it if you don’t have it:

$ yum install iotop

Running iotop without any options will result in a list all processes.

To view the processes that cause to disk activity, you should use -o option:

$ iotop -o

You can easily know what program is impacting the system.

ps command

We’ve talked about ps command before on a previous post and how to order the processes by memory usage and CPU usage.

Monitoring System Health with iostat and lsof

iostat command gives you CPU utilization report; it can be used with -c option to display the CPU utilization report.

$ iostat -c

The output result is easy to understand, but if the system is busy, you will see %iowait increases. That means the server is transferring or copying a lot of files.

With this command, you can check the read and write operations, so you should have a solid knowledge of what is hanging your disk and take the right decision.

Additionally, lsof command is used to list the open files:

lsof command shows which executable is using the file, the process ID, the user, and the name of the opened file.

Calculating the system load

Calculating system load is very important in Linux process management. The system load is the amount of processing for the system which is currently working. It is not the perfect way to measure system performance, but it gives you some evidence.

The load is calculated like this:

Actual Load = Total Load (uptime) / No. of CPUs

You can calculate the uptime by reviewing uptime command or top command:

$ uptime

$ top

The server load is shown in 1, 5, and 15 minutes.

As you can see, the average load is 0.00 at the first minute, 0.01 at the fifth minute, and 0.05 at fifteenth minutes.

When the load increases, processors are queued, and if there are many processor cores, the load is distributed equally across the server’s cores to balance the work.

You can say that the good load average is about 1. This does not mean if the load exceeds 1 that there is a problem, but if you begin to see higher numbers for a long time, that means a high load and there is a problem.

pgrep and systemctl

You can get the process ID using pgrep command followed by the service name.

$ pgrep servicename

This command shows the process ID or PID.

Note if this command shows more than process ID like httpd or SSH, the smallest process ID is the parent process ID.

On the other hand, you can use the systemctl command to get the main PID like this:

$ systemctl status <service_name>.service

There are more ways to obtain the required process ID or parent process ID, but this one is easy and straight.

Managing Services with systemd

If we are going to talk about Linux process management, we should take a look at systemd. The systemd is responsible for controlling how services are managed on modern Linux systems like CentOS 7.

You can start, stop and check the status like this:

$ systemctl status .service

$ systemctl stop .service

$ systemctl start .service

Instead of using chkconfig command to enable and disable a service during the boot, you can use the systemctl command:

$ systemctl enable .service

$ systemctl disable .service

Systemd also ships with its own version of the top command, and in order to show the processes that are associated with a specific service, you can use the system-cgtop command like this:

$ systemd-cgtop

As you can see, all associated processes, path, the number of tasks, the % of CPU used, memory allocation, and the inputs and outputs related.

This command can be used to output a recursive list of service content like this:

$ systemd-cgls

This command gives us very useful information that can be used to make your decision.

Nice and Renice Processes

The process nice value is a numeric indication that belongs to the process and how it’s fighting for the CPU.

A high nice value indicates a low priority for your process, so how nice you are going to be to other users, and from here the name came.

The nice range is from -20 to +19.

nice command sets the nice value for a process at creation time, while renice command adjusts the value later.

$ nice –n 5 ./myscript

This command increases the nice value which means lower priority by 5.

$ sudo renice -5 2213

This command decreases the nice value means increased priority and the number (2213) is the PID.

You can increase its nice value (lower priority) but cannot lower it (high priority) while root user can do both.

Sending the kill signal

To kill a service or application that causes a problem, you can issue a termination signal (SIGTERM). You can review the previous post about signals and jobs.

$ kill process ID

This method is called safe kill. However, depending on your situation, maybe you need to force a service or application to hang up like this:

$ kill -1 process ID

Sometimes the safe killing and reloading fail to do anything, you can send kill signal SIGKILL by using -9 option which is called forced kill.

$ kill -9 process ID

There are no cleanup operations or safe exit with this command and not preferred. However, you can do something more proper by using the pkill command.

$ pkill -9 serviceName

And you can use pgrep command to ensure that all associated processes are killed.

$ pgrep serviceName

I hope you have a good idea about Linux process management and how to make a good action to make the system healthy.

Thank you.

likegeeks.com

for myvar in vars

do

Code Here

done

In every loop, the variable myvar holds one of the values of the list. The loop iterates until the list is finished.

Iterating Over Simple Values

You can iterate over simple values like this:

#!/bin/bash

for var in first second third fourth fifth

do

echo The $var item

done

Check the results:

Iterating Over Complex Values

Your list maybe contains a comma or two words, and you want to deal with them as one item on the list.

Check the following example:

#!/bin/bash

for var in first "the second" "the third" "I’ll do it"

do

echo "This is: $var"

done

We quote our strings with double quotations.

We play nice till now, we always do. Just keep reading and practicing.

Command Substitution

By using command substitution using this format $(Linux command) you can store the result in a variable for later use.

#!/bin/bash

my_file="myfile"

for var in $(cat $my_file)

do

echo " $var"

done

Here we get the file content using cat command. Notice that our file contains one word per line, not separated by spaces.

Here we get the content of the file using command substitution then iterate over the result, assuming that each line has one word.

What about having spaces in one of these lines?

In this case, every word will be considered a field. You need to tell the shell to consider new lines as a separator instead of spaces.

The Field Separator

By default, the following characters treated as fields.

• Space
• Tab
• newline

If your text includes any of these characters, the shell will assume it’s a new field.

Well, you can change the internal field separator or IFS environment variable. like this:

IFS=$'\n'

It will consider new lines as a separator instead of spaces.

#!/bin/bash

file="/etc/passwd"

IFS=$'\n'

for var in $(cat $file)

do

echo " $var"

done

You got it. Bash scripting is easy.

The separator is colons in /etc/passwd file which contains the user’s information, you can assign it like this:

IFS=:

Bash scripting is awesome, right?

Iterating Over Directory Files

If you want to list the files in /home directory, you can use the for loop like this:

#!/bin/bash

for obj in /home/likegeeks/*

do

if [ -d "$obj" ]

then

echo "$obj is a folder"

elif [ -f "$obj" ]

then

echo "$obj is a file"

fi

done

From the previous post, you should know the if statement and how to check for files and folders, so if you don’t know, I recommend you to review it bash script step by step.

Here we use wildcard character which is the asterisk * and this is called in bash scripting file globbing which means All files with all names.

Notice that in the if statements here we quote our variables with quotations because maybe the file or the folder name contains spaces.

As you see the result, all files and directories in that folder are listed.

for Command C-Style

If you know C language, you may find that the for loop here is some weird because you are familiar with this syntax:

for (var= 0; var < 5; var++)

{

printf(“number is %d\n”, var);

}

Well, you can use the same syntax but with a little difference, here’s the syntax.

for (( variable = start ; condition ; iteration step))

So it looks like this:

for (( var = 1; var < 5; var++ ))

And this is an example:

#!/bin/bash

for (( var=1; var <= 10; var++ ))

do

echo "number is $var"

done

And this is the output:

The while Command

The for loop is not the only way for looping in bash scripting. The while loop does the same job but it checks for a condition before every iteration.

The while loop command takes the following structure:

while condition

do

commands

done

and here is an example:

#!/bin/bash

number=10

while [ $number -gt 4 ]

do

echo $number

number=$[ $number - 1 ]

done

The script is simple; it starts with the while command to check if number is greater than zero, then the loop will run and the number value will be decreased every time by 1 and on every loop iteration it will print the value of number, Once the number value is zero the loop will exit.

If we don’t decrease the value of var1, it will be the same value and the loop will be infinite.

Nesting Loops

You can type loops inside loops. This is called the nested loop.

Here’s an example of nested loops:

#!/bin/bash

for (( v1 = 1; v1 <= 5; v1++ ))

do

echo "Start $v1:"

for (( v2 = 1; v2 <= 5; v2++ ))

do

echo " Inner loop: $v2"

done

done

The outer loop hits first, then goes into the internal loop and completes it and go back to the outer loop and so on.

Iterate Over File Content

This is the most common usage for the for loop in bash scripting.

We can iterate over file content, for example, iterate over /etc/passwd file and see the output:

#!/bin/bash

IFS=$'\n'

for text in $(cat /etc/passwd)

do

echo "This line $text ++ contains"

IFS=:

for field in $text

do

echo " $field"

done

done

Here we have two loops, the first loop iterate over the lines of the file and the separator is the newline, the second iteration is over the words on the line itself and the separator is the colon :

You can apply this idea when you have a CSV or any comma separated values file. The idea is the same; you just have to change the separator to fit your needs.

Controlling the Loop

Maybe after the loop starts you want to stop at a specific value, will you wait until the loop is finished? Of course no, there are two commands help us in this:

• break command
• continue command

The break Command

The break command is used to exit from any loop, like the while and the until loop

#!/bin/bash

for number in 10 11 12 13 14 15

do

if [ $number -eq 14 ]

then

break

fi

echo "Number: $number"

done

The loop runs until it reaches 14 then the break command exits the loop.

And the same for the while loop:

#!/bin/bash

val=1

while [ $val -lt 5 ]

do

# Check number value

if [ $val -eq 4 ]

then

# The Code Breaks here <==

break

fi

# The Printed Message

echo "Iteration: $val"

val=$(( $val + 1 ))

done

The break command exits the while loop and that happens when the execution reaches the if statement.

The continue command

You can use the continue command to stop executing the remaining commands inside a loop without exiting the loop.

Check the following example:

#!/bin/bash

# The loop starts here

for (( number = 1; number < 10; number++ ))

do

# Check if number greater than 0 and less than 5

if [ $number -gt 0 ] && [ $number -lt 5 ]

then

continue

fi

# The printed message

echo "Iteration number: $number"

done

When the if condition is true, the continue command runs each iteration, and lines after the continue command never run until the condition becomes false.

Redirecting the Loop Output

You can use the done command to send the loop output to a file like this:

#!/bin/bash

for (( var = 1; var < 10; var++ )) do echo "Number is $var" done > myfile.txt

echo "finished."#!/bin/bash

for (( var = 1; var < 10; var++ )) do echo "Number is $var" done > myfile.txt

echo "finished."

The shell creates the file myfile.txt and the output is redirected to the file, and if we check that file we will find our loop output inside it.

Let’s employ our bash scripting knowledge in something useful.

Useful Examples

Finding executables

To get all executable files on your system, you can iterate over the directories in the PATH variable. We discussed for loop and if statements and file separator so our toolset is ready. Let’s combine them together and make something useful.

#!/bin/bash

IFS=:

for dir in $PATH

do

echo "$dir:"

for myfile in $dir/*

do

if [ -x $myfile ]

then

echo " $myfile"

fi

done

done

This is just awesome. We were able to get all the executables on the system that we can run.

Now nothing stops you except your imagination.

I hope you learn a new thing or at least review your knowledge if you forget it. My last word for you, keep reading and practicing.

Thank you.

likegeeks.com