Tag Archives | file

15+ examples for listing users in Linux

In this post, you will learn about listing users in Linux. Besides this, you will know other tricks about Linux users’ characteristics. There are 2 types of users in Linux, system users who are created by default with the system. On the other hand, there are regular users who are created by system administrators and can log in to the system and use it. Before we start listing users, we need to know where are these users saved on Linux? The users are stored in a text file on the system called the passwd file. This file is located in the /etc directory. The file is located on the following path:

Continue Reading →

/etc/passwd

In this file, you can find all the information about the users in the system.

List all users

Listing users is a the first step to manage them. This way we will know how many they are and who they are. In Linux, almost everything can be done in various ways and this is no exception.

To list all users, you can use the cat command:

cat /etc/passwd

list all users in Linux

As you can see in the image, there is all the information about the users.

1- In the first field, you will see the user name.

2- Then, a representation of the encrypted password (The x character). The encrypted password is stored in /etc/shadow file.

3- The UID or the user ID.

4- The next field refers to the primary group of the user.

5- Then, it shows user ID info such as the address, email, etc.

6- After this, you will see the home directory of the user.

7- The last field is the shell used by that user.

However, although the information is quite useful but if you only want to list users’ names in a basic way, you can use this command:

cut -d: -f1 /etc/passwd

Listing users in Linux

Now we have the names only by printing the first field of te file only.

List & sort users by name

The above command serves the purpose of listing users on Linux. But what about listing the users in alphabetical order?

To do this, we will use the previous command, but we will add the sort command.

So, the command will be like this:

cut -d: -f1 /etc/passwd | sort

Sort by name

As you can see in the image, the users are shown sorted.

Linux list users without password

It is important to know users who have no password and to take appropriate action. To list users who do not have a password, just use the following command:

sudo getent shadow | grep -Po '^[^:]*(?=:.?:)'

User with no password

The used regex will list all users with no password.

List users by disk usage

If you have a big directory and you want to know which user is flooding it, you can use the du command to get the disk usage.

With this, you can detect which of these users are misusing the disk space.

For it, it is enough to use the following command:

sudo du -smc /home/* | sort -n

List users by disk usage

In this way, you will have the users ordered by the disk usage for the /home directory.

We used the -n for the sort command to sort the output by numbers.

List the currently logged users

To list the currently logged in users, we have several ways to do it. The first method we can use the users command:

users

Users currently logged

It will list the users with open sessions in the system.

But this information is a little basic however, we have another command that gives more details. The command is simply w.

w

Using the w command to list users currently logged

With this command, we can have more information such as the exact time when the session was started and the terminal session he has available.

Finally, there is a command called who. It is available to the entire Unix family. So you can use it on other systems like FreeBSD.

who

The who command

With who command, we also have some information about currently logged in users. Of course, we can add the option -a and show all the details.

who -a

The who command with options

So, this way you know everything about the logged in users.

Linux list of users who recently logged into the system

We saw how to get the currently logged in users, what about listing the login history of users?

You can use the last command to get more info about the logins that took place:

last

The last command

Or the logins of a particular user

last [username]

For example:

last angelo

last command with specific user

These are the user login activity and when it was started and how long it took.

List users’ logins on a specific date or time

What about listing users’ logins on a specific date or time? To achieve this, we use the last command but with the -t parameter:

last -t YYMMDDHHMMSS
For example:
last -t 20190926110509

List users by a specific date

And now all you have to do is choose an exact date & time to list who logged on that time.

List all users in a group

There are 2 ways to list the members of a group in Linux, the easiest and most direct way is to get the users from the /etc/group file like this:

cat /etc/group | grep likegeeks

This command will list users in the likegeeks group.

The other way is by using commands like the members command in Debian based distros. However, it is not installed by default in Linux distributions.

To install it in Ubuntu / Linux Mint 19, just use APT:

sudo apt install members

Or in the case of CentOS:
sudo dnf install members

Once it’s installed, you can run the command then the name of the group you want to list the users to:

members [group_name]

For example:
members avahi

Using the members command

This way you can list users for a group in a Debian based distro. What about a RedHat based distro like CentOS?

You can use the following command:

getent group likegeeks

List users with UID

In Unix systems, each user has a user identifier or ID. It serves to manage and administer accounts internally in the operating system.

Generally, UIDs from 0 to 1000 are for system users. And thereafter for regular users. Always on Unix systems, UID zero belongs to the root users (You can have more than one user with UID of zero).

So now we will list the users with their respective UID using awk.

The command that performs the task is the following:

awk -F: '{printf "%s:%s\n",$1,$3}' /etc/passwd

List users with the UID

As you can see, each user with his UID.

List root users

In a Unix-like system like Linux, there is usually only one root user. If there are many, how to list them?

To do this, we can use this command:

grep 'x:0:' /etc/passwd

root users in the system

Here we are filtering the file to get users with UID of zero (root users).

Another way by checking the /etc/group file:

grep root /etc/group

The root users in Linux

Here we are getting users in the group root from the /etc/group file.

Also, you can check if any user can execute commands as root by checking the /etc/sudoers file:

cat /etc/sudoers

Get the total number of users

To get the total number of users in Linux, you can count lines in /etc/passwd file using the wc command like this:

cut -d: -f1 /etc/passwd | wc -l

List total number of users in Linux

Great! 43 users. But this includes system and regular users. What about getting the number of regular users only?

Easy! Since we know from above that regular users have UID of 1000 or greater, we can use awk to get them:

awk -F: '$3 >= 1000 {print $1}' /etc/passwd

List regular users

Cool!

List sudo users

Linux systems have a utility called sudo that allows you to execute commands as if you were another user who is usually the root user.

This should be handled with care in a professional environment.

Also, it is very important to know which users can run the sudo command. For this, it is enough to list the users that belong to the sudo group.

members sudo

sudo group users

Users in this group can execute commands as super users.

List users who have SSH access

SSH allows users to access remote computers over a network. It is quite secure and was born as a replacement for Telnet.

On Linux by default, all regular users can log in and use SSH. If you want to limit this, you can use the SSH configuration file (/etc/ssh/ssh_config) and add the following directive:

AllowUsers user1 user2 user3
Also, you can allow groups instead of allowing users only using the AllowGroups directive:
AllowGroups group1 group2 group3

These directives define who can access the service. Don’t forget to restart the SSH service.

List users who have permissions to a file or directory

We can give more than one user permission to access or modify files & directories in two ways.

The first method is by adding users to the group of the file or the directory.

This way, we can list the group members using the members utility as shown above.

Okay, but what if we just want this user to have access to this specific file only (Not all the group permissions)?

Here we can set the ACL for this file using setfacl command like this:

setfacl -m u:newuser:rwx myfile

Here we give the user called newser the permission for the file called myfile the permissions of read & write & execute.

Now the file can be accessed or modified by the owner and the user called newuser. So how to list them?

We can list them using the getfacl command like this:

getfacl myfile

This command will list all users who have permissions for the file with their corresponding permissions.

List locked (disabled) users

In Linux, as a security measure, we can lock users. This as a precaution if it is suspected that the user is doing things wrong and you don’t want to completely remove the user and just lock him for investigation.

To lock a user, you can use the following command:

usermod -L myuser

Now the user named myuser will no longer to able to login or use the system.

To list all locked users of the system, just use the following command:

cat /etc/passwd | cut -d : -f 1 | awk '{ system("passwd -S " $0) }' | grep locked

This will print all locked users including system users. What about listing regular users only?

As we saw above, using awk we can get locked regular users like this:

awk -F: '$3 >= 1000 {print $1}' /etc/passwd | cut -d : -f 1 | awk '{ system("passwd -S " $0) }' | grep locked

Very easy!

Listing remote users (LDAP)

Okay, now can list all system users (local users), but what about remote users or LDAP users? Well, we can use a tool like ldapsearch, but is there any other way?

Luckily yes! You can list local & remote users with one command called getent

getent passwd

This command lists both local system users and LDAP or NIS users or any other network users.

You can pipe the results of this command to any of the above-mentioned commands the same way.

Also, the getent command can list group accounts like this:

getent group

You can check the man page of the command to know the other databases the command can search in.

Conclusion

Listing users in the Linux system was fun! Besides this, we have learned some tips about users and how to manage them in different ways.

Finally, this knowledge will allow a better administration of the users of the system.

I hope you find the tutorial useful. keep coming back.

0

Recover deleted files on Linux (Beginners Tutorial)

Have you ever deleted any important files by mistake? Who doesn’t! Okay, but can I recover them? In this post, you will learn how to recover deleted files on Linux using various programs on different file systems. You will see how to recover deleted files from SD cards, HDDs, and deleted partitions on different Linux file systems such as EXT3, EXT4 and even from Windows file systems such as FAT32 & NTFS. This is quite a problem. Often, Linux users frequently install several systems at the same time and they may delete a partition bu mistake during the installation process. However, how to recover files from those deleted partitions? For this, we need to recover the partition using a tool called TestDisk. Testdisk is a powerful partition analysis and data recovery utility. It is shipped with a large number of Linux distributions such as Debian and Ubuntu. On the other hand, the application is cross-platform and supports a large number of partition tables such as Intel, MSDOS, and Mac. These are the most popular partition tables. Also, it supports many file systems such as NTFS, EXT4 and other nonpopular file systems such as BeOS and ReiserFS.

Continue Reading →

Recover files from deleted partition

When a file is deleted, the list of clusters occupied by the file is deleted, marking those sectors available for the use. If the clusters have not been overwritten, TestDisk can recover the files.

First, start the application like this:

$ testdisk

Next, you have the option to create a new file for the logs. If you want to create one, choose the create option and press Enter. If you don’t want a log file, select the No Log option.

Using testdisk

Next, the disks or partitions recognized by the system will be scanned. In this particular case, sda is the partition we want to recover.

Scan the devices

TestDisk recognizes various types of partition tables. It is usually Intel. Unless you are using a specialized one.

Select the partition table

In the next screen, you will see a series of options that the program has. For this particular case, we need to choose the Analyse option.

With this option, the program will exhaustively analyze the disk to find the structure.

Analyse the disk

Then, it will ask about the type of search you wish to do. Usually, choose the Quick Search option.

The partition structure

If you are lucky, you will see the deleted partition. If not, you will have to choose a deeper search.

Using TestDisk

Then, choose the Write option to write the partition table. When finished, restart the system and you will have your partition back!

Recover a partition

Reboot to apply the changes

Note that during these steps, it may take a long time. It depends on the disk size.

According to the type of file system, this partition may have, particular instructions will be followed. They will be detailed later.

Recover deleted files from an external drive

Now let us imagine we have an external flash drive and by mistake, you have deleted some files from it. How to restore them?

Thanks to TestDisk, the process becomes quite similar to that of a deleted partition. But there are some differences.

To start the program we will use the testdisk command. Also, we can add the flash drive as a parameter like this:

$ sudo testdisk /dev/sdb

TestDisk to recover files

Next, select proceed. Then, choose the partition table type.

Select the partition table

Then, select the Advanced options to recover files.

Advanced options on TestDisk

The next step is selecting the partition and the Undelete option.

Undelete files with TestDisk

Then, you will see all the deleted files on the partition.

Recover files with TestDisk

Now, select the destination folder to place the recovered files. You need to press C on the first option to place the files on the current directory.

Select the destination folder

Finally, you will see this message:

Everything OK with TestDisk

Congratulations! Files restored.

Recover deleted files from SD card

Usually, on an SD card, it is common to notice that they are used for multimedia files. Therefore, it is advisable to use a more specialized program for these files.

In this case, we will use the application called Photorec that comes incorporated in TestDisk.

First, insert the SD card on the PC. Next, run photorec as root:

$ sudo photorec [device]

Then, you will see the following image. Select the media and proceed and press Enter.

Using photorec to recover files from SD

Next, select the partition. And select Options and press enter.

Select the partition

There you will see the recovery options that will be performed on the SD card.

Photorec options

Press q to return to the previous screen. And there it is necessary to choose the types of files that we want to recover. This is achieved by selecting the File Opt option.

Formats to recover

Press the s key to select and deselect all formats. You can also select the types of files you want to recover using the right key. To save the selected options press the b key. Return to the main menu using the q key.

Then, on the main menu, choose the Search option to start the process. And choose the file system.

Select the file system

You will then be presented with two options. Free and Whole. Normally, Free is enough. If you want to do a deep analysis, choose Whole but keep in mind that it will slow down the process.

Now, it is necessary to choose the location where the files will be saved. To do this, press the c key.

Select the destination

After choosing the destination, the recovery process will start. Remember that the system will collapse and freeze. So be patient.

In the end, you will see a message informing you of everything that has happened.

Photorec report

Next, check the results.

Check the results

Recover deleted files from NTFS

NTFS is a Windows file system. If you are one of those who use both systems on the computer, then you may need to restore deleted files from a Windows partition with this file system.

To do this, we have a tool called ntfsundelete that is quite simple to use.

First, you need to scan the disk or partition. For example:

$ sudo ntfsundelete /dev/sda1

Using ntfsundelete

Then, we will be able to recover the deleted file with the following command:

$ sudo ntfsundelete [HD_Or_partition] -u -m [filename]

Recovering files using ntfsundelete 

 The recovered files now belong to the root user. The last step is to change the permissions and owners of the files using the chown command.

Recover Files from FAT32

Another common Windows file system is FAT32. You can recover files from FAT32 is by using TestDisk.

So again run testdisk as root user and pass the disk as a parameter:

$ sudo testdisk [partition/HD]

TestDisk is compatible with FAT32 

Then continue the steps as described above to restore of the files.

Recover on memory files (Using inode)

If you delete a file that is used by another process, you can restore it from the memory using inode.

To do this, some initial conditions must be established. First, the deleted file MUST remain open by another process. Then you have to verify the process and finally recover it and change its permissions.

In this case, I will create a file called example.txt using the nano editor and add some text:

$ nano example.txt

Then save the changes and open another terminal window and use the file. For example, with the less command.

$ less example.txt

Using the less command

 Open another terminal session, delete the file and make sure it’s deleted:

$ rm example.txt

$ ls example.txt

Delete the example file

As you can see, the file no longer exists. But we will be able to recover it. To do this, let’s get the number of the process associated with the inode of the file.

$ lsof | grep example.txt

Check the deleted file

You will notice the process and command that is using the file (the less command). From that image, we have to pay attention to the second and fourth values. These are the PID of the process and the descriptor of the file respectively.

Then, recover it with the following command:

$ ls -l /proc/2325/fd/4

Find the process of the deleted file

Then copy it to whatever location you want and that is enough to recover it.

$ sudo cp /proc/2325/fd/4 .

Next, check the results and open the file:

Recover a deleted file using inode  

This way we can recover a deleted file that still on memory and used by a process with the inode.

Recover Deleted Files from EXT4 (Using extundelete)

EXT4 is the default file system on most Linux distributions. It is quite fast and with technical features that are very well taken advantage of by the Linux kernel.

One of the used tools to recover files from EXT4 filesystem is extundelete.

Extundelete is an open-source application that allows recovering deleted files from a partition or a disk with EXT3 or EXT4 file system. It is simple to use and comes by default installed on most Linux distributions.

To recover a certain file, just use the following command:

$ sudo extundelete [device] -restore-file [pathfile]

For example:

$ sudo extundelete /dev/sdb1 -restore-file home/angelo/other.txt

If you want to recover all the files in a folder, use the wildcard character:

$ extundelete /dev/sda6 -restore-file home/angelo/*

But if you want to restore all files on the partition or disk, the next command would suffice:

$ extundelete /dev/sda6 -restore-all

Using extundelete to recover files

So, the recovered files will be on the RECOVERED_FILES directory. So this way, you can recover deleted files using extundelete.

Using debugfs

It is also possible to use the debugfs tool to recover deleted files. This tool also uses the inode number of the deleted file. However, it only works on EXT4 file systems.

Its operation is quite simple, too. First, you have to enter the partition or device.

$ debugfs [device]

For example,

$ sudo debugfs /dev/sdb1

Using debugfs

Then, after a while, you will be able to login to the debugfs console to search for recently deleted files.

$ debugfs: lsdel

inodes to recover

In the first column, you will see the inode number of the deleted files in that device. Then, restore it with the following command:

$ debugfs mi

And that is it. It is quite easy.

Using ext4magic

Another alternative way to recover deleted files on a disk with an Ext4 file system is to use Ext4magic. This application is also quite simple to use.

The most basic syntax of the application is the following:

$ sudo ext4magic [device] -f [folder_to_scan] -r -d [output_folder]

If I wanted to recover the deleted files from a folder called files, the command would be similar to this one:

$ sudo ext4magic /dev/sdb1 -r -d files

Using ext4magic to recover files

That is how easy it is to use ext4magic. All this thanks to the fact that Ext4 is a community and open source file system.

Recover overwritten files (Using Scalpel)

Scapel is another open-source tool that allows you to recover files from formatted drives, overwritten files and even damaged drives. It is well known for its speed and efficiency. In this sense, it emerges as an alternative to consider.

Scalpel carves files without the help of filesystems. It tries to extract headers and footers of files and tries to guess the entire file structure using some well-designed algorithms.

Like TestDisk, it is available in the official repositories of most Linux distributions. Therefore, its installation is reduced to the use of the terminal and the package manager of the distribution.

The fastest and easiest way to use Scapel is as follows:

$ scalpel [device] -o [output_folder]

The output_folder where scapel will place all recovered files. Note that Scalpel will create the output directory itself.

But how does Scapel know which files to recover? Well, that is defined in the application configuration file.

This configuration file is usually located at the following location:

/etc/scalpel/scalpel.conf

And you can open it with your favorite text editor and there you will only have to uncomment the lines to define the file formats to search.

Scalpel configuration file

The file formats you uncomment, Scalpel will search for it.

Next, run the full Scalpel command and in the output folder, you will see the recovered files.

$ sudo scalpel /dev/sdb1 -o recovered_files1

Using scalpel to recover files

Sometimes, Scalpel restores parts of the file. That depends on the health of the drive and how much data has been corrupted.

Also, there are many craving algorithms you can use, but we discussed here the basic way of craving data.

Recover files from a non-bootable system

This is a delicate case because we need to access from a Live cd of Ubuntu or another similar Linux distribution. Once we have boot, we could use TestDisk to try to recover the data.

In this case, we would have to use an external drive where to save the data. On the other hand, in case TestDisk can’t do the job, we can also try extundelete or ext4magic as long as the partition is Ext4.

If it does not work, you could try regenerating the partition using TestDisk as explained above.

Conclusion

It is possible to delete files accidentally. The idea is to know the appropriate tools and techniques to recover these files.

In this post, we have covered several circumstances and different file systems that could help avoid such problems.

Keep coming back.

Thank you.

0

Process Large Files Using PHP

If you want to process large files using PHP, you may use some of the ordinary PHP functions like file_get_contents() or file() which has a limitation when working with very large files. These functions rely on the memory_limit setting in php.ini file, you may increase the value but these functions still are not suitable for very large files because these functions will put the entire file content into memory at one point. Any file that has a size larger than memory_limit setting will not be loaded into memory, so what if you have 20 GB file and you want to process it using PHP? Another limitation is the speed of producing output. Let’s assume that you will accumulate the output in an array then output it at once which gives a bad user experience. For this limitation, we can use the yield keyword to generate an immediate result.

Continue Reading →

SplFileObject Class

In this post, we will use the SplFileObject class which is a part of Standard PHP Library.

For our demonstration, I will create a class to process large files using PHP.

The class will take the file name as input to the constructor:

class BigFile
{
protected $file;
public function __construct($filename, $mode = "r")
{
if (!file_exists($filename)) {
throw new Exception("File not found");
}
$this->file = new SplFileObject($filename, $mode);
}
}

Now we will define a method for iterating through the file, this method will use fgets() function to read one line at a time.

You can create another method that uses fread() function.

Read Text Files

The fgets() is suitable for parsing text files that include line feeds while fread() is suitable for parsing binary files.

protected function iterateText()
{
$count = 0;
while (!$this->file->eof()) {
yield $this->file->fgets();
$count++;
}
return $count;
}

This function will be used to iterate through lines of text files.

Read Binary Files

Another function which will be used for parsing binary files:

protected function iterateBinary($bytes)
{
$count = 0;
while (!$this->file->eof()) {
yield $this->file->fread($bytes);
$count++;
}
}

Read in One Direction

Now we will define a method that will take the iteration type and return NoRewindIterator instance.

We use the NoRewindIterator to enforce reading in one direction.

public function iterate($type = "Text", $bytes = NULL)
{
if ($type == "Text") {
return new NoRewindIterator($this->iterateText());
} else {
return new NoRewindIterator($this->iterateBinary($bytes));
}
}

Now the entire class will look like this:

class BigFile
{
protected $file;
public function __construct($filename, $mode = "r")
{
if (!file_exists($filename)) {
throw new Exception("File not found");
}
$this->file = new SplFileObject($filename, $mode);
}
protected function iterateText()
{
$count = 0;
while (!$this->file->eof()) {
yield $this->file->fgets();
$count++;
}
return $count;
}
protected function iterateBinary($bytes){
$count = 0;
while (!$this->file->eof()) {
yield $this->file->fread($bytes);
$count++;
}
}
public function iterate($type = "Text", $bytes = NULL)
{
if ($type == "Text") {
return new NoRewindIterator($this->iterateText());
} else {
return new NoRewindIterator($this->iterateBinary($bytes));
}
}
}

Parse large Files

Let’s test our class:

$largefile = new BigFile("file.csv");
$iterator = $largefile->iterate("Text"); // Text or Binary based on your file type
foreach ($iterator as $line) {
echo $line;
}

This class should read any large file without limitations Great!!

You can use this class in your Laravel projects by autoloading your class and add it to composer.json file.

Now you can parse and process large files using PHP easily.

Keep coming back.

Thank you.

0