Welcome back, aspiring cyberwarriors!

One of the fundamental principles that separates Linux from other operating systems is its philosophy that nearly everything is a file, and more often than not, these files contain plain text. Configuration files that control how services behave, log files that record system activity, and network captures that reveal traffic patterns. Countless other critical components of the system exist as text files that you can read, modify, and manipulate using simple command-line tools.

In this article, we will explore the essential commands and techniques for manipulating text in Linux. Let’s get rolling!

Step 1: Cat That File

For demonstration purposes, we will use files from the world’s best Network Intrusion Detection System (NIDS), Snort. If some of you have prior experience with Snort, you probably know that the configuration file was called snort.conf. However, in Snort 3, the traditional snort.conf file has been replaced by Lua-based configuration files in snort.lua serving as the primary configuration file.

Now, let’s move on to the cat command. As demonstrated in an earlier tutorial, cat is one of the most basic text display commands. Let’s use cat to view the Snort configuration file located in /etc/snort.

kali > cat /etc/snort/snort.lua

As you can see, snort.lua is displayed on our screen until it reaches the end of the file. Not the most convenient or practical way to work with this file.

Step 2: Take the Head

To view the beginning of a file, use the head command. This command displays the first 10 lines of a file, by default.

kali > head /etc/snort/snort.lua

If we want to see more or fewer than the default 10 lines, we can tell head how many lines to display by putting the number of lines we want (with the – n switch) between the command and the file name.

kali > head -20 /etc/snort/snort.lua

Here we can see that only the first 20 lines of snort.lua are displayed.

Step 3: Grab That Tail

Similar to the head command, we can view the last lines of a file with the tail command. Let’s use it on the snort.lua.

kali > tail /etc/snort/snort.lua

Note that it shows some logging configurations but not all. Let’s see if we can display all of them by retrieving the last 20 lines of the snort.lua file.

kali > tail -20 /etc/snort/snort.lua

Now we can view nearly all the logging rules on one screen.

Step 4: Numbering Those Lines

Sometimes—especially with very long files—we may want the file displayed with line numbers. This is probably the case with snort.lua, which has over 200 lines. This makes it easier to reference changes and come back to the same place within a file. To display a file with a line number, we simply type:

kali > nl /etc/snort/snort.lua

Note that each line now has a number, making referencing much easier.

Step 5: I Grep That

After cat, grep is the most widely used text manipulation command. It’s a filtering command; in other words, it enables us to filter the content of a file for display. If, for instance, we wanted to see all the instances of where the word “log” occurs in our snort.lua file, we could ask cat only to display those lines where it occurs by typing:

kali > cat /etc/snort/snort.lua | grep log

This command will first grab the snort.lua and then “pipe” it (|) to grep, which will take it as input and then look for the occurrences of the word “log” and only display those lines. Grep is a powerful and essential command for working in Linux, as it can save us hours searching for every occurrence of a word or command.

Step 6: I Sed That Works

The sed command allows us to search for occurrences of a word or text pattern and then perform some work on them. The name comes from the concept of a stream editor and is a contraction of those two words. At its most basic, sed operates like the Find and Replace function in Windows. Let’s search for the word “modbus” in the snort.lua file using grep.

kali > cat /etc/snort/snort.lua | grep modbus

We can see that the grep command found four occurrences of the word modbus.

Let’s say we want sed to replace every occurrence of modbus with Modbus (remember, Linux is case-sensitive) and then save the new file as snort2.lua.

We could do this by typing:

kali > sed s/modbus/Modbus/g /etc/snort/snort.lua > snort2.lua

This command says, “search (s) for the word modbus and replace it with the word Modbus globally (i.e., wherever you find it in the file).”

kali > cat /etc/snort/snort2.lua | grep Modbus

Now, when we grep snort2.lua for Modbus, we see that none were found, and when we grep for Modbus, we find four occurrences of Modbus.

If we want to replace only the first occurrence of the word modbus, we could leave out the trailing ‘g’ (for global), and it would replace only the first occurrence.

kali > sed s/modbus/Modbus/ snort.lua > snort2.lua

Step 7: Less is More

Although cat is a good utility for creating and displaying small files, it certainly has its shortcomings when displaying large ones. When we catch the snort.lua, it scrolls through numerous pages until it comes to the end. Not very practical.

For working with larger files, we have two other utilities, more and less. More came first and is the utility that the man (manual) pages use. Let’s open snort.lua with more commands.

kali > more /etc/snort/snort.lua

Notice that more displays only the first page and displays for us in the lower left corner how much of the file is shown (13% in this case).

To see additional lines or pages, we can use the ENTER key, PAGE UP, or PAGE DOWN to move an entire page. To exit more, type “q” for quit.

Less is very similar to more, but with additional functionality; hence the common Linux aficionado quip: “Less is more.”

Let’s open snort.lua with less.

kali > less /etc/snort/snort.lua

Notice near the bottom-left of the screen that ‘less’ has highlighted the file path.

If we hit the / key, less will enable us to search for terms in the file. For instance, let’s search for filters:

/filter

This will immediately take us to the first occurrence of output and highlight it.

We can then look for the next occurrence of filter by typing “n” for next.

As you can see, less took us to the next occurrence of the word filter, highlighted all the search terms, and in this case, took us directly to the filter section of Snort. How convenient!

Summary

This article illustrates essential commands and techniques for manipulating text in Linux, emphasizing the importance of text files in system configuration and logging. We explored commands such as cat, head, tail, nl, grep, and sed, demonstrating how they can enhance your efficiency in handling configuration files like snort.lua.

For more information on using Linux for hacking, check out the book “Linux Basics for Hackers” on Amazon or visit our training center.

The post Linux Basics for Hackers, Part 09: Text Manipulation first appeared on Hackers Arise.

Welcome back, aspiring cyberwarriors!

Among the areas that Linux newcomers find problematic, managing user environment variables is often the most obscure. Although Windows operating systems support environment variables, most users seldom—if ever—manage them. To get the most from our Linux hacking system, you need to both understand and manage environment variables for optimal performance, convenience, and possibly even stealth.

These environment variables are used in our particular user environment. In most cases, that environment will be your BASH shell. Each user, including root, has a set of environment variables with default values unless they’re changed. You can change these values to make our system work more efficiently and tailor our work environment to meet our individual needs best.

View Our Environment Variables

Let’s start by viewing all your environment variables by entering env.

Note that all environment variables are in all uppercase, such as HOME, PATH, SHELL, etc. As you will see later in this article, you can create your own user-defined variables (see below), and if you do, it is advisable—but not required—that they also be in all uppercase.

In addition, we can view all variables, including user-defined variables and command aliases, by entering the command set.

This command lists numerous variables specific to our system. In most cases, this list is so long that it can’t be viewed on a single page. To see all these variables line-by-line, you can pipe the output to the more command, such as:

Now, the list of variables fills up one screen and stops, waiting for us to hit the ENTER key to advance to the next line. You can do this until we come across any variable we are looking for. If we press ENTER a few times, we will find a variable named HISTSIZE. Hitting the ENTER key will take you through each of these variables, one by one. Whenever you use the more command for output, you can use the q to exit or quit and return to the command prompt.

Rather than scrolling through this long list of variables tediously looking for the variable of interest, you can use the filtering command grep to find it. For instance, as you saw above, there is a variable named HISTSIZE. This variable contains the number of commands stored in your command history file. That is, the commands that you have previously typed and can recall by using the UP and DOWN arrows from the BASH shell.

Let’s try to find it using set and filtering the output with grep to find the HISTSIZE variable.

As shown above, this command finds the variable HISTSIZE and displays its value. The default value of this variable is set to 1000 on your system. This means that the HISTSIZE variable stores your last 1000 commands by default.

Viewing Variables Values

The set command displays all your variable names, but if you want to see the value stored in the variable, you can use the keyword echo followed by the dollar sign $ and the variable name, such as:

It’s important to note that when you want to use the value stored in a variable, such as here, you need to put a $ before the variable name. The dollar sign ($) before the variable name indicates you want to work with the value inside the variable, rather than the label of the variable.

As I noted above, the HISTSIZE variable contains the number of commands stored in our history file. As you can see in this screenshot, the HISTSIZE variable is set to 1000. In some cases, we may NOT want our past commands stored in the history file. This may be because you don’t want to leave any evidence of your activity on the system. In that case, you can set your HISTSIZE variable to 0, and the system will NOT store any past commands.

Now, when we try to use the UP or DOWN arrows to recall commands, nothing happens because the system no longer stores them. Stealthy, but inconvenient.

Exporting our Environment Variables

When you change an environment variable, it’s only for that particular environment. In this case, that environment is the BASH shell. This means that once we close that terminal, any changes we made to these variables are lost or reset to their default values. If we want the value to remain for our next terminal session and another terminal session, we need to export the variable. Think of it as “exporting” the new value from your current environment (the BASH shell) to the rest of the system so that it is available in every environment.

We can do this by simply entering export and then the variable name, such as:

Now, the HISTSIZE variable is set to 0 when we leave this environment and return later. Of course, we can set the HISTSIZE variable back to 1000 by simply entering:

Changing Our Shell Prompt

The default shell prompt in Kali takes the following format;

username@hostname:current_directory>

If you are the root user, this translates to a default prompt of;

root@kali:current_directory

We can change the default command prompt by setting the PS1 variable. This variable has a specific set of placeholders for information to be inserted into the prompt. These include;

u =name of the current user

h = host name

W= current working directory

Let’s have a little fun and change the prompt in our terminal. The environment variable that contains our prompt for the first terminal is PS1. We can change it by typing:

Now, every time you open a terminal, you are reminded that you are “World’s Best Hacker”.

Remember that our pr ompt will now be “World’s Best Hacker” whenever we open the first terminal (PS1), but the second terminal will still be the default command prompt. This means that if we really like this new command prompt and want to keep it, we need to export the variable PS1 so that each time we open this terminal or any terminal, the prompt will be “World’s Best Hacker: #”

Changing Our Path Variable

Probably the most important variable in our environment is our PATH variable. This variable controls where your shell looks for the commands you type, such as cd, ls, and echo (they are usually located in the sbin or bin sub-directories, such as /usr/local/sbin or/usr/local/bin). If the BASH shell doesn’t find the command in one of the directories in our path, it returns an error “command not found” even if it DOES exist in another directory not in our PATH.

Let’s take a look at the contents of our PATH variable by echoing its contents:

Notice the directories included in our PATH variable. These are usually the/bin and /sbin directories, where our system commands are found. When we type ls, the system knows to look in each of these directories for the ls command, and when it does, it executes it.

If we were to download and install a new hacking tool named “newhackingtool” into the /root/newhackingtool directory, we could only use it when we were in that directory. This means that every time we wanted to use that tool, we had to navigate to /root/newhackingtool first. That might be just fine, but a bit inconvenient. To be able to use this new tool from ANY directory, you could add this directory to the PATH variable.

To add this newhackingtool directory to our PATH variable, you can enter:

In this command, you are saying “take the PATH variable (PATH) and assign it (=) the value of the old PATH variable ($PATH) and add /root/newhackingtool.”

It’s important to note here that we have appended the /root/newhackingtool directory to your PATH variable. If you now go back and examine the contents of the PATH variable, you will see that this directory has been appended to the end of the PATH.

This means when you want to run your newhackingtool, you won’t need to navigate to the /root/newhackingtool directory. You can now execute newhackingtool applications from anywhere on your system. The BASH shell will now look in that directory for our new tool!

A common mistake made by those new to Linux is to assign the new directory, /root/newhackingtool, to the PATH variable, such as;

kali > PATH=/root/newhackingtool

kali > echo $PATH

/root/newhackingtool

Now, your PATH command ONLY contains the/root/newhackingtool directory, not the system binaries directories such as /bin, /sbin, and others. This is NOT good. In this case, when you go to use any of the system commands, you are likely to receive the error “command not found” (unless in the unlikely case you are in the system binaries directories when you execute it).

kali > cd

bash: cd: command not found

kali >

Remember, you want to append to the PATH variable, not replace.

This can be a very useful technique for directories we use often, but be careful not to add too many directories to your PATH variable, as the system will have to search through each directory in the PATH to find commands, which could potentially slow down your terminal and your hacking.

Creating a New User-Defined Variable

You can create your own custom, user-defined variables in Linux by simply assigning a value to your new variable. The syntax is rather straightforward; first the name of your variable, then the assignment symbol “=”, and finally the value in the variable, such as;

kali > MYNEWVARIABLE = “Hacking is the most valuable skill set in the 21st century”

Now, to see the value in that variable, you can use the echo command followed by the $ and the variable name.

kali > echo $MYNEWVARIABLE

Hacking is the most valuable skill set in the 21st century

If you want to delete this new variable or any system- or user-defined variable, you can use the unset command. You should be cautious when deleting a system variable, as your system will likely operate very differently afterwards.

kali > unset MYNEWVARIABLE

Summary

Although environment variables seem a bit obscure, they can control the settings and appearance of your Linux working environment. You can manage them to tailor our environment to your needs by changing any of those variables and exporting the changes. In addition, we can create new variables to help manage your system.

For more information on using Linux for hacking, check out the book “Linux Basics for Hackers” on Amazon or visit our training center.

The post Linux Basics for Hackers, Part 08: Managing the User Environment first appeared on Hackers Arise.

Welcome back, aspiring cyberwarriors!

Any self-respecting hacker must be able to script. For that matter, any self-respecting Linux administrator must be able to script. With the arrival of Windows PowerShell, Windows administrators are increasingly required to script to automate tasks and become more efficient.

As hackers, we often need to automate running multiple commands, sometimes across multiple tools. To become an elite hacker, you not only need to have advanced shell scripting skills, but also the ability to script in one of the widely-used scripting languages, such as Ruby (Metasploit exploits are written in Ruby) or Python (many hacking tools are Python scripts).

We will start with basic shell scripting, move to advanced shell scripting, and then to each of these scripting languages, developing hacking tools as we go. Our ultimate goal is to develop enough scripting skills to develop our own exploits. Let’s get rolling!

Step 1: Types of Shells

A shell is an interface between the user and the operating system. This enables us to run commands, utilities, and programs, and to manipulate files, etc.

There are several shells available for Linux. These include the Korn shell, the Z shell, the C shell, and the Bourne Again Shell (or BASH). Kali Linux now uses Z Shell by default. But the BASH shell is available in Kali Linux and in nearly all Linux and UNIX distributions (including Mac OS X); we will be using BASH exclusively here.

To check the current shell on your system, write the following command:

kali> echo $SHELL

To switch to BASH, use the following:

kali> chsh -s /bin/bash

Then log out and back in (or reboot).

Step 2: BASH Basics

To create a shell script, we need to start with a text editor. You can use any text editor in Linux, including vi, vim, emacs, gedit, kate, etc., but I will be using Mousepad in these tutorials. Using a different editor should not affect your script or its functionality.

Besides running system commands, utilities, and applications from a BASH shell script, the BASH shell includes its own commands. These include:

:, ., break, cd, continue, eval, exec, exit, export, getopts, hash, pwd, readonly, return, set, shift, test, [, times, trap, umask, and unset, alias, bind, builtin, command, declare, echo, enable, help, let, local, logout, printf, read, shopt, type, typeset, ulimit, and unalias.

I will address these commands in a later tutorial, but I want you to know that this shell has built-in commands that have their functionality within the BASH shell.

Step 3: Comments

Like any coding, we may want to add comments. Comments are simply notes to ourselves or anyone else reading the code about what we were trying to do with the script or that section of the script. These notes or “comments” are not read or executed by the interpreter.

The BASH shell enables comments by preceding a line with the “#”, so if I wanted to note that this was my first script, I could write in my text editor:

This is my first script!

The interpreter would ignore everything after the # and then move to the next line.

Step 4: “Hello, Hackers-Arise!”

For our first script, we will start with a simple script that returns a message to the screen that says “Hello, Hackers-Arise!”.

We start by entering the shebang or “#!”. This tells the operating system that whatever follows the shebang is the interpreter we want to use for our script.

We then follow the shebang with /bin/bash, indicating that we want the operating system to use the BASH shell interpreter. As we will see in later tutorials, we can use other interpreters such as PERL or Python, but here we want to use the BASH interpreter.

#! /bin/bash

Next, we enter echo, a command in Linux that tells the system to simply repeat or “echo” back to our monitor (stdout) what follows. In this case, we want the system to echo back to us “Hello, Hackers-Arise!”. Note that the text or message we want to “echo back” is in double quotation marks.

echo “Hello, Hackers-Arise!”

Now, let’s save this file as HelloHackersArise. After saving, we can see that code highlighting appears.

Step 5: Set Execute Permissions

When we create a file, it’s not necessarily executable, not even by us, the owner. Let’s look at the permissions on our new file by typing ls -l in our directory.

As you can see, our new file has rw-rw-r– (664) permissions. The owner of this file only has read (r) and write (w) permissions, but no execute (x) permissions. The group has the same permissions, and all others have only read permission. We need to modify it to give us execute permissions in order to run this script. We do this with the chmod command. To give the owner, the group, and all others execute permissions, we type:

kali > chmod 755 HelloHackersArise

Now when we do a long listing (ls -l) on the file, we can see that we have execute permissions.

kali > ls -l

Step 6: Run HelloHackersArise

To run our simple script, we type:

kali > ./HelloHackersArise

The ./ before the file name tells the system to run the script in the current directory. This means don’t look in the directories in the PATH variable for this file, but rather look just in my current directory and run HelloHackersArise

When we then hit enter, our very simple script returns to our monitor.

Hello Hackers Arise!

Success! We just completed our first simple script!

Step 7: Using Variables

So, now we have a simple script. All it does is echo back a message. If we want to create more advanced scripts, we will likely want to add some variables.

Variables are areas of memory where we can store values. That “something” might be some letters or words (strings) or numbers. It can help to add functionality to a script that has values that might change.

Let’s go back to the script we wrote earlier to use nmap to scan for vulnerable machines with a particular port open. Remember the (in)famous hacker, Max Butler, used a similar script to find systems running Aloha POS, which he then hacked, exposing millions of credit card numbers.

As you can see, this script was written to scan a range of IP addresses for port 5505 (the port Aloha left open for tech support) and create a report of all IP addresses with this port open. The IP address range is “hard-coded” into the script and can only be changed by opening and editing the script file.

What if we altered this script to prompt the user for the range of IP addresses to scan and the port to scan for? Wouldn’t it be much easier if we were prompted for these values and they were entered into the script?

Let’s take a look at how we could do that.

Step 8: Adding Prompts & Variables to Our Script

First, we could replace the specified subnet with an IP range. We can do this with a variable called “FirstIP” and then a second variable named “LastIP” (the name of the variable is irrelevant, but best practice is to use a variable name that helps you remember what it holds).

Next, we can replace the port number with a variable named “port.” These variables will serve as storage areas for the user’s input before running the scan.

Next, we need to prompt the user for these values. We can do this by using the echo command we learned above in writing the HelloHackersArise script.

So, we can echo the prompt “Enter the starting IP address:” to display on the screen and prompt the user for the first IP address in their nmap scan.

echo “Enter the starting IP address:”

Now, when the user sees this prompt on the screen, they will enter the first IP address. We need a way, then, to capture the user’s input. We can do this by following the echo line with the read command and the variable name. The read command reads a value from the keyboard (stdin) and assigns it to a variable that follows it.

read FirstIP

The above command assigns the user-entered IP address to the variable FirstIP. Then we can use that value in FirstIP throughout our script.

Of course, we can do the same for each variable: first prompt the user to enter the information, then use a read command to capture it.

Next, we need to edit the nmap command in our script to use the variables we just created and filled. When we want the value stored in a variable, we can prefix the variable name with a $, such as $port.

So, to use nmap to scan a range of IP addresses starting with the first user input IP through the second user input IP and look for a port input by the user, we can rewrite the nmap command like this:

nmap -sT $FirstIP-$LastIP -p $port -oG Aloha

As written, the script will scan the IP address range from FirstIP to LastIP, looking for the port the user entered. Let’s now save our script file and name it Scannerscript.

Step 9: Run It with User Input Variables

Now we can run our simple scanner script with the variables specifying the IP address range and the port to scan, without having to edit the script.

kali > ./Scannerscript

As you can see, the script prompts for the starting IP address, the last IP address, and the port to scan for. To scan the full 10.0.2.0/24 network range with this simple script, enter 10.0.2. (including the trailing dot) as the starting IP address and 254 as the last IP address. This makes nmap interpret the target as 10.0.2.-254, which it correctly expands to scan from 10.0.2.1 to 10.0.2.254. After collecting this information, the script runs the nmap scan and produces a report in the file Aloha3 that shows every IP address in that range where the specified port is available.

Summary

Learning shell scripting is a crucial skill for anyone venturing into cybersecurity, whether as a hacker or a system administrator. The concepts and commands discussed in this tutorial serve as a foundation for learning advanced scripting techniques. Understanding how to create and execute shell scripts, manage file permissions, and use built-in BASH commands enables users to automate tasks effectively and efficiently.

For more information on using Linux for hacking, check out the book “Linux Basics for Hackers” on Amazon or visit our training center.

The post Linux Basics for Hackers, Part 07: BASH Scripting Basics first appeared on Hackers Arise.