User Tools

Site Tools


Table of Contents

Linux Frequently Used Terminal List

Blocking & Banning

Refrain from blocking or banning as the main strategy. It has unintended consequences and needlessly inhibits users most of the time.

  1. mod_qos, a quality of service module for the Apache web server
  2. mod_ratelimit, a bandwidth rate limiting for clients
  3. mod_evasive, a module for Apache that provides evasive action in the event of an HTTP Distributed Denial of Service (DDoS/DoS) attack or brute force attack.
  4. mod_security, a web application firewall for the Apache web server. In addition to providing logging capabilities, ModSecurity can monitor HTTP traffic in real time in order to detect attacks.

Manual Brute-Force Detection & Blocking

Log the IP's in the logs such as the access.log in Apache Sort IP list in readable form

cat access.log.14 | awk '{print $1}' | sort -n | uniq -c | sort -nr | head -20
grep 'bot' access.log
grep 'spider' access.log

Take each IP and search an abuse list such as or

They have an API that can monitor your server and report incidents automatically and even keep you aware of incidents happening around the world in real time.

If the IP has a rating of 100% with many incidents, ban it using a firewall

sudo iptables -I INPUT -s -j DROP
sudo iptables -I INPUT -s -j DROP

Save the IP list as IPtables command so you can redo with.

sudo service iptables save
sudo iptables-save > /etc/rules.v4
sudo iptables-restore < /etc/rules.v4
sudo ip6tables-save > /etc/rules.v6
sudo ip6tables-restore < /etc/rules.v6

Hall Of Fame List (old do not use)

sudo iptables -I INPUT -s,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, -j DROP

Understand That Exposure Can Bring Trouble

After a certain amount of terra bytes, your server becomes a sign post for all types. While most people it provides some value to their travel. Some types of people make their way in the world by making trouble. Lawyers want to turn the internet into a copyright scam so they can make money. Governments and religions want to keep people brainwashed and so are threatened by information because it expands the minds of the people. Some people can use the government to support their private business models.

Each person believes in their crusade but the server may contains millions and billions of pages of information, it is one word in one book in a library of millions of books. The two points of view, the scope are different.

You might need to ban companies that litigate, lawyers, government and police from being able to access the files you have on your computer.

To much exposure and trouble, there are people in the world that have their agenda and they need a muse to act out their agenda. For example legal people and their copyright scam, legal pepple do not care about your copyright it has instead become a must for a worldwide monetary scam. Therefore it may be suitable for your server to ban crawling and robots so that you do not attract mainstream attention. To do this you need to learn the ability to map the IP structure of organizations and use the above method to restrict them from your server. To be on the safe side, you will want to ban all law enforcement agencies including federal agencies, you will want to ban attorney operations especially those used by big corporations like Hollywood, Apple, Google and Microsoft and finally you will want to ban a new type of actor and that is a state backed operation that seek to take down servers that are either in competition with the state backed organizations or hold sensitive information that they do not want available. Some people will never understand the warfare is coming from state backed actors around the world. Google owns 14 million IP address' worldwide and if it were to DDOS a competitor it could do so at will. So you will need to be able to quickly reject their IP address range if you detect such a situation.

Mapping the IP range of an entity takes detective work which is called “enumeration”. You would find out the ASN and do a BGP lookup. However websites have completely automated the process.

(BGP Lookup Tool: ASN,, or you can research SPF of the DNS record using NKlookup, or you will have to do the slow time consuming detective work starting with one address and finding links)

Choosing a Unix/Linux

  1. Lightweight Linux with the best package manager
  2. LAMP
  3. Set up iptables
  4. Email anti-spam and secure configuration

Limiting Concurrent Connections Address

Some people bomb the server with 20 or 50 connections. This limits connections from each IP address to no more than 5 simultaneous connections. This sort of “rations” connections, and prevents crawlers from hitting the site simultaneously.

Do not use instead use specific server facilities such as Apache's mod_qos iptables -I INPUT -p tcp -m connlimit –connlimit-above 5 -j REJECT It blocks people excessively

You can use a command like…

netstat -nat

to monitor IP address and then do the iptables ip ban if you can determine with accuracy that it is a hacker.

Ban Spam Domains

smtpd_sender_restrictions = hash:/etc/postfix/access reject_unauth_destination = hash:/etc/postfix/access

Once this has been added to the, you need to create the /etc/postfix/access file and put entries in it like this:


So on and so forth until all the domains have been entered. Once you have created this file and the permissions are correct run the following command:


postmap hash:/etc/mail/access

and finally restart postfix.


for f in *.rar; do unrar e “$f”; done

Chmod Files or Directories Only

find /path/ -type f -exec chmod 644 {} +
find /path/ -type d -exec chmod 755 {} +

TV and Radio Server

The TV and Radio Ganino promotion for easier success uses Darwin Streaming Server (DSS) by Apple. Darwin Streaming Server (DSS), was the first open sourced RTP/RTSP streaming server. It was released March 16, 1999 and is a fully featured RTSP/RTP media streaming server capable of streaming a variety of media types including H.264/MPEG-4 AVC, MPEG-4 Part 2 and 3GP.

How to install the Darwin media streaming server on Linux (Ubuntu)

Rachael Bond Saturday, 26 June 2010 Linux

The instructions have now been updated for Ubuntu 12.04 LTS. Homepage for DSS is also tested on Linux Mint 14

Ensure that you having the following dependencies installed:

~$ sudo apt-get install gcc g++ make patch

You can download the latest version of the Darwin server package from Apple.

~$ wget
~$ wget

To additional patches are required to use Darwin on Ubuntu and a custom install script.

Once again, if you prefer to download straight to your server use either:

~$ wget
~$ wget

Before installing Darwin create its user and group (hyphen may to be change from – to -):

~$ sudo addgroup -system qtss
~$ sudo adduser -system -no-create-home -ingroup qtss qtss

Next unpack the .tar file:

~$ tar -xvf DarwinStreamingSrvr6.0.3-Source.tar

and apply the two patches:

~$ patch -p0 < dss-6.0.3.patch
~$ patch -p0 < dss-hh-20080728-1.patch

Move into the Darwin directory:

~$ cd DarwinStreamingSrvr6.0.3-Source

and delete the Install file:

~$ rm Install

Download the new Install file:

~$ wget

and change its permissions:

~$ chmod +x Install

For Ubuntu 12.04 it would appear that there is a compiling problem. This can be resolved by editing line 8 in Makefile.POSIX (in DarwinStreamingSrvr6.0.3-Source) and changing it from:

LIBS = $(CORE_LINK_LIBS) -lCommonUtilitiesLib -lQTFileLib


LIBS = $(CORE_LINK_LIBS) -lCommonUtilitiesLib -lQTFileLib -ldl

After this Darwin can be built:

~$ ./Buildit

and install:

~$ sudo ./Install

To allow Darwin to run automatically on boot, download either:

~$ wget

and change its permissions:

~$ chmod +x darwin-streaming-server

Move this file into /etc/init.d:

~$ sudo mv darwin-streaming-server /etc/init.d/darwin-streaming-server

and then update rc.d to autostart it:

~$ sudo update-rc.d darwin-streaming-server defaults

Darwin requires the ports TCP: 554, 1220 and UDP: 6970-6999 open in order to work. If you are using Shorewall edit /etc/shorewall/rules and add these lines:

ACCEPT net fw tcp 554 ACCEPT net fw tcp 1220 ACCEPT net fw udp 6970:6999

Finally direct your browser to http://yourserver:1220 to check that Darwin is working and to enter your admin passwords. If your server is going to be exclusively for RTSP streaming then select port 80 for use by Darwin – otherwise leave blank as this will interfere with Apache.

You are now ready to upload .mp4 and .mov files for streaming…. however, make sure that your .mp4 files have been correctly encoded and hinted.

Broadcasting a professional image ensures success and easier acceptance. It is accessible from any device that has internet anywhere and at anytime globally. Keep broadcasting.

Encoding for DSS - Re-encoding & Hinting

Encoding is key, encoding videos to specifications that server expects. For example encoding video in h264 and audio in aac. You will need to re-encode all files so that they are identical in every way. I use Wondershare Video Converter Ultimate.

The profile is Mp4:

Video Codec: H.263 Audio Codec: AA2 Video Size: Bit Rate 512Kbps (means every user that downloads the stream at 512kps will get smooth video, the server must also be able to stream @ 512kbps to every user.

After re-encoding you must hint the video. The file will not play until you hint the file.

~$ apt-get install gpac

will provide the MP4Box command

~$ MP4Box -hint thefiletohint.mp4

a check on the video will now show the rtp streams section that a non-hinted video does not have.

~$ ffmpeg -i file-we-require-info-on.ext

To extract Audio From A Video Using ffmpeg

~$ ffmpeg -i “whatever.format” -vn -ac 2 -ar 44100 -ab 320k -f mp3 output.mp3

starting the server point your browser to

to access the stream use standard rtp port


to access the mp3 stream

SELinux and Firewall will not let it through. Disable SELinux for this test by editing /etc/SElinux/config and setting it to disabled. /etc/init.d/iptables stop to stop the firewall, later adding the exceptions rather than disabling.

QuickTime Streaming Server 4 and Darwin Streaming Server 4 use the following TCP ports:

  • 80: HTTP
  • 554: RTSP
  • 6970 -9999: used for dynamic (announced) UDP broadcasts
  • 7070: RTSP
  • 8000, 8001: MP3 streaming
  • 10,000-20,000: buffer space
  • 20,000-65,535: static SDP default range (user defined)

the comments relating to chkconfig are important as chkconfig will read them regardless of the hash usually meaning comments are ok to disregard.

$ chkconfig --add myscript 
$ chkconfig --level 2345 myscript on 
$ chkconfig --list | grep myscript 

Editing Playlist With A Command

for f in /usr/local/movies//*.mp3; do echo "\"$f\" 5"; done >> radio.playlist

file being /var/streaming/playlists/radio/radio.playlist

Batch Rename Remove Char

for i in *\]* ; do mv -v "$i" "${i/\]/}" ; done

Create HTML page from directory

for f in *.mp3; do echo "<a href=\"/music/beethoven/$f\">$f</a><br />"; done >> page.txt

Copy Recursive Into One Directory

find . -iname '*.mp3' -type f -exec cp {} target_dir/ \;

PF Rules

pfctl -f /etc/pf.conf

Rename folders to numbers

for i in `find . -type d`; do
mv ${i} $COUNTER;

Remove spaces from directories

find -name "* *" -type f | rename 's/ /_/g

Shell command to bulk change file extensions in a directory (Linux)

1. Change from one extension to another The command below will rename all files with the extension .php4 to .php

for f in *.php4; do mv $f `basename $f .php4`.php; done;

2. Add (append) an extension to all files The command below add the extension .txt to all files in the directory

for f in *; do mv $f `basename $f `.txt; done;

2. Remove (delete) an extension from all files The command below remove the extension .txt from all files in the directory

for f in *.txt; do mv $f `basename $f .txt`; done;

Artists Scripts

#! /bin/bash

for entry in * 
echo "{{:artists:art_via_denmark:$entry?direct&100|}}" 


for a in {0..255}
  for b in {0..255}
    for c in {0..255}
      for d in {0..255}
        ping -c 4 -t 10 $a.$b.$c.$d >> $a.$b.$c.$d
       #echo $a.$b.$c.$d


for a in {0..255}
  for b in {0..255}
    for c in {0..255}
      for d in {0..255}
        echo "====== $a.$b.$c.$d ======" >> $a.$b.$c.$d
        echo "===== Ping =====" >> $a.$b.$c.$d
        ping -c 4 $a.$b.$c.$d >> $a.$b.$c.$d
        echo "===== Nmap =====" >> $a.$b.$c.$d
        nmap -T4 -A -v -p 1-65535
        echo "===== Associated IP =====" >> $a.$b.$c.$d
        dig  $a.$b.$c.$d >> $a.$b.$c.$d
        echo "===== Associated Websites =====" >> $a.$b.$c.$d
        dig -x $a.$b.$c.$d >> $a.$b.$c.$d
        echo "===== Activity =====" >> $a.$b.$c.$d
        echo "Edit page" >> $a.$b.$c.$d

Dokuwiki Permissions

 {DOCUMENT_ROOT}/internet/conf/ is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/pages is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/attic is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/media is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/media_attic is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/media_meta is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/meta is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/cache is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/locks is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/index is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/tmp is not writable by DokuWiki.

Dokuwiki Enwiki

 cat enwiki-latest-pages-articles.xml | ./mwimport | mysql -f -D database -u user -ppassword
 bzcat dewiki-20120603-pages-articles.xml.bz2 | perl | mysql -f -u [USERNAME] -p [DATABASE]

Replace Rename Folder Removing Whitespace

for f in *; do mv "$f" `echo $f | tr ' ' '_'`; done
for i in `find . -type d`; do new_name=`echo $i | tr '[A-Z]' '[a-z]'`; mv $i $new_name ; done
for i in `find . -type f`; do new_name=`echo $i | tr '[A-Z]' '[a-z]'`; mv $i $new_name ; done

Duplicate Concat

:g/^/norm yyp

Yet another one(shorter):


Another one:

:%!sed p

/etc/my.cnf bind-address = skip-external-locking This also prevents MySQL from listening on the external network interfaces; note: the skip-external-locking directive should be used instead of the deprecated skip-locking.

As a first step after basic configuration, we then need to install the default databases, change the password of the MySQL root user (don't take my passwords as an example!) and answer a few questions:

# /usr/local/bin/mysql_install_db [ … ] # /usr/local/bin/mysqld_safe & [ … ] # /usr/local/bin/mysql_secure_installation [ … ] Enter current password for root (enter for none): <Enter> OK, successfully used password, moving on… [ … ] Set root password? [Y/n] Y New password: root Re-enter new password: root Password updated successfully! [ … ] Remove anonymous users? [Y/n] Y … Success! [ … ] Disallow root login remotely? [Y/n] Y … Success! [ … ] Remove test database and access to it? [Y/n] Y - Dropping test database… … Success! - Removing privileges on test database… … Success! [ … ] Reload privilege tables now? [Y/n] Y … Success! [ … ] #

pf.conf ruleset

#### First declare a couple of variables ####
### Outgoing tcp / udp port ####
### 43 - whois, 22 - ssh ###
tcp_services = "{ ssh, smtp, domain, www, https, 22, ntp, 43,ftp, ftp-data}"
udp_services = "{ domain, ntp }"
### allow ping / pong ####
icmp_types = "{ echoreq, unreach }"
#### define tables. add all subnets and ips to block
table <blockedip> persist file "/etc/pf.block.ip.conf"
martians = "{,,,,,,, }"
### admin server ranges ###
adminrange = ""
# connected to internet
ext_if = "em1"  
# connected to vpn / lan
int_if = "em0"
##### ftp proxy
#### Normalization
#scrub provides a measure of protection against certain kinds of attacks based on incorrect handling of   packet fragments
scrub in all
#### NAT and RDR start
#nat-anchor "ftp-proxy/*"
#rdr-anchor "ftp-proxy/*"
# redirect ftp traffic
#rdr pass proto tcp from any to any port ftp -> $proxy port $proxyport
# Drop incoming everything 
block in all 
block return 
# keep stats of outgoing connections
pass out keep state
# We need to have an anchor for ftp-proxy
#anchor "ftp-proxy/*"
# unlimited traffic  for loopback and lan / vpn
set skip on {lo0, $int_if}
# activate spoofing protection for all interfaces
block in quick from urpf-failed
#antispoof is a common special case of filtering and blocking. This mechanism protects against activity from spoofed or forged IP addresses
antispoof log for $ext_if
#Block RFC 1918 addresses
block drop in log (all)  quick on $ext_if from $martians to any
block drop out log (all) quick on $ext_if from any to $martians
# Block all ips
# pfctl -t blockedip -T show
block drop in log (all)  quick on $ext_if from <blockedip> to any
block drop out log (all) quick on $ext_if from any to <blockedip>
# allow outgoing 
pass out on $ext_if proto tcp to any port $tcp_services
pass out on $ext_if proto udp to any port $udp_services
# Allow trace route
pass out on $ext_if inet proto udp from any to any port 33433 >< 33626 keep state
# Allow admin to get into box
pass in on $int_if from $adminrange to any
# Allow incoming ssh, http, bind traffic
# pass in  on $ext_if proto tcp from any to any port 25
pass in on $ext_if proto tcp from any to any port ssh  flags S/SA synproxy state
pass in on $ext_if proto udp from any to any port domain 
pass in on $ext_if proto tcp from any to any port domain flags S/SA synproxy state
pass in on $ext_if proto tcp from any to any port http flags S/SA synproxy modulate state
pass inet proto icmp all icmp-type $icmp_types keep state
## add your rule below ##

AVI to MP4

avconv -i A\ Blade\ in\ the\ Dark.avi -c:v libx264 -c:a copy A\ Blade\ in\ the\ Dark.mp4

Find Files Larger Than A Certain Size

find ./ -xdev -type f -size +500M

Some Useful Linux Unix Commands

Mono In The Background

nohup mono myexe.exe &

SSH Command In The Background Then Exit

tmux attach   #to come back to the command in the next session

Join Multiple MP4 Using GUI

mkvtoolnix-gui  #apt-get install ...

Video Convertor

sudo add-apt-repository ppa:ffmulticonverter/stable
sudo apt-get update
sudo apt-get install ffmulticonverter

Generate Thumbnails From Images

mogrify -resize 200 *.png

Ebay Search All (old)


CCTV recorder

Turn a webcam into a CCTV recorder

streamer -q -c /dev/video0 -f rgb24 -r 24 -t 02:30:00 -o outfile.avi &

BSD Nameserver Bind Reload

rndc reload

Restart SSHD

kill -HUP `cat /var/run/`

Chmod Directories

find ./ -type d -exec chmod 755 {} \;

Chmod Files

find ./ -type f -exec chmod 644 {} \;

Remove ^M


Make Thumbnails

mogrify -resize 80x80 -background white -gravity center -extent 80x80 -format png -quality 75 *.png

Images From PDF

convert -density 900 -depth 8 -quality 100 Voynich_Manuscript.pdf Voynich_Manuscript.png

Convert Directory to MP3

find -name "*.flac" -exec avconv -i {} -acodec libmp3lame -ab 320k {}.mp3 \;

Merge PDF

pdftk *.pdf cat output 1234567.pdf;

Sample Daemon Startup Script On Red Hat Systems

# chkconfig: 2345 20 80
# description: Description comes here....

# Source function library.
. /etc/init.d/functions

start() {
  # code to start app comes here 

stop() {
  # code to stop app comes here 

case "$1" in 
      echo "Usage: $0 {start|stop|restart}"

exit 0

Create Site Mirror

wget -mk -w 20

SCP Various Commands

Copy the file “foobar.txt” from a remote host to the local host

scp /some/local/directory

Copy the file “foobar.txt” from the local host to a remote host

scp foobar.txt

Copy the directory “foo” from the local host to a remote host's directory “bar”

scp -r foo

Copy the file “foobar.txt” from remote host “” to remote host “”


Bzcat & enWiki

bzcat dewiki-20120603-pages-articles.xml.bz2 | perl | mysql -f -u [USERNAME] -p [DATABASE]
cat enwiki-<date>.xml | perl | mysql -f -u<admin name> -p<admin password> --default-character-set=utf8 <database name>

Mount BSD Drive From Linux

sudo mount -t ufs -r -o ufstype=44bsd /dev/sdb1 /mnt/

Copy multiple files and append to end of filename

for f in /etc/*.conf; do cp -v -- "$f" "$f.orig"; done

How to file split at a line number

wc -l 100 file

Create Sitemap (Dirty)

find ./ > sitemap.test

Open in gvim search and replace




Open in vim and run command

 :%s/$/\**##**/g \\

Open in gvim search and replace

  • *##**



Add the open and closing tags

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="">

Vim Search and replace

 %s/<\/loc>/  \r<\/loc>/g

Split With Custom Prefix

 split -l 49999 --additional-suffix=.xml sitemap.test sitemap

Gzip Multiple

 gzip sitemap*.xml

Add Line TO start and end of mutiple file

 for file in sitemap*.xml; do
  sed -i '1i <?xml version="1.0" encoding="UTF-8"?>\r\n<urlset xmlns="">' "$file" &&
  echo '</urlset>' >> "$file"
linux_command_cheat_sheet.txt · Last modified: 2020/02/17 21:02 (external edit)