Deciphering an API documentation

JavaScript on the Web has a lot of APIs to work with. Some of them are fully supported some are still in draft. One of the things I worked with lately is the FileSystemDirectoryReader Interface of the File and Directory Entries API. It defines only one method called readEntries that returns an array containing some number of the directory’s entries. This is draft proposed API and is not supposed to be fully browser compatible. However I’ve tested it almost on all the latest versions of browsers and it works fine on each one except for the Safari browser (I think ;)).

This post will focus on one example that shows that sometimes reading an API documentation can be a little tricky. So the example in the documentation shows a common use of this API where the source of the FileSystemEntry items that we read from are passed with a (drag and) drop event in the browser. The FileSystemEntry can be either a file or a directory. What we want to do is build a file system tree of the dropped item . If the dropped item is a directory then the item is actually FileSystemDirectoryEntry object than defines the createRender method that creates the FileSystemDirectoryReader object on which we will call the readEntries method.

The demo example can be tested in this fiddle. What I want you to do is to drop a directory that contains more than 100 files. If you do that you can notice that the readEntries method returns only the first 100 queued files. That is the main reason for writing this post. The description on the successCallback argument of the readEntries method is a little bit confusing, it says: “A function which is called when the directory’s contents have been retrieved. The function receives a single input parameter: an array of file system entry objects, each based on FileSystemEntry. Generally, they are either FileSystemFileEntry objects, which represent standard files, or FileSystemDirectoryEntry objects, which represent directories. If there are no files left, or you’ve already called readEntries() on this FileSystemDirectoryReader, the array is empty.”

In their example we can see the scanFiles method that reads the items and creates html elements:

function scanFiles(item, container) {
        var elem = document.createElement("li");
        elem.innerHTML = item.name;
        container.appendChild(elem);

        if (item.isDirectory) {
            var directoryReader = item.createReader();
            var directoryContainer = document.createElement("ol");
            container.appendChild(directoryContainer);

            directoryReader.readEntries(function (entries) {
                entries.forEach(function (entry) {
                    scanFiles(entry, directoryContainer);
                });
            });
        }
    }

It seems that the successCallback functions returns the entries partially in a packages of 0 to max 100 items.  If we use this function we will never iterate more that 100 items in the given directory. What we need to do is to decipher this part: “If there are no files left, or you’ve already called readEntries() on this FileSystemDirectoryReader, the array is empty.”.  Translated this into JavaScript code is:

function scanFiles(item, container) {
        var elem = document.createElement("li");
        elem.innerHTML = item.name;
        container.appendChild(elem);

        if (item.isDirectory) {
            var directoryReader = item.createReader();
            var directoryContainer = document.createElement("ol");
            container.appendChild(directoryContainer);

            var fnReadEntries = (function () {
                return function (entries) {
                    entries.forEach(function (entry) {
                        scanFiles(entry, directoryContainer);
                    });
                    if (entries.length > 0) {
                        directoryReader.readEntries(fnReadEntries);
                    }
                };
            })();

            directoryReader.readEntries(fnReadEntries);
        }
    }

The change that we need to apply is to check after the iteration of the entries if the length of the entries array is bigger than 0.  Case that’s the truth then we should call the readEntries method again. If the entries size is zero, then the iteration is finished – all the file system items are iterated.

This fiddle has the improved version of the scan file method that will list all the files in the directory (more that 100), and won’t trick you.

Now finally we can conclude what the part “returns an array containing some number of the directory’s entries” meant. 🙂

Cheers

Dumping MongoDB database

Last couple of months I was wandering through the world of MongoDb, both as a developer and as a sysadmin. I won’t say how happy I’m with it or how much I got disappointed , I will just note that you don’t know how deep or serious you dived into a software product till the moment when you feel the need for a backup. And that day came, and I did something about it.

I’m not so much into installing backup software that will just do the things for me, I always start with the idea that I can do it with my own simple implementation that will fill my needs for saving data or doing something else. Mongo has become a huge software/database monster and there are a lot of different approaches regarding this question. Off course the right way for you depends of several factors. Some of them are: the infrastructure of the Mongo deployment, the importance of the data, the quantity of the data, the performance factor etc.

For my needs that are associated with a single cluster instance of Mongo database without replication or sharding I realized that using the “classical” dumping method will fulfil .
One of the things I like about Mongo is its nicely done documentation. A lot of information about backuping MongoDb can be found on its official site:

http://docs.mongodb.org/manual/core/backups/

The documentation is like a guide showing you the different ways of backuping. Generally there are 3 different approaches: doing file system snaphosts, using mongodump command and using MongoDB Management Service.

For my dumping approach I wrote simple and primitive bash shell script that you can use for local backups or as a push towards the idea how to backup mongo data. It is oriented around dumping databases. Mongo data can be dumped as whole instance, whole collection or part collection and as a database. Here is the scirpt, the product of it are BSON and metadata JSON files at the designated directory.

#!/bin/bash

#
# GNRP LICENSE: Script licensed by GOD
#
# @author: Igor Ivanovski <igor at genrepsoft.com>
#
# March, 2015
#

#
# Filesystem directory variables
#

# Now in format: YYYY-MM
DATEYYMM=`date +"%Y-%m"`

# Now in format: DD
DATEDD=`date +%d`

# Backup directory 
BACKUPDIR="/opt/backup/mongodumps"

# Daily backup directory
BACKUPDIRDAILY="$BACKUPDIR/$DATEYYMM/$DATEDD/"

#
# List of databases to backup
#
DBs="
admin
someDb
";

#
### Mongo Server Setup ###
#

# Don't forget to add adequate roles if you are using authenticatio
#use someDb db.createUser({user:"backup",pwd:"pwd",roles:["readWrite"]})
#use admin db.createUser({user:"backup",pwd:"pwd",roles:["backup"]})

# Mongo backup username  
MUSER="backup"

# Mongo backup password
MPASS="pwd"

# Mongo HOST  name
MHOST="localhost"

# Mongo PORT number 
MPORT="27017"

# Mongo dump binary 

# Check if mongodump is installed
STATUS=0
[ "Y`which mongodump`" != "Y" ] && STATUS=$? || STATUS=$?
[ $STATUS == 1 ] && echo "No mongodump found. Exiting"; exit 1 
MONGODUMP=`which mongodump`;

# Starting to dump databases one by one
if [ "X$DBs" != "X" ]; then
    for db in $DBs
        do
            echo "Backing up database $db"
            $MONGODUMP --host $MHOST --port $MPORT --username $MUSER --password $MPASS --out $BACKUPDIRDAILY --db $db
        done

else
        echo "All listed mongo databases dumped. Bye"
fi

Thank you for reading me, here is a bonus from me for swinging 😉

Deploying WordPress over Nginx and PHP-FPM

Welcome random web traveler. As the title suggests this post will deal with plain production ready examples of Nginx configuration (plus php-fpm) for WordPress site.

Before we move to the real thing, note that this examples are tested on both Debian 7 and CentOS 7 OSs. Since I don’t want to dive into setting up this servers for WordPress, I’m just giving your refined nginx configs that may be found useful. However the steps for building up WordPress on Linux are pretty simple:

– Installing Nginx from package repository or compiling it from scratch;

– Installing php5, php-mysql, php-fpm and other php libraries if needed (like php-gd);

– Installing MySql or Maria-DB;

– And off course setting up php, nginx and mysql/mariadb.

OK lets start with the nginx server configuration.  Found in /etc/nginx/nginx.conf

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;

events {
 worker_connections 768;
 # multi_accept on;
}

http {

 ##
 # Basic Settings
 ##

 sendfile on;
 tcp_nopush on;
 tcp_nodelay on;
 keepalive_timeout 65;
 types_hash_max_size 2048;
 server_tokens off;

 client_max_body_size 100m;

 client_header_buffer_size 1k;
 large_client_header_buffers 8 8k;

 # server_names_hash_bucket_size 64;
 # server_name_in_redirect off;

 include /etc/nginx/mime.types;
 default_type application/octet-stream;

 ##
 # Logging Settings
 ##

 access_log /var/log/nginx/access.log;
 error_log /var/log/nginx/error.log;

 ##
 # SSL settings
 ##
 ssl_session_cache shared:SSL:10m;
 ssl_session_timeout 10m;

 ##
 # Gzip Settings
 ##

 gzip on;
 gzip_disable &quot;msie6&quot;;

 # gzip_vary on;
 # gzip_proxied any;
 # gzip_comp_level 6;
 # gzip_buffers 16 8k;
 # gzip_http_version 1.1;
 # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

 ##
 # nginx-naxsi config
 ##
 # Uncomment it if you installed nginx-naxsi
 ##

 #include /etc/nginx/naxsi_core.rules;

 ##
 # nginx-passenger config
 ##
 # Uncomment it if you installed nginx-passenger
 ##

 #passenger_root /usr;
 #passenger_ruby /usr/bin/ruby;

 ##
 # Virtual Host Configs
 ##

 include /etc/nginx/conf.d/*.conf;
 include /etc/nginx/sites-enabled/*;
}

Pretty straightforward, right?

Next the Nginx magic behind WordPress. This example assumes that we want the administration of WordPress to go through SSL (https protocol). File: example.conf

server {
 ## Your website name goes here.
 server_name example.com www.example.com;
 listen 80;
 ## Your only path reference.
 root /opt/wordpress/;
 ## This should be in your http block and if it is, it's not needed here.
 index index.php;
 # port_in_redirect on;

 access_log /var/log/nginx/example_log;
 error_log /var/log/nginx/example_err warn;

 # rewrite all 403 to 404
 error_page 403 = 404;

 location = /favicon.ico {
 log_not_found off;
 access_log off;
 }

 location = /robots.txt {
 allow all;
 log_not_found off;
 access_log off;
 }

 # deny all access to .dot files
 location ~ /\. { access_log off; log_not_found off; deny all; }

 # deny access to files starting with a $, these are usually temp files
 location ~ ~$ { access_log off; log_not_found off; deny all; }

 location / {
 # This is cool because no php is touched for static content.
 # include the &quot;?$args&quot; part so non-default permalinks doesn't break when using query string
 try_files $uri $uri/ /index.php?$args;
 }

 location ~ /wp-admin/admin-ajax\.php {
 try_files $uri =404;

 # With php5-fpm
 fastcgi_intercept_errors on;
 fastcgi_pass 127.0.0.1:9000;

 fastcgi_index index.php;
 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
 include fastcgi_params;

 }

 # Request to wp-login to go through HTTPS protocol
 location ~ /(wp-admin/|wp-login\.php) {
 return 301 https://$host$request_uri;
 #rewrite /wp-(admin|login) $scheme://$host$request_uri/ permanent;
 }

 location ~ \.php$ {
 try_files $uri =404;
 #NOTE: You should have &quot;cgi.fix_pathinfo = 0;&quot; in php.ini

 # With php5-fpm
 fastcgi_intercept_errors on;
 fastcgi_pass 127.0.0.1:9000;

 fastcgi_index index.php;
 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
 include fastcgi_params;

 }

 location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
 expires max;
 log_not_found off;
 }

 error_page 500 502 503 504 /50x.html;
 location = /50x.html {
 root /usr/share/nginx/www;
 }

}

server {
 listen 443 ssl;
 server_name example.com www.example.com;
 index index.php;

 root /opt/wordpress/;

 # Logs
 access_log /var/log/nginx/example_ssl_access.log;
 error_log /var/log/nginx/example_ssl_error.log info;

 ssl on;
 ssl_certificate /etc/ssl/certs/example.com.crt;
 ssl_certificate_key /etc/ssl/keys/example.com.key;

 ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
 ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
 ssl_prefer_server_ciphers on;

 # Process requests to wp-admin/* and wp-login.php
 location ~ /wp-(admin|login|content|includes) {

 location ~ \.php$ {
 try_files $uri =404;
 #fastcgi_split_path_info ^(.+\.php)(/.+)$;

 # With php5-fpm
 fastcgi_intercept_errors on;
 fastcgi_pass 127.0.0.1:9000;

 fastcgi_index index.php;
 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
 fastcgi_param HTTPS on;
 include fastcgi_params;

 }
 }

 # redirect everyone back to the non-ssl page
 location / { return 301 http://$host$request_uri; }

 location ~ !^(/wp-admin/|wp-login\.php) { return 301 http://$host$request_uri; }

 # rewrite all 403 to 404
 error_page 403 = 404;

 # deny all access to .dot files
 location ~ /\. { access_log off; log_not_found off; deny all; }

 # deny access to files starting with a $, these are usually temp files
 location ~ ~$ { access_log off; log_not_found off; deny all; }

 # keep logs clean by not logging access to favicon.
 location = /favicon.ico { access_log off; log_not_found off; }

 # keep logs clean by not logging access to robots.txt
 location = /robots.txt { access_log off; log_not_found off; }

}

FastCGI params that are defined in Nginx (/etc/nginx/fastcgi_params)


fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;

fastcgi_param SCRIPT_FILENAME $request_filename;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;

fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx/$nginx_version;

fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;

fastcgi_param HTTPS $https;

# PHP only, required if PHP was built with --enable-force-cgi-redirect
fastcgi_param REDIRECT_STATUS 200;

You can see that 2 params are overwritten in the example.conf.

Blended with Nginx we use PHP FastCGI Process Manager or PHP-FPM. I like to start it like a daemon. For that reason we can use init script installed at /etc/init.d/php-fpm . Btw I borrowed it.


#!/bin/sh
### BEGIN INIT INFO
# Provides: php-fpm php5-fpm
# Required-Start: $remote_fs $network
# Required-Stop: $remote_fs $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: starts php5-fpm
# Description: Starts PHP5 FastCGI Process Manager Daemon
### END INIT INFO

# Author: Ondrej Sury &lt;ondrej@debian.org&gt;

PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC=&quot;PHP5 FastCGI Process Manager&quot;
NAME=php5-fpm
DAEMON=/usr/sbin/$NAME
DAEMON_ARGS=&quot;--fpm-config /etc/php5/fpm/php-fpm.conf&quot;
PIDFILE=/var/run/php5-fpm.pid
TIMEOUT=30
SCRIPTNAME=/etc/init.d/$NAME

# Exit if the package is not installed
[ -x &quot;$DAEMON&quot; ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] &amp;&amp; . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (&gt;= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

#
# Function to check the correctness of the config file
#
do_check()
{
[ &quot;$1&quot; != &quot;no&quot; ] &amp;&amp; $DAEMON $DAEMON_ARGS -t 2&gt;&amp;1 | grep -v &quot;\[ERROR\]&quot;
FPM_ERROR=$($DAEMON $DAEMON_ARGS -t 2&gt;&amp;1 | grep &quot;\[ERROR\]&quot;)

if [ -n &quot;${FPM_ERROR}&quot; ]; then
echo &quot;Please fix your configuration file...&quot;
$DAEMON $DAEMON_ARGS -t 2&gt;&amp;1 | grep &quot;\[ERROR\]&quot;
return 1
fi
return 0
}

#
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test &gt; /dev/null \
|| return 1
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
$DAEMON_ARGS 2&gt;/dev/null \
|| return 2
# Add code here, if necessary, that waits for the process to be ready
# to handle requests from services started subsequently which depend
# on this one. As a last resort, sleep for some time.
}

#
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon --stop --quiet --retry=QUIT/$TIMEOUT/TERM/5/KILL/5 --pidfile $PIDFILE --name $NAME
RETVAL=&quot;$?&quot;
[ &quot;$RETVAL&quot; = 2 ] &amp;&amp; return 2
# Wait for children to finish too if this is a daemon that forks
# and if the daemon is only ever run from this initscript.
# If the above conditions are not satisfied then add some other code
# that waits for the process to drop all resources that could be
# needed by services started subsequently. A last resort is to
# sleep for some time.
start-stop-daemon --stop --quiet --oknodo --retry=0/30/TERM/5/KILL/5 --exec $DAEMON
[ &quot;$?&quot; = 2 ] &amp;&amp; return 2
# Many daemons don't delete their pidfiles when they exit.
rm -f $PIDFILE
return &quot;$RETVAL&quot;
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
#
# If the daemon can reload its configuration without
# restarting (for example, when it is sent a SIGHUP),
# then implement that here.
#
start-stop-daemon --stop --signal USR2 --quiet --pidfile $PIDFILE --name $NAME
return 0
}

case &quot;$1&quot; in
start)
[ &quot;$VERBOSE&quot; != no ] &amp;&amp; log_daemon_msg &quot;Starting $DESC&quot; &quot;$NAME&quot;
do_check $VERBOSE
case &quot;$?&quot; in
0)
do_start
case &quot;$?&quot; in
0|1) [ &quot;$VERBOSE&quot; != no ] &amp;&amp; log_end_msg 0 ;;
2) [ &quot;$VERBOSE&quot; != no ] &amp;&amp; log_end_msg 1 ;;
esac
;;
1) [ &quot;$VERBOSE&quot; != no ] &amp;&amp; log_end_msg 1 ;;
esac
;;
stop)
[ &quot;$VERBOSE&quot; != no ] &amp;&amp; log_daemon_msg &quot;Stopping $DESC&quot; &quot;$NAME&quot;
do_stop
case &quot;$?&quot; in
0|1) [ &quot;$VERBOSE&quot; != no ] &amp;&amp; log_end_msg 0 ;;
2) [ &quot;$VERBOSE&quot; != no ] &amp;&amp; log_end_msg 1 ;;
esac
;;
status)
status_of_proc &quot;$DAEMON&quot; &quot;$NAME&quot; &amp;&amp; exit 0 || exit $?
;;
check)
do_check yes
;;
reload|force-reload)
log_daemon_msg &quot;Reloading $DESC&quot; &quot;$NAME&quot;
do_reload
log_end_msg $?
;;
reopen-logs)
log_daemon_msg &quot;Reopening $DESC logs&quot; $NAME
if start-stop-daemon --stop --signal USR1 --oknodo --quiet \
--pidfile $PIDFILE --exec $DAEMON
then
log_end_msg 0
else
log_end_msg 1
fi
;;
restart)
log_daemon_msg &quot;Restarting $DESC&quot; &quot;$NAME&quot;
do_stop
case &quot;$?&quot; in
0|1)
do_start
case &quot;$?&quot; in
0) log_end_msg 0 ;;
1) log_end_msg 1 ;; # Old process is still running
*) log_end_msg 1 ;; # Failed to start
esac
;;
*)
# Failed to stop
log_end_msg 1
;;
esac
;;
*)
echo &quot;Usage: $SCRIPTNAME {start|stop|status|restart|reload|force-reload}&quot; &gt;&amp;2
exit 1
;;
esac

:

This script assumes that we have one main php-fpm.conf file where we define pid file, log file, pools that we will use etc. Every change we do to php configuration like max_upload_size and else can be applied by reloading of this daemon.

That’s it my dearest. Don’t forget that WordPress requires additional settings for working with SSL too. Any questions or suggestions, please write.

Hopes I saved a little precious time of yours 😉

Cool usage of TimeCategory in Groovy

Groovy, the programming language based on JVM implements a feature called Categories. It is originally borrowed from Objective-C . Simple explanation for this feature can be the ability to implement new methods in existing classes without modifying their original code which in some way is injecting new methods through a Category class. For more information official documentation can be found here .

Rather interesting for me was playing with the TimeCategory class for writing a short and easy script for fixing some datetime columns in database. This class offers a convenient way of Date and Time manipulation.

General syntax for categories is the following:

use ( Class ) {
// Some code
}

Concrete usage of TimeCategory:

use ( TimeCategory ) {
// application on numbers:
println 1.minute.from.now
println 10.hours.ago
// application on dates
def someDate = new Date()
println someDate - 3.months
}

Seems weird? From when Integer has months, minutes, hours etc. methods ? Well it still doesn’t have any of that, however those methods are dynamically added with the TimeCategory use.

If you are interested how is this possible I suggest you to go through TimeCategory API and source code if possible. Also this forum post can be useful for deeper understanding of the groovy magic.

And last but not least, an example groovy script for your pleasure.


@GrabConfig(systemClassLoader=true)
@Grab(group='mysql', module='mysql-connector-java', version='5.1.27')

import groovy.time.TimeCategory
import java.sql.Timestamp

sql = groovy.sql.Sql.newInstance(
"jdbc:mysql://hostname:3306/DB_name?autoReconnect=true",
"user",
"password",
"com.mysql.jdbc.Driver")

def rows= [:]

// Select Data
sql.eachRow("select * from Table_Name"){
def impDates = new ImportedDates() // This is some custom Class found in the same package/directory if script
impDates.dateColumn = it.dateColumn

if(impDates.dateColumn!=null){
use(TimeCategory){
impDates.dateColumn = impDates.dateColumn - 1.day // Shift dateColumn for one day backwards in time
}
}

rows.put(it.UID,impDates) // Put private key and ImportedDate object in Map

}

// Update Data
rows.each {row-&gt;
ImportedDates id = row.value
// Check if value is different from null, if it is convert it to Timestamp(we use datetime column in db) and execute update query
dateColumn  = null
if(id.dateColumn) dateColumn = new Timestamp(id.dateColumn.getTime())

// Actual update query
sql.executeUpdate('update Table_Name set dateColumn = ? ' +
'where UID like ?',
[dateColumn, row.key.toString()])

}

Cheers.

Remove old backups with Bash Shell

Hello folks, I present you a bash script I wrote recently. It’s purpose  is deleting ‘old’ directories  marked by time period expressed in days. The script is simple and at its bases uses the command find to locate directories that we want to remove.

The general thinking on which this script is based are directories with backups categorized by some logical distinction and retention period for the second level of directories that are sorted by some other means like let’s say date. The script run by cron job searches the second level of directories and deletes the one older than some number of days n.

Example structure:

/mnt/backup/apps/2014-06-20/*

/mnt/backup/dbs/2014-06-1/*

/mnt/backup/logs/2014-06-1/*

 

If we want to exclude some of the directories in /mnt/backup , for example /mnt/backup/logs we can do that by putting the path into the variable EXCLUDED_DIRS. Note that only the directories from depth 1 can be excluded with this implementation (/mnt/backup/<> = OK , /mnt/backup/logs/<> != OK ).

EXCLUDED_DIRS=”/mnt/backup/logs /some/other/etc”

 

USAGE of the script:

<script_name> {arg1:/path/to/folder} {arg2:n days}  

or

./deleteOldDirs.sh         /mnt/backup           60

 

Note on find. For checking the timestamp of the directory I use ctime like option from find.  When this argument is used ,  find checks for the iNode pointing to the file/folder and collects information (like permissions, owner, etc) from which can be determinated when it was modified. In other hand if we want to check modification of the data in the backup itself we can use the argument mtime instead.

 

The script , beware of removing yourself.

May the force be with you !

#!/bin/bash

#
# GNRP LICENSE: Script licensed by GOD
#
# @author: Igor Ivanovski <igor at genrepsoft.net>
#
# July, 2014
#

#
# *** VARIABLES
#

PATH=$1 # First argument should be PATH of the backup folder
DAYS=$2 # Secound argument should be the max number of days we want to keep backups

GREP="/bin/grep"
MOUNT="/bin/mount"
FIND="/usr/bin/find"
RM="/bin/rm"

PATH_STATUS=0
MOUNT_STATUS=0

EXCLUDED_DIRS="/home/igor/Documents /home/igor/Programs"

#
# *** CHECKERS
#

if [ $# -eq 0 ]; then
    echo "No arguments provided"
    exit 1
fi

if [ -z $PATH ] || [ -z $DAYS ]; then
    echo "Not all arguments provided"
    exit 1
fi

#
# *** FUNCTIONS
#

function check_if_path_exists {

	[ ! -d $1 ] && echo "Path is invalild" || PATH_STATUS=1
}

function check_mount {
	 if [ -n "$($GREP $1 /etc/fstab)" ]; then
                echo "Mounting point exists..."
		if [ ! -n "$($MOUNT -l | $GREP $1)" ]; then
		    echo "...but not mounted."
		    echo "Mounting now."
                    $MOUNT $1
                    [ $? -eq 0 ] && { echo "Mounting was successfull!"; MOUNT_STATUS=1; } || echo "Can't mount";
		fi
         else
                echo "Mounting point doesn't exist"
         fi
}

# Excludes only if parent directory fits
function check_if_parent_dir_excluded { 
            for dir in $EXCLUDED_DIRS; do
		size=${#dir}
	        check=${1:0:size}
		#echo "$dir : ${#dir} == $check : ${#check}"
		if [ $dir ==  $check ]; then 
			echo "1"
		fi
	    done
}

function find_dirs {
	$FIND $PATH -mindepth 2 -maxdepth 2 -type d -ctime +$DAYS -print0
}

function print_dirs {
	while read -r n; do
                 result=$(check_if_parent_dir_excluded $n)
		 [ -z $result ] && printf '%q\n' "$n" || echo "Found EXCLUDE_DIR: $n" 
	done < <($FIND $PATH -mindepth 2 -maxdepth 2 -type d -ctime +$DAYS -print) # Use NL delimiter
}

function del_dirs {
        while read -r -d '' n; do
                 printf '%q\n' "$n"
                 result=$(check_if_parent_dir_excluded $n)
                 if [ -z $result ]; then
			 echo "Directory will be deleted now..."
			 $RM -rf $n
			 [ $? -eq 0 ] && echo "Gone!" || echo "Some error occured!"
		 else	
	 		echo "Found EXCLUDE_DIR: $n continue..." 
		 fi
        done < <($FIND $PATH -mindepth 2 -maxdepth 2 -type d -ctime +$DAYS -print0) # Use NUL delimiter
}

#
# *** MAIN
#

check_if_path_exists $PATH
[ $PATH_STATUS -eq 0 ] && check_mount $PATH
[ $MOUNT_STATUS -eq 1 ] && check_if_path_exists $PATH
[ $PATH_STATUS -eq 1 ] &&  del_dirs || { echo "Bye."; exit 0; }

#
# *** END
#