Settler can be used for defining application wide settings in Rails. Settings are loaded from a YAML file and stored in the database using ActiveRecord to allow users to update settings on the fly. The YAML configuration allows you to not only specify defaults, but setting value validations and typecasts as well!

Settler originated out of fact that I found that some common features were missing from to existing solutions like binarylogic’s settingslogic and Squeegy’s rails-settings:

• I was looking for something that would enable me to define settings in a file, but could be updated afterwards by users of a given system.
• I wanted to enable users to change settings and therefore had to make sure that these changes would be valid. It should therefore be possible to add validations to settings.
• Instead of being a simple key-value store, I did not want to cast a setting to its appropriate type (string, bool, integer or float) every time I used a certain setting. The configuration should therefore allow me to specify the data type of the setting, which should be properly typecasted every time I used it.

As I found that the aforementioned gems had their own strengths, I decided to use their functionality and combine it into one gem. The added value comes with the validation and typecasting functionality. Please see the Github project page for more info!

dynamic_attributes is a gem that lets you dynamically specify attributes on ActiveRecord models, which will be serialized and deserialized to a given text column. Think of it as having some document-database functionality in Rails while just using a relational database.

Example:

    >> dm = DynamicModel.new(:field_test => 'I am a dynamic attribute')
      +-------+-------------+--------------------------------------------+
      | title | description | dynamic_attributes                         |
      +-------+-------------+--------------------------------------------+
      |       |             | {"field_test"=>"I am a dynamic_attribute"} |
      +-------+-------------+--------------------------------------------+
    >> dm.field_test
      => "I am a dynamic_attribute"
    >> dm.field_test2
      NoMethodError: undefined method `field_test2'
    >> dm.field_test2 = 'I am too!'
      => 'I am too!'
    >> dm.field_test2
      => 'I am too!'
    >> dm.save
      +-------+-------------+------------------------------------------------------------------------+
      | title | description | dynamic_attributes                                                     |
      +-------+-------------+------------------------------------------------------------------------+
      |       |             | {"field_test2"=>"I am too!", "field_test"=>"I am a dynamic_attribute"} |
      +-------+-------------+------------------------------------------------------------------------+

I used this gem in our CMS to add non-standard attributes to upcoming events. Events usually have a start time, location and description, but some events needed some more attributes, like the name of the presenter or a case number.

A cool program that acts as a HTTP proxy for Safari is called GlimmerBlocker. Besides being a great adblocker, it supports keyword handling by passing the text entered in the Safari address bar through a list of filters. Some keywords that are common to Yubnub users are already defined, like for example ‘gim’ for google image search or ‘y’ for a Yahoo search.

It was an easy step to add a new filter that passes commands to Yubnub instead. This is by far the most elegant solution I found to have Yubnub integration into Safari that doesn’t hack Safari and works for any version of Safari.

Installation

The installation consists of installing GlimmerBlocker and defining one simple filter. Here are the steps to take:

• Download and install GlimmerBlocker
• Go to the GlimmerBlocker Preferences Pane (listed in the ‘Other’ section of your System Preferences
• Check ‘Activate GlimmerBlocker’ in the Setup tab
• Go to the Filters tab and create a new filter. Name the new filter ‘YubNub’
• Select the new filter by clicking on the name and enable it by checking the checkbox next to it
• In the bottom part of the pane, create a new rule (OR paste the rule XML described in the next paragraph!). Get it to look like the following screenshot:
  
  The keyword used is ‘.*’ (dot star), meaning that this will handle any keyword that is detected. Since it will accept any keyword, the priority of the rule is the lowest to give precedence to rules with more specific keywords.
• On the ‘expansion’ tab, which appears when ‘Use javascript expansion’ is selected in the previous pane, copy-paste the following script:
  gb.url = "http://www.yubnub.org/parser/parse?command=" + gb.keyword + "+" + gb.encodedTerm;. It should now look like this:
  
• Save the rule.
• Try it out by entering some YubNub commands in the Safari address bar! Safari doesn’t have to be restarted for new rules to be applied, so you can tinker with it very easily. Check out the YubNub website for some example commands!

YubNub Rule XML

A really cool feature of GlimmerBlocker is that rules can be shared. By pasting the following XML into the rules table of the YubNub filter, you can skip all rule configuration steps described above!

<?xml version="1.0" encoding="UTF-8"?>
<glimmerblocker-rules>
    <rule priority="1" type="keyword" keyword=".*" keyword-type="regexp">
        <comments><![CDATA[Send YubNub commands]]></comments>
        <keyword language="js" version="1" keyword-uses-js="1"><![CDATA[gb.url = "http://www.yubnub.org/parser/parse?command=" + gb.keyword + "+" + gb.encodedTerm;]]></keyword>
    </rule>
</glimmerblocker-rules>

The solution to logging what a user is actually doing can be achieved with Stackable VFS Modules, which is available since Samba 3. Unfortunately, the link there does not describe the full_audit module, which I highly recommend using instead of audit or extd_audit. The reason for this is that I couldn’t get those modules to log simple things like a file upload by a user, unless I chose VFS log level 10.

Using the Full_audit VFS Module

If you’re running Debian unstable like I do, then full_audit is included when installed from the APT. To find out which modules you have, take a look in /usr/lib/samba/vfs. When you’re sure you have the module, configure it as follows in smb.conf:

    vfs objects = full_audit

    full_audit:prefix = %u|%I|%m|%S
    full_audit:success = mkdir rename unlink rmdir pwrite
    full_audit:failure = none
    full_audit:facility = LOCAL7
    full_audit:priority = NOTICE

Let’s go through it one line at a time.

• vfs objects: we’d like to use the full_audit module.
• full_audit:prefix: Every line that full_audit outputs will be prefixed by this line, in which you can use Samba variables. This line will prefix the username, IP, machine name and share name, separated by pipes.
• full_audit:success: This specifies which actions will actually be logged when it has successfully been completed. unlink is in this case a delete action and pwrite is an upload action.
• full_audit:failure: Specifies which actions should be logged, but which have resulted in a failure. Since a failure will often mean that nothing has been changed, I found that it is not interesting to log any of these actions.
• full_audit:facility: By default, full_audit will only write to the system syslog, but you can specify a different ‘syslog facility’ to write all output to a different log file. Custom syslog facilities should be named local[number], where number is a number between 0 and 7 (don’t ask me why syslogd doesn’t support any name). I’ll get back on this later.
• full_audit:priority: This line sets the severity of the log messages that are generated, like ‘notice’, ‘info’, ‘warning’, ‘debug’, ‘alert’. There are probably more, but these are the most well-known ones.

Creating a Syslogd Facility

To specify a custom log file to which full_audit should write, you should create a new syslogd facility. A facility can be described in syslog.conf. Since I had chosen the facility local7, I can add that facility to the configuration like this:

local7.*                        /var/log/samba/log.audit

This line means that all log messages of facility local7 will be written to /var/log/samba/log.audit. The star is needed to say that I’d like to log messages of any severity to the same log file. Finally, restart the syslogd daemon: /etc/init.d/sysklogd restart


Final Words

• When things don’t seem to be working, ensure you have restarted/reloaded syslogd and samba.
• If there is anything bad you could say about full_audit, then it would be that it can’t output log messages to the log file specified in smb.conf. I always found it very useful that Samba could log by machine name by specifying log file = /var/log/samba/log.%m, but full_audit cannot use this. If you find a way though, please let me know!
• Other references: A blog in a language I can’t read where I took the configuration part from: Monitoring Aktivitas Samba

Output Example

Here is an example of what the configuration just explained generates:

Aug 10 11:52:52 rhino smbd_audit: moiristo|123.45.67.89|moiristo|moiristo|unlink|ok|public/Upload/hypnotoad.gif
Aug 10 11:52:59 rhino smbd_audit: moiristo|123.45.67.89|moiristo|moiristo|pwrite|ok|public/Upload/hypnotoad.gif
Aug 10 11:53:41 rhino smbd_audit: moiristo|123.45.67.89|moiristo|moiristo|rename|ok|public/Upload/hypnotoad.gif|public/Upload/hypnotoads.gif
Aug 10 11:53:51 rhino smbd_audit: moiristo|123.45.67.89|moiristo|moiristo|rename|ok|public/Upload/hypnotoads.gif|public/Upload/hypnotoad.gif

def all_letters(str)
	# Use 'str[/[a-zA-Z]*/] == str' to let all_letters
	# yield true for the empty string
	str[/[a-zA-Z]+/]  == str
end

Analogous, we can do this with digits as well:

def all_digits(str)
	str[/[0-9]+/]  == str
end

def all_letters_or_digits(str)
	str[/[a-zA-Z0-9]+/]  == str
end

Of course, it’s even better if you make these methods a core extension of String itself

Knowing of the function string.succ in Ruby and the ability to specify string ranges, I figured that something similar would also exist in Java. After browsing the webs for hours, I didn’t find anything however. Moreover, even C# with all its extra convenience methods doesn’t come with this (see also Mack Allen’s post here)!

The code below is my stab at creating string.succ in Java. It (mostly) does exactly the same thing as the Ruby version. Mostly I say, try for example the range ("test1".."test10") in irb and Successor.range("test1","test10",false) in a java console.

See the main() function for examples!

Update 5-6-2009: Made it a lot shorter by getting rid of the helper classes, which turned out to be overkill.

/** 
 * Successor.java: A utility class for generating string successors.
 *
 * Copyright (c) 2009, R de Lange
 * 
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * as published by the Free Software Foundation; either version 3
 * of the License, or (at your option) any later version.
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 */
import java.util.*;
import java.io.*;

/**
* The Successor utility class can be used to find string successors or
* to generate lists of successive strings.
* 
* @author    R de Lange
* @created   June 5, 2009
*/
public class Successor {
	
	/**
	*	Main class to test Successor. Examples taken from Ruby string#succ.
	*	
	*	Call						should yield
	*	Successor.succ("abcd")      #=> "abce"
	*	Successor.succ("THX1138")   #=> "THX1139"
	*	Successor.succ("<<koala>>") #=> "<<koalb>>"
	*	Successor.succ("1999zzz")   #=> "2000aaa"
	*	Successor.succ("ZZZ9999")   #=> "AAAA0000"
	*	Successor.succ("***")       #=> "**+"
	*
	*	- For an interactive console interface for testing Successor.succ(str), specify the -i argument
	*	- To test successor ranges using Successor.range(str), specify the -r argument. Adding an arbitrary 
	*	extra argument (e.g. -r e) will toggle the 'exclusive' boolean for successor ranges.
	*/	
	public static void main(String[] args) throws Exception{
		if(args.length == 0){
			System.out.println("Running Successor examples...");
			System.out.println("abcd #=> " + Successor.succ("abcd"));
			System.out.println("THX1138 #=> " + Successor.succ("THX1138"));
			System.out.println("<<koala>> #=> " + Successor.succ("<<koala>>"));
			System.out.println("1999zzz #=> " + Successor.succ("1999zzz"));
			System.out.println("ZZZ9999 #=> " + Successor.succ("ZZZ9999"));
			System.out.println("*** #=> " + Successor.succ("***"));			
		}
		else {
			BufferedReader in = new BufferedReader(new InputStreamReader(System.in));				
			String command = args[0];
			if(command.equals("-i")) {
				String line = "";
				while(!line.equals("quit")){
					System.out.print("String? ");
					line = in.readLine();
					if(!line.equals("quit")) System.out.println(Successor.succ(line));
				}	
			}
			else if(command.equals("-r")) {
				String left = "", right="";
				while(true){
					System.out.print("Left? ");
					left = in.readLine();
					System.out.print("Right? ");
					right = in.readLine();
					System.out.println(Arrays.deepToString(Successor.range(left, right, args.length == 2).toArray()));
				}	
			}				
		}
	}
	
	/**
	* Returns the successor to str. The successor is calculated by incrementing characters starting from the rightmost 
	* alphanumeric (or the rightmost character if there are no alphanumerics) in the string. Incrementing a digit always 
	* results in another digit, and incrementing a letter results in another letter of the same case. Incrementing 
	* nonalphanumerics uses the system default character set's collating sequence. If the increment generates a 'carry', 
	* the character to the left of it is incremented. This process repeats until there is no carry, adding an additional 
	* character if necessary. 
	*/
	public static String succ(String str){
		char[] cstr = str.toCharArray();
		if(cstr.length == 0) return "";
		
		int letterOrDigitIndex = getLastLetterOrDigitIndex(cstr, cstr.length-1);	
		if(letterOrDigitIndex >= 0)
			return alphanumSucc(cstr, letterOrDigitIndex);
		else {
            // Increment rightmost character
            cstr[cstr.length-1] = (char) (cstr[cstr.length-1] + 1);
            return new String(cstr);
        }
	}
	
	/**
	* Alias of Successor.succ(str)
	*/
	public static String next(String str){ return Successor.succ(str); }
	
	/**
	* Returns a range of string successors as a list. Starting from str1, this method will add all successors	
	* to a list until the successor string equals str2. If the 'exclusive' boolean is true, str2 will be included
	* in the resulting list, otherwise excluded. If the successor str2 is not found after generating half a million
	* successors, the range is assumed to be invalid, in which case it will return an empty list.
	*/
	public static List<String> range(String str1, String str2, boolean exclusive){
		int counter, limit = 500000;
		ArrayList<String> result = new ArrayList<String>();
		
		result.add(str1);
		String currentSuccessor = str1;
		for(counter = 0;counter<=limit && !currentSuccessor.equals(str2); counter++){
			currentSuccessor = Successor.succ(currentSuccessor);
			if((currentSuccessor.equals(str2) && !exclusive)|| !currentSuccessor.equals(str2)) 
				result.add(currentSuccessor);
		}
		if(counter > limit) {
			System.err.println("Range could not be created, level too deep.");
			result.clear();
		}
		return result;	
	}
	
	/**
	* Calculates the successive string for strings that contain alphanumeric characters. 
	* See the description of Successor.succ(str) for more informtionon this function
	*/
	private static String alphanumSucc(char[] cstr, int lastLetterOrDigitIndex){
		char succ = charSucc(cstr[lastLetterOrDigitIndex]);

		int currentIndex = lastLetterOrDigitIndex;
        cstr[currentIndex] = succ;
		
		// Loop while we have a carry
		while(succHasCarry(succ) && lastLetterOrDigitIndex >= 0){
			lastLetterOrDigitIndex = getLastLetterOrDigitIndex(cstr, lastLetterOrDigitIndex-1);		
			if(lastLetterOrDigitIndex >= 0) {
				currentIndex = lastLetterOrDigitIndex;
				succ = charSucc(cstr[currentIndex]);
				cstr[currentIndex] = succ;
			}
		}

		// Check if we still have a carry left
		if(succHasCarry(succ)) cstr = insertCarry(cstr, succ, currentIndex);
		
		// Finished!
		return new String(cstr);
	}
	
	/**
	* Create a new array in which the given carry is inserted at the given index
	* @return A new char array of length cstr.length+1, which contains all values of
	* cstr + the carry char inserted at the given index.
	*/
	private static char[] insertCarry(char[] cstr, char carry, int index){
		char[] result = new char[cstr.length+1];
		System.arraycopy(cstr,0,result,0,index+1);
		result[index] = carry;
		System.arraycopy(cstr,index,result,index+1,cstr.length);
		return result;
	}
	
	/**
	* Determines whether the given char array contains alphanumeric characters
	* @return the highest index at which an alphanumeric character can be found, or -1 if none were found 
	*/
	private static int getLastLetterOrDigitIndex(char[] cstr, int lastIndex){
		int result = -1;
		for(int i=lastIndex;i>=0 && result < 0;i--) {
			if(Character.isLetterOrDigit(cstr[i])) result = i;
		}
		return result;
	}

    /**
     * Determines if a given *successor* character (returned by charSucc(char c)) is a
     * character with a carry.
     *
     * @param c - the successor char
     * @return true if c is '0', 'a' or 'A'.
     */
    private static boolean succHasCarry(char c){
        return c == '0' || c == 'a' || c == 'A';
    }

    /**
     * Determines the successor of the given char
     * @param c - the char
     * @return the successor of char c
     */
    private static char charSucc(char c) {
        char result;

        switch(c) {
            case '9': result = '0'; break;
            case 'z': result = 'a'; break;
            case 'Z': result = 'A'; break;
            default: result = (char) (c + 1);
        }

        return result;
    }
}

Making smbstatus suid root unfortunately didn’t do the trick. Taking a closer look at the error suggested that it got to do with the *.tdb files that samba is using. So, I went to the folder containing these files (for me /var/run/samba/*.tdb). All these database files were chmodded to 644, so only enabled rw permissions to root.

By changing the permissions of messages.tdb to 666 (rw permissions on world, if you find this to devil-ish, 646 would also do the trick ) a normal user can also open the database file and thus everything worked again. Of course, you should keep in mind that this may pose a security risk.

<?
error_reporting(0);

if(isset($_GET['kill'])){
	echo shell_exec("sudo /scripts/smbkill ".$_GET['kill']." 2>&1");
	}

exec("smbstatus -S",$log);
exec("smbstatus -L",$lock);

$pid_us = array();
for($i=3;$i<sizeof($log);$i++)
	if(!trim($log[$i])=="") {
		$users[] = split("[ ]+",$log[$i],4);
		$pid_us[$users[sizeof($users)-1][1]] = $users[sizeof($users)-1][2];  
		}
for($i=3;$i<sizeof($lock);$i++)
	if(!trim($lock[$i])=="") {			
			$line = split("[ ]+",$lock[$i],7);
			$usr = $pid_us[$line[0]];
			if(trim($usr)=="") $usr = $line[0];									
			$locks[$line[0]][] = array($line[0],$usr,substr_replace($line[6],"",-25));	//Remove date (25 chars from the right
		}
	
//die("");

echo "<br />";
echo "<font style=\"font-family: Verdana;\">Connected users:</font>\n";
echo "<br /><br />";
echo "<table>\n";
foreach($users as $user){
	list($service,$pid,$mach,$date) = $user;
	echo "<tr\n>";
	echo "<td><a onclick=\"return confirm('Kick this user?');\" href=\"".$PHP_SELF."?kill=".$pid."\"><img border=\"0\" src=\"remove.gif\" /></a></td>\n";
	echo "<td class=\"user\">".$mach."</td>\n";
	echo "<td class=\"pid\">".$pid."</td>\n";
	echo "<td>".$service."</td>\n";
	echo "<td>".$date."</td>\n";
	echo "</tr>\n";
}
echo "</table>\n\n";

echo "<br /><br />";
echo "<font style=\"font-family: Verdana;\">Locked files:</font>\n";
echo "<br /><br />";
echo "<table>\n";

$cuser = "";
$first = true;
foreach($locks as $l){
	foreach($l as $lck){
		list($pid, $user,$file) = $lck;
		if($user != $cuser){		
			if(!$first){
				echo "<tr>\n";
				echo "<td>&nbsp;</td>\n";
				echo "<td>&nbsp;</td>\n";
				echo "<td>&nbsp;</td>\n";
				echo "</tr>\n";				
				}
			echo "<tr>\n";			
			echo "<td class=\"pid\"><a onclick=\"return confirm('Kick this user?');\" href=\"".$PHP_SELF."?kill=".$pid."\"><img border=\"0\" src=\"remove.gif\" /></a> ".$pid."</td>\n";
			echo "<td class=\"user\">".$user."</td>\n";
			echo "<td>".$file."</td>\n";
			echo "</tr>\n";
			$cuser = $user;
			}
		else {
			echo "<tr>\n";
			echo "<td>&nbsp;</td>\n";
			echo "<td>&nbsp;</td>\n";
			echo "<td>".$file."</td>\n";	
			echo "</tr>\n";				
			}
		$first = false;
		}
}
echo "</table>\n";
?>

Note that you should of course make sure that only you are able to access the page and execute the kill command, so you’ll need to get creative with sudo and htaccess

Image used:

As a solution, I left one computer always running with Thunderbird active, but since I already have a linux server running (terminal access only), I was thinking whether there was something available that was able to sort my messages, which I can alter from the command line and doesn’t involve a graphical frontend.

Imapfilter did the job for me. Although I don’t have any experience in Lua, in which it is written, I managed to successfully sort my mail. You can install it from source, but there is also a Debian package available from the apt (at least, from repository unstable). Below I’ll describe how I configured imapfilter, since there is not much information around on this (there are however some small examples described in a sample config file).

After install, first run imapfilter as a normal user. This will create the directory ~/.imapfilter with appropriate permissions. Enter that directory and create a new file named config.lua. In this file, you should first define some global options, then your IMAP accounts, followed by the filters (rules) that should be applied to the messages in your mailboxes.

The global options are defined in a lua table variable named options. A table in lua is much like a hash in Ruby. I have defined the following options:

  options.timeout = 120
  options.subscribe = true

The meaning of these options is described on the imapfilter homepage.
Definitions of IMAP accounts look like this:

account1 = IMAP {
	   	server = 'imap.example.com',
		username = 'user',
		password = 'pass',
		ssl = 'ssl3'
	}

account2 = IMAP {
	   	server = 'imap2.example.com',
		username = 'user',
		password = 'pass'
	}

As you can see, imapfilter also supports SSL (you need to have OpenSSL installed though). You can define all your IMAP accounts like this.

Next, I define the sorting rules. Note that I mostly move messages from the inbox of an account to a folder of the same account. This is however not necessary, it is also possible to transfer messages from one account to the other. Example 1:

   msgs = account1.INBOX:contain_from('MediaPortal')
   account1.INBOX:move_messages(account1.MediaPortal, msgs)

The first example is simple: find all messages in the INBOX in which the sender contains the name MediaPortal and move those messages to the MediaPortal folder.
Now, a more complex example:

 msgs = 	account2.INBOX:contain_from('Cron') +
		account2.INBOX:contain_from('Log') +
		account2.INBOX:contain_subject('yum')
 account2.INBOX:mark_seen(msgs)
 account2.INBOX:move_messages(account2['Maildir/Admin Notifications'], msgs)

What you need to know here is that the ‘+’ operator means ‘or’. So, I put all messages that are from ‘Cron’, ‘Log’ or have the subject ‘yum’ in the variable msgs, I mark all those messages as read and finally move them to the ‘Admin Notifications folder’. As you can see, this account uses the Maildir directory.

You can debug your configuration by running imapfilter -d or imapfilter -v. To run these rules frequently, I finally had to run this code in daemon mode. To achieve this, I had to enclose all the above code in a function and pass that function in the become_daemon() function:

-- global options

function sortMail()
   -- IMAP account definitions
   -- rules
end

become_daemon(300, sortMail)

Note that I had to include the account definitions in the function; it seems the accounts are also initialized when defined and it wouldn’t work anymore when these definitions were outside the function. The above example will run imapfilter as a daemon and execute the sortMail function every 300 seconds.

UPDATE 03-02-2010: Sometimes, an error may occur while processing the messages, causing imapfilter to terminate. To solve this, I created a function that wraps the sortMail function and ensures errors are ignored:

function sort_mail_and_ignore_errors()
        pcall(sort_mail())
end

daemon_mode(300, sort_mail_and_ignore_errors)

Some final remarks that may save you some time:

• There are problems selecting/searching messages when you’re running an Exchange server. I tried several things to get it working properly, but nothing worked. This could be dependent on the configuration or version of the Exchange server however.
• Be sure you add the IMAP definitions to the function called in daemon mode
• if you don’t know exactly which folders you have or should define, you can add some debugging code to find out:

-- Inbox information
account1.INBOX:check_status()

-- Get and print available mailboxes and folders
mailboxes, folders = account1:list_all()
table.foreach(mailboxes,print)
table.foreach(folders,print)

• Want to filter out spam? This can be done like this:

-- Select all messages marked as spam       
msgs = account1.INBOX:match_header('^.+MailScanner.*Check: [Ss]pam.*$')
-- Throw them away 
account1.INBOX:delete_messages(msgs)

Fore more information and examples, see also the imapfilter homepage: http://imapfilter.hellug.gr