For the dynastie project, I need to load data encoded in UTF-8 with Python minidom XML parser. But when I wrote node.toxml('utf-8') to display the XML tree, I get this error :

UnicodeDecodeError at /generate/1

'ascii' codec can't decode byte 0xc2 in position 187: ordinal not in range(128)

In facts Python thinks that all data in XML tree are in ASCII and try to encode it into UTF-8 (or anything else you supplied). The solution is to use your own writer that will convert all non utf-8 strings in unicode string which can be then re-encoded in every format (like utf-8). This doesn't appears in Python 3 because, in Python 3, all strings are already in unicode. Add the following class to your code :

class UnicodeWriter(codecs.StreamWriter):
    encode = codecs.utf_8_encode

    def __init__(self):
        self.value = u''

    def write(self, object):
        if not type(object) == unicode:
            self.value = self.value + unicode(object, 'utf-8')
        else:
            self.value = self.value + object
        return self.value

    def reset(self):
	self.value = u''

    def getvalue(self):
	return self.value

And our node.toxml('utf-8') becomes :

writer = UnicodeWriter()
node.writexml(writer)
writer.getvalue().encode('utf-8')

A lot of things has been written about nginx and Apache : proxy_pass, proxy_redirect, subdomains... I just want to publish my configuration that is in test but works. This is a response to my requirements : I want nginx to serve the subdomain blog.soutade.fr but transfer all other requests to an Apache server (soutade.fr, www.soutade.fr, indefero.soutade.fr ...).

First step is to install nginx. Nginx current version is 0.7.23 (thanks to debian stable). Then edit /etc/nginx/sites-available/default :

server {
        listen   80 default; ## listen for ipv4

        server_name  soutade.fr *.soutade.fr;

        access_log  /var/log/nginx/soutade.fr.access.log;

        location / {
                proxy_set_header X-Real-IP  $remote_addr;
                proxy_set_header X-Forwarded-For $remote_addr;

                proxy_set_header Host $host;

                resolver         localhost;
                proxy_pass       http://$host:8000;
        }
}

server {
        listen   80; ## listen for ipv4

	server_name  blog.soutade.fr;

        access_log  /var/log/nginx/soutade.fr.access.log;

        location / {
                root   /var/www/blog;
                index  index.html;
	}

        location = /favicon.ico { access_log off; log_not_found off; }
        location ~ /\. { deny all; access_log off; log_not_found off; }
}

The first block tells nginx to redirect all request from soutade.fr and *.soutade.fr to a local Apache server listening on port 8000. The second block creates a special rule for blog.soutade.fr : files will be served by nginx server and no redirection will be applied. We also disable favicon.ico error log and deny serving \.* files. There is a tip in the first block : we need to set up a local DNS server ("resolver localhost" directive). If this option is not set, it will try to do another DNS request to resolve $host address. So we'll simply install bind9 and configure it. Edit /etc/bind/named.conf.local

zone "soutade.fr" {
     type master;
     file "/etc/bind/db.soutade.fr";
};

Finally edit /etc/bind/db.soutade.fr with :

$TTL    604800
@       IN      SOA     soutade.fr. root.soutade.fr. (
                      07082012          ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
;
@       IN      NS      localhost.
@       IN      A       127.0.0.1
*       IN      CNAME   soutade.fr.
@       IN      AAAA    ::1

And restart bind : sudo service bind9 restart. root.soutade.fr. is the mail address of the administrator. Now everything might be ok.

Today a little Python tip. This is not very complex, but not so intuitive due to conversions.

def xor_buffer(buf):
    res = ''
    for i in range(len(buf)):
    	res = res + chr(ord(buf[i]) ^ 0xff)

    return res

At work we're not root in our machines. This simplify the administrator's works and we don't need to be root for usual tasks. If I need to, I can ask my colleague two desks in front of me. But for a special case I was given modprobe access via sudo. I said to him "If I can use modprobe, I'm root !", he replied "No because modprobe only loads modules from /lib/modules, but this is your challenge !".

Challenge accepted and successfully completed ! I'm running an Ubuntu 10.4 LTS with a 2.6.32-41-generic kernel.

Module source

The module was easy to develop (even if it has been a while I havn't do kernel module). It was named beyrouth for "be root", but it's more generic. Indeed you can change UID and GID of any running process !

/*  
 *  beyrouth.c - Change task's UID and GID
 */
#include /* Needed by all modules */
#include /* Needed for KERN_INFO */
#include 
#include 
#include 
#include 

// Copied from linux/cred.h
#define __task_cred(task) \
  ((const struct cred *)(rcu_dereference((task)->real_cred)))

// Copied from linux/cred.h
#define get_task_cred(task)                             \
  ({							\
    struct cred *__cred;				\
    rcu_read_lock();					\
    __cred = (struct cred *) __task_cred((task));	\
    get_cred(__cred);					\
    rcu_read_unlock();					\
    __cred;						\
  })

int pid = -1;
int uid = -1;
int gid = -1;

module_param(pid, int, 0);
module_param(uid, int, 0);
module_param(gid, int, 0);

int init_module(void)
{
  struct cred *_cred;
  struct pid* _pid;
  struct task_struct* task;

  printk(KERN_ERR "Hello world\n");

  if (pid == -1)
    {
      printk(KERN_ERR "PID is missing\n");
      return -2;
    }

  _pid = find_get_pid(pid);
  if (!_pid)
    {
      printk(KERN_ERR "PID not found\n");
      return -3;
    }

  task = pid_task(_pid, PIDTYPE_PID);
  if (!task)
    {
      printk(KERN_ERR "Task not found\n");
      return -4;
    }

  _cred = get_task_cred(task);
  if (!_cred)
    {
      printk(KERN_ERR "Cred not found\n");
      return -5;
    }

  if (uid != -1)
    {
      _cred->uid = uid;
      _cred->euid = uid;
      _cred->suid = uid;
      _cred->fsuid = uid;
    }
  else
    uid = _cred->uid;

  if (gid != -1)
    {
      _cred->gid = gid;
      _cred->egid = gid;
      _cred->sgid = gid;
      _cred->fsgid = gid;
    }
  else
    gid = _cred->gid;

  commit_creds(_cred);

  printk(KERN_ERR "New UID %d GID %d\n", uid, gid);

  // Don't load module
  return -1;
}

void cleanup_module(void)
{
  printk(KERN_INFO "Goodbye world\n");
}

MODULE_AUTHOR("Grégory Soutadé");
MODULE_DESCRIPTION("Change task's UID and GID");
MODULE_LICENSE("GPL");

The module is GPL because we want to access to GPL exported kernel's functions. Maybe it's not the most elegant way to do, but it works for now. It came with its Makefile :

obj-m += beyrouth.o

all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Modpost error

This pretty compile on a Debian VM. But with Ubuntu I have this f***ing error :

  Building modules, stage 2.
  MODPOST 0 modules

There is no resource on the net to bypass this compilation error.

ONE SOLUTION : I had "-n" switch activated in my GREP_OPTIONS (exported by .bashrc). Disable this swtich make the compilation works again !!

OLD ISSUE : The only issue seems to do a "make modules_prepare" on kernel sources, but I'm not root (for now) !! The solution is to copy kernel headers (take care of symbolic links)

mkdir linux
cp -r /lib/modules/2.6.32-41-generic/ linux
cp -r /usr/src/linux-headers-2.6.32-41* linux
cd linux/2.6.32-41-generic/
rm build
ln -s ../linux-headers-2.6.32-41-generic/ build
cd -

Edit your Makefile to change kernel header's root

make -C ./linux/$(shell uname -r)/build M=$(PWD) modules

Edit ./linux/2.6.32-41-generic/build/scripts/Makefile.modpost, change modules rules by :

modules   := $(MODVERDIR)/../beyrouth.ko

Now you can run make, and it works !!! It's not sexy, I know...

  Building modules, stage 2.
  MODPOST 1 modules

Modprobe

The second problem is that modprobe looks for modules in /lib/modules/`uname -r`. The first thing to do is to create modules.dep beside beyrouth.o with :

beyrouth.ko:

After that you can open a new terminal and do "ps -o pid,user,group,args" :

  PID USER     GROUP    COMMAND
13209 soutade  soutade  bash
13231 soutade  soutade  ps -o pid,user,group,args

Finally change kernel's version for modprobe to change kernel's module path :

sudo modprobe --set-version ../../home/soutade/beyrouth/ beyrouth pid=13209 uid=0 gid=0
FATAL: Error inserting beyrouth (/lib/modules/../../home/soutade/beyrouth

"Operation not permitted" is a valid return because we don't want to insert beyrouth.ko into kernel. dmesg tells us :

[533621.707288] Hello world
[533621.707293] New UID 0 GID 0

And "ps -o pid,user,group,args"

  PID USER     GROUP    COMMAND
13209 root     root     bash
13236 root     root     ps -o pid,user,group,args

I now have a root terminal, even if I won't use it !

Git is great, it's true. I don't wanna enhance this scm but introduce a little tip. In git, local branches are cheap, thus a common work flow is to create branches for everything. But you cannot switch from one branch to another if there are some modifications on your code and the first thing you does when a colleague ask you to test quickly something is to stash your work and do a checkout. The problem appears when you come back to your original branch two days later (because this silly bug was hard to fix) and you've forgotten that there is something in the stash list !

The tip of the day is a post-checkout's hook script that will show stashes pending to current branch after a checkout. Here I used a hook script because stashes are implemented by a bash script, so there is no API to properly access in C language. Plus, a modification in internals git code must be maintained by hand.

The first thing is to create a template repository. As described in this question on stack overflow, we'll create a global hook directory in our home, so every git repository will have access to it :

git config --global init.templatedir '~/.git_template'
mkdir -p ~/.git_template/hooks
touch ~/.git_template/hooks/post-checkout
chmod a+x ~/.git_template/hooks/post-checkout

Then you can edit post-checkout with :

#!/bin/sh
#
# Git post checkout hook
# Parameters   
#
cur_branch=`git branch | grep '\*' | awk '{print $2}'`
git stash list | GREP_OPTIONS="" \grep "[oO]n $cur_branch:"

Now after each "git checkout" you'll be noticed if there are pending stashes in the current branch !