Activate eSata on Sheevaplug with Debian I recently bought an external hard disk with an eSata interface, it was not easy to find (almost are with USB2/3, other are expensive advanced NAS), but I did. The purpose of this disk is to make backups. But, on my Sheevaplug, the main partitions (/root, /boot...) are on an USB key (Toshiba 16 GB) running Debian stable. When I plugged my new hdd it was not recognized ! Actually I first configured my sheevaplug following some tutorials (http://www.cyrius.com/debian/kirkwood/sheevaplug/ for example). It was said to set the boot variable "arcNumber" to 2097. Why ? In facts ARM SoC doesn't have peripherals discovery mode, so you need to tell which board you're running on.

After looking a bit into Debian's kernel, it seems that eSata interface is activated only if arcNumber is set to 2678 ! If I do that, original Ubuntu on NAND flash (factory installation) doesn't recognize the current SoC because arcNumber 2678 is a patch from Debian (in original installation, eSata is activated by default). The second point is that if you set the board as an eSata board, Debian will try to boot on the eSata hard disk (even if you specify different kernel root=XX values).

So what to do ? The solution is to specify your partitions not using classic /dev/sdXXX format, but using UUID numbers. They are not human readable, nevertheless they refer to an unique partition ! The first step consists in listing your partitions UUID :

ls -l /dev/disk/by-uuid/
lrwxrwxrwx 1 root root 10 Sep 27 07:34 1642ad57-77aa-494c-aa77-6998d420eb8f -> ../../sda3
lrwxrwxrwx 1 root root 10 Sep 27 07:34 198239b4-ff16-4dda-8df0-37b106005817 -> ../../sda1
lrwxrwxrwx 1 root root 10 Sep 27 07:34 2e0cd399-3839-4e4e-bc57-5e6628841bc1 -> ../../sda2
lrwxrwxrwx 1 root root 10 Sep 27 07:34 dd27350b-2522-46a6-862e-0cbc072b535f -> ../../sda4

Then, edit /etc/fstab to use UUID and not /dev/sdXXX (it's fastidious I know) After that, you need to reboot with the serial console connected and stop automatic boot (type a key) to edit uBoot configuration. We'll set arcNumber to 2678 by default.

setenv arcNumber 2678

Then edit bootargs_options (for me it's bootargs_options_usb) to set correct UUID value

setenv usb_bootargs_root  "root=UUID=2e0cd399-3839-4e4e-bc57-5e6628841bc1"

Last step is to edit the global boot_cmd to set arcNumber to 2097 before booting to NAND (in my case, if USB boot fails it will try to boot on MMC then on NAND) :

setenv bootcmd 'setenv arcNumber 2678; saveenv; run usb_boot; setenv arcNumber 2097; saveenv; run bootcmd_mmc; run bootcmd_nand'

Finally save environment variables to flash and boot

saveenv
boot

My final environment variables

ethact=egiga0
bootargs_root=ubi.mtd=1 root=ubi0:rootfs rootfstype=ubifs
mtdpartitions=mtdparts=orion_nand:0x400000@0x100000(uImage),0x1fb00000@0x500000(rootfs)
ethaddr=00:50:43:01:4C:56
bootargs_console=console=ttyS0,115200
bootargs_root_nand=ubi.mtd=1 root=ubi0:rootfs rootfstype=ubifs
bootcmd_nand=setenv bootargs $(bootargs_console) $(mtdpartitions) $(bootargs_root_nand); \
nand read.e 0x00800000 0x00100000 0x00400000; bootm 0x00800000
bootargs_root_mmc=root=/dev/mmcblk0p2 rootdelay=5
bootcmd_mmc=setenv bootargs $(bootargs_console) $(bootargs_root_mmc); mmcinit;\
ext2load mmc 0:1 0x800000 /uImage; bootm 0x00800000
real_bootcmd=run bootcmd_mmc; run bootcmd_nand
filesize=32D62A
usb_bootargs_console=console=ttyS0,115200
usb_bootcmd_usb=usb start; ext2load usb 0:1 0x01100000 /uInitrd; ext2load usb 0:1 0x00800000 /uImage
usb_boot=setenv bootargs $(usb_bootargs_console) $(usb_bootargs_root); run usb_bootcmd_usb;\
bootm 0x00800000 0x01100000
mainlineLinus=yes
bustargs_root_usbroot=/dev/sda2
usb_bootargs="root=UUID=2e0cd399-3839-4e4e-bc57-5e6628841bc1"
stdin=serial
stdout=serial
stderr=serial
mainlineLinux=yes
enaMonExt=no
enaCpuStream=no
enaWrAllo=no
pexMode=RC
disL2Cache=no
setL2CacheWT=yes
disL2Prefetch=yes
enaICPref=yes
enaDCPref=yes
sata_dma_mode=yes
netbsd_en=no
vxworks_en=no
bootdelay=3
disaMvPnp=no
enaAutoRecovery=yes

I added a rule in fstab to mount my hdd at startup

UUID=590f30b1-7727-4d0a-a86a-2360ec0b3f88  /media/backup   ext4    defaults  0    1

A simple backup script based on rsync that power down disk after backup is done.

For the dynastie project, I need to load data encoded in UTF-8 with Python minidom XML parser. But when I wrote node.toxml('utf-8') to display the XML tree, I get this error :

UnicodeDecodeError at /generate/1

'ascii' codec can't decode byte 0xc2 in position 187: ordinal not in range(128)

In facts Python thinks that all data in XML tree are in ASCII and try to encode it into UTF-8 (or anything else you supplied). The solution is to use your own writer that will convert all non utf-8 strings in unicode string which can be then re-encoded in every format (like utf-8). This doesn't appears in Python 3 because, in Python 3, all strings are already in unicode. Add the following class to your code :

class UnicodeWriter(codecs.StreamWriter):
    encode = codecs.utf_8_encode

    def __init__(self):
        self.value = u''

    def write(self, object):
        if not type(object) == unicode:
            self.value = self.value + unicode(object, 'utf-8')
        else:
            self.value = self.value + object
        return self.value

    def reset(self):
	self.value = u''

    def getvalue(self):
	return self.value

And our node.toxml('utf-8') becomes :

writer = UnicodeWriter()
node.writexml(writer)
writer.getvalue().encode('utf-8')

A lot of things has been written about nginx and Apache : proxy_pass, proxy_redirect, subdomains... I just want to publish my configuration that is in test but works. This is a response to my requirements : I want nginx to serve the subdomain blog.soutade.fr but transfer all other requests to an Apache server (soutade.fr, www.soutade.fr, indefero.soutade.fr ...).

First step is to install nginx. Nginx current version is 0.7.23 (thanks to debian stable). Then edit /etc/nginx/sites-available/default :

server {
        listen   80 default; ## listen for ipv4

        server_name  soutade.fr *.soutade.fr;

        access_log  /var/log/nginx/soutade.fr.access.log;

        location / {
                proxy_set_header X-Real-IP  $remote_addr;
                proxy_set_header X-Forwarded-For $remote_addr;

                proxy_set_header Host $host;

                resolver         localhost;
                proxy_pass       http://$host:8000;
        }
}

server {
        listen   80; ## listen for ipv4

	server_name  blog.soutade.fr;

        access_log  /var/log/nginx/soutade.fr.access.log;

        location / {
                root   /var/www/blog;
                index  index.html;
	}

        location = /favicon.ico { access_log off; log_not_found off; }
        location ~ /\. { deny all; access_log off; log_not_found off; }
}

The first block tells nginx to redirect all request from soutade.fr and *.soutade.fr to a local Apache server listening on port 8000. The second block creates a special rule for blog.soutade.fr : files will be served by nginx server and no redirection will be applied. We also disable favicon.ico error log and deny serving \.* files. There is a tip in the first block : we need to set up a local DNS server ("resolver localhost" directive). If this option is not set, it will try to do another DNS request to resolve $host address. So we'll simply install bind9 and configure it. Edit /etc/bind/named.conf.local

zone "soutade.fr" {
     type master;
     file "/etc/bind/db.soutade.fr";
};

Finally edit /etc/bind/db.soutade.fr with :

$TTL    604800
@       IN      SOA     soutade.fr. root.soutade.fr. (
                      07082012          ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
;
@       IN      NS      localhost.
@       IN      A       127.0.0.1
*       IN      CNAME   soutade.fr.
@       IN      AAAA    ::1

And restart bind : sudo service bind9 restart. root.soutade.fr. is the mail address of the administrator. Now everything might be ok.

Today a little Python tip. This is not very complex, but not so intuitive due to conversions.

def xor_buffer(buf):
    res = ''
    for i in range(len(buf)):
    	res = res + chr(ord(buf[i]) ^ 0xff)

    return res

At work we're not root in our machines. This simplify the administrator's works and we don't need to be root for usual tasks. If I need to, I can ask my colleague two desks in front of me. But for a special case I was given modprobe access via sudo. I said to him "If I can use modprobe, I'm root !", he replied "No because modprobe only loads modules from /lib/modules, but this is your challenge !".

Challenge accepted and successfully completed ! I'm running an Ubuntu 10.4 LTS with a 2.6.32-41-generic kernel.

Module source

The module was easy to develop (even if it has been a while I havn't do kernel module). It was named beyrouth for "be root", but it's more generic. Indeed you can change UID and GID of any running process !

/*  
 *  beyrouth.c - Change task's UID and GID
 */
#include /* Needed by all modules */
#include /* Needed for KERN_INFO */
#include 
#include 
#include 
#include 

// Copied from linux/cred.h
#define __task_cred(task) \
  ((const struct cred *)(rcu_dereference((task)->real_cred)))

// Copied from linux/cred.h
#define get_task_cred(task)                             \
  ({							\
    struct cred *__cred;				\
    rcu_read_lock();					\
    __cred = (struct cred *) __task_cred((task));	\
    get_cred(__cred);					\
    rcu_read_unlock();					\
    __cred;						\
  })

int pid = -1;
int uid = -1;
int gid = -1;

module_param(pid, int, 0);
module_param(uid, int, 0);
module_param(gid, int, 0);

int init_module(void)
{
  struct cred *_cred;
  struct pid* _pid;
  struct task_struct* task;

  printk(KERN_ERR "Hello world\n");

  if (pid == -1)
    {
      printk(KERN_ERR "PID is missing\n");
      return -2;
    }

  _pid = find_get_pid(pid);
  if (!_pid)
    {
      printk(KERN_ERR "PID not found\n");
      return -3;
    }

  task = pid_task(_pid, PIDTYPE_PID);
  if (!task)
    {
      printk(KERN_ERR "Task not found\n");
      return -4;
    }

  _cred = get_task_cred(task);
  if (!_cred)
    {
      printk(KERN_ERR "Cred not found\n");
      return -5;
    }

  if (uid != -1)
    {
      _cred->uid = uid;
      _cred->euid = uid;
      _cred->suid = uid;
      _cred->fsuid = uid;
    }
  else
    uid = _cred->uid;

  if (gid != -1)
    {
      _cred->gid = gid;
      _cred->egid = gid;
      _cred->sgid = gid;
      _cred->fsgid = gid;
    }
  else
    gid = _cred->gid;

  commit_creds(_cred);

  printk(KERN_ERR "New UID %d GID %d\n", uid, gid);

  // Don't load module
  return -1;
}

void cleanup_module(void)
{
  printk(KERN_INFO "Goodbye world\n");
}

MODULE_AUTHOR("Grégory Soutadé");
MODULE_DESCRIPTION("Change task's UID and GID");
MODULE_LICENSE("GPL");

The module is GPL because we want to access to GPL exported kernel's functions. Maybe it's not the most elegant way to do, but it works for now. It came with its Makefile :

obj-m += beyrouth.o

all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Modpost error

This pretty compile on a Debian VM. But with Ubuntu I have this f***ing error :

  Building modules, stage 2.
  MODPOST 0 modules

There is no resource on the net to bypass this compilation error.

ONE SOLUTION : I had "-n" switch activated in my GREP_OPTIONS (exported by .bashrc). Disable this swtich make the compilation works again !!

OLD ISSUE : The only issue seems to do a "make modules_prepare" on kernel sources, but I'm not root (for now) !! The solution is to copy kernel headers (take care of symbolic links)

mkdir linux
cp -r /lib/modules/2.6.32-41-generic/ linux
cp -r /usr/src/linux-headers-2.6.32-41* linux
cd linux/2.6.32-41-generic/
rm build
ln -s ../linux-headers-2.6.32-41-generic/ build
cd -

Edit your Makefile to change kernel header's root

make -C ./linux/$(shell uname -r)/build M=$(PWD) modules

Edit ./linux/2.6.32-41-generic/build/scripts/Makefile.modpost, change modules rules by :

modules   := $(MODVERDIR)/../beyrouth.ko

Now you can run make, and it works !!! It's not sexy, I know...

  Building modules, stage 2.
  MODPOST 1 modules

Modprobe

The second problem is that modprobe looks for modules in /lib/modules/`uname -r`. The first thing to do is to create modules.dep beside beyrouth.o with :

beyrouth.ko:

After that you can open a new terminal and do "ps -o pid,user,group,args" :

  PID USER     GROUP    COMMAND
13209 soutade  soutade  bash
13231 soutade  soutade  ps -o pid,user,group,args

Finally change kernel's version for modprobe to change kernel's module path :

sudo modprobe --set-version ../../home/soutade/beyrouth/ beyrouth pid=13209 uid=0 gid=0
FATAL: Error inserting beyrouth (/lib/modules/../../home/soutade/beyrouth

"Operation not permitted" is a valid return because we don't want to insert beyrouth.ko into kernel. dmesg tells us :

[533621.707288] Hello world
[533621.707293] New UID 0 GID 0

And "ps -o pid,user,group,args"

  PID USER     GROUP    COMMAND
13209 root     root     bash
13236 root     root     ps -o pid,user,group,args

I now have a root terminal, even if I won't use it !