Activate eSata on Sheevaplug with Debian I recently bought an external hard disk with an eSata interface, it was not easy to find (almost are with USB2/3, other are expensive advanced NAS), but I did. The purpose of this disk is to make backups. But, on my Sheevaplug, the main partitions (/root, /boot...) are on an USB key (Toshiba 16 GB) running Debian stable. When I plugged my new hdd it was not recognized ! Actually I first configured my sheevaplug following some tutorials (http://www.cyrius.com/debian/kirkwood/sheevaplug/ for example). It was said to set the boot variable "arcNumber" to 2097. Why ? In facts ARM SoC doesn't have peripherals discovery mode, so you need to tell which board you're running on.

After looking a bit into Debian's kernel, it seems that eSata interface is activated only if arcNumber is set to 2678 ! If I do that, original Ubuntu on NAND flash (factory installation) doesn't recognize the current SoC because arcNumber 2678 is a patch from Debian (in original installation, eSata is activated by default). The second point is that if you set the board as an eSata board, Debian will try to boot on the eSata hard disk (even if you specify different kernel root=XX values).

So what to do ? The solution is to specify your partitions not using classic /dev/sdXXX format, but using UUID numbers. They are not human readable, nevertheless they refer to an unique partition ! The first step consists in listing your partitions UUID :

ls -l /dev/disk/by-uuid/
lrwxrwxrwx 1 root root 10 Sep 27 07:34 1642ad57-77aa-494c-aa77-6998d420eb8f -> ../../sda3
lrwxrwxrwx 1 root root 10 Sep 27 07:34 198239b4-ff16-4dda-8df0-37b106005817 -> ../../sda1
lrwxrwxrwx 1 root root 10 Sep 27 07:34 2e0cd399-3839-4e4e-bc57-5e6628841bc1 -> ../../sda2
lrwxrwxrwx 1 root root 10 Sep 27 07:34 dd27350b-2522-46a6-862e-0cbc072b535f -> ../../sda4

Then, edit /etc/fstab to use UUID and not /dev/sdXXX (it's fastidious I know) After that, you need to reboot with the serial console connected and stop automatic boot (type a key) to edit uBoot configuration. We'll set arcNumber to 2678 by default.

setenv arcNumber 2678

Then edit bootargs_options (for me it's bootargs_options_usb) to set correct UUID value

setenv usb_bootargs_root  "root=UUID=2e0cd399-3839-4e4e-bc57-5e6628841bc1"

Last step is to edit the global boot_cmd to set arcNumber to 2097 before booting to NAND (in my case, if USB boot fails it will try to boot on MMC then on NAND) :

setenv bootcmd 'setenv arcNumber 2678; saveenv; run usb_boot; setenv arcNumber 2097; saveenv; run bootcmd_mmc; run bootcmd_nand'

Finally save environment variables to flash and boot

saveenv
boot

My final environment variables

ethact=egiga0
bootargs_root=ubi.mtd=1 root=ubi0:rootfs rootfstype=ubifs
mtdpartitions=mtdparts=orion_nand:0x400000@0x100000(uImage),0x1fb00000@0x500000(rootfs)
ethaddr=00:50:43:01:4C:56
bootargs_console=console=ttyS0,115200
bootargs_root_nand=ubi.mtd=1 root=ubi0:rootfs rootfstype=ubifs
bootcmd_nand=setenv bootargs $(bootargs_console) $(mtdpartitions) $(bootargs_root_nand); \
nand read.e 0x00800000 0x00100000 0x00400000; bootm 0x00800000
bootargs_root_mmc=root=/dev/mmcblk0p2 rootdelay=5
bootcmd_mmc=setenv bootargs $(bootargs_console) $(bootargs_root_mmc); mmcinit;\
ext2load mmc 0:1 0x800000 /uImage; bootm 0x00800000
real_bootcmd=run bootcmd_mmc; run bootcmd_nand
filesize=32D62A
usb_bootargs_console=console=ttyS0,115200
usb_bootcmd_usb=usb start; ext2load usb 0:1 0x01100000 /uInitrd; ext2load usb 0:1 0x00800000 /uImage
usb_boot=setenv bootargs $(usb_bootargs_console) $(usb_bootargs_root); run usb_bootcmd_usb;\
bootm 0x00800000 0x01100000
mainlineLinus=yes
bustargs_root_usbroot=/dev/sda2
usb_bootargs="root=UUID=2e0cd399-3839-4e4e-bc57-5e6628841bc1"
stdin=serial
stdout=serial
stderr=serial
mainlineLinux=yes
enaMonExt=no
enaCpuStream=no
enaWrAllo=no
pexMode=RC
disL2Cache=no
setL2CacheWT=yes
disL2Prefetch=yes
enaICPref=yes
enaDCPref=yes
sata_dma_mode=yes
netbsd_en=no
vxworks_en=no
bootdelay=3
disaMvPnp=no
enaAutoRecovery=yes

I added a rule in fstab to mount my hdd at startup

UUID=590f30b1-7727-4d0a-a86a-2360ec0b3f88  /media/backup   ext4    defaults  0    1

A simple backup script based on rsync that power down disk after backup is done.

For the dynastie project, I need to load data encoded in UTF-8 with Python minidom XML parser. But when I wrote node.toxml('utf-8') to display the XML tree, I get this error :

UnicodeDecodeError at /generate/1

'ascii' codec can't decode byte 0xc2 in position 187: ordinal not in range(128)

In facts Python thinks that all data in XML tree are in ASCII and try to encode it into UTF-8 (or anything else you supplied). The solution is to use your own writer that will convert all non utf-8 strings in unicode string which can be then re-encoded in every format (like utf-8). This doesn't appears in Python 3 because, in Python 3, all strings are already in unicode. Add the following class to your code :

class UnicodeWriter(codecs.StreamWriter):
    encode = codecs.utf_8_encode

    def __init__(self):
        self.value = u''

    def write(self, object):
        if not type(object) == unicode:
            self.value = self.value + unicode(object, 'utf-8')
        else:
            self.value = self.value + object
        return self.value

    def reset(self):
	self.value = u''

    def getvalue(self):
	return self.value

And our node.toxml('utf-8') becomes :

writer = UnicodeWriter()
node.writexml(writer)
writer.getvalue().encode('utf-8')

A lot of things has been written about nginx and Apache : proxy_pass, proxy_redirect, subdomains... I just want to publish my configuration that is in test but works. This is a response to my requirements : I want nginx to serve the subdomain blog.soutade.fr but transfer all other requests to an Apache server (soutade.fr, www.soutade.fr, indefero.soutade.fr ...).

First step is to install nginx. Nginx current version is 0.7.23 (thanks to debian stable). Then edit /etc/nginx/sites-available/default :

server {
        listen   80 default; ## listen for ipv4

        server_name  soutade.fr *.soutade.fr;

        access_log  /var/log/nginx/soutade.fr.access.log;

        location / {
                proxy_set_header X-Real-IP  $remote_addr;
                proxy_set_header X-Forwarded-For $remote_addr;

                proxy_set_header Host $host;

                resolver         localhost;
                proxy_pass       http://$host:8000;
        }
}

server {
        listen   80; ## listen for ipv4

	server_name  blog.soutade.fr;

        access_log  /var/log/nginx/soutade.fr.access.log;

        location / {
                root   /var/www/blog;
                index  index.html;
	}

        location = /favicon.ico { access_log off; log_not_found off; }
        location ~ /\. { deny all; access_log off; log_not_found off; }
}

The first block tells nginx to redirect all request from soutade.fr and *.soutade.fr to a local Apache server listening on port 8000. The second block creates a special rule for blog.soutade.fr : files will be served by nginx server and no redirection will be applied. We also disable favicon.ico error log and deny serving \.* files. There is a tip in the first block : we need to set up a local DNS server ("resolver localhost" directive). If this option is not set, it will try to do another DNS request to resolve $host address. So we'll simply install bind9 and configure it. Edit /etc/bind/named.conf.local

zone "soutade.fr" {
     type master;
     file "/etc/bind/db.soutade.fr";
};

Finally edit /etc/bind/db.soutade.fr with :

$TTL    604800
@       IN      SOA     soutade.fr. root.soutade.fr. (
                      07082012          ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
;
@       IN      NS      localhost.
@       IN      A       127.0.0.1
*       IN      CNAME   soutade.fr.
@       IN      AAAA    ::1

And restart bind : sudo service bind9 restart. root.soutade.fr. is the mail address of the administrator. Now everything might be ok.

Today a little Python tip. This is not very complex, but not so intuitive due to conversions.

def xor_buffer(buf):
    res = ''
    for i in range(len(buf)):
    	res = res + chr(ord(buf[i]) ^ 0xff)

    return res

Alors que mon certificat SSL expire dans deux mois, OVH m'envoie un mail m'indiquant qu'ils abandonnent la certification SSL. Je trouve dommage de ne plus proposer de certification SSL quand on veut être un acteur majeur de l'Internet... Bref je n'ai pas envie d'avoir deux contrats dans deux sociétés différentes (pour mon certificat et mon nom de domaine), j'ai donc pris la décision (contestable) de tout migrer chez Gandi. À la base je voulais m'enregistrer chez Gandi car ils ont une bonne renommée, mais j'ai finalement choisi OVH. La migration s'est faite en huit jours une fois que j'ai eu désactivé la protection du domaine et récupéré la clé secrète chez OVH.

Après quelques jours d'utilisation, je me rend compte qu'OVH est supérieur techniquement à Gandi dans les outils qu'ils proposent, ils sont beaucoup plus simples et intuitifs (même si le support est loin d'être terrible). Bon, une fois configuré, on ne touche plus à rien, mais quand même... Et surtout un GROS manque chez Gandi est qu'ils ne gèrent pas nativement une ip dynamique (DynDNS). Évidemment je m'en suis rendu compte après migration...

La solution proposé sur les forums est de faire pointer un enregistrement IP (A) vers un DynDNS ... ce n'est pas acceptable. Heureusement, après quelques recherches, je découvre que Gandi met a disposition une API XML permettant de manipuler son compte ! Après m'être battu inutilement avec la version OT&E (qui ne permet de rien faire), j'ai finalement réussit à obtenir l'équivalent d'un DynDNS avec un petit script python que l'on mettra en cron :

#!/usr/bin/env python

import xmlrpclib
import urllib2
import re
import sys

#
# Update Gandi DNS with dynamic IP
#
# This doesn't work with OT&E platform !!
#

DEBUG = False

zone_to_edit = 'Soutade'

apikey = 'Your private API key'
api = xmlrpclib.ServerProxy('https://rpc.gandi.net/xmlrpc/')

def debug(str):
    if DEBUG:
        print str

# Get current IP
current_ip = ''
try:
    dyndns_url = 'http://checkip.dyndns.com/'
    f = urllib2.urlopen(dyndns_url)
    data = f.read()
    f.close()
    pattern = re.compile('[^:]*(\d+\.\d+\.\d+\.\d+)')
    result = pattern.search(data, 0)
    if result == None:
        print "No IP found"
        sys.exit(1) 
    else:
        current_ip = result.group(0).strip()
except urllib2.HTTPError:
    print "Connexion error"
    sys.exit(1)

debug('Your current IP is \'' + current_ip + '\'')

# Update Gandi DNS
zones = api.domain.zone.list(apikey, {'name' : zone_to_edit})

if len(zones) == 0:
    print 'Zone \'' + zone_to_edit + '\' not found'
    sys.exit(1)

zone = zones[0]
debug('Zone to be edited')
debug(zone)
records = api.domain.zone.record.list(apikey, zone['id'], zone['version'])
edited = False
for record in records:
    if record['type'] == 'A' and current_ip != record['value']: 
        debug('Edit record')
        debug(record)
        record['value'] = current_ip
        edited = True
    # Delete all id keys
    del record['id']

if edited:
    debug('Edit result')
    debug(records)
    if zone['version'] == 1:
        zone['version'] = 2
    else:
        zone['version'] = 1
    debug('New version is ' + str(zone['version']))
    api.domain.zone.record.set(apikey, zone['id'], zone['version'], records)
    api.domain.zone.version.set(apikey, zone['id'], zone['version'])

debug('New zone values')
debug(api.domain.zone.record.list(apikey, zone['id'], zone['version']))

Le script ne se veut pas générique, mais il fonctionne ! Il faut l'adapter en changeant la clé d'API et le nom de la zone à éditer. Il faut aussi créer deux versions (1 et 2) de la zone à modifier avec au moins un champs A et attacher le nom de domaine à cette zone. On pourra activer les traces en changeant la valeur de DEBUG. Pour info, mes paramètres DNS sont (mode expert) :

@ 1080 IN A 86.193.241.253
* 1080 IN CNAME soutade.fr.
@ 1080 IN MX 10 soutade.fr.

La première ligne fait pointer soutade.fr (nom de domaine attaché = @) à une IP que l'on mettra à jour dynamiquement. La seconde ligne fait pointer n'importe quel sous domaine de soutade.fr (blog.soutade.fr par exemple) vers soutade.fr qui point lui même vers une IP. La troisième ligne permet d'indiquer que le serveur mail se trouve également sur l'IP pointée par soutade.fr.

La bonne nouvelle est que j'ai droit à un certificat SSL gratuit pendant un an et le renouvellement de mon nom de domaine pour un an également, le tout pour 15€ ! Je reviendrai peut être chez OVH pour Internet... un jour...