Deuxième version d'IWLA, le clone d'AWSTATSl'analyseur de log web écrit en Python.

Qu'est ce qu'on trouve dans cette version ? Et bien ... tout. Cette version arrive au niveau d'AWSTATS en ce qui concerne mon usage, et même le dépasse !

Quelques fonctions non présentes dans AWSTATS :

• (Meilleure) Détection des robots
• Affichage des différences entre deux analyses
• Ajout facile des ses propres moteurs de recherche
• Traçage d'IP
• Détection de lecteur de flux RSS
• Support des fichiers de log compressés

Pour le reste, on retrouve toutes les briques de bases. Mon seul regret est de n'avoir pas encore mis en place les tests automatiques.

Bookeen doesn't provides any way to hack their e-ink e-readers. Nevertheless, after they introduce a secured update file format for the first Cybook serie, they come back to a "tar-like" format for their e-readers based on Allwiner platform (new Odyssey, Muse, Nolimbook...). With this format, they try to be closest to Android platform.

Be careful : handling all bookeen's readers is difficult because there is a lot of derived/customized readers available ! For this, Bookeen has its own server that checks for serial number to determine which update to serve. A special attention must be done with manipulating boot and bootloader images, it can bricks your e-reader !

This tutorial will only works with UNIX/Linux tools, I do not plan to do it for Windows.

So, let's start. An archive is commonly named CybUpdate.bin. In facts it's a .tar.bz2 file.

First, decompresse it :

mkdir decompressed
cd decompressed
tar -jxvf ../CybUpdate.bin

The following content should be created :

contents
bootloader.fex
boot.fex
rootfs.fex

I think that boot.fex and bootloader.fex are optional, but not sure. Two types of files are present :

• contents that contains meta information
• fex files

Contents has the following format :

<ident>|<filename>|<length>|<sha256sum>|<version>

idents are :

• LOAD for bootloader
• BOOT for boot partition
• ROOT for rootfs

Fex extension is a generic one that actually is flash images in different format (vfat, ext...).

In bootloader.fex and rootfs.fex we have a file "/version" specifying the current version (allowing to do checks). Mounting bootloader and rootfs is quite easy :

mkdir root
sudo mount -t ext2 rootfs.fex root -o loop

mkdir bootloader
sudo mount -t vfat bootloader.fex bootloader -o loop

After doing modifications, just unmount the directory and the image is automatically generated ! (Don't forget to update contents metadata).

boot.fex is more complex, it has Android bootloader format. You have to use split_bootimg.pl to decompress it.

mkdir boot
cd boot
../split_bootimg.pl ../boot.fex
> Page size: 2048 (0x00000800)
> Kernel size: 10863524 (0x00a5c3a4)
> Ramdisk size: 2253456 (0x00226290)
> Second size: 0 (0x00000000)
> Board name: sun5i
> Command line: 
> Writing boot.fex-kernel ... complete.
> Writing boot.fex-ramdisk.gz ... complete.

boot.fex-kernel is a binary version (compiled) of Linux.

Then we decompress ramdisk :

mkdir ramdisk_decompressed
cd ramdisk_decompressed
gzip -dc ../boot.fex-ramdisk.gz | cpio -i

You can re compress it with :

find | cpio -o | gzip -c > ../boot.fex-ramdisk.gz

Rebuilding boot is done with mkbootimg. Be careful to use the same parameters split_bootimg.pl displayed !

./mkbootimg.py --kernel boot.fex-kernel --ramdisk boot.fex-ramdisk.gz --pagesize 2048 --board sun5i -o ../boot.fex

As you see, there is nothing complicated here, but mistakes with boot/bootloader or init scripts can bricks your e-reader.

Have fun !

Sortie de la version 0.3 de Dynastie. Pour rappel, il s'agit de mon générateur de blog. En test depuis quelques mois, c'est maintenant officiel. Les différences ?

• Support des brouillons
• Ajout du marqueur "dyn:firstpageonly" : le sous-arbre n'apparaît que sur la première page
• Ajout du marqueur "dyn:ljdc_last" : dernière image "les joies du code"
• Quelques corrections de bug

Suite à l'introduction de la syntaxe Markdown, je n'utilise plus que cette dernière. Si un jour j'ai du temps, il faudra que je teste avec Django 1.8 et que je passe le code en Python 3.

Après une année blanche, voici la sortie de la version 0.5 de KissCount. Déjà 6 ans ! Le code est devenu assez mâture. Tellement qu'il faudrait VRAIMENT passer en Qt 5... Mais cette version n'est pas dénuée de nouveauté : Les tags (étiquettes) viennent rejoindre l'ensemble des fonctionnalités présentes. Il y a aussi quelques corrections de bugs.

Petit point faible côté empaquetage : seule la version GNU/Linux 64 bits est disponible. Il y a une version cross-compilée pour Windows qui malheureusement plante au démarrage (impossible de savoir pourquoi).

À priori, la version suivante ne sortira pas avant longtemps vu le peu de temps disponible que j'ai et de la maintenance/développement sur mes autres projets...

My configuration is a bit specific. I have an Apache server for PHP and Python stuff with nginx as a frontend (and for serving static files like this blog). This part seems normal for most of people. BUT, in my case, I want that the two servers write the access logs IN THE SAME FILE (/var/log/apache2/access.log)

I run a Debian stable on my server. With the previous version (7.0), all was fine, but, until Jessie (8.0), I found a very strange behaviour with my web statistics analyzer. I usually have from 60 to 80 visits per day. In may (after the upgrade to Jessie), I get only 2 or 3 visits per day, using only one subdomain. Plus, it's a period where I have no internet at home and no time to investigate...

Finally, I found the problem ! It's located in logrotate. The two files are :

/etc/logrotate.d/apache2

/var/log/apache2/*.log {
    daily
    missingok
    rotate 14
...
    postrotate
            if /etc/init.d/apache2 status > /dev/null ; then \
                /etc/init.d/apache2 reload > /dev/null; \
            fi;
    endscript
...
}

And /etc/logrotate.d/nginx

/var/log/nginx/*.log {
    weekly
    missingok
    rotate 52
...
    postrotate
            invoke-rc.d nginx rotate >/dev/null 2>&1
    endscript
}

The first one do a logrotate every day of all files in /var/log/apache2/*.log (that contains our . The second one, do only one logrotate every week. But, the most important line is located in postrotate. Why it's important ? Because it says to nginx and Apache "Now the log file has changed, close the previous and open the new one". Seems correct, but without that, nginx write in /var/log/apache2/access.log.1 and not /var/log/apache2/access.log until it gets rotated (after a week) because it will keep the handle on access.log that is renamed to access.log.1 by logrotate.

The solution is to add

invoke-rc.d nginx rotate >/dev/null 2>&1

In the postrotate section of /etc/logrotate.d/apache2 (after the if). Then, restart nginx :

systemctl restart nginx