BSD systems are known to have the highest uptime, but our favorite GNU/Linux system can do the same if we want. At work my computer runs since 260 days without any reboot thanks to Ubuntu (LTS 10.04).

This is cool, nevertheless having software launched for a long time consume a lot of memory (I like to keep terminals and emacs open to save history), especially Firefox uses one to three giga bytes of memory (even in version 30.0). Software not used everyday goes to swap and come back when needed. This process can slow my computer for minutes (when I unlock my session).

The only solution is to kill these applications and restart them properly. I used to track memory eaters with "top" command, but it's hard to see in one time which software is using most of memory.

I wanted to do it for long time, and now it's out ! I wrote a PERL script called memstats that list processes by memory usage (and not by CPU usage like top).

It reads information from /proc/PID/stat and /proc/PID/status, so it's very Linux dependant (sorry BSDs...).

memstats output looks like "top" (because it's a good one).

soutade@cybelle> memstats
  PID    OWNER  VIRT   RES S       TIME         COMMAND
20098  soutade 3.10g 1.80g S   09:00:59         firefox
 1285     root  526m  259m S 6214:00:04            Xorg
 2555  soutade  862m  200m S 6214:00:10  gnome-terminal
 2113  soutade  827m  181m S 6214:00:20     gnome-panel
20081  soutade  866m  153m S   10:00:12     thunderbird
 2122  soutade 1.97g   98m S 6214:00:20        nautilus
 2192  soutade  591m   86m S 6214:00:13 indicator-apple
20104  soutade 1.07g   42m S   09:00:58          pidgin
26536  soutade  627m   40m S   06:00:22           gedit
 6517  soutade  529m   30m S 2816:00:40           emacs
 2845  soutade  442m   27m S   00:00:50           emacs
22935  soutade  449m   23m S  103:00:06           emacs
 2094  soutade  899m   23m S 6214:00:21        metacity
 2207  soutade  393m   11m S 6214:00:13 indicator-messa
 2762  soutade  114m   10m S   00:00:48          python
 9752  soutade  401m    8m S   96:00:18 gnome-screensav
 2070  soutade  766m    7m S 6214:00:22 gnome-settings-
 2914  soutade   94m    6m R   00:00:00        memstats
 2121  soutade  477m    6m S 6214:00:20 notification-da
 2489  soutade  414m    6m S 6214:00:20 update-notifier
23305  soutade  391m    5m S 5641:00:44 ubuntuone-syncd
 2280  soutade  418m    4m S 6214:00:10 gdu-notificatio
 2061  soutade   22m    4m S 6214:00:22     dbus-daemon
 1973     root  192m    4m S 6214:00:32         lightdm
 6886  soutade  581m    4m S 3797:00:46 gnome-keyring-d
 2134  soutade  539m    4m S 6214:00:19 gnome-fallback-
28423  soutade   85m    4m S   05:00:43            bash
 2389  soutade  335m    4m S 6214:00:00 zeitgeist-daemo
 6080  soutade  332m    4m S 3624:00:44      pulseaudio
 2382  soutade  400m    3m S 6214:00:00 zeitgeist-datah

memstats is licenced under GNU GPL v3 and available in my inDefero forge. Have fun !

As you may know, my work consists in developing software for embedded devices. We usually says that an embedded device is a peace of hardware with low resources (memory, CPU, flash...) taking in example phones. Nowaday, smartphones have only core software that is really embedded, the rest is sometimes more powerful than the computer I wrote this post. But, don't care, I don't work in telephony. Here (at Neotion), we do really embedded software, with chipsets clocked from 100Mhz to 200Mhz, with available RAM from 1MB to 32MB, and flash up to 8MB.

After years of developments, we can have one or more software that became too big to fit in its allocated flash partition. So, to find the guilty functions, I wrote a simple PERL script (~130 lines) that will count number of instructions for each function from objdump's output (with -ld switch) and displays statistics per function and per file (it doesn't focus on .data or .bss section). To correctly use the script, you have to compile your program with -ggdb option (to have line numbers and file paths), but you can also set optimisations (-OX).

Example with main.c

#include <stdio.h>

int function1(int a, int b)
{
    return a*b+4;
}

int function2(int a, int b, char* c)
{
    printf("Result %d*%d+4 = %s\n", a, b, c);

    return 0;
}

int main(int argc, char** argv)
{
    char buf[32];

    sprintf(buf, "%d", function1(5, 4));

    function2(5, 4, buf);
}
    

> gcc main.c -ggdb -o test
> objdump -ld test > test.txt
> ./binstats.pl --in test.txt

Total instructions 63

63	(100.00%)   /home/soutade/main.c
	38	 main
	16	 function2
	9	 function1

There are also options to filter small files, small functions and paths that helps to focus on big ones. Have fun !

Wireshark (previously Ethereal) is the best open source protocol dissector/analyzer. You can analyze an incredible amount of protocols, not only Internet ones, but every stream based protocols. Moreover you can add your own filters/dissectors written either in C or in Lua. Nevertheless, the documentation on the net concerning Lua dissectors is light and sparse. It's been hard for me to make something that works even if it's, at the end, not really complicated. I'll try to explain the basis of Lua dissectors.

1) Installation

You need to have a wireshark that supports Lua support (wireshark -v). After that, create or edit ~/.wireshark/init.lua. To load a new plugin, just type

dofile("mydissector.lua")

Create the new file ~/.wireshark/mydissector.lua

2) Post Dissector

There are three types of dissectors :

• Dissector : you add your own protocol
• Chained dissector : you add new fields to an existing protocol
• Post dissector : you interact after all packets has been parsed

An example for each of one can be found here. I'll describe a post dissector, but other types of dissectors has pretty the same format.

Tip 1
If you want to display someting on the console, just do

print(something)

Tip 2
If the base array is not defined, add this to ~/.wireshark/init.lua

-- Display Bases
 base = {
    ["NONE"] = 0,
    ["DEC"] = 1,
    ["HEX"] = 2,
    ["OCT"] = 3,
    ["DEC_HEX"] = 4,
    ["HEX_DEC"] = 5,
}

First, you need to define your protocol : Proto(<internal name>, <displayed name>)

p_dummyproto = Proto("dummyproto","DummyProto")

Then, define your fields : ProtoField.TYPE(<internal name>, <displayed name>, [base], [values], [mask])
TYPE are defined here

-- Simple field without value
local f_X =  ProtoField.uint16("dummyproto.f_X","Field X")
-- Simple field displayed in hex format
local f_Y =  ProtoField.uint8("dummyproto.f_Y","Field Y", base.HEX)
-- Field with precomputed values and bitfield
local VALS_ZZ    = {[0] = "Single", [1] = "Dual"}
local f_Z =  ProtoField.uint8("dummyproto.f_Z","Field Z", base.HEX, VALS_ZZ, 0x3)

Third step is to register each field

p_dummyproto.fields = {f_X, f_Y, f_Z}

After that, the big part : protocol dissection. Fields are organized as a tree. You have to parse each byte (or range of bytes) in the given buffer and append your fields. Be careful : objects returned by :add function has userdata type and cannot be directly manipulated.

function p_dummyproto.dissector(buffer,pinfo,tree)
   -- Access to another field
   local f_udp_port = Field.new("udp.port")

   -- If it exists and has the right value
   if f_udp_port and tostring(f_udp_port) == tostring(5555) then
     -- Add our protocol dissection with data in buffer[17, 17+14]
     local subtree = tree:add(p_dummyproto, buffer(17,14))

     -- Add a subtree to our root for the first two bytes
     local t = subtree:add(f_X, buffer(17, 2))

     -- Add a sub subtree
     local t2 = t:add(f_Y, buffer(17, 1))

     -- Parse sub data
     parse_data(t, buffer, 18)
   end
end

function parse_data(tree, buffer, start)
     -- Wireshark integrate bitop from luajit http://bitop.luajit.org/
     field_1 = buffer(start, 1):uint()
     field_1 = bit.band(field_1, 0x3)

     -- You can also append free text information to current field
     if field_1 < 16 then
         tree:append_text(" field information")
     end
end

Finally register your post dissector

register_postdissector(p_dummyproto)

A complete example can be found here. It's a full reimplementation of ARP protocol dissector in Lua.

After 6 years of wonderful services I decided to change my superb Nokia 7373 by a Nexus 4 (thanks to the last price down). Welcome to the 21th century with a big smartphone running Android. One problem : how to transfer my contacts from Nokia to Android ? After reading some articles on the net I didn't find any simple solution. So I developped a perl script that will convert contacts extracted from Nokia Suite to vCad format (.vcf). The procedure is bellow :

• Extract your contacts with Nokia Suite into a CSV file
• Run this script (you can see help with -h switch)
• Copy vcf files to your phone (you can copy them in DCIM directory if you have a Nexus 4, it works)
• Go to "contact" application and import all vcf files

Now you can use your mobile phone as before. Personally I deactivated contact synchronisation because I do not want to give phone numbers of my friends to Google (even if I suppose it already has). *Beware* this script has been developped for French people : default call prefix is +33 (you can change it) and a cell phone numbers starts with 6 in France (you can change it). If you're in another country you may have to tweak it. Currently it can extract first name, last name, phone numbers and email. Have fun !

If I often compile under emacs, I use a lot the terminal. Most of people ignore how it's fast and practical to work, not for every tasks but it's perfect for programing. I already talks about Autojump2, today I will present colout. It's true that the terminal miss some colors. For better displays I personaly use black over white, but when you compile some projects, it's hard to see if errors happens.

I tweak colout (written by nojhan) to fit my needs. It's a simple Python software that will color terminal output if it found specific words. The list of words is fix and embedded into the script (unlike the original one that takes them in parameter). My script is available here.

You only need to add colout file into a bin directory (referenced in your $PATH) and edit your .bashrc with (don't forget to make it executable) :

function make() { /usr/bin/make "$@" 2>&1 | colout ;}

So, when you type "make", it will transparently use colout. The only bad thing is that you loss make's returns value.

Example with :

int main()
{
    int a, b;

    b = a+1;
    c = a;

    return 0;
}

It's better, isn't it ?