Using an android smartphone as a drawing tablet

This is a simple hack that I developed in one afternoon. While reading up about android stuff I ended up finding the getevent command for adb and it inspired me to work on a project to transform my smartphone into a drawing tablet. The result is kinda slow and it's a little bit buggy but works quite well for a quick prototype. See the video:

For this hack you're going to need adb. adb is a command line tool for debugging Android systems, it can be installed from your package manager.

You're going to have your phone connected to your computer via usb and enable the usb debugging mode from it's configuration. If you search the web for it, it's easy to find.

From your computer, running the command adb shell getevent -lp lists the input devices for your phone and provides useful information about variables. More info on getevent.

This is the touchscreen device on my phone:

add device 1: /dev/input/event3
name:     "synaptics_dsx_i2c"
events:
    ABS (0003): ABS_MT_SLOT           : value 0, min 0, max 9, fuzz 0, flat 0, resolution 0
                ABS_MT_TOUCH_MAJOR    : value 0, min 0, max 255, fuzz 0, flat 0, resolution 0
                ABS_MT_POSITION_X     : value 0, min 0, max 720, fuzz 0, flat 0, resolution 0
                ABS_MT_POSITION_Y     : value 0, min 0, max 1280, fuzz 0, flat 0, resolution 0
                ABS_MT_TRACKING_ID    : value 0, min 0, max 65535, fuzz 0, f    lat 0, resolution 0
                ABS_MT_PRESSURE       : value 0, min 0, max 255, fuzz 0, flat 0, resolution 0
input props:
    INPUT_PROP_DIRECT

Running adb shell getevent -l /dev/input/event3, it interactively prints events from this device, like in the example below. The first column is the event type, the second one is the parameter and the third the value. I'm only interested on the latter two.

EV_ABS       ABS_MT_TRACKING_ID   0000190f            
EV_ABS       ABS_MT_POSITION_X    0000022a            
EV_ABS       ABS_MT_POSITION_Y    000002ea            
EV_ABS       ABS_MT_PRESSURE      00000031            
EV_ABS       ABS_MT_TOUCH_MAJOR   00000031            
EV_SYN       SYN_REPORT           00000000            
EV_ABS       ABS_MT_TRACKING_ID   ffffffff            
EV_SYN       SYN_REPORT           00000000

After tapping on my phone and analyzing the output, I found that I could use the variable ABS_MT_TRACKING_ID as a key down / key up event. The value ffffffff seems to be associated with key up, everything else I considered key down. The ABS_MT_POSITION_X and ABS_MT_POSITION_Y parameters are obvious, as they represent the x,y coordinates.

xdotool is a command line tool for simulating input devices on your computer. I used it to move the mouse around my computer. The syntax is as it follows:

# Move mouse to the (x,y) (100,100) coordinates on the screen
xdotool mousemove 100 100;
# Set the mouse button 1 down
xdotool mousedown 1;
# Set the mouse button 1 up
xdotool mouseup 1;

When converting the values from the x,y coordinates, I noticed they are huge. In my device the vertical coordinate seem to vary from 0 to 12360, possibly reflecting the sensitivity of the device. I had to scale these numbers to the resolution of my screem. It is baasic old school math, like so:

scaledCoords = (originalCoords/phone_maxCoords)*computerResolution;

Now all I had to do was to pipe the output of getevent, parse the parameters, scale and convert the x,y coordinates to decimal and use them on xdotool. I made a shell script for it, this is the final version: 

#! /bin/bash


x=0;
y=0;
phone_maxY="7080";
phone_maxX="12360";
computer_maxX="1920";
computer_maxY="1080";
scale="0.75";
i=0;
mouse="up";
adb shell getevent -l /dev/input/event3 | while read type param value
do
    if [[ $param == "ABS_MT_TRACKING_ID" ]]
    then
        if [[ ${value:4:4} == "ffff" ]]
        then
            mouse="up"; 
            xdotool mouseup 1;
            echo "up";
        else
            mouse="down";
            echo "down";
        fi
    fi
    if [[ $param == "ABS_MT_POSITION_X" || $param == "ABS_MT_POSITION_Y" ]]
    then
        ((i++));

        [[ $param == "ABS_MT_POSITION_X" ]] && y=$(printf "%d" 0x$value 2>/dev/null);
        [[ $param == "ABS_MT_POSITION_Y" ]] && x=$(printf "%d" 0x$value 2>/dev/null);

        if ((i == 2))
        then
            # scale
            newX=$(lua -e "print($x/$phone_maxX);");
            newY=$(lua -e "print(($phone_maxY-$y)/$phone_maxY);");
            # zoom (centralize)
            newX=$(lua -e "print($newX*$scale+0.5*(1-$scale));");
            newY=$(lua -e "print($newY*$scale+0.5*(1-$scale));");
            # map to screen coordinates
            newX=$(lua -e "print(math.floor($newX*$computer_maxX));");
            newY=$(lua -e "print(math.floor($newY*$computer_maxY));");

            xdotool mousemove $newX $newY mousedown 1;
            echo $mouse $newX $newY;

            i=0;
        fi
    fi
done;

Of course because I did it via a shell script and had to pipe stuff around, the prototype is a little bit slow, but it is just a proof of concept I did in the afternoon. You're free to improve it and make it better.

Download the source code.