Random notes about Android

Table of contents

• Table of contents
• Testing menu
• Boot image format
• Fastboot IMG format
• Property system
• About the package manager
• Dumping XML files from an APK
• The app_process utility
• The dumpsys utility
• Notes about wake locks
• Keyboard tables
• Recompiling/modifying the keyboard app (com.android.inputmethod.latin)
• A note about tethering and the DUN APN type
• A note about the Google OTA updates
• System fonts
• Wifi on the Nexus 4

Testing menu

Accessible by typing *#*#4636#*#* in the telephony app.

Also, to access developer settings, repeatedly tap the build number in the about phone menu.

Boot image format

How to change the Linux kernel command line used in Android:

Install the abootimg program under Debian or Ubuntu.

Using TWRP recovery (or possibly some other one), back up the boot image. Copy the boot image from this backup (TWRP calls it boot.emmc.win — others might call it something like boot.img). Feed it to abootimg -x which will extract three files (bootimg.cfg, initrd.img, zImage). Edit the cmdline from the bootimg.cfg file and run abootimg -u with the -f argument to update the boot image file. It can then be flashed using flashboot flash boot when the phone is in bootloader.

Fastboot IMG format

Fastboot IMG files, at least those which are ext4 images, are not mountable (“raw”) partition images, but a special (sparse) format. To convert from and to that format, one needs the simg2img and make_ext4fs programs (see below). Use with something like ./simg2img input.img output.raw.img to convert a sparse image to a mountable (“raw”) image. Conversely, something like ./make_ext4fs -s -l $SIZE -a userdata userdata-new.img mnt-point/ (as root, where $SIZE is the size of the mountable image) to reconstruct a sparse image file from a mounted directory; it may be necessary to specify an SElinux file context file using the -S option. See also this page on OMAPpedia.

These simg2img and make_ext4fs utilities are distributed by various Linux distros (e.g., as part of the Debian android-tools-fsutils package). Unfortunately, the version in question might be too old, or not compiled with the right options (some versions are simply not functional; others might lack SElinux support).

Sadly, compiling simg2img and make_ext4fs from the Android source is messy. Here are some hints:

• In the case of simg2img, it's not too difficult, but note that it seems to have disappeared from Android KitKat on. The source resides in the ext4_utils directory of the platform/system/extras Git repository. Compile with gcc -o simg2img simg2img.c sparse_crc32.c -lm -lz (simple enough)!
• On the other hand, ext4_utils is a mess. Here are some hints on compiling the Jelly Bean version: the source can also be obtained from the ext4_utils directory in the platform/system/extras Git repository of the Android sources, and it also needs some header files from the platform/system/core repository. One is compiled with gcc -o simg2img simg2img.c sparse_crc32.c -lm -lz and the other with gcc -I /somewhere/core/include -o make_ext4fs make_ext4fs_main.c make_ext4fs.c sparse_crc32.c contents.c allocate.c indirect.c extent.c backed_block.c ext4_utils.c output_file.c uuid.c sha1.c wipe.c -lm -lz -DANDROID (why does it always have to be so complicated?). Sadly, this version does not have SElinux support (if you compile with -DHAVE_SELINUX, it will simply segfault).
• Now here are some hints on compiling the KitKat version. The source is in the same ext4_utils directory, but now we will also need the Android version of libselinux and more core files (those in the include but also the libsparse directories). First compile libselinux with something like this (run from the src/ directory in its source): gcc -I ../include -I /somewhere/core/include -c *.c -DHOST; ar cru libselinux.a *.o (the -DHOST is essential to avoid an error about gettid() not found). Next compile libsparse with: gcc -I include -I /somewhere/core/include -c *.c; ar cru libsparse.a *.o (of course, /somewhere/core/include should be the same as ../include here). Finally, compile the program with gcc -I /somewhere/core/include -I /somewhere/core/libsparse/include -I /somewhere/libselinux/include -o make_ext4fs make_ext4fs_main.c make_ext4fs.c contents.c allocate.c indirect.c extent.c ext4_utils.c uuid.c sha1.c wipe.c crc16.c -lm -lz /somewhere/libselinux/src/libselinux.a /somewhere/core/libsparse/libsparse.a -DANDROID -DHAVE_SELINUX (whew!). But that's not all: the -S option to specify SElinux file contexts requires a file that can be found in the boot.img file (see below).

Finally, the boot.img file uses a different format! This one is documented in the bootimg.h header file.

Property system

This page is a good introduction to the Android property system. The getprop and setprop utilities (in /system/bin/, at least under CM) can be used to manipulate them from the command line.

About the package manager

This page contains a good starting introduction to the Android package manager. The key file is /data/system/packages.xml; it can, to some extent, be manipulated by the pm utility (found in /system/bin/ — at least under CM; but really just a wrapper to CLASSPATH=/system/framework/pm.jar app_process /system/bin com.android.commands.pm.Pm). For example, to find the .apk file corresponding to a package, run something like pm path com.example.somepackage (one can then use this to read its manifest).

It seems one should be able to remove individual permissions from a package by manipulating the /data/system/packages.xml file; I haven't actually tried this.

Package restrictions are in /data/system/users/0/package-restrictions.xml (here, 0 is the default user ID because these are, in fact, per-user restrictions): enabled or disabled individual components are listed there (to find the components of a package, look inside its manifest file), but of course it is probably better to enable/disable components using something like pm disable com.example.somepackage/com.example.somepackage.SomeComponent (omit the part after the / for disabling a whole package).

Dumping XML files from an APK

The .apk files contain XML in a special binary format. The aapt utility found in the Android development tools can be used to dump it in a human-readable form, e.g.: aapt d xmltree Whatever.apk AndroidManifest.xml (see also here).

See here for the official documentation of the AndroidManifest.xml file format.

The app_process utility

The app_process utility seems to be used to run Java code at a fairly low level: this page speaks a bit about it. It's the program that starts the main zygote: see here for a description of the Android zygote startup.

The dumpsys utility

The dumpsys program is a fantastically useful utility which can dump a huge amount of information concerning an Android system. With no parameter, it will dump information on all “services” it knows about, which is a huge amount, starting with a list of all the services in question (i.e.: look at the beginning of the output for a list of all useful parameters which can be passed to dumpsys). Among other things, dumpsys power gives information about power management, dumpsys cpuinfo about the CPU usage, dumpsys battery about battery state, dumpsys activity about the current activities, dumpsys package about all installed packages, dumpsys meminfo com.example.somepackage shows memory usage by a running app, while dumpsys batteryinfo displays package by package battery usage. Some services take parameters: try dumpsys someservice -h for usage.

Notes about wake locks

To enquire about the current wake locks, read /sys/power/wake_lock to find the active locks, or /proc/wakelocks to find stats about past locks — note that one can create and destroy a wake lock by simply writing an arbitrary name into /sys/power/wake_lock and /sys/power/wake_unlock respectively. Also note that Android does not automatically release wake locks when a process dies (i.e., they are not garbage-collected in any sense). See here and there for more details about Android wake locks.

The above concerns kernel wake locks. Android also has userland wake locks, which are all reflected as a single kernel wake lock, PowerManagerService: to enquire about the userland wake locks, type dumpsys power and look for the Wake Locks section of the output (it should list the number of userland wake locks held, with the process corresponding to each).

Keyboard tables

The .kl file in /system/usr/keylayout/ defines the scancode→keycode correspondence table and the .kcm file in /system/usr/keychars/ defines the corresponding characters. More information here.

Recompiling/modifying the keyboard app (com.android.inputmethod.latin)

Warning: This is a mess and these instructions are probably incomplete, brittle and subject to random change; moreover, they depend on the Android SDK and NDK being installed in /opt/android-* like I have them. Use this as a guideline, do not copy blindly!

Get the source using Git:

git clone git://github.com/CyanogenMod/android_packages_inputmethods_LatinIME
cd android_packages_inputmethods_LatinIME
git checkout cm-10.1.2

Then we need to copy the Android support library and also the com.android.inputmethodcommon source:

cd java
cp /opt/android-sdk-linux_86/extras/android/compatibility/v4/android-support-v4.jar libs/
cp -a /opt/android-sdk-linux_86/samples/android-17/SoftKeyboard/src/com/android/inputmethodcommon src/com/android/

Generate the standard build.xml file (still in the java/ dir):

/opt/android-sdk-linux_86/tools/android update project --path . --target android-17

To avoid compressing the .dict files in the resources:

Copy the section of /opt/android-sdk-linux_86/tools/ant/build.xml that starts with <target name="-package-resources" depends="-crunch"> and copy it in a <project> element in a file custom_rules.xml then uncomment <nocompress extension="dict" />

To compile the JNI interface:

pushd ../native/jni
/opt/android-ndk-r8e/ndk-build
popd
mkdir libs/armeabi
cp -a ../native/libs/armeabi/libjni_latinime.so libs/armeabi/

Finally, we should be able to do:

ant release

A note about tethering and the DUN APN type

The following is probably obsolete in Android 6 or later; at any rate, the settings database now seems to be in /data/system/users/0/settings_global.xml and whereabouts instead of settings.db as indicated below.

This blog post (in French) speaks about it in more detail. For the benefit of those who can't read French:

The tether_dun_required preference seems to be used to lock the phone into using an APN of type dun when tethering. Relevant code is in frameworks/base/services/java/com/android/server/connectivity/Tethering.java; the APN type is maybe transmitted to the network operator, I'm not sure (relevant code possibly somewhere around frameworks/opt/telephony/src/java/com/android/internal/telephony/DataConnection.java and frameworks/opt/telephony/src/java/com/android/internal/telephony/gsm/GsmDataConnection.java). For some reason, some people report tether_dun_required being in the secure preferences whereas I've found it in the global preferences. At any rate, one can edit the settings database using

sqlite3 /data/data/com.android.providers.settings/databases/settings.db

and insert the value in the global preferences using:

INSERT INTO global ( name, value ) VALUES ( 'tether_dun_required', 0 ) ;

A note about the Google OTA updates

Probably obsolete.

The Google services framework tries to download and install OTA firmware updates on certain devices (e.g., Nexus 4). On CM, this will fail, retry indefinitely, and use up a lot of bandwidth for nothing. See here for details. The following commands should disable the phenomenon:

pm disable com.google.android.gsf/com.google.android.gsf.update.SystemUpdateActivity
pm disable com.google.android.gsf/com.google.android.gsf.update.SystemUpdateService
pm disable com.google.android.gsf/com.google.android.gsf.update.SystemUpdateService\$Receiver

(See above for more about the pm command.) The (open source) Autostarts Android app can also be used to disable the three components in question.

I'm not sure whether it's a permanent fix, however (it should persist over reboot, but who knows whether the Google services framework has a way of re-enabling itself; check in /data/system/users/0/package-restrictions.xml after a reboot to make sure the components are, indeed, disabled). Also, some people have reported a wake lock in Google services (i.e., it prevents the phone from sleeping) with a similar solution, but perhaps only because only part of the above components were disabled.

System fonts

They are contained in /system/fonts/; the files /system/etc/system_fonts.xml and /system/etc/fallback_fonts.xml specify which fonts are used (the former defines the main font families, the latter which fonts are used as fallbacks for missing characters).

Wifi on the Nexus 4

The default configuration for the Wifi driver on the Nexus 4 enables extremely aggressive power-saving features which help preserve battery life but break networking badly, and IPv6 even more: see here for specific Nexus 4 issues, and here for a related issue. A much better behavior can be obtained by putting both McastBcastFilter and gEnableBmps to 0 in /system/etc/wifi/WCNSS_qcom_cfg.ini (the former prevents dropping of broadcast and multicast frames, something which completely and utterly breaks networking and IPv6 in particular; the second disables Broadcast Mode Power Management: I don't understand the exact relation between the two); perhaps also set gEnableSuspend to 2 to mitigate some of the battery drain (I'm not at all sure). This, of course, will have important consequences on battery life when Wifi is on: there does not seem to be a way to have both satisfactory Wifi behavior and satisfactory battery life, unfortunately — one has to choose which is the lesser evil.