Android Marshmallow
This guide gives detailed step-by-step instructions on building and Android Marshmallow image along with a working kernel and .dtb file that work with gem5.
Contents
Overview
To successfully run Android in gem5, an image, a compatible kernel and a device tree blob .dtb) file configured for the simulator are necessary. This guide shows how to build Android Marshmallow 32bit version using a 3.14 kernel with Mali support. An extra section will be added in the future on how to build the 4.4 kernel with Mali.
Pre-requisites
This guide assumes a 64-bit system running 14.04 LTS Ubuntu. Before starting it is important first to set up our system correctly. To do this the following packages need to be installed through shell.
Tip: Always check for the up-to-date prerequisites at the Android build page.
Update and install all the dependencies. This can be done with the following commands:
sudo apt-get update
sudo apt-get install openjdk-7-jdk git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev ccache libgl1-mesa-dev libxml2-utils xsltproc unzip
Also, make sure to have repo correctly installed (instructions here).
Ensure that the default is OpenJDK 1.7
javac -version
To cross-compile the kernel (32bit) and for the device tree we will need the following packages to be installed:
sudo apt-get install gcc-arm-linux-gnueabihf device-tree-compiler
Before getting started, as a final step make sure to have the gem5 binaries and busybox for 32-bit ARM.
- For the gem5 binaries just do the following starting from your gem5 directory:
.cd util/m5make -f Makefile.armcd ../termmakecd ../../system/arm/simple_bootloader/make
- For busybox you can find the guide here
Building Android
We build Android Marshmallow using an AOSP running build based on the release for the Pixel C. The AOSP provides other builds, which are untested with this guide.
Tip: Synching with repo will take a long time. Use -jN to speed up the make process, where N number of parallel jobs to run.
- Make a directory and pull the android repository. Type in the following:
mkdir androidcd androidrepo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r63repo sync -c -jN
- Before you start the AOSP build, you will need to make one change to the build system to enable building libion.so, which is used by the Mali driver. Edit the file aosp/system/core/libion/Android.mk to change LOCAL_MODULE_TAGS for libion from 'optional' to 'debug'. Here is the output of repo diff:
--- a/system/core/libion/Android.mk +++ b/system/core/libion/Android.mk @@ -3,7 +3,7 @@ LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_SRC_FILES := ion.c LOCAL_MODULE := libion -LOCAL_MODULE_TAGS := optional +LOCAL_MODULE_TAGS := debug LOCAL_SHARED_LIBRARIES := liblog LOCAL_C_INCLUDES := $(LOCAL_PATH)/include $(LOCAL_PATH)/kernel-headers LOCAL_EXPORT_C_INCLUDE_DIRS := $(LOCAL_PATH)/include $(LOCAL_PATH)/kernel-headers
- Source the environment setup and build Android. Make sure to do this in a bash shell.
Tip: For root access and "debuggability" [sic] we choose userdebug. Build can be done in different modes as seen here
Tip: Making Android will take a long time. Use -jN to speed up the make process, where N number of parallel jobs to run.
-
source build/envsetup.sh
-
lunch aosp_arm-userdebug
-
make -jN
Creating an Android image
After a successful build, we create an image of Android and add the init files and binaries that configure the system for gem5.
- Create an empty image to flash the Android build and attach the image to a loopback device. The commands below create a 3GB image.
Tip: If you want to add applications or data, make the image large enough to fit the build and anything else that is meant to be written into it.
-
dd if=/dev/zero of=myimage.img bs=1M count=2560
-
sudo losetup /dev/loop0 myimage.img
- Make three partitions AndroidRoot (1.5GB), AndroidData (1GB), AndroidCache (512MB):
- Partition the device:
-
sudo fdisk /dev/loop0
- Update the partition table:
-
sudo partprobe /dev/loop0
- Name the partitions / Define filesystem as ext4:
-
sudo mkfs.ext4 -L AndroidRoot /dev/loop0p1
-
sudo mkfs.ext4 -L AndroidData /dev/loop0p2
-
sudo mkfs.ext4 -L AndroidCache /dev/loop0p3
- Mount the Root partition to a directory:
-
sudo mkdir -p /mnt/androidRoot
-
sudo mount /dev/loop0p1 /mnt/androidRoot
- Load the build to the partition:
-
cd /mnt/androidRoot
-
sudo zcat <path/to/build/android>/out/target/product/generic/ramdisk.img | sudo cpio -i
-
sudo mkdir cache
-
sudo mkdir /mnt/tmp
-
sudo mount -oro,loop <path/to/build/android>/out/target/product/generic/system.img /mnt/tmp
-
sudo cp -a /mnt/tmp/* system/
-
sudo umount /mnt/tmp
- Download and unpack the overlays that are necessary from the gem5 Android KitKat page and make the following changes to the init.gem5.rc file. Here is the output of repo diff:
--- /kitkat_overlay/init.gem5.rc
+++ /m_overlay/init.gem5.rc
@@ -1,21 +1,13 @@
+
on early-init
mount debugfs debugfs /sys/kernel/debug
on init
- export LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:/vendor/lib/egl
-
- # See storage config details at http://source.android.com/tech/storage/
- mkdir /mnt/media_rw/sdcard 0700 media_rw media_rw
- mkdir /storage/sdcard 0700 root root
+ # Support legacy paths
+ symlink /sdcard /mnt/sdcard
chmod 0666 /dev/mali0
chmod 0666 /dev/ion
-
- export EXTERNAL_STORAGE /storage/sdcard
-
- # Support legacy paths
- symlink /storage/sdcard /sdcard
- symlink /storage/sdcard /mnt/sdcard
on fs
mount_all /fstab.gem5
@@ -60,7 +52,6 @@
group root
oneshot
-# fusewrapped external sdcard daemon running as media_rw (1023)
-service fuse_sdcard /system/bin/sdcard -u 1023 -g 1023 -d
/mnt/media_rw/sdcard /storage/sdcard
+service fingerprintd /system/bin/fingerprintd
class late_start
- disabled
+ user system
- Add the Android overlays and configure their permissions:
-
sudo cp -r <path/to/android/overlays>/* /mnt/androidRoot/
-
sudo chmod ug+x /mnt/androidRoot/init.gem5.rc
-
/mnt/androidRoot/gem5/postboot.sh
- Add the m5 and busybox binaries under the sbin directory and make them executable:
-
sudo cp <path/to/gem5>/util/m5/m5 /mnt/androidRoot/sbin
-
sudo cp <path/to/busybox>/busybox /mnt/androidRoot/sbin
-
sudo chmod a+x /mnt/androidRoot/sbin/busybox /mnt/androidRoot/sbin/m5
- Make the directories readable and searchable:
-
sudo chmod a+rx /mnt/androidRoot/sbin/ /mnt/androidRoot/gem5/
- Remove the boot animation:
-
sudo rm /mnt/androidRoot/system/bin/bootanimation
- Download and unpack the Mali drivers (for gem5 Android 4.4) from here. Add the drivers and adjust their permissions:
- Make the directories for the drivers and copy them:
-
sudo mkdir -p /mnt/androidRoot/system/vendor/lib/egl
-
sudo mkdir -p /mnt/androidRoot/system/vendor/lib/hw
-
sudo cp <path/to/userspace/Mali/drivers>/lib/egl/libGLES_mali.so /mnt/androidRoot/system/vendor/lib/egl
-
sudo cp <path/to/userspace/Mali/drivers>/lib/hw/gralloc.default.so /mnt/androidRoot/system/vendor/lib/hw
- Change the permissions
-
sudo chmod 0755 /mnt/androidRoot/system/vendor/lib/hw
-
sudo chmod 0755 /mnt/androidRoot/system/vendor/lib/egl
-
sudo chmod 0644 /mnt/androidRoot/system/vendor/lib/egl/libGLES_mali.so
-
sudo chmod 0644 /mnt/androidRoot/system/vendor/lib/hw/gralloc.default.so
- Unmount and remove loopback device:
-
cd /..
-
sudo umount /mnt/androidRoot
-
sudo losetup -d /dev/loop0
Building the Kernel (3.14)
After successfully setting up the image, a compatible kernel needs to be built and a .dtb file generated.
- Clone the repository containing the gem5 specific kernel:
-
git clone -b ll_20140416.0-gem5 https://github.com/gem5/linux-arm-gem5.git
- Make the following changes to the kernel gem5 config file at <path/to/kernel/repo>/arch/arm/configs/vexpress_gem5_defconfig. Here is the output of repo diff:
--- a/arch/arm/configs/vexpress_gem5_defconfig +++ b/arch/arm/configs/vexpress_gem5_defconfig @@ -200,4 +200,15 @@ CONFIG_EARLY_PRINTK=y CONFIG_DEBUG_PREEMPT=n # CONFIG_CRYPTO_ANSI_CPRNG is not set # CONFIG_CRYPTO_HW is not set +CONFIG_MALI_MIDGARD=y +CONFIG_MALI_MIDGARD_DEBUG_SYS=y +CONFIG_ION=y +CONFIG_ION_DUMMY=y CONFIG_BINARY_PRINTF=y +CONFIG_NET_9P=y +CONFIG_NET_9P_VIRTIO=y +CONFIG_9P_FS=y +CONFIG_9P_FS_POSIX_ACL=y +CONFIG_9P_FS_SECURITY=y +CONFIG_VIRTIO_BLK=y +CONFIG_VMSPLIT_3G=y +CONFIG_DNOTIFY=y +CONFIG_FUSE_FS=y
- For the device tree, add the Mali GPU device and increase the memory to 1.8GB. Do this with the following changes at <path/to/kernel/repo>/arch/arm/boot/dts/vexpress-v2p-ca15-tc1-gem5.dts. Here is the output of repo diff:
--- a/arch/arm/boot/dts/vexpress-v2p-ca15-tc1-gem5.dts
+++ b/arch/arm/boot/dts/vexpress-v2p-ca15-tc1-gem5.dts
@@ -45,7 +45,7 @@
memory@80000000 {
device_type = "memory";
- reg = <0 0x80000000 0 0x40000000>;
+ reg = <0 0x80000000 0 0x74000000>;
};
hdlcd@2b000000 {
@@ -59,6 +59,14 @@
// mode = "3840x2160MR-16@60"; // UHD4K mode string
framebuffer = <0 0x8f000000 0 0x01000000>;
};
+
+ gpu@0x2d000000 {
+ compatible = "arm,mali-midgard";
+ reg = <0 0x2b400000 0 0x4000>;
+ interrupts = <0 86 4>, <0 87 4>, <0 88 4>;
+ interrupt-names = "JOB", "MMU", "GPU";
+ };
+
/*
memory-controller@2b0a0000 {
compatible = "arm,pl341", "arm,primecell";
- Download and unpack the userspace matching Mali kernel drivers for gem5 from here. Copy them to the gpu driver directory:
-
cp -r <path/to/kernelspace/Mali/drivers>/driver/product/kernel/drivers/gpu/arm/ drivers/gpu
- Change the following in <path/to/kernelspace/Mali/drivers>/drivers/video/Kconfig and <path/to/kernelspace/Mali/drivers>/drivers/gpu/Makefile:
Here is the output of the Kconfig repo diff:
--- a/drivers/video/Kconfig
+++ b/drivers/video/Kconfig
@@ -23,6 +23,8 @@ source "drivers/gpu/host1x/Kconfig"
source "drivers/gpu/drm/Kconfig"
+source "drivers/gpu/arm/Kconfig"
+
config VGASTATE
tristate
default n
Here is the output of the drivers/gpu/Makefile repo diff:
--- a/drivers/gpu/Makefile +++ b/drivers/gpu/Makefile @@ -1,2 +1,2 @@ -obj-y += drm/ vga/ +obj-y += drm/ vga/ arm/
- Finally, build the kernel and the .dtb file:
- Build the kernel
Tip: Use -jN to speed up the make process, where N number of parallel jobs to run.
-
make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm vexpress_gem5_defconfig
-
make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm vmlinux -jN
- Create the .dtb file:
-
dtc -I dts -O dtb arch/arm/boot/dts/vexpress-v2p-ca15-tc1-gem5.dts > vexpress-v2p-ca15-tc1-gem5.dtb
Testing the build
Make the following changes to example/fs.py. Here is the output repo diff:
--- a/configs/example/fs.py Thu Jun 02 20:34:39 2016 +0100
+++ b/configs/example/fs.py Fri Jun 10 15:37:29 2016 -0700
@@ -144,6 +144,13 @@
if is_kvm_cpu(TestCPUClass) or is_kvm_cpu(FutureClass):
test_sys.vm = KvmVM()
+ test_sys.gpu = NoMaliGpu(
+ gpu_type="T760",
+ ver_maj=0, ver_min=0, ver_status=1,
+ int_job=118, int_mmu=119, int_gpu=120,
+ pio_addr=0x2b400000,
+ pio=test_sys.membus.master)
+
if options.ruby:
# Check for timing mode because ruby does not support atomic accesses
if not (options.cpu_type == "detailed" or options.cpu_type == "timing"):
And the changes to FS config to either enable or disable software rendering.
--- a/configs/common/FSConfig.py Thu Jun 02 20:34:39 2016 +0100
+++ b/configs/common/FSConfig.py Thu Jun 16 10:23:44 2016 -0700
@@ -345,7 +345,7 @@
# release-specific tweaks
if 'kitkat' in mdesc.os_type():
- cmdline += " androidboot.hardware=gem5 qemu=1 qemu.gles=0 " + \
+ cmdline += " androidboot.hardware=gem5 qemu=1 qemu.gles=1 " + \
"android.bootanim=0"
self.boot_osflags = fillInCmdline(mdesc, cmdline
Set the following M5_PATH:
M5_PATH=. build/ARM/gem5.opt configs/example/fs.py --cpu-type=atomic --mem-type=SimpleMemory --os-type=android-kitkat --disk-image=myimage.img --machine-type=VExpress_EMM --dtb-filename=vexpress-v2p-ca15-tc1-gem5.dtb -n 1 --mem-size=1800MB
