[HOWTO] Benchmark App Startup

I wanted to look for methods how users can compare subjective “slowness” and benchmark their experience for others.

Android documentation has some acronyms for performance “vitals”, like “time to initial display” (TTID) for App startup, “time to full display” (TTFD) interaction time and “jank” for visible lag when rendering as in scrolling, see these two links:

I created a list from the apps (and resolved the main activitiy where necessary) present on the default install launcher screen of /e/ upper left to bottom right. You can either grep logcat for the “Displayed” string to get timings, or provide activitymanager (am) with the -W flag to get it in the loop. Get the appid via adb shell pm list packages or by checking logcat while starting it via touch.

adb shell am start -W <appid/app-main-activity>

Script

The script puts screen lock time to a few minutes for the time of the test. It’s best to run it after a reboot for a baseline to compare to. If the Android Runtime / jvm already has lots of Apps running in the background and not yet evicted timings can differ.

#!/usr/bin/env bash

echo "# set screen power and screen lock times to 10 minutes #"
adb shell settings put secure lock_screen_lock_after_timeout 600000
adb shell settings put global screen_off_timeout 600000

activities='foundation.e.apps/foundation.e.apps.MainActivity
foundation.e.tasks/org.dmfs.tasks.TaskListActivity
com.android.documentsui/.files.FilesActivity
foundation.e.mail
com.android.settings
com.android.gallery3d
foundation.e.calendar
com.android.contacts
com.generalmagic.magicearth/com.generalmagic.android.map.MapActivity
org.lineageos.eleven
foundation.e.notes/.android.activity.NotesListViewActivity
com.android.calculator2/.Calculator
org.lineageos.recorder
com.android.deskclock
com.android.dialer
foundation.e.message/com.moez.QKSMS.feature.main.MainActivity
foundation.e.browser
foundation.e.camera'

for activity in $activities
do
  echo "# $activity #"
  adb shell am start -W "$activity" | tee -a benchmark.log
  sleep 3
done

echo "# reset screen power and screen lock times #"
adb shell settings put secure lock_screen_lock_after_timeout 30000
adb shell settings put global screen_off_timeout 15000

Sample Ouput

LaunchState: COLD
Activity: foundation.e.browser/com.google.android.apps.chrome.Main
TotalTime: 690
WaitTime: 693

How to read results?

If it is a complex app that needs the network to update, look for TotalTime, as it outputs the TTFD moment after the initial display that also can just be a loading symbol. The TTFD is when the network info for a UI listing is complete and a user can really start to interact. That is the TotalTime or in logcat “Fully drawn” for the main activity.

Android vitals considers your app’s startup times excessive when the app’s:

  • Cold startup takes 5 seconds or longer.
  • Warm startup takes 2 seconds or longer.
  • Hot startup takes 1.5 seconds or longer.

5 Likes

some example output. I’m aware that totalTime can be deceptive for apps that go into a loading screen. On paper those are fully-drawn activities, but not from a user-interaction perspective. The default Apps of /e/ are not suspectible to this but the Maps app.

Having JSON output that could be drag+dropped to a bit of locally working javascript would be helpful, but then, for commercial apps on would need to measure the activity that allows for final user-interaction.

"FP3" after boot
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=foundation.e.apps/.MainActivity }
Status: ok
LaunchState: WARM
Activity: foundation.e.apps/.MainActivity
TotalTime: 1587
WaitTime: 1591
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=foundation.e.tasks/org.dmfs.tasks.TaskListActivity }
Status: ok
LaunchState: WARM
Activity: foundation.e.tasks/org.dmfs.tasks.TaskListActivity
TotalTime: 395
WaitTime: 399
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.android.documentsui/.files.FilesActivity }
Status: ok
LaunchState: COLD
Activity: com.android.documentsui/.files.FilesActivity
TotalTime: 723
WaitTime: 729
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=foundation.e.mail }
Status: ok
LaunchState: COLD
Activity: foundation.e.mail/com.fsck.k9.activity.setup.AccountSetupBasics
TotalTime: 762
WaitTime: 766
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.settings }
Status: ok
LaunchState: WARM
Activity: com.android.settings/.Settings
TotalTime: 611
WaitTime: 623
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.gallery3d }
Status: ok
LaunchState: COLD
Activity: com.android.gallery3d/.app.GalleryActivity
TotalTime: 480
WaitTime: 521
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=foundation.e.calendar }
Status: ok
LaunchState: COLD
Activity: foundation.e.calendar/com.android.calendar.AllInOneActivity
TotalTime: 584
WaitTime: 587
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.contacts }
Status: ok
LaunchState: COLD
Activity: com.android.contacts/.activities.PeopleActivity
TotalTime: 677
WaitTime: 680
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.generalmagic.magicearth/com.generalmagic.android.map.MapActivity }
Status: ok
LaunchState: WARM
Activity: com.generalmagic.magicearth/com.generalmagic.android.map.MainMapActivity
TotalTime: 449
WaitTime: 750
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=org.lineageos.eleven }
Status: ok
LaunchState: COLD
Activity: org.lineageos.eleven/.ui.activities.HomeActivity
TotalTime: 817
WaitTime: 820
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=foundation.e.notes/.android.activity.NotesListViewActivity }
Status: ok
LaunchState: COLD
Activity: foundation.e.notes/.android.activity.AccountActivity
TotalTime: 904
WaitTime: 912
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.android.calculator2/.Calculator }
Status: ok
LaunchState: COLD
Activity: com.android.calculator2/.Calculator
TotalTime: 408
WaitTime: 414
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=org.lineageos.recorder }
Status: ok
LaunchState: COLD
Activity: org.lineageos.recorder/.RecorderActivity
TotalTime: 406
WaitTime: 411
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.deskclock }
Status: ok
LaunchState: COLD
Activity: com.android.deskclock/.DeskClock
TotalTime: 555
WaitTime: 558
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.dialer }
Status: ok
LaunchState: WARM
Activity: com.android.dialer/.main.impl.MainActivity
TotalTime: 480
WaitTime: 483
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=foundation.e.message/com.moez.QKSMS.feature.main.MainActivity }
Status: ok
LaunchState: WARM
Activity: foundation.e.message/com.moez.QKSMS.feature.main.MainActivity
TotalTime: 404
WaitTime: 407
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=foundation.e.browser }
Status: ok
LaunchState: COLD
Activity: foundation.e.browser/com.google.android.apps.chrome.Main
TotalTime: 688
WaitTime: 691
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=foundation.e.camera }
Status: ok
LaunchState: COLD
Activity: foundation.e.camera/net.sourceforge.opencamera.MainActivity
TotalTime: 680
WaitTime: 683
Complete

"Moto G4 Play" after boot
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=foundation.e.apps/.MainActivity }
Status: ok
LaunchState: COLD
Activity: foundation.e.apps/.MainActivity
TotalTime: 3609
WaitTime: 3616
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=foundation.e.tasks/org.dmfs.tasks.TaskListActivity }
Status: ok
LaunchState: WARM
Activity: foundation.e.tasks/org.dmfs.tasks.TaskListActivity
TotalTime: 1006
WaitTime: 1015
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.android.documentsui/.files.FilesActivity }
Status: ok
LaunchState: COLD
Activity: com.android.documentsui/.files.FilesActivity
TotalTime: 1865
WaitTime: 1869
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=foundation.e.mail }
Status: ok
LaunchState: COLD
Activity: foundation.e.mail/com.fsck.k9.activity.MessageList
TotalTime: 1866
WaitTime: 1873
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.settings }
Status: ok
LaunchState: WARM
Activity: com.android.settings/.Settings
TotalTime: 583
WaitTime: 584
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.gallery3d }
Status: ok
LaunchState: UNKNOWN (-1)
Activity: com.android.gallery3d/.app.GalleryActivity
WaitTime: 7861
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=foundation.e.calendar }
Status: ok
LaunchState: COLD
Activity: foundation.e.calendar/com.android.calendar.AllInOneActivity
TotalTime: 1165
WaitTime: 1168
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.contacts }
Status: ok
LaunchState: COLD
Activity: com.android.contacts/.activities.PeopleActivity
TotalTime: 1088
WaitTime: 1091
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.generalmagic.magicearth/com.generalmagic.android.map.MapActivity }
Status: ok
LaunchState: UNKNOWN (-1)
Activity: com.generalmagic.magicearth/com.generalmagic.android.map.MainMapActivity
WaitTime: 1443
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=org.lineageos.eleven }
Status: ok
LaunchState: COLD
Activity: org.lineageos.eleven/.ui.activities.HomeActivity
TotalTime: 1951
WaitTime: 1956
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=foundation.e.notes/.android.activity.NotesListViewActivity }
Status: ok
LaunchState: COLD
Activity: foundation.e.notes/.android.activity.NotesListViewActivity
TotalTime: 1652
WaitTime: 1657
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.android.calculator2/.Calculator }
Status: ok
LaunchState: COLD
Activity: com.android.calculator2/.Calculator
TotalTime: 880
WaitTime: 884
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=org.lineageos.recorder }
Status: ok
LaunchState: COLD
Activity: org.lineageos.recorder/.RecorderActivity
TotalTime: 716
WaitTime: 720
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.deskclock }
Status: ok
LaunchState: COLD
Activity: com.android.deskclock/.DeskClock
TotalTime: 917
WaitTime: 922
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.dialer }
Status: ok
LaunchState: COLD
Activity: com.android.dialer/.main.impl.MainActivity
TotalTime: 1906
WaitTime: 1910
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=foundation.e.message/com.moez.QKSMS.feature.main.MainActivity }
Status: ok
LaunchState: COLD
Activity: foundation.e.message/com.moez.QKSMS.feature.main.MainActivity
TotalTime: 2032
WaitTime: 2042
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=foundation.e.browser }
Status: ok
LaunchState: COLD
Activity: foundation.e.browser/com.google.android.apps.chrome.Main
TotalTime: 1299
WaitTime: 1303
Complete
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=foundation.e.camera }
Status: ok
LaunchState: COLD
Activity: foundation.e.camera/net.sourceforge.opencamera.MainActivity
TotalTime: 2745
WaitTime: 2753
Complete

2 Likes

This script is really nice.
I tried it with my FP3 and my startup times are always worse than yours, but still way faster than the “excessive” values from above. However, the screen timeout setting somehow failed silently and the screen locked while it was running. I simply used the “caffeine” feature of Android to keep it on.

It would be really nice if one could develop this into an app. Then we could have objective measurements whenever we get in the “my phone got slow after the recent update” situation.