Introducing adb-enhanced: A swiss army knife for Android development

Android development requires tons of disconnected approaches for the development and testing. Consider some scenarios

  1. To test runtime permission – Go to Settings -> Applications -> Application info of the app you are looking for and disable that permission.
  2. To test a fresh install – adb shell pm clear-data com.example
  3. To test your app under the battery saver mode – turn on the battery saver mode by expanding the notification bar
  4. To stop the execution of an app –  kill it via activity manager, adb shell am kill com.example
  5. To test your app under doze mode – first, make the device believe that it is unplugged via “adb shell dumpsys battery unplug”, then, make it think that it is discharging via “adb shell dumpsys battery set status 3”, and then enable doze mode via “adb shell dumpsys deviceidle force-idle”. And don’t forget to execute a set of unrelated complementary commands once you are done to bring the device back to the normal state.
  6. To see the overdraw of the app – Go to the developer options and enable/disable it there.
Over time, this became a significant mental burden that I first wrote some of these flows in a text file and then converted them to automated shell scripts. But when even that felt insufficient, I created a tool for myself called adb-enhanced.
How it works:
First, install the tool. I wrote this in Python, so, if the following command does not work, install Python

Now, let’s look at the about use-cases again with this tool:

  1. To test a runtime permission :
  2. To test a fresh install –
  3. To test your app under the battery saver mode –
  4. To stop the execution of an app –
  5. To test your app under doze mode
  6. To see the overdraw of the app
I open-sourced the code at https://github.com/ashishb/adb-enhanced. See the GitHub repository for what all this tool can do. Feedbacks and pull requests are welcome.

Swift, Kotlin, and Go

It is impressive to see the amount of similarity which exists in Swift, Kotlin and Go, the three new languages for iOS, Android, and server-development respectively.

Consider a simple, Hello World program.

Swift

Kotlin

Go

 

Server vs mobile development: Where the code runs matter

When the code runs on your servers, you have much more control over the “context” in which it runs. On the mobile devices, the device OS and the user control the context. This difference leads to some subtle implications.

One significant set of differences comes from the lack of control of the platform. For server-side code, one can choose from a wide array of languages. For the mobile code, however, the best choice would almost always be the one dictated by the platform – Java/Kotlin on Android and  Objective-C/Swift on iOS. Further, for the server-side where one can stick to a particular version of the language. In the case of mobile, the platform controls the language version. Same goes regarding the hardware choices – one can choose to use different types of server machines specialized in handling those jobs, eg. GPUs for math-intensive computes. While for the mobile-code, one had to write a good enough fallback to support a wide-enough set the devices. Similarly, the server-side has to rarely worry about the server killing a running process while it is normal for mobile OSes to kill backgrounded processes eventually.

The other set of differences comes from the temporary changes to the platform introduced by the carrier and the user. A network request running on mobile has to be robust enough to deal with intermittent broken connectivity to outright unavailability of a data connection, e.g., due to the user switching to the airplane mode. On mobile, network type matters as well. A network request on cellular would usually cost more than a network request on Wi-Fi to the user and a network request on roaming even more. On mobile, the code has to be aware of not doing unnecessary work when the user is low on battery. Server-side code is rarely subjected to such constraints.

Lastly, it is possible to parallelize and speed up the server-side code by adding more resources like RAM or better CPUs. If a certain number of servers are not enough, you can add more. There isn’t usually a way to offload compute-intensive or memory-intensive work off of the mobile devices without trading it off for network latency and sometimes, user’s privacy as well. While it might be preferable to go with a multi-processing approach in the server code to avoid concurrency issues, on the mobile, however, multi-threading being more straightforward and less resource-intensive is almost always the choice.

Google I/O 2018: Android Notes

Highlights

  1. All android support library code is moving to androidx namespace. No more android.support.v4 or android.support.v7 namespaces.
  2. Android app bundle to split the app into downloadable modules
  3. Navigation library to set up navigation between different activities/fragments.
  4. Use WorkMananger for background work – this is an improvement to JobScheduler
  5. Major improvements to Android Studio. Most standalone tools deprecated in favor of adding the same functionality into Android Studio.
  6. Major improvements to Android Vitals which in Google Play to learn more about what’s going on with Android app’s performance.
  7. Android P does more profiling to improve code locality for faster execution.

Modern Android Development

  1. hierarchy viewer replaced by ViewTree in Android Studio
  2. TraceView replaced by Systrace in Android Studio
  3. Profiler (DDMS) replaced by Android Profiler in Android Studio
  4. Dalvik replaced by ART
  5. Java replaced by Kotlin as preferred language
  6. Layouts
    1. Absolute Layout – deprecated
    2. Linear Layout – still OK
    3. Frame Layout – still OK
    4. Grid Layout – discouraged
    5. Relative Layout – discouraged
    6. ConstraintLayout – recommended
  7. ListView, GridView, and Gallery are deprecated. RecyclerView is recommended with ListAdapter for updations.
  8. Platform Fragments (android.app.fragments) are deprecated. Support library fragments recommended.
  9. Single activity app recommended
  10. Rather than managing activity lifecycle use Lifecycle to observe state changes
  11. Rather than keeping plain data with views, use LiveData to update views automatically. Use ViewModel to deal with screen rotation.
  12. Room is recommended in place of SQLite
  13. CursorAdapter and AsyncListUtil are discouraged. Use Paging instead with a neat, graceful offline use-case handling trick.
  14. Don’t do manual bitmap management. Use Glide, Picasso, or Lottie instead.
  15. Between TextureView and SurfaceView, use SurfaceView.
  16. Nine-patches are discouraged. Use Vector Drawables.
  17. Use FusedLocationProviderClient for fetching device location. Wi-Fi access triangulation is coming to Android P. FusedLocationProvider will eventually add support for that for indoor location tracking.
  18. MediaPlayer is discouraged, use ExoPlayer instead. A detailed hands-on presentation on ExoPlayer

Performance

  1. 42% 1-star reviews mention crashes/bugs
  2. ANR/crash rate reduces engagement – use StrictMode to catch them early
  3. Avoid IPC on the main thread, StrictMode won’t catch the violation, and you would never know what’s happening on the other side of the IPC call which can block the main thread
  4. If BroadcastReceiver#onReceive is going to take more than 10 seconds, then call goAsync for background processing and call PendingResult#onFinish() once that’s finished
  5. WakeLocks – avoid using them. All except PARTIAL_WAKE_LOCK are deprecated with Android P.
    1. Use FLAG_KEEP_SCREEN_ON for keeping the screen on in an activity
    2. Use WorkMananger for background work
    3. Use AlarmManager for short interval callbacks
  6. enums are not discouraged anymore. Platform code avoids it but unlike in prior Android versions, the penalty for enums is low now.
  7. A good introduction to Android’s display rendering
  8. A good introduction to Android’s text handling
  9. Text handling issues
    1. Only framework spans can be parceled, don’t mix custom spans with framework spans since only the frameworks will be part of the copied text
    2. After ~250 spans or more, SpannableStringBuilder ends up being more performant than SpannableString since the former internally uses trees.
    3. Metrics affecting spans cause measure-layout-draw calls, while appearance affecting spans cause layout-draw calls, therefore, the former is more expensive.
    4. Text measurement takes a lot of time on the UI thread, consider creating PrecomputedText on the background thread.
    5. android:autoLink = true works via RegEx and has bad performance. Consider using Linkify on the background thread. Android P also supports deep learning based TextClassifier which is also invoked on the background thread.
  10. Testing your app for background restriction – adb shell appops set <package name> RUN_ANY_IN_BACKGROUND ignore # applies background restriction (change “ignore” to “allow” to remove restriction)

Next Billion Users

  1. 28% searches in India are voice searches
  2. 2-wheeler mode added to Google Maps
  3. Google Tez uses Ultrasound for pairing and sending money to the nearby receiver
  4. Files Go – Folder based hierarchy won’t work for users who have never had PC before. Therefore, shows photos based on associations like “WhatsApp Images” or “Camera”. Supports P2P file sharing for fast transfers.
  5. Design for next billion users – http://design.google/nbu
  6. Android Go
    1. 25% devices in 2018 shipped with <= 1GB RAM (target for Android Go)
    2. Every 6 MB app size reduces install rate by 1%
    3. India – largest market for Go. The USA – second largest market for Go.
    4. 5 seconds cold start goal
    5. Users opt for smaller apk size with a lower rating.
    6. Average apk size is 30 MB. Go recommends 40MB max app size for non-game and 65MB for game apps.

MLKit

  1. On-device + cloud APIs. On device APIs are free.
  2. Firebase support to download model and even do A/B testing.
  3. Experimental support to convert TensorFlow models to TensorFlow Lite models.

Compiler

  1. D8 – new Dexer, now default. Enable/disable via “android.enableD8 = true/false”
  2. R8 – new optimizer, now default replacing proguard, still opt-in in 3.2. Enable via android.enableR8 = true

Kotlin – talk can be here

  1. Use properties in lieu of default getters/setters
  2. Use data classes to generate equals, hash method etc.
  3. Use default args instead of method overloading
  4. Use top-level functions as well as local functions (function inside functions for better encapsulation
  5.  Use function extensions to make the code more readable, eg. extend String class with isDigit to be able to later write the code like “1234”.isDigit
  6. Use smart casts
  7. Use sealed classes for a superclass which should not be extendable beyond the compile-time
  8. Use string interpolation in println, eg. println(“name is ${name}”)
  9. Use val (read-only values) by default, only use var (read-write variable) when there is a strong reason
  10. Declare lambda functions as inline to remove the cost of having an extra class
  11. Use co-routines via async instead of creating new threads
  12. Use suspend + async to convert callbacks into more readable synchronously written code

Diagnosing Mac apps which won’t open (error -10810)

Occasionally, my mac applications end up in a corrupt state where they won’t open. I recently encountered this with Deluge. The first step to diagnose is to open Terminal and open them in the terminal via

Now, the error is more diagnosable but still cryptic. Deluge.app above is a directory and we can navigate to the binary located at Deluge.app/Contents/MacOS/Deluge and execute the actual binary to see a more actionable error

Now, this is something meaningful and Google search worthy. Googling for “Symbol not found: _inflateValidate” takes us to the most likely cause. In my case, uninstall followed by the reinstall of Deluge fixed it.

Java Musings – initializing a final variable with an uninitialized final variable

Java has fewer quirks compared to C++, but sometimes I do come across surprises.
A code like following will fail to compile since you are trying to initialize a variable with an uninitialized variable.

But if instead of directly referencing mField1, you reference indirectly via a getter method code will compile, and mField2 will get “null” value for mField1.

Google I/O 2017: Android Notes

Infrastructure – Architecture & Performance

  1. Android Vitals – More visibility in Google Play dev console on battery drain, wakelocks being held for too long, ANRs, crashes, dropped frames, and frozen frames.
  2. Architecture components – better handling of the lifecycle, Room (ORM for Sqlite), live data observers. The API looks clunky though.

Performance

  1. 50% 1-star reviews mention stability & bugs.
  2. 60% 5-star reviews mention speed, design, or reliability.
  3. Apps with > 5% crash rate have 30% higher uninstall rate.

Emerging Markets

  1. > 100M users came online in 2016.
  2. 1B 2G devices expected in 2020.
  3. 50% of India is on 2G
  4. 33% users run out of storage in India every day.
  5. Data is expensive – it costs ~$2 to download a 40MB free app in India
  6. 53% users abandon websites if it takes more than 3 seconds to load

Action items

  1. Remove barriers – app size, always online requirement
  2. Optimize for 2G speeds
  3. Build for intermittent connectivity – offline is not a bug, its a state for the users
  4. Provide better multilingual support
  5. Guide new users – Case study: minimalistic empty chrome screen is unwelcoming in India. Adding links to recent websites and news articles made it more welcoming.

Android Go

Rechristened versions of Android One, to be installed on all devices with less than 1GB RAM going forward. Consists of Lite apps. More visibility into data usage, and easy mobile top-ups.

  1. Youtube Go, and Play Go will support P2P video & file sharing.
  2. Chrome Data saver on by default. Related: “Save-Data” header
  3. 10MB per app download size goal
  4. Multi-lingual GBoard
  5. More severe limits on background
  6. DEX reordering to increase disk locality of the content

Slimming down app size

Installed size consists of download size, followed by unpacking, compilation & optimization phase.
20% downloads are canceled or fail if app size >= 100MB

Action items

  1. minification via proguard
  2. split-density apk , can be automated by Google Play if you are willing to hand them your key
  3. vector drawable (API 14+), this might increase CPU & RAM usage for complex drawable though
  4. Exclude sparse translations
  5. Downloadable fonts
  6. AAPT2 – dead version elimination, resource de-duplication, and smarter image cruncher

Misc

  1. Kotlin officially supported in Android. Recommended talk: Introduction to Kotlin
  2. App overlay not allowed on the system UI anymore
  3. Android Studio Apk Analyzer is proguard-aware
  4. Brotli compression for Google Play app updates
  5. Instant apps

New Features (only in Android O)

  1.  WebView improvements – safe browsing support, multi-process, crashes + low-mem handling
  2. Auto-fill
  3. Fonts as first class resources
  4. Improved Media file access – ability to share big files like videos from one app to another without fully downloading them first.
  5. New Skia Renderer to cut down jank.
  6. Color management for different color spaces. Support for wide color Gamut. I would highly recommend watching Understanding Color talk.
  7. Even more rigorous clamp down on background processes & wake locks usage to save battery
  8. ART – use JIT profiling to relayout dex file reduces RAM & I/O, new concurrent Garbage Collector (using read barrier) to remove GC pauses,  more code inlining, code relocation, class hierarchy analysis to optimistically decide when classes & methods can be assumed final, automated SIMD instruction generation.
  9. App icons – background & foreground layers from the app (with mask from the system)
  10. Notifications  – Separated into 4 chunks – ongoing, people-to-people, general, and BTW. Notifications won’t reorder while you are looking at them. Notification Channels (notification categories which users can block/mute), mandatory for targeting API 26. App icon badging (“dot” in Google’ parlance).

New Features (backported via support library v26)

  1. Support library going API 14+ only. 1% users (~20M by Google’s count) are still on API 10.
  2. Downloadable fonts
  3. EmojiCompat for backward emoji support
  4. Better use of RenderThread to cut down on jank (frame drops)

Demystifying Android rendering: Jank and ANR

Almost everyone developing an Android app has seen something like this in their logs.

On most devices, the Android platform tries to render a new frame every 16 milliseconds (60 fps).   The rendering requires that whatever work is happening on the UI thread should finish in that timeframe (well, actually in less than that). Any unit of work (== Runnable) scheduled on the UI thread has to fit in that. When the work takes longer, then frames are skipped. One skipped frame is 16 ms of the hung screen. The UI looks janky and unresponsive and if the user interacts with the screen and application does not respond in time (5 seconds) then Application Not Responding (ANR) shows up.

Another interesting scenario where the work can take longer is when the work involves acquiring a lock (for example, for executing synchronized code). Any synchronized code on the main thread is almost always a bad idea since you have no control over who could be holding the lock. And the developer writing the code to run on a background thread has no idea that holding a particular lock can cause any problems. Android has two tools to debug this. One of them is dumpsys for the statistical info; the other one is Method profiling via Android Device Monitor for identifying the culprits.

And, by the way, Choreographer is the class which does the actual rendering work.

 

Floating point in user-facing strings

Priceline floating point mistake

Incorrect floating representation of 507.45

%f in user-facing strings is dangerous. Depending on the architecture, programming language involved, version of that language and compiler optimization flags, results can vary slightly. And if there are multiple languages involved in the serving stack, it is almost impossible to argue with the outcome. If those variations are immaterial, then use %.1f or %.2f to get one or two digits of precision after the decimal point, respectively. Otherwise, don’t use %f at all.