Circle CI vs Travis CI

I maintain a somewhat popular Android developer tool (adb-enhanced). The tool is written in Python, supporting both Python 2 and 3. Testing the tool requires both Python runtime as well a running Android emulator. I, initially, used Travis CI for setting up continuous testing of this tool. Later, I felt that Travis CI was too slow and when I came across Circle CI, I decided to give it a try. As of now, both Travis and Circle CI are used for testing. Here is what I learned from my experience.

Read More

Stanford CS251: Cryptocurrencies, blockchains, and smart contracts

Lectures

  1. Introduction
  2. Creating a Digital currency
  3. Bitcoin Overview
  4. Bitcoin Blockchain
  5. Bitcoin Mining
  6. Bitcoin Miner interactions and Game Theory
  7. Cryptocurrencies: Community, Economics, and Politics
  8. Alternative Consensus
  9. Wallet & Anonymity
  10. Anonymity on Blockchain
  11. Altcoins
  12. Ethereum
  13. Ethereum
  14. Ethereum Governance
  15. Bitcoin Side-chains (guest talk)
  16. Bitcoin Payment channel
  17. Guest talk on Legal by Ben Lawsky  – does not seem worthy of transcribing
  18. Advanced Topics – Quantum Computing, Threshold Signatures, and storing secret state on public chains
  19. Advanced Topics – Smart property, publicly verifiable randomness, and prediction markets
  20. Guest talk by Adam Ludwin (CEO, chain.com) – does not seem worthy of transcribing

The notes are based on the 2016 version of the course CS251

Android: Fragment related pitfalls and how to avoid them

  1. Don’t use platform fragments (android.app.Fragment), they have been deprecated and can trigger version-specific bugs. Use the support library fragments (android.support.v4.app.Fragment) instead.
  2. A Fragment is created explicitly via your code or recreated implicitly by the FragmentManager. The FragmentManager can only recreate a Fragment if it’s a public non-anonymous class. To test for this, rotate your screen while the Fragment is visible.
  3. FragmentTransaction#commit can fail if the activity has been destroyed.
    “java.lang.IllegalStateException: Activity has been destroyed”
    Why – This can happen in the wild where say right before FragmentTransaction#commit() executes, the user gets a phone call and your activity is backgrounded and destroyed.
    How to trigger manually – The easy way to manually test this is to add a call to Activity#finish() right before FragmentTransaction#commit.
    Fix – Before doing FragmentTransaction#commit(), check that the activity has not been destroyed –Activity#isDestroyed() should return false.
  4. FragmentTransaction#commit can fail if onSaveInstanceState has been called.
    “java.lang.IllegalStateException: Can not perform this action after onSaveInstanceState”
    Why – This can happen in the wild where say right before FragmentTransaction#commit() executes, the user gets a phone call and your activity is backgrounded and paused.
    How to trigger manually – The easiest way to manually trigger this behavior is to call Activity#onSaveInstanceState in your uncommitted code right before the call to FragmentTransaction#commit
    Fix 1 – call FragmentTransaction#commitAllowingStateLoss but that implies that your fragment would be in a different state then the user expects it to be.
    Fix 2 – The better way is to ensure that the code path L which leads to FragmentTransaction#commit is not invoked once Activity’s onSaveInstanceState has been called but that’s not always easy to do.
  5. FragmentManager is null after Activity is destroyed.
    “java.lang.NullPointerException: … at getSupportFragmentManager().beginTransaction()”
    Why – This can happen when the activity has been destroyed before getSupportFragmentManager() is invoked. The common cause of this is when a new fragment has to be added in response to a user action and the user immediately backgrounds the app, again, say due to a phone call, after clicking the button before getSupportFragmentManager() is invoked. Another common case is where an AsyncTask which will call getSupportFragmentManager() in onPostExecute and while the task is engaged in the background processing (doInBackground), the activity is destroyed.
    How to trigger manually – call Activity#finish() before getSupportFragmentManager().beginTransaction()
    Fix – If getSupportFragmentManager() is being invoked in the Activity, check if it’s null. If it is being invoked inside a Fragment check if isAdded() of the Fragment returns true before calling this.
  6. Avoid UI modifications which are not related to a FragmentTransaction with FragmentTransaction committed using commitAllowStateLoss
    Why – Any UI modifications like modifications of the text in a TextView are synchronous while the execution of a FragmentTransaction via FragmentTransaction#commitAllowStateLoss() is asynchronous. If the activity’s onSaveInstanceState is invoked after the UI changes have been made but before commitAllowStateLoss is called then the user can end up seeing a UI state which you never expected them to see.
    Fix – use commitNow() or hook into FragmentManager.FragmentLifecycleCallbacks#onFragmentAttached(). I will admit this I haven’t found a simpler fix for this. And this issue is definitely an edge case.
  7. Saving Fragment State
    As mentioned earlier, a Fragment is re-created on activity recreation by FragmentManager which will invoke it’s default no-parameter constructor. If you have no such constructor then on Fragment recreation, the app will crash with “java.lang.InstantiationException: MyFragment has no zero argument constructor”. If you try to fix this by adding a no argument constructor then the app will not crash but on activity recreation say due to screen rotation, the Fragment will lose its state. The right way to serialize a Fragment’s state is to pass arguments in a Bundle via setArguments.

    The Fragment code should then use getArguments() method to fetch the arguments. In fact, I would recommend a Builder pattern to hide all this complexity.

    Consider this complete example,

  8. Inside your Fragment code, if you want to decide whether it is safe to execute a UI code or not, rely on isAdded(), if it returns true, it is safe to perform UI modifications, if it returns false, then your Fragment has been detached from the activity either because it has been removed or because the host (Fragment/Activity) is being destroyed.
  9. Callbacks
    To callback into the parent activity/fragment in case of action inside your Fragment, say, a user click, provide an interface (say, MyFragmentListener) which the holding activity/Fragment should implement. In Fragment#onCreateView() get the host via getHost(), cast it to MyFragmentListener, and store it in the instance variable of your Fragment class. Set that instance variable to null in Fragment#onDestroyView(). Now, you can invoke callbacks on this MyFragmentListener instance variable.
  10. Backstack
    Backstack is nuanced and my grasp of it is still limited. What I do understand is that if you want your Fragment to react to the back key press then you should call FragmentTransaction#addToBackStack(backStackStateName) while adding the Fragment via FragmentTransaction and remove it while removing it. Removal from the back stack is a bit more nuanced. Note that, manual removal of a fragment from the back stack is not required in Activity#onBackPressed() as long as your Activity inherits from FragmentActivity.

 

Android: Handling JPEG images with Exif orientation flags

A JPEG file can have Exif metadata which can provide the rotation/translation field information for a raw JPEG image. So, a landscape raw JPEG image could actually be a portrait because it’s EXIF orientation could be set to ORIENTATION_ROTATE_90, the best way to handle such scenarios is to either use a library like Picasso or Glide or at least learn from them. Here is a piece of code from Picasso which loads a JPEG as an in-memory bitmap and performs the right translation/rotation.

 

Mac OS: App Translocation and Android Studio updates failure

I installed Android Studio via homebrew “brew cask install android-studio” as a part of my automated Mac OS setup. Recently, Android Studio prompted me that an update is available. When I accepted to update, it failed with an error “Studio does not have write access to /private/var/folders/wt/rjv6_wcn4f97_2nth7fqftqh0000gn/T/AppTranslocation/19A80F28-865B-41FC-AA87-B8E43C826FCB/d/Android Studio.app/Contents. Please run it by a privileged user to update.” This error was confusing; I was running Android Studio as myself, a nonprivileged user and the same user owned this directory. Googling it a bit for AppTranslocation took me here.

From this point onward, the issue and the fix were relatively straightforward. Apple marked Android Studio as quarantine and hence was in a read-only directory. Quarantine status was confirmed by

The fix was relatively simple as well

Update worked as expected after this.

Setting HOMEBREW_CASK_OPTS=–no-quarantine seems to enable this behavior for all the future installs.

Cross-language bridge error handling: JS-to-Java Example

All languages have certain semantics for dealing with error cases. C deals with them by setting error codes. Java deals with them by throwing exceptions. JavaScript deals with them by throwing exceptions as well but unlike Java, it does have any concept of checked Exceptions. The JS interpreter just stops. And this has some interesting implications in hybrid scenarios like a Webview based app.

Consider a simple Android app where most of the code is in JavaScript but is making a request to Java layer.

 

After starting the app, check that the READ_CONTACTS is revoked or revoke it with adbe

Now, when you click the “Click me” button, you will see a SecurityException in the logs but the app won’t crash. If you check Thread’s name via Thread.currentThread().name, it will return JavaBridge. It seems any Exception thrown on this thread is simply swallowed. This won’t show up in analytics or crash log reports. Your Javascript code on return simply won’t be executed. And your app will appear unusable. This is worse than crashes. crashes at least give the app a chance to get out of a bad state.

This is worse than crashes. crashes at least give the app a chance to get out of a bad state.

Remedy

Sending error information across languages is hard. At the bare minimum, every such call should be encapsulated with a try-catch which catches Exception. For severe unexpected errors, it might not be bad to let the app crash, as you would, while writing the Java code.

 

Full code for this blog post is posted on Github

Testing resumable uploads

The core idea behind resumable upload is straightforward if you are uploading a big file, then you are going to encounter users in the network conditions where they cannot upload the file in a single network session. The client-side code, to avoid restarting the file upload from the beginning, must figure out what portion of the file was uploaded and “resume” the upload of the rest.

How to do resumable upload

Before starting the upload, send a unique ID generated from the file contents to the server like MD-5 or SHA-256. The server decides and declares what the format of that unique ID is. Next, the server responds with an offset which indicates how many bytes server already has. The client uploads rest of the bytes with a Content-Range header.

How to test resumable upload

The right way to verify that this code works is to break the upload intentionally and randomly in the middle and check that the next upload session does not start from zero. Note that, it might not start from the exact byte offset where it was disconnected since the client network stack can have its buffer size to fill and it might discard the buffered bytes in case of an exception.  Therefore, the ratio of the number of bytes read to the file size should be close to one but might not be one.

The right way to verify that this code works is to break the upload intentionally and randomly in the middle and check that the next upload session does not start from zero.

Sample skeleton codes

 

How to speed up HTML5 videos

Some video streaming websites like YouTube provides an option for speeding up/slowing down videos; some don’t. The trick is simple, find out the Video object via

and then set its playbackRate property to the desired value

These changes have to be made in the Developer console in Chrome or Browser console in Firefox.

Introducing adb-enhanced: A swiss army knife for Android development

Android development requires tons of disconnected approaches for the development and testing. Consider some scenarios

  1. To test runtime permission – Go to Settings -> Applications -> Application info of the app you are looking for and disable that permission.
  2. To test a fresh install – adb shell pm clear-data com.example
  3. To test your app under the battery saver mode – turn on the battery saver mode by expanding the notification bar
  4. To stop the execution of an app –  kill it via activity manager, adb shell am kill com.example
  5. To test your app under doze mode – first, make the device believe that it is unplugged via “adb shell dumpsys battery unplug”, then, make it think that it is discharging via “adb shell dumpsys battery set status 3”, and then enable doze mode via “adb shell dumpsys deviceidle force-idle”. And don’t forget to execute a set of unrelated complementary commands once you are done to bring the device back to the normal state.
  6. To see the overdraw of the app – Go to the developer options and enable/disable it there.
Over time, this became a significant mental burden that I first wrote some of these flows in a text file and then converted them to automated shell scripts. But when even that felt insufficient, I created a tool for myself called adb-enhanced.
How it works:
First, install the tool. I wrote this in Python, so, if the following command does not work, install Python

Now, let’s look at the about use-cases again with this tool:

  1. To test a runtime permission :
  2. To test a fresh install –
  3. To test your app under the battery saver mode –
  4. To stop the execution of an app –
  5. To test your app under doze mode
  6. To see the overdraw of the app
I open-sourced the code at https://github.com/ashishb/adb-enhanced. See the GitHub repository for what all this tool can do. Feedbacks and pull requests are welcome.