The first two statements of your BASH script should be…

#!/usr/bin/env bash
set -euo pipefail

The first statement is a Mac, GNU/Linux, and BSD portable way of finding the location of the bash interpreter. The second statement combines

    1. “set -e” which ensures that your script stops on first command failure. By default, when a command fails, BASH executes the next command. Looking at the logs, you might feel that the script executed successfully while some commands might have failed. Caveat: Be careful about applying it to existing scripts.
    2. “set -u” which ensures that your script exits on the first unset variable encountered. Otherwise, bash replaces the unset variables with empty default values.
    3. “set -o pipefail” which ensures that if any command in a set of piped commands failed, the overall exit status is the status of the failed command. Otherwise, the exit status is the status of the last command.


  1. Unofficial Bash strict mode
  2. ExplainShell

Keep your dotfiles bug-free with Continuous Integration

Update: As of April 2020, I have switched over to GitHub Actions. Travis CI has become buggy and flaky over time and I got tired of trying to keep the builds green. My GitHub action scripts can be seen here.

Just like many software engineers, I maintain my config files for GNU/Linux and Mac OS in a git repository. Given that, I wrote a fair bit of them in interpreted code, notably, Bash, it is a bit hard to ensure that it is bug-free. The other problem I face is that packages on homebrew, the Mac OS package manager becomes obsolete and gets deleted from time to time.

I added CI testing on Travis CI to prevent these breakages and to ensure that my dotfiles are always in good shape for installation. The great thing about Travis CI is that it is entirely free for open-source repositories even for testing on Mac OS containers.

Read More

Stanford CS251: Cryptocurrencies, blockchains, and smart contracts


  1. Introduction
  2. Creating a Digital currency
  3. Bitcoin Overview
  4. Bitcoin Blockchain
  5. Bitcoin Mining
  6. Bitcoin Miner interactions and Game Theory
  7. Cryptocurrencies: Community, Economics, and Politics
  8. Alternative Consensus
  9. Wallet & Anonymity
  10. Anonymity on Blockchain
  11. Altcoins
  12. Ethereum
  13. Ethereum
  14. Ethereum Governance
  15. Bitcoin Side-chains (guest talk)
  16. Bitcoin Payment channel
  17. Guest talk on Legal by Ben Lawsky  – does not seem worthy of transcribing
  18. Advanced Topics – Quantum Computing, Threshold Signatures, and storing secret state on public chains
  19. Advanced Topics – Smart property, publicly verifiable randomness, and prediction markets
  20. Guest talk by Adam Ludwin (CEO, – does not seem worthy of transcribing

The notes are based on the 2016 version of the course CS251

Android: Fragment related pitfalls and how to avoid them

  1. Don’t use platform fragments (, they have been deprecated and can trigger version-specific bugs. Use the support library fragments ( instead.
  2. A Fragment is created explicitly via your code or recreated implicitly by the FragmentManager. The FragmentManager can only recreate a Fragment if it’s a public non-anonymous class. To test for this, rotate your screen while the Fragment is visible.
  3. FragmentTransaction#commit can fail if the activity has been destroyed.
    “java.lang.IllegalStateException: Activity has been destroyed”
    Why – This can happen in the wild where say right before FragmentTransaction#commit() executes, the user gets a phone call and your activity is backgrounded and destroyed.
    How to trigger manually – The easy way to manually test this is to add a call to Activity#finish() right before FragmentTransaction#commit.
    Fix – Before doing FragmentTransaction#commit(), check that the activity has not been destroyed –Activity#isDestroyed() should return false.
  4. FragmentTransaction#commit can fail if onSaveInstanceState has been called.
    “java.lang.IllegalStateException: Can not perform this action after onSaveInstanceState”
    Why – This can happen in the wild where say right before FragmentTransaction#commit() executes, the user gets a phone call and your activity is backgrounded and paused.
    How to trigger manually – The easiest way to manually trigger this behavior is to call Activity#onSaveInstanceState in your uncommitted code right before the call to FragmentTransaction#commit
    Fix 1 – call FragmentTransaction#commitAllowingStateLoss but that implies that your fragment would be in a different state then the user expects it to be.
    Fix 2 – The better way is to ensure that the code path L which leads to FragmentTransaction#commit is not invoked once Activity’s onSaveInstanceState has been called but that’s not always easy to do.
  5. FragmentManager is null after Activity is destroyed.
    “java.lang.NullPointerException: … at getSupportFragmentManager().beginTransaction()”
    Why – This can happen when the activity has been destroyed before getSupportFragmentManager() is invoked. The common cause of this is when a new fragment has to be added in response to a user action and the user immediately backgrounds the app, again, say due to a phone call, after clicking the button before getSupportFragmentManager() is invoked. Another common case is where an AsyncTask which will call getSupportFragmentManager() in onPostExecute and while the task is engaged in the background processing (doInBackground), the activity is destroyed.
    How to trigger manually – call `Activity#finish()` before `getSupportFragmentManager().beginTransaction()`
    Fix – If getSupportFragmentManager() is being invoked in the Activity, check if it’s null. If it is being invoked inside a Fragment check if isAdded() of the Fragment returns true before calling this.
  6. Avoid UI modifications which are not related to a FragmentTransaction with FragmentTransaction committed using commitAllowStateLoss
    Why – Any UI modifications like modifications of the text in a TextView are synchronous while the execution of a FragmentTransaction via `FragmentTransaction#commitAllowStateLoss()` is asynchronous. If the activity’s onSaveInstanceState is invoked after the UI changes have been made but before commitAllowStateLoss is called then the user can end up seeing a UI state which you never expected them to see.
    Fix – use commitNow() or hook into FragmentManager.FragmentLifecycleCallbacks#onFragmentAttached(). I will admit this I haven’t found a simpler fix for this. And this issue is definitely an edge case.
  7. Saving Fragment State
    As mentioned earlier, a Fragment is re-created on activity recreation by FragmentManager which will invoke it’s default no-parameter constructor. If you have no such constructor then on Fragment recreation, the app will crash with “java.lang.InstantiationException: MyFragment has no zero argument constructor”. If you try to fix this by adding a no argument constructor then the app will not crash but on activity recreation say due to screen rotation, the Fragment will lose its state. The right way to serialize a Fragment’s state is to pass arguments in a Bundle via setArguments.

    MyFragment myFragment = new MyFragment();
    return myFragment;

    The Fragment code should then use getArguments() method to fetch the arguments. In fact, I would recommend a Builder pattern to hide all this complexity.

    Consider this complete example,

    public class MyFragment {
    		private static final String KEY_NAME = "name";
    		public static class MyFragment.Builder {
    		   private final Bundle mBundle = new Bundle();
    		   public Builder setName(String username) {
    		       mBundle.putInt(KEY_NAME, clickCount);
    		       return this;
    		   public MyFragment build() {
    		      MyFragment myFragment = new MyFragment();
    		      // Set the username
    		      return myFragment;
    		public MyFragment() {
    			// Get the user name
    			@Nullable String username = getArguments() != null ? getArguments().getString(KEY_NAME, null) : null;
    	if (!isDestroyed()) {
    	   // Create the Fragment
    		MyFragment myFragment = new MyFragment().Builder().setName(username).build();
    		// Add the Fragment
    		FragmentTransaction ft = getSupportFragmentManager().beingTransaction();
  8. Inside your Fragment code, if you want to decide whether it is safe to execute a UI code or not, rely on isAdded(), if it returns true, it is safe to perform UI modifications, if it returns false, then your Fragment has been detached from the activity either because it has been removed or because the host (Fragment/Activity) is being destroyed.
  9. Callbacks
    To callback into the parent activity/fragment in case of action inside your Fragment, say, a user click, provide an interface (say, MyFragmentListener) which the holding activity/Fragment should implement. In Fragment#onCreateView() get the host via getHost(), cast it to MyFragmentListener, and store it in the instance variable of your Fragment class. Set that instance variable to null in Fragment#onDestroyView(). Now, you can invoke callbacks on this MyFragmentListener instance variable.
  10. Backstack
    Backstack is nuanced and my grasp of it is still limited. What I do understand is that if you want your Fragment to react to the back key press then you should call FragmentTransaction#addToBackStack(backStackStateName) while adding the Fragment via FragmentTransaction and remove it while removing it. Removal from the back stack is a bit more nuanced. Note that, manual removal of a fragment from the back stack is not required in Activity#onBackPressed() as long as your Activity inherits from FragmentActivity.

    SupportFragmentManager manager = getSupportFragmentManager();
    	FragmentManager.BackStackEntry entry = manager.getBackStackEntryAt(manager.getBackStackEntryCount() - 1);
    	if (backStackStateName.equals(entry.getName())) {
    	   manager.popBackStack(backStackStateName, FragmentManager.POP_BACK_STACK_INCLUSIVE);


Android: Handling JPEG images with Exif orientation flags

A JPEG file can have Exif metadata which can provide the rotation/translation field information for a raw JPEG image. So, a landscape raw JPEG image could actually be a portrait because it’s EXIF orientation could be set to ORIENTATION_ROTATE_90, the best way to handle such scenarios is to either use a library like Picasso or Glide or at least learn from them. Here is a piece of code from Picasso which loads a JPEG as an in-memory bitmap and performs the right translation/rotation.

// Get the orientation
ExifInterface exifInterface = new ExifInterface(imageFilePath);
int exifOrientation = exifInterface.getAttributeInt(TAG_ORIENTATION, ORIENTATION_NORMAL)

// Take the source of these methods from 
int exifRotation = getExifRotation(exifOrientation);
int exifTranslation = getExifTranslation(exifOrientation);

Matrix matrix = new Matrix();
if (exifRotation != 0) {
if (exifTranslation != 1) {
  matrix.postScale(exifTranslation, 1);

// Now use this matrix to create a new Bitmap from the existing Bitmap


Testing resumable uploads

The core idea behind resumable upload is straightforward if you are uploading a big file, then you are going to encounter users in the network conditions where they cannot upload the file in a single network session. The client-side code, to avoid restarting the file upload from the beginning, must figure out what portion of the file was uploaded and “resume” the upload of the rest.

How to do resumable upload

Before starting the upload, send a unique ID generated from the file contents to the server like MD-5 or SHA-256. The server decides and declares what the format of that unique ID is. Next, the server responds with an offset which indicates how many bytes server already has. The client uploads rest of the bytes with a Content-Range header.

How to test resumable upload

The right way to verify that this code works is to break the upload intentionally and randomly in the middle and check that the next upload session does not start from zero. Note that, it might not start from the exact byte offset where it was disconnected since the client network stack can have its buffer size to fill and it might discard the buffered bytes in case of an exception.  Therefore, the ratio of the number of bytes read to the file size should be close to one but might not be one.

The right way to verify that this code works is to break the upload intentionally and randomly in the middle and check that the next upload session does not start from zero.

Sample skeleton codes

// SHA-256 or MD-5 whatever the server decides
String uniqueId = generateUniqueId(File file);
// Number of already uploaded bytes (could be zero)
int alreadyUploaded = getAlreadyUploadedByteCount(uniqueId);
// Upload rest of the bytes
upload(file, uniqueId, alreadyUploaded);

boolean upload(File file, String uniqueId, int alreadyUploaded)
 throws IOException {
  InputStream in = getInputStream(file);
  uploadViaHttp(file, uniqueId, alreadyUploaded);
int readCount;

boolean upload(File file, String uniqueId, int alreadyUploaded) 
throws IOException {
  InputStream in = getInputStream(file);
  readCount = 0;  // Testing only
  uploadViaHttp(in, uniqueId, alreadyUploaded);
  double ratio = ((double)readCount)/file.size();  // Testing only

// Standard method
InputStream getInputStream(File file) throws IOException {
    return new FileInputStream(file);

// Enhanced method for testing
InputStream getInputStream(File file) throws IOException {
    return new FileInputStream(file) {
        int read(byte[] b, int off, int len) throws IOException {
            if (random.nextBoolean()) {
                throw new IOException("Intentionally broke the stream");
            int tmp =, 0, b.length());
            readCount += tmp;
            return tmp;

        int read(byte[] b) throws IOException {
            return read(b, 0, b.length);

        int read() throws IOException {


How to speed up HTML5 videos

Some video streaming websites like YouTube provides an option for speeding up/slowing down videos; some don’t. The trick is simple, find out the Video object via


and then set its playbackRate property to the desired value

document.querySelector("video").playbackRate = 2.0;   // For 2X speed-up

These changes have to be made in the Developer console in Chrome or Browser console in Firefox.

Java Musings – initializing a final variable with an uninitialized final variable

Java has fewer quirks compared to C++, but sometimes I do come across surprises.
A code like following will fail to compile since you are trying to initialize a variable with an uninitialized variable.

public class Sample {

  private final String mField1;
  private final String mField2 = mField1 + " two";

  private Sample(String field1) {
    mField1 = field1;

But if instead of directly referencing mField1, you reference indirectly via a getter method code will compile, and mField2 will get “null” value for mField1.

public class Sample {

  private final String mField1;
  private final String mField2 = getField1() + " two";

  private Sample(String field1) {
    mField1 = field1;

  private String getField1() {
    return mField1;

  private String getField2() {
    return mField2;

  public static void main (String[] args) {
    Sample sample = new Sample("one");
    // prints "Field 1 is one"
    System.out.println("Field 1 is " + sample.getField1());
    // prints "Field 2 is null two"!!!
    System.out.println("Field 2 is " + sample.getField2());

Google I/O 2017: Android Notes

Infrastructure – Architecture & Performance

  1. Android Vitals – More visibility in Google Play dev console on battery drain, wakelocks being held for too long, ANRs, crashes, dropped frames, and frozen frames.
  2. Architecture components – better handling of the lifecycle, Room (ORM for Sqlite), live data observers. The API looks clunky though.


  1. 50% 1-star reviews mention stability & bugs.
  2. 60% 5-star reviews mention speed, design, or reliability.
  3. Apps with > 5% crash rate have 30% higher uninstall rate.

Emerging Markets

  1. > 100M users came online in 2016.
  2. 1B 2G devices expected in 2020.
  3. 50% of India is on 2G
  4. 33% users run out of storage in India every day.
  5. Data is expensive – it costs ~$2 to download a 40MB free app in India
  6. 53% users abandon websites if it takes more than 3 seconds to load

Action items

  1. Remove barriers – app size, always online requirement
  2. Optimize for 2G speeds
  3. Build for intermittent connectivity – offline is not a bug, its a state for the users
  4. Provide better multilingual support
  5. Guide new users – Case study: minimalistic empty chrome screen is unwelcoming in India. Adding links to recent websites and news articles made it more welcoming.

Android Go

Rechristened versions of Android One, to be installed on all devices with less than 1GB RAM going forward. Consists of Lite apps. More visibility into data usage, and easy mobile top-ups.

  1. Youtube Go, and Play Go will support P2P video & file sharing.
  2. Chrome Data saver on by default. Related: “Save-Data” header
  3. 10MB per app download size goal
  4. Multi-lingual GBoard
  5. More severe limits on background
  6. DEX reordering to increase disk locality of the content

Slimming down app size

Installed size consists of download size, followed by unpacking, compilation & optimization phase.
20% downloads are canceled or fail if app size >= 100MB

Action items

  1. minification via proguard
  2. split-density apk , can be automated by Google Play if you are willing to hand them your key
  3. vector drawable (API 14+), this might increase CPU & RAM usage for complex drawable though
  4. Exclude sparse translations
  5. Downloadable fonts
  6. AAPT2 – dead version elimination, resource de-duplication, and smarter image cruncher


  1. Kotlin officially supported in Android. Recommended talk: Introduction to Kotlin
  2. App overlay not allowed on the system UI anymore
  3. Android Studio Apk Analyzer is proguard-aware
  4. Brotli compression for Google Play app updates
  5. Instant apps

New Features (only in Android O)

  1.  WebView improvements – safe browsing support, multi-process, crashes + low-mem handling
  2. Auto-fill
  3. Fonts as first class resources
  4. Improved Media file access – ability to share big files like videos from one app to another without fully downloading them first.
  5. New Skia Renderer to cut down jank.
  6. Color management for different color spaces. Support for wide color Gamut. I would highly recommend watching Understanding Color talk.
  7. Even more rigorous clamp down on background processes & wake locks usage to save battery
  8. ART – use JIT profiling to relayout dex file reduces RAM & I/O, new concurrent Garbage Collector (using read barrier) to remove GC pauses,  more code inlining, code relocation, class hierarchy analysis to optimistically decide when classes & methods can be assumed final, automated SIMD instruction generation.
  9. App icons – background & foreground layers from the app (with mask from the system)
  10. Notifications  – Separated into 4 chunks – ongoing, people-to-people, general, and BTW. Notifications won’t reorder while you are looking at them. Notification Channels (notification categories which users can block/mute), mandatory for targeting API 26. App icon badging (“dot” in Google’ parlance).

New Features (backported via support library v26)

  1. Support library going API 14+ only. 1% users (~20M by Google’s count) are still on API 10.
  2. Downloadable fonts
  3. EmojiCompat for backward emoji support
  4. Better use of RenderThread to cut down on jank (frame drops)

Floating point in user-facing strings

Priceline floating point mistake

Incorrect floating representation of 507.45

%f in user-facing strings is dangerous. Depending on the architecture, programming language involved, version of that language and compiler optimization flags, results can vary slightly. And if there are multiple languages involved in the serving stack, it is almost impossible to argue with the outcome. If those variations are immaterial, then use %.1f or %.2f to get one or two digits of precision after the decimal point, respectively. Otherwise, don’t use %f at all.