Timeout & Retries – learnings

1. As an API developer, my first goal is to provide robust client side implementation for request timeout and it is different from connection timeout. Both too high or too low is generally considered bad. It should be based on your promised SLA on latency. You may eventually end up tuning connection and other underlying platform level configurations that are optimal for your use case.
2. Retries are my client’s unintentional punitive actions against you for violating or not improving SLA. This outcome becomes severe when your service starts failing under load and the retries increases exponentially though there is no point of beating a dead horse, hence comes backoff.
3. Another important point is about cumulative retries from chain of services. You might be having complete useless timeouts as one of the way down dependency does not even honour you min timeout.
4. Exponential backoff is good to reduce contention but does not do anything to distribute the incoming traffic into equal time slices. This is why the leaky bucket or fixed window or the token bucket algorithm performs poorly on edge cases. On the other hand jittered backoff helps to shape traffic using randomness and it could seen in sliding window rate limiting.
5. Be extreme careful not to degrade customer experience while using Jitter, you do not want to lose them with delays but rather give them frustration free simple in and out experience.
6. Another important API property to pay attention is idempotent. If you are using any identifier based approach to identify duplicate requests then be sure to include that in retry request.

Software engineering Best practices – Configuration

1. Making configuration explicit but simple at the same time to avoid client surprises. The goal should be, if it is read by multiple audience, it should convey the same meaning and should be agnostic to the audience/engineer’s experience.
2. In modern distributed system, which are divided into control plane & data plane, any configuration to an API or service, is a control plane operation and gets better suited with control plane APIs and not to be confused with data plane API input filter.

Control plane vs Data plane

In my experience, this distintiction exists because it helps you to set right design and development goals for scalability, latency & different other properties of the. Control plane is a collection of services that is used to create & re/configure environments & runtimes, where data plane is responsible for delivering service/s for get/read or set/write/update data required to perform an end user action. In simpler form, Control plane services are used to commission/de and manage data plane services.

1. Usually APIs/ports exposed by data plane services are separate from data plane service APIs as these are really control APIs, e.g. these APIs are side car implementation of control plane pieces lying with data plane service.
2. Intuitively, control plane processes should call data plane control ports/APIs. It could be tempting to make the configuration pull from data plane services rather than push from control plane services but that loses meaning of the word ‘control’ w.r.t how control plane should dictate configuration push, load on data plane services or controlled configuration rollout.
3. On the other hand inversion of control makes the data plane services more independency on choosing the when part of equation of adopting change.

memcached and distribution or assignment

I have been using Memcached in a lot of different type of caching solutions for web services and thought of sharing some basic facts,

1. Memcached servers are just very efficient key value storage providing APIs to get and set values over the network.
2. Client makes the solution distributed. The client has the logic to select or choose the destination host for get or set operation using consistent hashing.
3. The memcached client library supports a number of different distribution algorithms that are used in multi-server configurations. Please read the protocol.
4.  The cache key is used in the consistent hashing to determine distrubution. The same server is selected during both set and get operations. The algorithms are Ketama and Wheel in code.
5. following diagram is for simplicity in understanding the process.

In the next post, I will show how to use Guava and Memcached together to implement a multi layered caching solution.

0 1 knapsack Java Code

     /**
     *
     * @param w weight for items
     * @param v values for items
     * @param i number of items
     * @param W the max weight
     * @return what knapsack/0-1 should return
     */
    static int knapsack(int[]w, int []v, int i, int W){
        if(i < 0) return 0;         if(w[i] > W){
            return knapsack(w,v, i-1, W);
        }else {
            return Math.max(knapsack(w,v, i-1, W), knapsack(w,v, i-1, W-w[i])+ v[i] );
        }
    }

2.11 years without a post

I like writing about what I am thinking but somehow I have stopped for past 2.11 years after I changed my job. I did not do well though to put an excuse for not writing so I am going to write about many interesting things I found along the way and about my new research on dimensions “From 3 to 2 or 3 to 4“

Writing your blog posts in Assamese

Ok, There is no direct support to write blog posts (atleast in wordpress or blogspot) in assamese but you can write with help of some unicode editors. One important thing to remember that your browser need to support unicode formats. Well, IE 6+ and Mozilla Firefox supports unicode character formats.

Steps:

• Open an online unicode editor ( Editor : http://www.assamesemail.com/editor/im3.htm Main Page – Unicode Online : http://www.unicodeonline.com/assamese/index.htm) type the words you want to type. You do not need an assamese keyboard for the same (I doubt if there is one). For example to type মই, you have to type “moi”. but the editor is pathetic and you will lose your mind so better use http://ekushey.org/projects/browser_ime/qwerty-phonetic or http://www.lipikaar.com/. Both of these sites a specially for bengali so, we need just replace “R” in app
• select and copy the words and paste it in your blog editor. You are done.

In assamese , “O mor apunar desh”…

অ’ মোর আপোনার দেশ,

অ’ মোর চিকুনি দেশ,

এনে খন সুব্ লা , এনে খন সুফলা, এনে খন মরমর দেশ।

এনে খন সুব্ লা , এনে খন সুফলা, এনে খন মরমর দেশ।

…..