I have been wearing an Omega Seamaster daily for several years. But a few months ago, I switched to the Pebble Steel. Since then, the Omega watch has been sitting in the winder all the time because the Pebble gives me one thing really important: I am not missing any calls or meeting reminders any more.

In the past, I missed a lot calls and meeting reminders because the phone was put in vibration mode or I didn’t hear it ringing because of other noises. For example:

• My phone was being charged in my office and I was talking to someone in the next door. Unfortunately, in the previous meeting, I turned it to vibration mode. I was so focused on the discussion and lost track of time. I missed the reminder of the next meeting and only realized it when it’s already half way through. I felt awful to be so much late to the meeting. Sometime the meeting had to be rescheduled due to my no show and it was me that wasted others’ time.


• My phone was being charged in the family room when I was cooking dinner in the kitchen. The range hood was running right in front of me and I can’t hear anything else. My wife called me on her way home to ask me if I want her to pick up any grocery or bakery. I missed her call (and the first thing she said when she came in was “why didn’t you answer my call”).


• My phone was in vibration mode and unfortunately put on an ottoman. The soft cushion of the ottoman absorbed all the vibrations when a call came in and I missed the call.


• My wife and I were skiing and we put our phone in the pocket of the ski jacket. When the ski school called to tell us to pick up our boy earlier, we both missed the call. We couldn’t hear the ring because we had helmet. We didn’t feel the vibration because we were in motion and we had layers under the ski jacket.

Those are not happening any more since I got the Pebble. When there is an incoming call or meeting reminder, it vibrates and I can feel it because it’s tied to my wrist and touches my skin directly. But I am now in constant struggling. When it comes to the craftsmanship, the Pebble can’t compare to my Omega. The Omega Seamaster feels much better and looks much more beautiful.

But I end up now wearing the Pebble everyday because it solves a big problem for me. So when it comes to “How should luxury watch brands like Rolex strategically respond to the launch of the Apple Watch?”, I hope they won’t try to wedge some crappy half-baked smartwatch features into their beautiful watches. On the other hand, I hope they can add two things to the traditional watches:

1. Sync up with my phone and vibrate when there is a notification. Vibration would be enough. It doesn't need to display anything. As long as I feel the vibration, I know I should check my phone.
2. Have health sensors and send data back to my phone as how Pebble, Fitbit, Jawbone and soon Apple Watch do. I love to see how many steps I have walked each day, how long I slept last night and how many hours were deep sleep.

Well, adding these two things to my Omega Seamaster means adding battery to it and it needs to be charged every a few days. That's OK with me. Even if I forget to charge the "smart Omega" and the battery runs out, it’s still a really nice Omega like what it is today. If Pebble or Apple Watch's battery is dead, it will be a piece of useless metal, or a beautiful bracelet at the best.

Bottom line: as Michael Wolfe said, they should keep Rolex watches Rolex-y.

Recently in his blog post "Microservices. The good, the bad and the ugly", Sander Hoogendoorn warned us about the versioning hell of microservices. He wrote:

I wrote almost the same in an internal discussion late last year in my team. In my internal memo “Transient States Testing Challenge”, I warned that this problem is emerging as a component gets split into a few smaller pieces (aka. microservices), the testing cost may increase substantially and we must understand and prepare for it. Here is the full text of the memo (with sensitive details removed):

---

I have heard people say "Don't tell me this is achievable. Tell me how to achieve." So is it useless to know the achievability while not knowing how to? Not really. Knowing how to achieve will definitely make things more straightforward, but when you don't know how to, there is still a great value to know the achievability.

Jared Diamond, in his book "Guns, Germs, and Steel: The Fates of Human Societies", said that in the history, there were two ways how a society learned skills (e.g. grow wheat, tame cows, etc.) from another society in a nearby region:

1. They learned the skills directly.


2. They didn't learned the skills directly, but they learned the achievability of certain things, then figured out the exact method by themselves.

For example, a society learned from a nearby society that cows are tamable and make good milk. They probably didn't get to learn the exact method of domesticating cows from the nearby society, due to the language barrier, the nearby society's unwillingness to share knowledge in order to keep competitive advantages, or whatever reasons. But having seen that the nearby society has successfully domesticated cows gave them the faith that cows are tamable and conviction to search for ways to do it. Also, this society wouldn't potentially waste time trying to tame zebra or antelope. Jared pointed out that knowing which way is a dead end vs. which way can go through helped a lot of societies significantly shorten the time it took for them to advance their developments.

Knowing the achievability has great value in software engineering as well.

Speaking from my own experience. In my current group, a full test pass have about 4,000 test cases in total. When I joined, it took more than 24 hours to run and had only 80%-90% passing rate during most part of a release cycle. People were not happy with it but most of them seemed to think that's just how it supposed to be. I told them no, it definitely can be much better. I told them that when I joined my prior group, things were in bad shape, too: it took >24 hours to run 12,000 test cases and similarly had only 80%-90% rate, but later we fixed it: the test duration shortened to sub-day (somewhere around 16-18 hours) and the failure rate dropped to low single digit %. I kept repeating this to everybody, telling them that since it was achieved in my prior group, we must be able to achieve it here as well. I also told them to have the right expectation of time: in my prior group, it took more than a year to fix it, so we should also anticipate similar amount of time to fix it here.

Knowing the achievability helped. Knowing approximately how long it will take to get there also helped. My team committed to improve the test efficiency and agility and made small and big investments one after another. After about 15 months, we were able to finish the same full test pass in just 7.5 hours. After about 2 years since we started the endeavor, 98-99% passing rate has become the norm. Had we not known the achievability, my team probably would have hesitated to make the investment, or pulled some resource out in the middle of the endeavor, due to having not (yet) seen the light of the end of the tunnel.

In my 10+ years’ experience in software engineering, I have seen that for many times, seemingly good (or at least harmless) ideas, from architecture design to engineering process, caused severe problems after several years. Some of them repeated multiple times or happened in more than one group. So I learned that those are things to avoid.

For example, I learned that in service oriented architecture (or microservices, to use the latest buzz words), 1-to-1 relationship between services is a bad design. It usually doesn't seem harmful in the v1 stage, but it always causes pain in the ass down the road. I shared this learning in Azure last year and it was also shared by Mark Russinovich in 2014 //BUILD conference (“Avoid Cloud Fail”, slice 16-18 in his deck): “Rule: make components capable of talking to multiple versions concurrently”; “Use registration with ‘return address’ pattern”; “Adheres to layering extension model where dependencies are one way”.

The challenge that I have is when I share my learning with people and advise them to do things differently, some of them don’t seem to be able to take the digested knowledge very well, especially when it’s against their intuition. Sometimes it would take quite a lot explanations to help them see why a seemingly good idea is actually a bad idea. It’s like playing the chess. In middlegame, a seemingly wise move may cause a devastating consequence in several steps later. An experienced player A can quickly foresee this and advise the less experienced player B not to make that move. Although B is a world champion poker player, due to having less experience in chess, he is not able to see the trap that far away. To explain it to B, A needs to show B step-by-step how will that move play out. In real life, some people in B’s position are lack of respect of A’s experience, doesn't trust A’s digested knowledge and are not willing to listen to A’s explanation.

For the worse, under the name of being data driven, some of them feel they are doing the right to reject the digested knowledge because it’s not backed up by data. Of course it’s not. It’s not always practical to have as much data. As we move from one project to another, from one company to another, we lose the access to the detail information of the past projects. So it’s very common that we only have the conclusions (e.g. “1-to-1 relationship is a bad design”), and can briefly talk about what happened in the past at a high level, but couldn't present any charts with a lot of numbers, any bug details, or side-by-side performance data comparison. A truly learning organization shouldn't and wouldn't let the pride of their data culture get in the way of learning from the others’ past experience.

Data driven is good (and popular these days), but experience still matters (a lot). As organizations have more metrics and push for data driven design and decision making, they should continue to respect and value experiences. Because experience (or digested knowledge) is the main way how we learn in our life. We were told by master investors that patience is important and buy-and-hold works. We were told that a rule of thumb in real estate is “location, location, location”. We were told that vaccines are safe and children should get vaccination. We don’t ask for data behind these. We know these wisdom came from decades of research, practice and experiments. It’s silly to ignore these only because “I haven’t seen the data”.

For many times when I see someone says "I am blocked", I want to ask them: "In what sense?" In many cases, they are not truly fully blocked. It's just one of their projects has paused, waiting for another thing to finish to continue. For the time being, they could have worked on other projects before the "blocked" one is unblocked.

Too much "I am blocked" is an indicator that the person is unwilling or incapable of adjusting course as needed. It's much easier to say "I am blocked" than to put thoughts in how to shuffle the original schedule and re-prioritize work items, which requires additional brain cycles. It's easier to blame others: "I can't get my job done because I am blocked".

When I closely examine some organizations where there are a lot of "I am blocked", I also find some engineering culture problems behind it. In those organizations, people try to impress their management by promising more things (than what they can deliver). Therefore, during the development cycle, they are overbooked. They can barely keep up with their own original commitments and have little time left to help others. As people have hard time to get traction when they need unplanned work from each other, they exaggerate, hoping that saying "I am blocked" can help get some traction.

In engineering teams which use agile development, it's more likely to hear people say "I am blocked". It's not that agile development is the culprit. It's mainly because many people mistakenly think agile means they only need to think about what to do in the current sprint and not need to worry about the next sprint at all. When the next sprint comes, they start to realize some critical dependency is not ready. So they scream "I am blocked".

In an organization where "I am blocked" is abused, it becomes harder for people to differentiate which one truly needs help immediately and which one can wait. It's like if everybody calls 911 to say "I am dying", it will be hard for 911 to decide whom to send ambulance to first. In the real life, 911 may ask a couple questions to find out whether the man is truly in danger or just in panic. In engineering teams, asking such probing questions may be seen as unwilling to help and bureaucracy: "why waste time asking so many questions, rather than just fix it for us?" It just makes the team culture more unhealthy.

In my observation, high efficiency people are much more unlikely to say "I am blocked". They know how to keep themselves productive. Likewise, an efficient manager knows how to keep his team productivity and keep making progress. They also plan ahead really well. They anticipate that they will need some thing from the other team six weeks from now. So they make the request to that team six weeks earlier, which in turn also helps that team to better arrange their work. If everybody can do more delicate coordination and planning, better tracking and early communicate, there will be less "I am blocked", projects will be less stressful and the team culture will be healthier.

For many times, I found that testing a solution could be equally hard as writing a solution, if not harder. Here is an example: how to randomly generate test cases for a HasCycle() function.

HasCycle() is a coding question that I used for many times in the past: write a function to detect whether there is a cycle in a directed graph. Some candidates finished the code pretty quickly, so my follow-up question would be "how would you test it". For those who manually enumerated test cases quickly and thoroughly, my next question was "how would you randomly generate test cases".

A wrong answer would be to write the solution again, say HasCycle2(), and see if HasCycle() produces the same results as HasCycle2() does for any randomly generated directed graph. That's a wrong method because it's circular proof: using this method requires the proof of HasCycle2() being fully correct, which is the same as to prove HasCycle() being fully correct. Someone argued that he could write HasCycle2() in a different algorithm to make it not a circular proof. No, that would be still a circular proof, just in a disguised form.

There wasn't a satisfactory answer by any candidate, as far as I remembered. To be honest, I didn't know the answer in the first place, either. It wasn't about the answer itself, anyway. It was mainly to figure out how the candidate approaches problems. As long as the reasoning is sound, it would be cool if the candidate told me "no, there is no way, because blah blah blah". But soon I started to think: to be fair, maybe I should figure that out myself first, to make sure either there is an answer, or the answer is "no, there isn't an answer".

I did come up with a way to randomly generate test cases for HasCycle(). Here are the steps:

1. Pick a random integer N. Create a N-nodes directed graph with no arcs.


2. Pick a pair of nodes X and Y randomly, as long as there isn't an X->Y arc in the graph.


3. Add an X->Y arc to the graph and color it as blue.


4. Add the red arcs, too. A red arc is the last arc to form a cycle, in which all other arcs are blue.


5. Repeat 2-4 for a random number of times (or until there is no more blue arc can be added)


6. On the outcome of (5), just remove all the red arcs and the nodes and the blue arcs will become a graph without cycle. Pass it into HasCycle() and it should return false.


7. Or, on the outcome of (5), randomly change the color of one or many red arcs to blue. Remove the remaining red arcs. The nodes and all the blue arcs will become a directed graph with cycle. Pass it into HasCycle() and it should return true.

The crux is how to identify which red arcs to add in step (4). Here is how:

1. Get all the nodes that are reachable from Y by blue arcs, including Y itself, and put them into a list from-Y. This can be done by a simple traversal, following the blue arcs draining from Y.


2. Similarly, get all the nodes from which X is reachable by blue arcs, including X itself, and put them into a list to-X. This can be done by reverse-traversal on the blue arcs sinking into X.


3. For each node i in from-Y and each node j in to-X, create a red arc i->j.

Use an example to illustrate how this works:

First, randomly pick N=11, to generate a 11-nodes directed graph with no arcs:

Add the first blue arc: A and K are randomly picked. A blue arc A->K is added. At the same time, a red arc K->A is added. K->A is red because it will form a cycle together with a blue arc A->K:

Add the second blue arc randomly: B->A. This time, not only A->B should be added as a red arc (because A->B and B->A together will be a cycle), but also K->B should be added ad red arc as well, because B->A, A->K and K->B will be a cycle:

Keep doing this. When G->H is added as blue arc, three red arcs need to be added: H->G, H->A, H->B. Because each of them will form a cycle with other blue arc(s):

Now let's examine a more general example of how to add red arcs. This time, a blue arc G->F is randomly added:

In this case, from-Y is F, C, E and to-X is G, A, B. So we should add nice arcs: F->G, F->A, F->B, C->G, C->A, C->B, E->G, E->A, E->B:

So the outcome of step (5) is this:

Now, if we want a random graph without cycle, we just need to remove all the red arcs:

If we want a random graph with cycle, we will randomly convert one red arc to blue and remove all the rest red arcs. For example, we convert C->A to blue. That will give us a cycle A->G->F->C->A:

I believe this method doesn't have any blind spot. Given enough time, it should be able to generate all possible kinds of directed graph, with 0, 1 or multiple cycles.

//the end