Monthly Archives: September 2013

Breaking Rank: A Hacker’s Way of Defeating Stack Ranking

The Problem

Alarm has been raised in the developer community over stack-ranking of employees: managers are required to rank their employees and report the top and bottom performers, usually done by percentages (top 10%, bottom 10%, etc.). Among other serious problems, this creates an atmosphere of competition and sabotage, and destroys employee morale. Employees at Microsoft, infamous among developers for this system, call the stack-ranking system the “most destructive” process (source). Employees are often given raises – or fired – because they fell into certain brackets.

My dad, a manager for a certain aerospace corporation, mentioned to me over lunch today that his company also does this ranking system, although fortunately his company doesn’t give too much credit in the form of raises or punishments based on this system. As a hacker, my first thought was: how can I break this, and how can I break it in a way that benefits the employees, removes the toxicity of the system, and of course, casts the manager in a positive light?

A naïve solution

Let’s assume that a manager can deal out merits/demerits on their own terms, and that they have hired properly so that their team is full of good developers. As this manager, let’s go to our employees and say, “Hey, we’re going to draw straws instead of do stack ranking. The short straw gets the bottom 10% and the long straw gets the top 10%.” This means that on average, our employees will get an equal number of raises and demerits over time, and our entire team will get the average raise.

Our top performers might complain, but they’ll probably be happier to remove the toxic environment of stack ranking and work in an overall better environment. Plus, they’ll still draw the top 10% straw regularly. If an employee starts slacking off because they’re no longer motivated, it’s up to the manager to provide that incentive to keep working hard – just like in a company without stack-ranking.

But this doesn’t work for everyone: Some employees might draw the short straw too often and get fired by the powers-that-be. So how can we ensure that we give all our employees equal numbers of raises and demerits over the period of a couple of rankings? And how do we make sure that upper management never notices that we’re breaking their system?

Lava Lamp Algorithm

I’m calling this a Lava Lamp algorithm for lack of a better word – if there’s an official name for this kind of thing, please leave a comment. You can find the code for this on my github.

The basic idea is that we create a random list of employees. Let’s take some “random” names and put them in a random order for example purposes:

  • David
  • Zach
  • Tom
  • Nick
  • Mary
  • Allison
  • Alan
  • Sonali

Then we assign each employee a weight based on their position in their stack. Employees at the bottom get “lighter” and employees at the top get “heavier” (like the wax in a lava lamp). Here, I’m using a weight of 2 as a maximum, which worked well in my testing with the size of this list (8).

  • David [-2]
  • Zach [-1.43]
  • Tom [-0.86]
  • Nick [-0.29]
  • Mary [0.29]
  • Allison [0.86]
  • Alan [1.43]
  • Sonali [+2]

Here’s the tricky part – we use the absolute value of their weight to determine the order in which we move them – with smaller values going earlier. This keeps the list from getting stuck in loops where an employee moves up, only to be pushed back down by the next movement. There might be a better way to do this – this is definitely a first shot at the algorithm.

To keep things super deterministic, in the event of a tie of the absolute values, positive weights move first, and in the event of a real tie, the employee with the lexicographically first name goes first. This way the list will always move the same way no matter the implementation. The employees will move in this order:

  • Mary [0.29]
  • Nick [-0.29]
  • Allison [0.86]
  • Tom [-0.86]
  • Alan [+1.43]
  • Zach [-1.43]
  • Sonali [+2]
  • David [-2]

By “move”, I mean that the users will bubble through the list a number of positions determined by their weight as an integer. Our list above would become the following after one iteration:

  • Tom
  • Zach
  • David
  • Nick
  • Mary
  • Sonali
  • Alan
  • Allison

Then we re-add the weights for each spot (“heavier” top, “lighter” bottom):

  • Tom [-2.86]
  • Zach [-1.43]
  • David [-2.86]
  • Nick [-0.57]
  • Mary [0.57]
  • Sonali [2.86]
  • Alan [2.86]
  • Allison [2.86]

Nobody has jumped from the top to the bottom or vice versa, or any other kind of thing that might raise flags – it just looks like the list has shuffled somewhat. Allison has the bottom spot for now, but she also has a weight of +2.86, meaning she’ll move up quite a bit on the next iteration.

I’ve run the algorithm out to 100,000 steps, and as far as I can tell, the employees end up cycling nicely from top to bottom.

Problems

  • There’s always a to-do list with any code, and this is no exception. I wrote this in an afternoon, so there’s going to be some obvious bugs and weirdness. Python isn’t my main language, either, so any help making things more pythonic would be greatly appreciated (as long as the suggestions are put nicely, please!).
  • There’s also a lot of times where very small numbers (due to rounding errors) are introduced. I probably don’t need to remove these, but if there’s an easy way, I’d love to know.
  • There also needs to be more testing of both the code and the algorithm. Everything I’ve thrown at it seems to produce good results, but I’d love for people to come up with corner cases for this kind of thing. Even if this specific algorithm ends up not working, there’s no reason why someone shouldn’t write an edition that actually does work 🙂

Shout-Outs

  • My dad for helping me with some bugs and providing the inspiration for this problem
  • Eric van der Heide (my roommate) for helping me with the heart of the algorithm.
  • Hacker School (which I just attended) for being awesome and helping give me the skills to do something like this