Category Archives: Articles

Scale teams at breakneck speed — without breaking necks

“When an organization grows in size, things that were previously easy become difficult.” — Ben Horowitz

Getting people to work together is harder when they can’t all fit in the same room anymore.

This is a big problem for high-growth startups: the rate at which teams change is so extreme it is difficult to keep up.

Doing something about this involves changes to processes and roles, which is tricky — according to the Harvard Business Review, the standard failure rate for change management projects is 60–70%.

In a former role at a successful IT software startup, I worked with a number of different departments as well as senior leadership. I observed many different solutions employed to deal with the substantial change we went through — from spinning out of a parent company, undergoing leadership change to being acquired ourselves.

In this series, I’d like to share what I’ve learned.

Managing yourself

M.C. Escher — Hand with Reflecting Sphere 1935 Lithograph

People are not robots: we don’t think and act differently just because someone tells us we should. If we’re trying to implement change in an organization, we need to be able to use persuasion to change the way they think and act.

Maintaining a positive mental attitude and setting a clear example with your behaviour are crucial.

Maintaining a positive mental attitude

No matter how well-implemented, change is stressful. Letting it make you disaffected or negative is entirely unproductive — everyone avoids such people. This makes it more difficult to maintain the healthy relationships you need to be able influence people.

The good news is that there is a lot you can do to manage stress. While finding what works for you requires experimentation, “How much exercise are you doing?” is the first thing I ask.

One of the common side-effects of overwork or disruption is reduced exercise. As stress is chemical and exercise flushes your system of stress hormones like cortisol, it is vital that we maintain a basic exercise regime.

This was a hard-learnt lesson for me. During a time of substantial upheaval, I found I was getting seriously distracted thinking about the future of my team — getting to the stage where my sleep was being affected. Unsurprisingly, this also affected the way I was communicating — including, crucially, new senior management. This all changed after I took up Brazilian Jiu Jitsu. The physical release this gave me translated to an almost immediate improvement in how I felt, which greatly improved my ability to advocate for my team, and was ultimately reflected in my appraisals.

It is also important to have the right perspective on what you should focus on — namely those things which matter and are within your sphere of control. If there is nothing you can do to change something from happening, your energy is best spent reacting in the most appropriate way.

Stress management isn’t a panacea of course — there may be misalignment between your personal goals and where your job is currently taking you. If after reflection you conclude this is the case, you should figure out the necessary steps to change your role (or your employer).

Setting an example

The people around you take cues from your behavior. Our former CEO was the hardest working, most enthusiastic person in the company. The fact that he was in the office earlier than anyone else in the morning was a daily reminder that what we were doing mattered. These sorts of signals help keep up the energy and momentum needed to drive change. It’s important for you to ask the question: “What can I do to help motivate and provide clarity to my co-workers?”

 

Communication

Internal communication is the glue that holds an organization together and should not be treated as an after-thought. Without it, a company is just a collection of disconnected individuals each working individually at his or her own job. With it, a company is a unit with power far beyond the sum of its parts.

Conor Neill, Senior Lecturer at IESE Business School

While we know that healthy internal communication is vital for organizations to be effective, discussions around the topic tend to be somewhat superficial. “Poor communication” is a common complaint on websites like Glassdoor (bringing up 465,000 results at last check), but reviews rarely go into any detail.

Sample “poor communication” review

Businesses can only benefit from taking retrospectives more seriously, spending more time thinking about the process of communication rather than tools, encouraging better meeting etiquette, and ensuring good manager-managee feedback.

Retrospection needs to be non-negotiable

Those who cannot remember the past are condemned to repeat it

George Santayana

A reliable indicator for team performance improvement is the execution of retrospectives. These should be iterative and incremental to fix current problems rather than post-mortems at the end of a project which are too late to help.

With the constant stream of new tasks and crises, it can be very tempting to skirt doing these, but this is a big mistake — busy people can’t afford not to review — the improvement in performance is too significant.

In the team I was part of, we took a pretty disciplined SCRUM approach with our retrospectives. This enabled us to incrementally improve the way we worked together sprint by sprint, getting to a stage where we able to completely transform the quality of work we were delivering. Instead of “finger biting” every time there was a new release, regular unexplained downtime, and team members taking time off due to stress we ended up with deployment of new features being on-time and on-spec, high stability, and amongst the best team morale in the company.

We used the following basic process:

  1. Thank top contributors — which helped build morale by demonstrating appreciation for individual contributors
  2. Highlight what went well — which again was good for morale but also helped make sure we preserved the progress we were making sprint to sprint
  3. Highlight what did not go so well — which was a socially accepted way to identify where we were falling down
  4. Determine actions to improve working processes — where we made sure to convert the issues that people raised into constructive change

Process before tools

There are many software companies with great marketing. They will tell you inspirational stories about how using the awesome power of AI/Machine Learning/Natural Language Processing/etc. will transform your organization and make it more competitive than you ever thought possible. The team behind the product all have incredible pedigrees from well-respected universities and market-leading businesses. There will be glowing testimonials and case studies from people who look just like you who work at businesses which seem very similar to yours. The weight of all this makes it seem as though this product will solve all of your issues.

It is important not to get carried away with the hype — while there are certainly some solutions out there which can be very helpful, the most common bottlenecks we face with communication at least are rarely technological.

It is much less important whether you use Slack, Email, IRC, or Yammer than knowing what communication you plan on managing. If you don’t know which groups should know which things & what the most important communication paths are, you don’t have the necessary information to evaluate whether a particular technology is right for you anyway.

However you decide to manage this, it is good practice to make the designated process as explicit as possible so that all employees can understand how communication is supposed to work. The more specific these channels can be the less employees are inundated with correspondence. Also, expectations regarding feedback and response times should be explicit so that people are able to contribute to decisions which affect them.

Meeting etiquette

Meetings are very expensive. If you have 10 developers in a meeting at $100 p/hour, that meeting costs $1000. This could be worth it, but all too often meetings are far from organized.

It is easy to lapse from good meeting etiquette, which ends up wasting a lot of time and energy, so it is important to remind ourselves to be disciplined about how these are run.

Every meeting must have a clear purpose, attendees should be punctual and do the necessary preparation ahead of time, and notes should be taken including follow-up actions. Everyone knows this, but unfortunately, these rules are not always followed due to lack of enforcement. While some of us may balk at openly confronting non-contributing meeting attendees like Elon Musk, requiring attendees to provide the points they want to discuss ahead of time is a good start.

Manager-managee feedback

Employees often leave managers not companies, so getting this communication path right is especially important. One of my managers was great at this. In particular, there were three things from a communication perspective that other managers should emulate:

  • Treat one-to-ones seriously (see meeting etiquette section).
  • Understand the personal goals of the people in your team and align their responsibilities as far as possible.
  • Avoid always acting as the intermediary between your team and other members of the organization.

Common knowledge

Common knowledge refers to the shared norms, values and expertise that enable employees to do their jobs and work together with minimal friction. Without these, it is difficult to create a productive work environment. For example, someone who is extremely direct will inevitably cause a lot of friction in a team that prides itself on sensitivity. This friction can easily become a distraction from getting work done.

Company values

Company values clarify the direction an organization is trying to go in. Unfortunately, in many organizations there is a gap between aspiration and reality. This can lead to distrust of leadership and poor morale. To stop this from happening, leaders need to consider the “cost” of proposed values before sharing them with the company, and to think about the necessary mechanisms to bring about alignment.

“Cost” refers to the downside of having a particular value. To illustrate, we can look at a sample value from Amazon:

Customer Obsession

Leaders start with the customer and work backwards. They work vigorously to earn and keep customer trust. Although leaders pay attention to competitors, they obsess over customers.

Amazon Leadership Principles

Most businesses would agree that caring about their customers is important. However, few would be willing to pay the same cost as Amazon. If Amazon pledges to deliver products which it doesn’t have or can’t find in its warehouses, employees will actually buy those products from rival storesrather than fail to deliver on its promises. This is unprofitable in the short-term, but helps maintain the reputation that has propelled Amazon to become one of the biggest companies in the world.

“Mechanisms to bring alignment” refers to incentives and processes within a organisation aimed at ensuring everyone in the company works according to the same company values. A famous example of this is when Zappos offered to pay people to quit after a philosophical change/reorganization of the company — and 14% took up the offer. This is a big sacrifice — having such a large percentage of your workforce leave couldn’t have been easy to handle. However, in the long run this will mean Zappos will find it easier to move forward, as the the people who would be tempted to go would likely have created friction for the radical change that Zappos was undertaking.

Onboarding

The process of getting new employees up and running provides an excellent opportunity to instruct new starts not only what to do but how they should do it and why it matters. This is clearly a really important focus with 1 in 25 employees leaving due to poor onboarding.

It doesn’t need to be complicated — Google found that they were able to improve the time taken for an employee to reach full productivity by 25% simply by sending the hiring manager a reminder checklist just before the new hire started.

Google splits its onboarding into two parts — a centrally organized corporate phase which lasts for around three days, and the team-based phase led by the new employee’s new line manager.

Rather than inundate the manager with lengthy emails and content, the HR team simply provide a simple checklist of five critical tasks:

Have a role and responsibilities discussion.

Match your new hire with a peer buddy.

Help your new hire build a social network.

Set up onboarding check-ins once a month for your new hire’s first six months.

Encourage open dialogue

Work Rules! by Laszlo Bock, Former SVP of People Operations at Google

This is smart approach because it emphasises the importance of new starts understanding how to answer their own questions. If candidates are expected to “pick it up as they go along” by themselves, they take considerably longer to gain the necessary context they need to do their jobs effectively and generally settle in better.

There is of course plenty you can add to this — for example in the onboarding I had in my team there were objectives around directly communicating to customers and passing exams which demonstrated product-knowledge, but this really is icing on the cake if all of the points in the Google checklist have been met.

Decision making

Jean-Pierre Dalbéra from Paris, France — Le penseur de la Porte de l’Enfer (musée Rodin), CC BY 2.0

When a company is small, it is easy to reach consensus, carry out consultation of stakeholders, and for one person to get all of the necessary information to make a decision.

This changes as team sizes get bigger. To best manage this, it is a good idea to distribute authority and be specific about decision making processes.

Distribute authority

While it can be difficult to loosen the reins if you are used to being in control of every decision in your team, failing to do so will lead to you becoming a bottleneck. Better then to empower more junior employees to make decisions themselves even if you don’t always agree. Figuring out how to do this will also help clear up friction-generating fuzziness around roles and responsibilities — job descriptions tend to be overly general, providing a breeding ground for unhealthy office politics.

Be specific about decision making processes

In order to make sure that everyone is on the same page, it is important that there is a clear method for making decisions, especially when they are high profile.

Deciding on company/product names is an example of one such decision that is often highly contentious. Naming is emotive, so there is real danger of getting stuck in a loop of endlessly generating ideas in a misguided attempt to reach consensus. Better to commit to a process that everyone will respect, where stakeholders “disagree and commit” even if they are not fans of whatever the outcome may end up being. You won’t get perfection, but especially in fast moving companies, perfection doesn’t exist anyway.

To illustrate what such a process may look like, here is a basic outline:

  1. Purpose

Select new company and product names

  1. Measures and Constraints

must be in-line with company values

.com domains must be available

name must be no longer than 4 syllables

Decision maker not able to contribute own ideas to avoid self-preference.

  1. Timeline

Week 1- 2: Company wide email to contribute/vote on ideas, review measures/constraints

Week 2–3: Marketing assistant review of submitted ideas, excluding those which are ineligible. Stakeholder panel votes on preferred ideas, generating shortlist of top 10 ideas

Week 3–4: Decision maker reviews shortlist. If happy with one of options, selects. Otherwise, requests additional round of suggestions from stakeholder panel.

Week 4–5: (If necessary) Stakeholder panel send new shortlist. Decision maker chooses best of options supplied.

  1. Roles

Decision maker: B Baggins

Marketing assistant: G White

Unless measures and constraints are specified ahead of time, it is difficult to to keep them unbiased and data-informed. Without a timeline, the decision is unlikely to happen in a reasonable amount of time.

Being clear about roles and timelines is vital to make sure there is accountability for different stages in the process. Unless this is done it is easy for employees to become cynical and habituated to dysfunction.

 

Can you honestly sell?

Every now and then, someone will give you a piece of advice that hits your brain like a lightning bolt.

It happened for me when I was preparing for an interview and my former co-founder Bjorn said “When you’re doing sales, find a way to like the person you are speaking to”.

Instantly I understood what had been niggling me about sales techniques — at a fundamental level I had been feeling somehow dishonest.

The reams of articles on building rapport overwhelmingly focus on teaching you to be better at giving the appearance of someone who is interested. But I’ve realized if you’re genuinely intrigued by the person you are speaking to, the behaviours they recommend, such as “matching and mirroring” take care of themselves.

More importantly, if any part of your behaviour is pure pretence, adopted in order to manipulate, there’s a strong chance something in your tone, expression, choice of words or body language will give the game away. No one likes being taken for a fool.

If you’re not interested in the person you are speaking to, as opposed to what they can give you, they won’t be interested in you

So how do you take a genuine interest in someone very different from yourself? Many people find this hard. The much vaunted connectivity of social media has reinforced our tendency to focus on people with the same beliefs and opinions as ourselves and to follow news reported with our preferred bias. If you are living in a ‘progressive’ cocoon, for example, how are you equipped to interact positively with a religious conservative?

The answer is — you work at it. You decide to learn from the person. You research them, you find out what is interesting about them, what they could teach you, and how you can help them. Without doing anything else, you have laid the foundation for a productive relationship.

 

Key Performance Indicator (KPI) frameworks

Original publish date: 28th January 2014. This article is still relevant, but please let me know if there are more relevant references to include. 

Key Performance Indicator (KPI) frameworks are very important. As far as Web Analytics is concerned, they have been referred to as the “difference between winners and losers” by none other than Avinash Kaushik, the Digital Marketing Evangelist for Google – one of the most influential people in digital marketing.

What KPI frameworks are, and why exactly we need them

 In a recent article by DM news (http://www.dmnews.com/infographic-marketers-struggle-to-make-the-data-driven-grade/article/322172/), it was revealed that the average grade marketers received when being asked to evaluate their relationship with data was a worrying “B-”. This serious lack of confidence is in some ways unsurprising. Marketers are overwhelmed by data – there are increasing quantities of information from an ever-widening variety of sources which we have the theoretical opportunity to interrogate, but the practical impossibility of being able to examine.

Infographic by DM NEWS

This has serious ramifications – paraphrasing Avinash: the principle cause of failure for most digital marketing campaigns is the inability to assess this data and to determine its significance. Creating a KPI framework, also referred to as a Digital Marketing and Measurement Model, addresses this by organising desired business outcomes such as “increased brand awareness” with things that we can measure such as “branded traffic”. This is done via various processes which vary based on the nature of the particular type of KPI framework used, but all have the same intended outcome: to expose opportunity by helping us prioritise what metrics we should pay the most attention to. Besides helping to direct optimisation, these frameworks have the additional benefits of helping to align thinking across an organisation, and creating consistency with reporting which makes it much easier to quickly gain a holistic view of all digital marketing activity. Hopefully, you are now intrigued by how this tool can help you improve your “data grade”. In the next section we will help you get started by going through the step-by-step process for generating a full KPI framework

The process of building KPI frameworks

In the previous section I outlined what KPI frameworks are, and why exactly we need them. To summarise, KPI Frameworks are an antidote to what Avinash Kaushik calls “the root cause of failure” for many digital marketing campaigns, namely “the lack of structured thinking about what the real purpose of [a] campaign is and a lack of an objective set of measures with which to identify success or failure.”.

In this section, I outline a structured process for generating a Top Down KPI framework (one of the several varieties available), borrowing heavily from Avinash’s article on the subject (http://www.kaushik.net/avinash/digital-marketing-and-measurement-model/), and input from the great guys at Lynchpin (http://www.lynchpin.com/). The different stages involved in generating such a framework are as follows:

  1. Identification of business objectives.
  2. Identification of goals for each objective.
  3. Identification of Key Performance Indicators (KPIs) for each goal.
  4. Identification of targets for each KPI.
  5. Identification of important segments to help interpret progress towards targets.
  6. Identification of Critical Success Factors (CSFs).
  7. Achieving final approval.

The example used to illustrate each of these stages is for a website-based freemium game which monetises through subscriptions. This is motivated by Ambergreen’s work with various gaming clients.

Identification of business objectives

What are business objectives?

Business objectives are the answer to the question, “Why does this website/marketing campaign exist?”. Often, the answer to this question is surprisingly difficult to discern, varying significantly from stakeholder to stakeholder.

Why is setting business objectives important?

Formally specifying business objectives is important to make sure there is true buy-in from decision makers into the purpose of the website/marketing campaign, and that the overall focus is correct. It provides a foundation for the rest of the framework which makes it easier to quickly interpret why particular goals/KPIs exist. It is also an early check on unrealistic expectations so long as the objectives are “DUMB”, or:

  • Doable

The objectives should be within the realm of possibility.

  • Understandable

The objectives should require little explanation, even to those with a cursory understanding of the website/marketing campaign.

  • Manageable

The objectives should be achievable given the constraints on resource.

  • Beneficial

The completion of the objectives should genuinely help the organization that is intended to profit from the website/marketing campaign.

How do you go about determining business objectives?

We recommend going through the following steps when determining business objectives:

  1. Explore the context of the website/marketing campaign

It is important to understand the challenges being faced by the organisation in charge of the website/marketing campaign in order to make sure that the most promising opportunities are considered.

  1. Identify key stakeholders

It is important to make sure that the people who are most involved in the website/campaign are formally identified so that the right people are asked the crucial questions as to what the website/campaign is there to do.

  1. Solicit business objectives from key stakeholders

Once they have been identified, the next stage is to hold discussions with key stakeholders in order to extract what they think the business objectives should be. This is not always easy, as it is often difficult to get hold of senior decision makers, but is very important for discovering true success criteria.

  1. Interrogate the business objectives

It is often useful for whoever is leading the KPI framework generation to identify additional opportunities based on the previous steps. This helps ensure that marketing gets the full credit for what it is responsible for.

  1. Gather business objectives and get sign off from key stakeholders.

Once the business objectives have been collected together into a formal document, getting sign off from the key stakeholders is useful to make sure that subsequent work is not misspent on objectives that lack buy-in.

While going through this process, it is helpful to ignore anything to do with how these objectives are to be achieved. For example, if you immediately get bogged down too soon in the details of planning a PPC campaign targeting generic search terms, it is very easy to get distracted from the fundamental questions regarding why the activity should be done in the first place.

What are some examples of business objectives?

Sample business objectives could be:

  • Grow brand.
  • Generate paying players.
  • Increase Customer Lifetime Value (CLV)

Identification of goals for each objective

What are goals?

Goals are “specific strategies that are necessary to accomplish business objectives” (http://www.kaushik.net/avinash/web-analytics-101-definitions-goals-metrics-kpis-dimensions-targets/#goals). These are the next level breakdown from the top-level business objectives. It is not possible to implement business objectives directly – they are too general. For example, “Generating more awareness” might translate to something like “Reinforce online/offline advertising”.

Why is setting goals important?

Formally identifying goals helps make sure you can see all your options for achieving business objectives. Often people will focus on reducing spend for example, rather than improving conversion rates. Having goals as part of your KPI framework acts as a constant reminder of the different approaches you can take, and thus makes it more likely that resource will be allocated efficiently.

How do you go about determining goals?

The process for generating goals is very similar to that for business objectives, involving discussions with key stakeholders as well as independent research by the analyst generating the KPI framework.

It is important that you focus here on “micro goals” (“the relationship building activities that lead up to a macro conversion” – https://analyticsacademy.withgoogle.com/assets/pdf/DigitalAnalyticsFundamentals-Lesson4.4SettingupgoalsandecommerceText.pdf) as well as “macro goals” (goals that lead directly to your primary business objectives). For example, from a gaming perspective, instead of only thinking about getting players to give money (a macro goal), think in terms of getting players to intermediary stages on the way to giving money, for example, achieving a certain level of social engagement with other players (an example of a micro conversion). this is useful for identifying all opportunities for improvement.

Sometimes, the value of these micro-conversions can be multiples of the macro-conversion For an excellent article on how to determine the value of these micro-conversions, including how to put a value on Youtube views, see http://www.kaushik.net/avinash/web-analytics-tips-identify-website-goal-values/.

What are some examples of goals?

For the Grow brand objective, sample goals might include

  • Macro – Increase the number of people who come to the website
  • Macro – Increase the extent to which people come back to the site
  • Micro – Increase owned channel (e.g. Youtube) reach

Identification of Key Performance Indicators (KPIs) for each goal

What are KPIs?

KPIs are the measures used to determine performance toward goals. It is important to determine the most relevant KPIs for your goals to make sure that a change in those metrics is truly significant in terms of the goal with which it is associated.

Why is setting KPIs important?

Setting KPIs is important because they enable measurement towards goal completion in an objective fashion.

How do you go about determining KPIs?

In order to determine what KPIs to use for a particular goal, answer the following question: “what metrics would give a complete, and specific representation of progress towards my goal?”. This often leads to a series of important questions. For example, are you really interested in pageviews? If someone went to your site to find a particular piece of information, multiple pageviews could be a bad sign, as it required visitors to go through irrelevant content before they found the information they needed. Depending on the scope of your project, you may not be able to use the most appropriate KPIs due to tracking limitations.

What are some examples of KPIs?

For the “Increase the number of people who come to the website” goal, a sample KPI could be:

  • Increase the number of visitors

Identification of targets for each KPI

What are targets?

To borrow from Avinash: “targets are numerical values you’ve pre-determined as indicators of success or failure”.

Why is setting targets important?

Setting targets helps enforce discipline with the way marketing plans are carried out. We all have far too little time to do everything we want to do in marketing – setting targets makes us carry out cost/benefit analyses on a regular basis, helping us to ensure our time is well spent. For example, if we have to choose between spending time trying to get more visitors from organic search, or improving conversion rates, the knowledge that there is very little incremental traffic available would certainly influence that decision – as would be revealed by the process of generating targets. It also lets everyone know instantly whether marketing is on track or not, which makes it easier for people to buy into this data-driven approach.

How do you determine targets?

There are several sources of information to help set targets.

These include:

  • Historical performance.
  • Estimates based on research tools (e.g. using the Adwords keyword planner tool to forecast PPC traffic).
  • Consulting other parties within the organisation (e.g. Finance, people who have run similar websites/campaigns previously).

If there is insufficient data available to create reliable number-based targets, it is best practice to go through the process of trying to set targets regardless until the first batch of actual data comes in to focus marketing effort.

What are some examples of targets?

For the “Increase the number of visitors” KPI, the target could be

  • 10% increase in visitors per month within 6 months.

Identification of important segments to help interpret progress towards the set targets

What are segments?

A segment is a group united by some set of shared characteristics.

Why is setting them important?

Quoting from Avinash (again): ”all data in aggregate is crap”. Fluctuations in business metrics are caused by groups of certain people doing things. It is important to break down overall numbers to help identify precisely where opportunity lies.

How do you determine which segments to use?

Again, consulting key stakeholders will provide valuable insight into which segments to choose for each goal. To help stimulate this conversation, try asking the following questions:

  • Who are we trying to attract?
  • Should we have different outcomes based on the different types of people coming to the website?
  • What are the key traffic drivers?
  • Do we expect traffic behaviour to be different by device?

What are some examples of segments?

For the number of visitors KPI, segments could include:

  • New/returning visitors
  • Traffic source
  • Specific referrers
  • Landing page

Identification of Critical Success Factors (CSFs)

What are CSFs?

A critical success factor is an element of a marketing campaign which is necessary (but not sufficient) to complete its goals.

Why is setting them important?

Without understanding what is necessary to complete marketing goals, these crucial elements may not be dealt with properly, which in turn can critically affect the ability of marketing teams to execute on goals.

How do you determine what your CSFs are?

To identify CSFs, try brainstorming around the following key types of CSFs, as presented by Rockart and Bullen:

  • Industry
  • Competitive strategy and industry position.
  • Environmental factors
  • Temporal factors
  • Managerial position (if considered from an individual’s point of view).

See http://rapidbi.com/how-to-write-a-critical-success-factor-csf/ for more information.

What are some examples of CSFs?

An example of a CSF which applies to the “10% increase in visitors per month goal” could be:

  • Implementation of click tracking (e.g. Bit.ly) to assess traffic sources.

Achieving final approval

Once the full plan has been created, make sure the key stakeholders provide the final sign off on the document. Producing a KPI framework is not an easy thing to do, and will certainly take substantial time to produce, but it is time that you would be hard pressed to find a better use for; a KPI framework acts as the beating heart of well managed digital campaigns.  In terms of estimating how much time to put aside for this process, this is generally proportional to the number of conflicting views/business models in the organisation and how many iterations are required to bring out the common ground. Commonly this in the range of 1-3 days. Also,  as a general rule, if you can’t fit the KPI framework on a single side of A4 then it has become too complex to manage.

It is also worth noting that CSFs can change over time whereas ideally the rest of the framework is reasonably static. We like to connect CSFs to the KPI framework to emphasize that in order for targets to be met, certain things need to happen, but due to the volatility of CSFs this requires regular review.

In the next section we will look at real life examples and sample models for making the most of analytics and KPIs.

Sample models and further discussion

In this section I provide an overview of the alternatives to top-down models, discuss some issues that people commonly encounter when setting up these frameworks and provide pointers to further reading to learn more about this subject.

Consequently this document is split into the following sections:

  • Alternative models
  • Common pitfalls
  • Further reading

Alternative models

While KPI frameworks contain much of the same information, different approaches can be taken based on personal preference. The three alternatives to “Top-Down” are as follows:

Lifecycle-based models

Lifecycle-based frameworks are structured around the different stages of a visitor’s relationship with websites. Consequently goals are bucketed by these different stages, rather than being derived from explicit business objectives as is the case with Top-Down models. There are multiple varieties in this category, including Xavier Blanc’s REAN model, and Dave Chaffey’s RACE model. These are often named according to the way the different visitor interactions are modelled.

For example, in the case of RACE, the stages are:

The definitions provided below are largely taken from http://www.smartinsights.com/digital-marketing-strategy/race-a-practical-framework-to-improve-your-digital-marketing/.

Reach

“Refers to building awareness of a brand … in order to build traffic by driving visits to different web presences”.

Act

“Act is short for Interact. It’s about persuading site visitors or prospects to take the next step, the next Action on their journey when they initially reach your site or social network presence”.

Convert

This is where users reach the “conversion point” e.g. a sale for an e-commerce company.

Engage

This stage is about “building customer relationships over time through multiple interactions using different paid, owned and earned media touchpoints such as your site, social presence, email and direct interactions to boost customer lifetime value”.

Here is an example RACE framework.

While this does not have the segments and targets we see in the Top-Down model, these can (and should) be included in an accompanying document. Dave Chaffey manages this in a formal way with the “SOSTAC®” planning model (http://www.smartinsights.com/digital-marketing-strategy/sostac-model/).

Persona-based models

Persona based frameworks are as the name suggests: based on personas, or “fictional characters created to represent different user types within a targeted demographic, attitude, and/or behaviour set that might use a site, brand or product in a similar way” (http://en.wikipedia.org/wiki/Marketing_persona). Persona-based planning is commonly used in offline marketing and can provide an easy-starting point to digital measurement. The process for creating a persona-based framework is as follows (credit to Lynchpin):

Step 1:  Define key site personas (this can be done via research/workshops) by determining why people might be on the site.

Step 2: Define how each persona may be identified online (i.e. what does the behavioural segment look like).

Step 3 : Define what a successful visit looks like for each persona based on the visitor’s objectives. This involves setting KPIs for each persona.

Step 4 : Use data to verify those personas are accurate. This can be done either by directly asking your visitors e.g. via onsite surveys or email, or by determining whether the given personas show behaviour onsite which is in keeping with the way they have been categorised.

Composite models

Composite models combine different frameworks for the goal of ensuring completeness. E.g. Top-down & persona-based or RACE for each persona.

Common Pitfalls

Successfully implementing KPI frameworks can be tricky. In this section I explore some of the most common issues and offer some solutions.

Too many unknowns and data trust issues

If a company is relatively new to structured approaches to digital measurement, there may be tracking limitations which prevent analysts from being able to see enough of the user journey to assign credible values to each of the conversion types. This makes it difficult to justify decisions which are based on numbers rather than “expert-opinion”.

Related to this issue is a lack of “data trust” – where tracking has been installed, but there are doubts about the accuracy of the data, either because of technical issues with the analytics program/implementation, or because of the assumptions which have been built into it. Attribution is a common area where such doubts arise. This again has the potential to turn people away from data-based decisions entirely.

The answer in both situations is the same – provide a roadmap that addresses the issues that cause the problem or doubts and facilitates a robust framework to be established. Using an interim model, with caveats that cover these issues, is sensible where possible.  It should be noted that Analytics is data is  never 100% accurate – it is intended to provide insights into trends and allow for approximate comparison. Do not become bogged down in calculations if data is marginally out. For further information on this subject, see Avinash Kaushik’s post “Data Quality Sucks, Let’s Just Get Over it” (http://www.kaushik.net/avinash/data-quality-sucks-lets-just-get-over-it/).

Using KPIs that are too high/low for level of reporting

Certain metrics are useless at an aggregate level. One prime example is bounce rate . If a visitor bounces after having read the latest blog post, it would be hard to say for sure whether that it was a “bad” bounce. However, if a visitor came on transactional keywords, landed on a product page and then bounced before adding the item to their shopping cart, it is clear that is not the ideal outcome. Here context is critical to interpretation.

Similarly, trying to apply high-level metrics to low-level activity does not work. For example, attempting to determine the ROI of an individual tweet is a waste of resource as it offers little insight. This is because there are so many influencing factors that ROI measurement becomes futile. A far better approach is to analyse ROI at campaign level as this substantially lessens the issues around attribution.

To summarise; it is important to consider the scope of measures and how meaningful they are when including in reports.

Neglecting to gain buy-in

It is absolutely critical that key stakeholders involved in a project understand the importance of the KPI framework, and are provided with the opportunity of feeding into it. If an individual independently creates a framework without the appropriate feedback it can often result in resistance from the people it affects – even if the framework accurately captures all of the necessary objectives/goals/KPIs. This is because people want to be able to contribute or influence the areas in which they operate, particularly something as important as how results are recorded and performance is measured. Not facilitating this can result in a negative emotional response.

Not knowing what you can/can’t measure

There are limits to what you can/can’t measure, especially with standard analytics installations. For example, it is not possible to measure time on site for visitors who bounce in Google Analytics (given a standard, out-of the box implementation). This is because of the way this metric is calculated using time stamps every time a visitor visits a new page. As with bouncing visitors there is no second pageview: time on site cannot be calculated. Consequently, it is useful to have a technical analyst on hand to be able to report on whether the desired KPIs are immediately measurable and if not, whether it would be possible to customise tracking.

Mixing strategic KPIs and tactical metrics

It is important not to mix strategic KPIs (e.g. ROI) with tactical metrics (e.g. Google PageRank). Tactical metrics are only of use to the extent that they contribute to strategic KPIs. For example PageRank is only useful to the extent it helps garner organic search traffic. Consequently tactical metrics do not belong on the KPI framework.

Further reading

Race planning framework by Steve Chaffey.

Cult of Analytics by Steve Jackson

Web Analytics 2.0: The Art of Online Accountability and Science of Customer Centricity by Avinash Kaushik

Google Analytics by Justin Cutroni

Personas: The Art and Science of Understanding the Person Behind the Visit by Will King

Why NPS is great even though it doesn’t work

Last month I was lucky enough to hear a talk given by Simon Lyons, the director of Marketing & Communications at Aggreko plc. at the Prince’s Scottish Youth Business Trust (PSYBT) Elevator networking event.

Simon had one piece of advice for the roomful of young entrepreneurs eager to grow their businesses: listen to your current customers. It may not be a new idea, but his method for tracking customers’ opinions was novel to many of the people in the room. Simon recommends using something called “NPS” or Net Promoter Score.

What is it?
The Net Promotor Score is an indicator of how loyal your customers are, and can be worked out easily from one simple question.

Ask your customers “Would you recommend [my company] to a friend or colleague?”, giving them a scale from 0 (definitely not) to 10 (very definitely).

Then you can categorise respondents as follows:

0-6: Detractors – unhappy customers
7-8: Passives – satisfied but unenthusiastic customers
9-10: Promoters – evangelists for your company

Subtract the percentage of “Detractors” you have from the percentage of “Promoters”, and you have your NPS. This score can thus range from -100 (meaning all your customers would actively dissuade others from using your company) to +100 (meaning all your customers would strongly recommend using your company).

What is it supposed to do?
Enthusiasts see the NPS as a reliable indicator of future growth.

As the Net Promoter website points out, there are problems with using current sales to indicate future performance. Conventional accounting doesn’t take into account the difference between “Good profits” and “Bad profits”.

The idea behind this distinction is that “Good profits” (those generated by making the customer happy) lead to future growth because customers will continue to buy from them, and promote the company to their associates, leading to referral sales. “Bad profits” (those generated by cutting back services or the abuse of market power) will ultimately lead to a reduction in future sales, as the customers who are being exploited will change suppliers as soon as they can, and actively recommend against using the company to the people they know.

By measuring their Net Promoter Score, managers theoretically gain insight into customers’ reactions to changes they are making. This can help them decide whether to backtrack or push forward. For more information about the way NPS can be incorporated into the way a business is run, see Net Promoter Operating Model.

Criticism
Since Fred Reichheld unveiled the Net Promoter Score in “The One Number You Need to Grow”, the idea has been a magnet for controversy.

Perhaps the most heavyweight opponent is Tim Keiningham, Global Chief Strategy Officer and EVP at IPSOS Loyalty, one of the world’s largest market research firms.

Keiningham says the primary reason why NPS has become so popular is that it was backed up by extensive scientific research suggesting that NPS was always the best indicator of future firm growth.

However, Keiningham et al’s own research found otherwise. It showed that the “recommend intention” was not universally the best single predictor of future growth, and more sophisticated indicators which take multiple variables into account out-performed NPS without exception.

For more detail about these criticisms, see Keiningham’s interview with Admap

Why these criticisms don’t matter
Fair enough, some of the claims for NPS verge on the bombastic. NPS may well not be the best indicator of future loyalty. However, getting into the habit of asking customers what they think and responding to that feedback is extremely important. The complications of setting up traditional surveys puts some businesses off the whole process, and certainly makes it more difficult to rally employees to improve service.  NPS is not the “The Only Number You Need to Grow”, and “Would you recommend [my company] to a friend or colleague?” might not even be the best single question you can ask your customers, but it seems like a good start.

What do you think?
Is this a fair assessment of the Net Promoter Score?

Do you think your business should use Net Promoter Score?

If not NPS, what customer satisfaction metrics does your business use (if any), and how is that data  incorporated into the decision making process?

Net Promoter, Net Promoter Score and NPS are registered trademarks of Bain & Company, Inc., Satmetrix Systems, Inc., and Fred Reichheld.

Stop reading Mashable

Sign of deafness
Be deaf to Mashable

Just because you are being followed by some massive celebrity, does not mean that they are paying any attention to what you say.

It’s not physically possible for the likes of Christopher Penn to read even a small amount of the content produced by the 22,197 people he follows on twitter.

So who does he pay attention to?

Well, according to Klout, @cc_chapman (K:76), @chelpixie (K:52) @chrisbrogan (K:77), @ambercadabra (K:70), @scottmonty (K:76)

K here refers to Klout score – an automatically calculated measure of influence. I have some major concerns about the way these scores are used to rank people, as opposed to help group tweeters into fairly broad buckets, but that’s another post. @cspenn himself has a Klout score of 71.

The answer is: other people with a high degree of influence like himself. While he might reply if a low-influencer sends him a message or mentioned him, the chances are low that he will pay much attention to that person’s normal stream. Why? Even people as well known as @cspenn and @chrisbrogan are aiming to extend their own reach. @cspenn can market who he is to the large/important communities around @chrisbrogan because he can reciprocate. That’s why you’ll see @cspenn mentioned in @chrisbrogan’s blog posts and vice-versa.

If you produce really great content you’ll have more chance of getting their attention, sure, but a lot of people are producing good content out there, so the reach dimension still comes into play. You might be better off generating heat from up-and-comers, at least as a prelude to trying to get the bigger guys interested.

How do you do that?

Stop paying attention to so many big producers like Mashable. Turn off some RSS subscriptions.  Do some removals from your must-read twitter list(s). Interact more with smaller content producers! Could be you’ll get the sort of outcome you’re after.

Information layer cake

I recently found Glen Cathey’s blog post on LinkedIn search rankings and something struck me instantly.

This is a long article.

At about five pages, it’s still manageable, but I came to the site wanting answers quickly, and I’m sure the length of the article turns a good percentage of readers off. Don’t get me wrong – it’s great that Glen produces such in-depth articles in a space dominated by “Top 5 Tips”, and even better that he shared it with us all, but I think he missed a trick. Knowing that there a lot of people out there with low attention spans should mean that you have different versions of the text to keep that traffic around.

One site that is occasionally good at this is Wikipedia with the English/Simple English explanations.

Compare the simple Wikipedia explanation of probability

Probability is a part of mathematics. It has to do with chance, the study of things that might happen or might not happen.

with the normal English explanation:

Probability is a way of expressing knowledge or belief that an event will occur or has occurred.

I find the simple English explanation easier to understand, and reading it actually makes it more likely I’ll check the more detailed version.

The fact that I know I can find useful information about a subject, regardless of my literacy in that particular field, makes Wikipedia a more valuable resource for me. I think for us to communicate as effectively as possible, we have to apply that lesson ourselves.

Basics of a social media campaign

A social media campaign can be summarised in the following flowchart:

This graphic could be pretty much used to illustrate quite a few processes, but it’s important to make sure that when we are trying to use social media for marketing, we have to be able to learn from and improve the efficacy of the work that we do.

Some further detail about what is involved in each stage:

Audit:
An audit is the essential first step of any marketing campaign, and can also be a useful way for you to dip your toe in the water without any risk or commitment. Creating an audit involves investigating a client’s current online presence / products / competition and carrying out industrial research.

Plan:
The plan converts the opportunities outlined in the Audit to a sequence of actions – tied to specific milestones.

Implement:
Actually carrying out the steps that have been laid out.

Evaluate:
Using an array of analytics tools, you can measure the magnitude of a variety of different effects that your social media efforts might have. Based on how the results relate to the expectations set, you may want to alter the original plan.

A tip on how to organise your Twitter presence

I recently had the opportunity to ask Chris Brogan (@chrisbrogan) about something which I’d be thinking about for a while – how to represent yourself/your company on Twitter. There are a quite a few different ways to do this – here’s what Chris had to say.

.@– I much prefer single user accounts, plus 1 brand account to oversee conversations. Depends on branding principals. #b2beu

@chrisbrogan

Chris Brogan

(apologies for the typo:)

@ Could you expand in “oversee conversations”? #b2beu

@zeal_doug

Douglas Holmes

.@ – someone has to have a governing role, to say what’s good or not for a company’s “voice.” #b2beu

@chrisbrogan

Chris Brogan

So what does Chris mean by a ‘governing role’?

I think it’s to do with focus. People expect that when they are interacting with a person, not everything is going to be about their job, whereas from an official company mouthpiece (such as a corporate twitter account), the content is going to be much more focussed. However there should be a link between the two different channels, so even if someone is interacting with a corporate account, they know who is behind the logo.

A canonical example of this is the @ScottMonty / @Ford relationship. You get a real sense of who Scott is through his personal account which makes interactions with the guy on @Ford much deeper.

LinkedIn for the minimalist

With LinkedIn, there are only two things you need to be doing right now:

i) Adding connections, and

ii) Getting recommendations

You need to be doing these things on an ongoing basis because people’s willingness to connect/recommend you decreases the longer they haven’t spoken to you.

You need these things in the first place because recommendations/connections are a great way for people to find you, the more of them you have the more important you seem, and connecting with people enables you to better understand your own network, and thus tap it for whatever reason you need.

So if you’re not a big LinkedIn user, here’s what I recommend:

1) Try importing from your address book/faceboook etc… there might be people who are on LinkedIn whom you know, but have not connected yet.

2) Make sure when you meet someone who might be useful, you try to connect with them on LinkedIn. If you manage to make the connection, you are giving yourself a platform to deepen that relationship at a later date, if you so desire. This is made easier by having a smartphone with LinkedIn installed, and adding them in “deadtime” when you might be travelling or waiting for something.

3) Try asking for recommendations from former/current employers/colleagues. If you offer to reciprocate, the likelihood that they’ll accept will be greater.