Brian Massey, quoted by Peep Laja:

Great website optimizers can be found in the most unusual of places, because they are currently among the most unusual of individuals. Great website optimizers did not go to school for optimization. They are grown, not found. They can be found in libraries and in psychology schools. They tend bees, arrange flowers, and any other number of hobbies that involve visual differential analysis.

The whole thing is worth a read if you’re testing online.

If you wanted to see how your nonprofit’s fundraising stacks up to “industry standards,” you’d likely turn to a benchmark report like the M+R 2015 Benchmark report.

That’s fine if you know what you’re looking at. But nonprofit benchmark reports have several flaws, and if you’re not careful you can draw the wrong conclusions.

1. Benchmark reports are biased

For one thing, the results are biased. I don’t mean that the researchers behind the M+R study and others like it deliberately skew their results. Instead I mean biased in a statistical sense: these reports aren’t based on a random sample of nonprofits. 

Studies like M+R’s draw their data from organizations that volunteered to share their information with the authors. The authors of some studies even examine organizations they have preexisting relationships with, i.e. their clients.

That means most benchmark studies aren’t representative of all nonprofits—which is problematic since the conceit of such studies, their disclaimers aside, is that they are representative.

About all you can say is that the results of the study are true of the organizations studied. That’s interesting but not very helpful to evaluating your own marketing.

2. You can’t draw useful conclusions from benchmark data

Let’s say you work at an animal welfare group. You’re looking to run some Google ads and want to see if there’ll be competition from your peer groups.

Turn to page 38 of the M+R 2015 Benchmark report. You’ll find that 63 percent of animal welfare groups run paid search ads. So you’re behind the curve.

But wait! Is that what the report really says? Let’s take a look at the data.

The M+R report looks at 85 nonprofits, grouped into seven sectors. Of these, only eight are classified as “wildlife and animal welfare.”

What does it mean that there are 85 total nonprofits and just eight animal welfare groups? It means we can’t draw useful conclusions about animal welfare groups (or any other group) from this data. 

Not with any reliability, anyway. For animal welfare groups, the margin of error is ±34 percent at a 95 percent confidence interval.¹ For everyone studied, it’s ±10 percent. And this assumes we are dealing with a random sample, which we aren’t. 

So according to M+R’s report, anywhere from 29 to 97 percent of animal groups use paid search ads. Between 48 and 68 percent of all nonprofits do so.

Yeah. That narrows it down.

In fact, you’d need a sample of 370 (randomly sampled) nonprofits just to get to a margin of error of ±5%. That would bring the range down—to between 58 and 68 percent of animal groups.

3. Sector-by-sector breakdowns are meaningless

Nonprofit benchmark studies typically categorize their results into sectors: education nonprofits, internationally-focused nonprofits, environmental nonprofits, and so forth.

This breakdown is useful only insofar as groups that focus on the same set of issues are in any way comparable when it comes to their business and marketing practices. Which is a ridiculous assumption to make.

Can you really compare the ASPCA, which had $166 million in revenue in 2013, to your local animal shelter? By the usual benchmarking taxonomy, you should, because both are animal-welfare groups. 

Or let’s think about the retail world for a minute. If you ran the Apple Store, would you benchmark your sales results against other technology stores, like Best Buy? This is a really odd comparison to make, as this table shows:

Sources: Forbes/​eMarketer; Seeking Alpha

The Apple Store isn’t even in the same league as Best Buy, given sales per square foot, a common measure of retail performance. Yet a simplistic definition of sectors, like that used by nonprofit benchmark studies, would lead you believe they’re peers.

So make sure that when you’re comparing yourself to your peers that they really are your peers. Benchmark reports may not give you the data you need to do that.

Why are you measuring yourself against other nonprofits at all?

Charities are notoriously basket cases, with poor management and bad practices. Because they don’t face the same market pressures as for-profit industries, these bad practices can continue for years before anyone fixes them.

So don’t measure your success as a fundraiser against biased data in benchmark studies. Instead, measure your success against your goals: are you meeting revenue targets, retaining enough donors, and acquiring enough donors?

I welcome your feedback. Tell me what you think in the comments.

1. In simple terms, margin of error measures how precise your data is. A bigger sample size will typically reduce your margin of error, meaning your data is more precise.

Let’s say a poll reports that a politician is favored by 50 percent of voters, with a margin of error of ±3 percent. That means his actual popularity (if you were somehow able to talk to every single voter instead of a random sample) is somewhere between 47 and 53 percent. The data captured can’t pinpoint it exactly.

A margin of error of ±34 percent means that your sample data is basically meaningless at predicting the actual data. 

Customer service is a critical part of your customers’ interaction with your company. It’s not just about the product you sell.

Brian X. Chen:

As it stands, there is no one-stop website to see reliable ratings for customer service. In part, that is probably because customer service can be such a challenge to measure, said Matthew Dixon, an executive at CEB, a business advisory firm. His studies found that when people have positive customer service experiences, they tend not to share them.

“But when they’ve been wronged, they literally will tell anybody who will listen,” Mr. Dixon said.

Mr. Dixon’s studies found that customers stayed loyal to brands that offered hassle-free service interactions. His studies also found that customers were four times as likely to become disloyal to a brand after any service interaction at all — because so many service centers drag out people’s issues.

It seems like a no-brainer that consumers stick with brands offering solid customer service. Apple, which has more than $190 billion in cash, is well known for its Genius Bar, the service stations at Apple stores where customers can seek help directly from the company’s trained technicians. Amazon, the largest online shopping site in the United States, is celebrated for its customer service.

Seth Stevenson:

Do you like smooth, responsive touch screens? How about intuitive menu systems? Do you enjoy taking crisp photos and looking at big, sharp images?

If the answer to these questions is yes, I can assure you that you don’t want a BlackBerry Classic.

I’m a big believer staying organized, especially the Getting Things Done approach to organization. I maintain a single task list for both work and personal to-dos, and make every effort to keep my inbox at zero. And to ensure my task list is always handy wherever I am, I manage it electronically.

Since 2008, I have used Remember the Milk to maintain my task list. It allows me to organize my tasks by project, tag, due date, and location, and its real-time sync lets me add and complete tasks from anywhere. Better yet, RTM offers an unparalleled quick-add feature: typing in a single field, I can create a task, set a due date, add it to a project, set its location, and assign tags.

Nearly seven years later, RTM offers these same features—and hasn’t really added anything else. Its web app hasn’t been updated since I began using it, though there are indications that a new version is coming eventually. The iOS app has seen just two minor updates since 2012—one to ensure Retina compatibility and the other to fix a bug in iOS 8.

Meanwhile, other task-management apps have continued to innovate, and have long passed RTM in terms of functionality. While RTM may eventually catch up, it’s far from clear that it will. There certainly hasn’t been any public communication to that effect: the company hasn’t talked about upcoming features, and they’ve even stopped their weekly tips and tricks blog posts. Unfortunately, all indications are that RTM has fallen behind for good.

Why I picked Todoist

Starting last summer, I explored several of the major task management apps. My principal criteria: the app had to offer functionality equal or superior to Remember the Milk, especially when it came to task entry and cross-platform sync; and it needed to be regularly maintained and updated. Among those I test-drove were Wunderlist, Todoist, Apple Reminders, Asana, and Evernote’s reminders.

After a few weeks of research and experimentation, and after a thorough trial earlier this month, I decided on Todoist, for five big reasons:

1. Quick-add. At least in the desktop and web apps, Todoist’s quick-add feature allows task entry without having to use a mouse. On my Mac, ⌘-Shift-A is an elegant way to add quick tasks without having to switch away from my current window. Like RTM, Todoist allows me to set most task attributes without a mouse, but it adds support for linking and even text formatting within task names.
2. Web access and native apps. Todoist offers a robust web app as well as native apps for both iOS and OS X, all of which are delightfully simple in design. Both native apps include Notification Center support, and iOS features like time– and location-based reminders are extraordinarily helpful. My tasks sync quickly and seamlessly across devices.
3. Active maintenance. Todoist regularly adds new features, and communicates with customers about upcoming releases. In just the last six months, Todoist launched features like location-based reminders, a new “karma” progress-tracking tool, Yosemite support, iOS 8 support (taking advantage of all the new features), and IFTTT integration. They’ve also announced their major initiatives for 2015, giving me confidence that the platform will continue to strengthen and evolve.
4. The small things. In many cases, it’s the small details that can make or break a user experience. For example, on a recent trip to Dallas, I loaded up Todoist on my phone and it immediately prompted me to switch the time zones used to set my task due dates. RTM’s iPhone app, by contrast, not only failed to adjust time zones but inexplicably showed the wrong day’s tasks when note in the home time zone.

None of the other apps matched my workflow. Wunderlist task entry is clumsy, and the interface is visually off-putting; Reminders lacks the features I desired, like subtasks, tagging, and more; Asana is powerful, but it’s really designed for project collaboration rather than daily tasks; and Evernote’s reminders function isn’t really useful for a GTD setup. Federico Viticci reviews Todoist in-depth if you’re looking for a deeper dive into Todoist and its features.

After just a few days of using Todoist full-time, Todoist has improved my workflow and productivity. I find myself regularly using features like quick-add on my Mac and location-based reminders on my phone, while the karma tool is a fun way to keep me motivated to stay organized.

How do you keep your tasks organized?