On its web site, GreatNonprofits bills itself as "a place to find trustworthy nonprofits." The site allows people—from donors to clients, volunteers to employees—to submit reviews of nonprofits. These reviews are prominently featured on web sites like Guidestar, Charity Navigator, and JustGive. At first, this may seem like a good idea. After all, consumers commonly use reviews on sites like Yelp and Trip Advisor to help them choose restaurants and hotels. What is wrong with using crowdsourced reviews to help donors pick nonprofits?

One problem is that reviewing nonprofits is far more complex than reviewing consumer products and services. When a customer at a restaurant pays for a meal, he can smell, taste, and experience it. In contrast, when a donor gives to a charity, he pays for goods or services that someone else receives. His review is often not based on any firsthand knowledge of the quality or efficiency of the charity's programs.

Another problem is the lack of consistency in what is being measured. Six positive reviews about a charity's polite telephone operators might be averaged together with one negative review alleging its consistent racial discrimination against grant recipients. It is difficult to imagine that most potential donors or clients of such a charity would consider it to be a "trustworthy nonprofit," even though this is what its high average star rating would portend to reflect. A donor also might give a positive review to a charity because he likes its cause, not based on any knowledge about how efficiently or effectively that specific charity works to forward that cause.

As with any review site, it is also important to consider the reviewers' backgrounds and qualifications. Even someone who receives services from a charity or volunteers with one often has only a narrow understanding of the organization, not a clear picture of how well it operates on the whole. A charity may provide grants to a handful of needy veterans, sick kids, etc., while wasting most of its resources on overhead. The recipients of the assistance may be quite happy and post positive reviews even though the charity could have helped many more people with the donations it received had it been operating more efficiently.

For example, one reviewer of the Childhood Leukemia Foundation, which receives a nearly perfect score from GreatNonprofits but is F rated by CharityWatch, praises the charity for distributing "Hope Binders" that help families of cancer patients organize bills and insurance information. While these binders may be useful, they represent just one tiny aspect of the charity's operations. This reviewer did not weigh more substantial factors that are representative of how well the charity operates overall. For example, a note in the group's 2010 audit describes the charity's promise to pay 75% to 85% of future revenues to outside fundraisers. This leaves a paltry portion of donations available to help children with leukemia.

An outstanding charity that is unlucky enough to get a few bad reviews can receive a mediocre rating from GreatNonprofits. The highly regarded Doctors Without Borders, which receives an A rating from CharityWatch, receives only 3.5 out of 5 stars from GreatNonprofits as of February 2012. Why? Largely because two of only six reviewers downgraded the charity—one for a billing error that was not immediately resolved, and one for a misunderstanding that the charity's low cost of raising funds was too high.

On sites like Yelp or Trip Advisor, there are often enough reviews of a particular restaurant or hotel to get a general idea of how happy people are with the quality of the food or accommodations they received. But most people know that if any business has only a handful of reviews, these are far less reliable since the unreasonable opinions of just one or two people, either too positive or too negative, can greatly skew the rating. GreatNonprofits, founded in 2007, boasts that it lists 1.8 million nonprofits on its site, but only about 12,000 of these have been reviewed. Of those, many have been reviewed by just one or two people. For example, the Parkinson Research Foundation, which is F rated by CharityWatch, gets a perfect 5-star rating on GreatNonprofits. This rating is based on a single review written by an employee of the charity—not exactly an objective or independent source.

GreatNonprofits encourages reviews by biased or interested parties, like the one above. On its website it states that groups that are not happy with a negative review can "rally people…and ask them to provide alternate perspectives to this critical review." By encouraging groups to drum up glowing reviews from their supporters, GreatNonprofits is letting charities know how easily they can drown out their critics and receive high ratings. This may be good for charities, but it is not helpful for donors who want an honest and objective view.

The Internet offers a bounty of information about charities, but this can be a double-edged sword for donors. On the one hand, it is now easier than ever for donors to research charities. On the other hand, donors must sift through mountains of information that is often superficial, biased, or simply incorrect. As a resource for donors, CharityWatch analysts scrutinize charity financials to provide the donating public with independent, meaningful information about how charities spend their money. This kind of information only comes from a rigorous analysis of a charity's finances; it cannot be gleaned from a few short reviews posted by donors and volunteers, or by self-interested charity employees and their hired public relations agents.