Understanding Tucows Program Reviews

In the Tell Me About... article last week I talked about the Tucows rating system. This week I'm going to talk about some of the criteria I use when I review software and how that differs from a rating.
Published: Oct 6, 2005
Author: Michael E, Callahan

This question submitted by Ralph Anderson, Kathy Vane, Allison Saunders, James Schmidt, Ron Bannister, and numerous others

In the Tell Me About... article last week I talked about the Tucows rating system. You can read that article by clicking HERE. This week I'm going to talk about some of the criteria I use when I review software and how that differs from a rating.

I'm often asked a question like, "You selected that program as a Pick, but it only got 3 cows on Tucows. How is that possible?" Well,really the answer is quite simple. As I explained in the other article,the reviewers here at Tucows are using a specific set of criteria. With many of the things in that criteria the reviewer has no options. That is to say, a program either has something or it doesn't. It's totally based on objective criteria. At the same time other parts of the process are more subjective - they do depend on the authors opinion. Think about it. If a program loses too many points in the objectiveportion of the review process, they may be able to get all of the points from the subjective part and still get 3 cows. A ratingadheres to specific rules for the reviewer. I'm not bound by those rules. So, here is some of the criteria I've used for 23 years in reviewing software.


The one thing I've consistently tried to do is to maintain my objectivity. Something I've always said is,"All things depend on point-of-view!" and I believe that's very true. How you "see"something depends on where you view it from. I feel it's very important for a software reviewer not to forget what it's like to be a new user. What it's like not to understand everything that's in front of you on the machine. People like me have essentially "grown up"with computers. I've seen things change and evolve and I've been able to learn things gradually. I can only imagine how overwhelming it must before someone who gets into computers today. So much to learn and know about.

For this reason when I evaluate software I try to look at it from the perspective of a new user. Someone who has had a computer for 3 months.Someone who doesn't understand all the conventions and other things that a longtime computer user knows. When I reply to an author who has asked me to look at their product, I also keep this perspective. I'll say,"How do you do that in the program?" when really I know how, but I don't think a newer user would.

It's my belief that more software authors need to adopt this same kind of perspective. If you create a piece of software of course it's going to make sense to you, but what about me? What about the lady who just got a computer in May? What about the older couple who got a computer 4 months ago as a gift? Will the program make sense to them? Thatis what software authors need to ask themselves.

Is It Intuitive?

In my mind, that's the question that needs to be answered. Is the program intuitive? Is it easy-to-use? Can a novice start the program and use it? Can they use it without having to read a long document first?Can they use it without having to constantly look at the help file? Yes,some programs are complex in what they do, but they should still be intuitive to use.

To find out how intuitive a program is I never look at the help,online or otherwise. I will read a "Quick Start" guide that comes up with the program, but I don't do anything else. I won't look at the program documentation. The first thing I do is look at the program from my new user perspective. I see how easy the program is to use. Does it make sense? Does it use terms that are complex or confusing? Does it lay things out where one might think to look for them? One of the goals with Windows was to have a program interface that was more standardized. So that if you learned one program you would have an easier time with the next. Does the software author adhere to those conventions?

My goal is to determine just how intuitive a program is so I can tell you about that program. So I can tell the author whether I think their program is intuitive or not. Ultimately it doesn't matter how good a program is at what it does if it's too hard to use.

Is It Configurable?

It is the nature of people to have things the way they want them.People like being able to configure something to suit themselves. We order pizzas with the toppings we like, we order cars with the options we want, we get a hamburger and we want it "our way." It is the same with software. People like to be able to configure a program to do certain things the way they would like. More configurable options means a program can appeal to more people. It's my feeling that software authors should give users as many options as possible. So, I look for that in all the programs I evaluate and you'll notice that I comment on it in my reviews. To me "configurable options" is a plus.

Does It Do The Job?

That's the next thing I ask. Does the program perform the tasks it says it does? And does it perform those tasks efficiently? In a way that makes sense? Does the program have enough options and features? Does it have too many? Some programs suffer from what I call feature overload. It's the graphics program that also plays CDs, acts as a screen saver, serves as a macro program, and lets you keep notes. It's the product that tries to do too many things. Do one thing, or a few things, and do them well. It doesn't matter how intuitive a program isif it doesn't do the job.

Summing It Up

Reviewing anything involves an element of responsibility. Whether a person reviews movies, or plays, or restaurants, or software, they are influencing the decisions people make. Give something a "thumbs down"and there are people who won't look at it. A rating is very different from a review because the ratings have certain criteria that are set. I have developed my own criteria over time, but I don't care about some of the objective criteria -- like is there an icon for the "uninstall"? When I review a program I am reviewing just the software, and how it works, and how intuitive it is, and how much you can configure it.

I've made it a point throughout my career of evaluating software to never do negative reviews. To me that is a waste of my time and yours. I only review programs that I think are good and worthy of your time. I evaluate well over 200 programs each and every week, yet I only" PicK" about 7. In 23 years of reviews I have less than 100 Dr. File Finder Favorites. Having evaluated over218,000 programs I have a unique perspective. You'll note that even whenI do a Dr. File Finder's Picks Roundup!that I don't compare one program to the others... I review each one on its own merits and let you decide. Why? Because I realize that your opinion may differ from mine. That you may like certain features that I don't care about.I view my role as this: I sort through all the software I can and I tell you about the best. Hopefully that makes your decision easier.

By December I'll have evaluated 220,000 software programs and I still never get tired of it. Each day it's something new and something different. Yes, I look at a lot of awful software, but at the same time,I've also found some real gems. Those are the ones I tell you about. The key thing to remember is that "Software is always a matter of personal taste." I'll continue to sort through the software and tell you about the ones I think are worthy of your time.

I'd like to thank Ralph Anderson, Kathy Vane, Allison Saunders, James Schmidt, Ron Bannister, and numerous others for asking this question.

If you have a question on any technology topic that you'd like someone to tell you about you can submit it via email by clicking HERE You will not receive a reply, but all topics will be considered.

About Michael E, Callahan

Michael E. Callahan, known around the world by the trademarked name Dr. File Finder, is regarded as the world's leading expert on shareware. Dr. File Finder works with software programs and developers full-time, and in the average year he evaluates 10,000 programs. Since 1982 he has evaluated over 250,000 software and hardware products. Mr. Callahan began evaluating software online in 1982 and no one has been at it longer. He currently works doing online PR and marketing for software companies, and is the Senior Content Producer for Butterscotch.Com.

Digg This
Please login to add your comment
Leave A Comment