Lots of great ideas so far. I have a rough idea on how I want to put this project together, so this helps a lot.
The ranking is rather subjective. But it's kinda like Amazon; they depend on lots of data before it's useful information. Let's say we get 100+ people to rank each category for the Dewalt 788, the average number is pretty good data. If we only has a couple people rating the Craftsman 16", it's not as much to go on. You'll be able to see how many people voted and the distribution of votes per category.
Plus, after the writeup, owners of the saw can write in their reviews, tips, and extra tidbits. That's the idea, anyway.