TTAC is the third most critical reviewers of automobiles, according to Motor Mouths, an ambitious project which seeks to aggregate and rate car reviews. With an average review score of 75.52, TTAC is the least forgiving non-print outlet that MM rates. Only Consumer Reports (70) and the Wall Street Journal (73.5) boast a more carmudgeonly average score of vehicles they review, with Automobile.com and the NY Times rounding out the top five. Of course, standardizing reviews is a tricky undertaking, making comparisons a bit of an apples-oranges exercise. Still, MM goes there, even comparing reviewers with owner feedback to create an index of “reviewer credibility.” Sadly, TTAC reviewers don’t rate in the top three “most credible,” but since MM has only been around to gather owner feedback since February, we’ll give it some time. Not that we’d necessarily expect to replace the LA Times’ Pulitzer Prize-winning Dan Neil at the top of that list.
The Truth About Cars: “Screw your print publications, and screw your cars.”
Still, it’s funny: call a car ugly, say it has the structural integrity of a Lincoln Log House, and all of the sudden, say it’s not worth the rubber it sits on, and all of the sudden – you’re an asshole. Go figure?!
Motor Mouths = Rottentomatoes of auto sites?
Well, when you consider the pablum that most automobile reviewers serve up, being in the bottom 3 is badge of honour.
Well, the reviews here might not seem a credible because different authors hold different standards to the cars.
I remember the current-gen Maxima’s interior being called cheap, when it was freaking AWESOME.
S-type gets a good interior score, though!
Not saying the reviewers are wrong, because you guys have awesome writers who know alot about cars and test them often, but maybe creidbility is low because the standard the cars are held up to aren’t consistent?
I love all of your reviews!! The ones on Monday/Wednesday especially get me through the slow week. :P Btw, where did Megan Benoit go? She was awesome. :)
Eddy,
I told you I didn’t like automotive awards.
Interesting site concept.
Owners rate reviewers on their credibility. You can count on postive reviews getting higher credibility scores.
You’d get a different result if people who considered but rejected a model were asked to rate the credibility of reviewers.
That said, my lowish score of 5.5 ties with PJ’s and is ahead of RF’s, Jonny’s, Sajeev’s, Megan’s, and Bill’s!
And we’re all ahead of the reviewer at Motherproof!
Only 10 points separates TTAC’s hardcore haters from Road and Track’s old softy’s. 85 points is the highest and TTAC awards 75 on average. 75 is a C which is average on the bell curve and therefore probably accurate for the average car. Also TTAC tends to review more rental or “average” cars so…that’s about right.
Does that place TTAc squarely in the mainstream?
IGB: I think it means MM might have to tweak its secret sauce a little. I, for one, am very curious as to how they fit all those reviews into their 100 point score. Especially considering the stylistic etc differences between such diverse review sources. Interesting concept though.
Sorry boss, I just wanted to use the Obamicon one more time!
Interesting concept and worth a read. However, as a psychologist, I have an issue with their credibility ratings. Owners of vehicles are inherently biased (because they either like the car or can’t admit that they make a $30+k screw up). Thus, the data will skewed toward the more positive reviewers. To be accurate, they need a sample of people that have test driven the car (whether they purchase or not).
I am assuming it’s only averaging reviews from when the star ratings were implemented (which wasn’t that long ago), which helps not MM’s credibility.
Also, what a dumb thing to measure, and of such questionable utility.
Hi guys. I’m the founder of MotorMouths, and I thought I’d pop in with a quick explanation. As someone up higher suggested, yes, we’re essentially the RottenTomatoes.com of new car reviews. The idea was to build a fast, efficient site that let shoppers quickly see the critical consensus for any new car.
We’re just now emerging from a quiet beta, but in fact we’ve been building a database and tracking reviews for almost a year — some 10,000 worth. We only opened up the credibility system about a month ago, and so, yes, the cred scores are still in their infancy. As more verified owners weigh in, the scores will have a little more resonance. (And Michael K., some of the toughest credibility scores are coming from owners who love their car, and yet take issue with specific aspects of a generally positive review of that car — if the critic makes an error of fact, or offers a sloppy or badly-reasoned opinion, they get dinged.) By the way, if a critic’s score falls below 3, we remove them from the site; the idea here is not to embarrass any critics, just to make the site a resource for credible reviews.
A quick explanation of the algorithm we use to derive scores from reviews. Edward, we do indeed run all reviews through the same algorithm, no matter if the critic has already attempted to quantify their review with a scoring system. Stars, grades, thumbs-up, thumbs-down — all of these are tossed out. We then break down every review, parsing specific pieces of positive and negative opinion. Each of these is placed into one of four categories that speak to a vehicle’s design, performance, build quality, and relative value, and graded on a 20-point scale. The algorithm then yields a specific score that can indeed allow for comparison across different publications. In other words, TTAC.com’s reviews can be placed alongside, say, MotherProof.com for actual comparison. The only weighting of values in this algorithm occurs between categories; performance vehicles have a slightly different value system than standard vehicles, to allow for different levels of buyer interest in the specific categories.
We played with semantic analysis for some of this, and there’s still a bit of that going on. But final analysis and scoring of reviews is done by a team of professional editors, and thus we’ve very quickly picked up on the specific likes, dislikes, quirks, and biases of various critics and publications.
Well, that turned out to be not-so-quick. But I hope it answered some questions. And we’d be happy to hear any additional feedback. Cheers.
@MotorMouths.com : Interesting concept, it sounds like you’ve thought this through pretty well. Good luck, you might’ve found your niche.
On a separate note, I don’t think it’s fair to compare Consumer Reports to … well anything really. I’ve never been comortable with the way they review cars. They’re very good at objective info, like reviewing an appliance. But in my opinion, when reviewing cars a lot of the important information is subjective. That’s where TTAC really shines: understanding who a car is designed for, and what that says about the company that made it.
Although if you feel a car IS an appliance, I’m sure CR is very helpful to you. But it isn’t to me.
Aeroelastic: Since CR.org is behind a pay wall (and thus not very useful to our users), the only reviews of theirs that we include are those from their blog, which tend to be a little more conversational and, well, TTAC-like than the somewhat more antiseptic official breakdowns they do in the magazine.
That said, I do find their blog interesting, from a meta-perspective.
This site is pretty cool. Even though I think distilling reviews into numerals from a mysterious/subjective weighted formula is pretty lame. I’m torn.
But who knew I was 0.2 points more credible than Jonny Lieberman? Seriously???
I’m glad this information is available on the Interweb!
I’m in to this Motormouths thing – a review aggregator takes care of most of the work of digging up car reviews. It’s going up in my favorites tab, that’s for sure.
Really, it’s a great niche and I hope they don’t stray too far from the concept, because it’s easy to use & understand.