Sunday, June 03, 2012

Is One Review Just Like Another?

Marshmallow Sheep

Recently, Harvard Business School published a report entitled What Makes a Critic Tick!

The purpose of the report,  in the words of the authors, was to "investigate the determinants of expert reviews in the book industry."

The report analyzes the 100 top-rated, non-fiction book reviews from over 40 publications ("expert reviews") during the years 2004-2007.  The findings are then compared to an analysis of "consumer reviews",  represented by Amazon user reviews.

The Executive Summary of the report lists the following key concepts:

  • The data suggest that media outlets do not simply seek to isolate high-quality books, but also to find books that are a good fit for their readers. This is a potential advantage for professional critics, one that cannot be easily replicated by consumer reviews.
  • Expert ratings are correlated with Amazon ratings, suggesting that experts and consumers tend to agree in aggregate about the quality of a book. However, there are systematic differences between these sets of reviews.
  • Relative to consumer reviews, professional critics are less favorable to first-time authors. This suggests that one potential advantage of consumer reviews is that they are quicker to identify new and unknown books.
  • Relative to consumer reviews, professional critics are more favorable to authors who have garnered other attention in the press (as measured by number of media mentions outside of the review) and who have won book prizes.

Bill Marx, writing in The Arts Fuse, is aghast that people are taking this study seriously. He is especially dismayed at the editors of The Guardian, who published a short column in which their non-fiction book review editor assures us that she ranges far and wide to find books to assign to reviewers, (though she adds that she does look for books that would be of interest to Guardian readers.)

However, Marx reserves most of his ire for point number two in the Executive Summary:

I have read the HBS study (by Loretti I. Dobrescu, Michael Luca, and Alberto Motta), and it is clueless on so many levels about the craft and mechanics of reviewing...
Criticism should be fair, but it is never unbiased or objective, as if reviews were a form of scientific experiment. Readers want to hear the critic’s personal judgement —negative, positive, or mixed —as shaped by his or her sensibility and depth of knowledge. Critics are individuals, not measuring sticks. The authors of the study have no idea of what a book review is supposed to do: they supply judgments with reasons. They do not summarize quality, whatever in the world that means. Predictably,the bibliography of the HBS report doesn’t list a single publication that deals with the crafts of reviewing—to them, a sentence long “thumbs up” on Amazon is the same as a review in The New York Times. Because it does not discriminate between substantial reviews and guttural opinions, sensible considerations and hit jobs, the study proves nothing.
I wondered if, based on the sort of definitions that Bill puts forward in his column, how much of a difference one could find between Amazon reviews and those in major publications.

Since the study by HBS focused on non-fiction, I decided to look at the reviews of a recently published non-fiction book in which I was interested, but had not yet read.

Robert Caro just published his latest installment of his massive biography of Lyndon Johnson, The Passage To Power; The Years of Lyndon Johnson.   It has been reviewed extensively in major media outlets and also has a healthy number of reviews on

First, I read through the many "expert reviews" from newspapers, magazines and other online sites, starting with The New York Times.

The Times review, strangely enough, was by William Jefferson Clinton. Yes, that Bill Clinton.

As far as biased judgments, (and I don't mean that in a perjorative sense,) this is as good an example as any. Clinton is upfront about the lens through which he read Caro's account of LBJ:

Even as Barry Goldwater was midwifing the antigovernment movement that would grow to such dominance decades later, L.B.J., Shriver and other giants of the civil rights and anti­poverty movements seemed to rise all around me as I was beginning my political involvement. They believed government had an essential part to play in expanding civil rights and reducing poverty and inequality. It soon became clear that hearts needed to be changed, along with laws. Not just Congress, but the American people themselves needed to be got to.
L.B.J. got to me, and after all these years, he still does. With this fascinating and meticulous account of how and why he did it, Robert Caro has once again done America a great service.
Aside from Clinton's personal reminiscence and political thoughts, his review takes the same structure as most "expert reviews" I read.

Almost each and every review outlines Caro's vast project and then relates the way in which Passage to Power covers this particular time in Johnson's career.  Most all of the reviews give an idea of the way Caro posits and contextualizes LBJ's story within certain themes. For example, this is from the Los Angeles Times review:

 Johnson had been urged to back off the civil rights bill by advisors who deemed it impossible to overcome the implacable opposition of the Senate's powerful Southern Democrats. "A President shouldn't waste his power on lost causes," one of them said. "Well, what the hell's the presidency for?" Johnson replied. In the past, ambition had always trumped his compassion for the poor and persecuted; now his superb political skills could be dedicated to something other than his personal advancement, and they were.
"The cliché says that power always corrupts," Caro writes, "but what is equally true is that power always reveals." The unswerving commitment to civil rights and the eradication of poverty revealed in "The Passage of Power" give it a different tone from its predecessors. Its tone of sympathy and admiration for a man who "not only had held the country steady during a difficult time but had set it on a new course, a course toward social justice." To do that, Johnson wrought astonishing changes within himself. His brutal arrogance, lachrymose self-pity, cruelty to subordinates, gloating boastfulness, penchant for secrecy and deception — qualities Caro has unsparingly depicted elsewhere and catalogs again in the last chapter — were suppressed as Johnson assumed command with an air of "disciplined calmness" that suggested he was fulfilled at last.

Very few reviews go further than this.

What do I mean by "further?"

While most reviewers have a firm handle on Caro's views, (varied as they are,) of LBJ, very few seem to contextualize these. In other words, very few provide a lens through which we can view Caro.

One of the exceptions is this review in the Washington Post:

Caro’s ability to show these many sides of Johnson — good, bad, ugly — rebuts the rap that he paints his characters only in black and white. But if Caro’s personalities are multidimensional, they’re nonetheless overdrawn in a way that sows a nagging distrust. At any moment, he showcases only one element of Johnson (or of RFK, or of other characters); typically, it is a portrait of an extreme. The young Bobby Kennedy is not portrayed just as ruthless; one instance of ruthlessness after another is recounted, amid countless repetitions of the word “ruthless.”
In the end, the thought occurs that Caro, in all these volumes, is at some level writing about himself — working out on the canvas of Johnson’s life the questions and themes of power, greatness and tragedy that have long occupied him. The LBJ that Caro gives us is not an inaccurate portrait, but it’s certainly a subjective one — an idiosyncratic expression of Caro’s own sensibility. The Rabelaisian Johnson, the man of extremes and excess, finds his correlative in the biographer who works in a torrential style, capturing as few writers can the richness and magnitude of his subject but also sliding at times into melodrama and belabored points.
Caro’s sprawling, sparkling, theatrical opus, rather, calls to mind a work such as Carl Sandburg’s six-volume life of Abraham Lincoln, which also took years of prodigious labor and displayed, as the historian James G. Randall wrote, “a poet’s sense of language . . . and an ability to combine realistic detail with emotional appreciation.” Today, Sandburg’s work is read more for literary pleasure than historical authority. 

In this review, David Greenberg tries to position this biography within the genre as whole, and to make an informed judgement on the craftsmanship.  I found very few mainstream reviewers who even attempted this.  Although, I will say Greenberg seemed to be allowed more space for his assessment than others.

Another interesting exception is Tom Carson writing in GQ.  Carson's review is almost the flipside of most other reviews in that he is very Caro-heavy:

(Caro) would have benefited from keeping Vladimir Nabokov's dandy advice to anyone writing about America—"Nothing is more exhilarating than philistine vulgarity"—tacked above his desk. Yet as the great Garry Wills may have been the first to point out, Caro also seemed to hate the whole concept of power. Not just its misuse, understand—its use, along with the psychologies of those drawn to seek it.
This was a fairly standard attitude among liberal intellectuals in the 1950s, the era Caro was formed by. They loved Adlai Stevenson because he was so gosh darn noble he didn't even act like he wanted to be president. And they quickly forgot that Franklin D. Roosevelt, the most successful Democratic POTUS of the 20th century—and young Congressman Johnson's patron and role model in the 1930s—was not only as conniving and duplicitous a politician as any who ever lived, but enjoyed every second of it. In a much diminished and less consequential way, that was also true of Bill Clinton, whose own review of The Passage of Power in last Sunday's New York Times Book Review respected the rule of omerta about how this jazz really works with enough circumspection to make any Mafioso proud.

Both Carson and Greenberg are among the very few reviewers to even mention Robert Dallek's two volume biography of Johnson.  They both advocate it as authoritative and definitive.

I want to add that that a few of the other reviewers, most notably Erik Nelson writing in Salon, were critical of the book's length, and of Caro's habit of recounting large passages from earlier volumes in the series.

Now, looking at Amazon, I find something striking.   Most of the five star and four star user reviews, while not exactly as extensive or as well-written as some of the "expert reviews" cover most of the same ground. And, just as with the major media outlet reviews, very few these consumer reviews venture into criticism of Caro's narrative voice.

Here is one example that breaks out of a simple laudatory summary of the book:

The source material of The Passage of Power has been picked over at an order of magnitude far beyond that of the first three volumes. That presents Caro with certain difficulties he had not earlier had, as he more often relies on quotes. Because this is so much a story of Kennedy v. Johnson, the two warring camps have tended to give their conflicting versions and stuck to them, leaving no real way to nail down the truth. Many more call-backs to the previous books are also made. Caro's trademark style is left somewhat hobbled (and having recently read The Power Broker, it seems that Caro's style has become muted over the years). But Caro is still Caro, and his prose should remain the envy of his fellow popular historians.
Caro's feelings toward Johnson are understandably complex, and his treatment of Johnson has vacillated from volume to volume based on how he feels about Johnson's actions during that period. To Caro, the Johnson of The Passage of Power was Johnson at his best; it was "a moment not only masterful but, in its way, heroic." His leadership and liberal principles were on full display, and his abusiveness and deceit were (temporarily) held in check. Johnson comes off very well here, particularly to a reader who hasn't read the first three volumes. The great divide between the Kennedy and Johnson factions leaves a historian with little choice but to pick between the two, and Caro usually comes down on the side of the Johnson men. But he by no means gives Johnson a pass. He shows Johnson's nastier strong-arm tactics and makes a convincing case that Illinois wasn't the only state stolen in the 1960 election--Johnson helped Kennedy steal Texas too.
Here is another:
Caro dismisses other historians and journalists who agree with his assessments of LBJ's motives, but he doesn't back up his evidence. For example, Caro claims LBJ really believed in civil rights because during a meeting with civil rights leaders he emphasized his commitment to civil rights. While I don't necessarily doubt LBJ's commitment to the cause, his attempts to convince liberals and civil rights leaders could just as easily have been an attempt to shore up his base for the 1964 election - especially being the first southerner to become president in over a century. Especially given that LBJ was a master at modulating his sales pitch to different audiences, I'd have thought Caro would have been a bit more thoughtful in using this as evidence.
I'm not sure I came across one mention of the Dallek biography in the Amazon reviews.

However, I did notice that most every one of the one-star Amazon reviews (there are 11 at this writing) were very concerned with the Kennedy assassination and accused Caro of covering up Johnson's involvement.

So, I read about 20 "expert reviews" and 69 "consumer reviews."  My findings are not very surprising.

The "expert reviews" are better written and have much better style than Amazon reviews.  However, I would have to say that the content of the reviews is very similar in both the major outlet reviews and the better Amazon reviews.

There is nothing at all in the Amazon reviews approaching the combined content, style, reason and judgement displayed in the Greenberg review in the Washington Post or the Carson review in Esquire. But, then again, those reviews stood out among their expert company as well.

Of course, this is just one book and one set of reviews, so it is kind of hard to extrapolate a great deal.

By the way, if you are interested in what it is like for a writer to devote his entire existence to documenting one man, you can read a profile of Robert Caro in Esquire.

No comments: