Wegman and Said on social networks: More dubious scholarship
Posted on April 22, 2010 by Deep Climate| 105 Comments
Today I continue my exploration of the dubious scholarship in the contrarian touchstone known as the Wegman report, this time focusing on the report’s background section on social network analysis. As many readers may recall, Wegman et al used a simplistic analysis of co-author relationships to speculate about supposed lack of independence between researchers in paleoclimatology, accompanied by lapses of rigour in the peer review process. This, of course, echoed similar accusations by self-styled climate auditor Steve McIntyre.
In both the original Wegman report and a subsequent follow-up paper by Yasmin Said, Wegman and two others, the background sections on social network research show clear and compelling instances of apparent plagiarism. The three main sources, used almost verbatim and without attribution, have now been identified. These include a Wikipedia article and a classic sociology text book by Wasserman and Faust. But the papers rely even more on the third source, a hands-on text book that explores social network concepts via the Pajek analysis software package – the same tool used by the Wegman team to analyze “hockey stick” author Michael Mann’s co-author network.
Not only that, but the later Said et al paper acknowledges support from the National Institutes on Alcohol Abuse and Alcoholism, as well as the Army Research Laboratory, raising a host of new issues and questions. And chief among those questions is this: Will George Mason University now finally do the right thing and launch a complete investigation of the actions and scholarship of Wegman and Said?
I first explored apparent plagiarism in the Wegman report late last year. At that time I analyzed wholesale cribbing from, and distortion of, material from “hockey stick” co-author Raymond Bradley. But I also pointed out that at least some of the background material on social network analysis appeared to be taken from other sources, without attribution.
I have now done a complete analysis of section 2.3 of the Wegman Report, which runs close to five pages. As we shall see, a careful sentence by sentence analysis shows that almost all of the material can be traced to one of the three antecedents named above.
Said et al 2007
Before turning to the details of that analysis, I’ll touch on a little known follow up article, namely Social Networks of Author–Coauthor Relationships, published in July 2007 in the journal Computational Statistics and Data Analysis (a year after the Wegman report).
The four authors were all affiliated with George Mason University and are all connected to the Wegman panel and report. They include two of the Wegman report authors, namely Wegman protoge Yasmin Said and Wegman himself. The third author, John Rigsby, was acknowledged as a contributor to the Wegman report (in fact he performed the block model analysis of Michael Mann’s coauthor network) and is now apparently working on his PhD under Wegman at GMU. The fourth author, Walid Sharabati, did not participate in the report, but did supply an analysis of Wegman’s own co-author network for Wegman’s written response to questions posed by Rep. Bart Stupak in connection with the “hockey stick” congressional hearings [PDF 2.7 Mb]. Like Said, he was a Wegman PhD student; he’s now at Purdue University.
Said et al describe the genesis of their paper this way:
Wegman et al. (2006) [i.e. the Wegman report] undertook a social network analysis of a segment of the paleoclimate research community. This analysis met with considerable criticism in some circles, but it did clearly point out a style of co-authorship that led to intriguing speculation about implications of peer review. Based on this analysis and the concomitant criticism, we undertook to examine a number of author–coauthor networks in order to see if there are other styles of authorship. Based on our analysis we identify four basic styles of co-authorship, which we label, respectively, solo, entrepreneurial, mentor, and laboratory. The individuals we have chosen to represent the styles of co-authorship all have outstanding reputations as publishing scholars. Because of potential for awkwardness in social relationships, we do not identify any of the individuals or their co-authors.
Of course, the “entrepreneurial” style network was already identified in Wegman et al as that of “hockey stick” paleoclimatologist Michael Mann. Perhaps more “awkward” is the fact that the mentor style network described is that of Wegman himself and is obviously based on Sharabati’s earlier analysis of his own mentor’s coauthor network.
Clearly, any analysis of Wegman et al’s speculation on peer review in paleoclimatology must also cover the Stupak response and the Said et al paper, a task I’ll leave to another time. For now, I’ll note that it’s hard to imagine a more self-serving approach to scholarship concerning peer review issues than the one emerging here.
Wegman’s sources
Let us now turn to the analysis of the antecedents of the Wegman et al background section on social networks in greater detail. (I’ll follow that with an analysis of the introduction section of Said et al 2007, which is essentially the same lifted material in reduced form). If you want to follow along, here is a side-by-side comparison of Wegman et al and the sources I’ve identified.
Right off the top, Wegman et al quote the favourite source of scholars in a hurry: Wikipedia (more precisely, the 2006 version of the Wikipedia article on social networks).
Here’s the opening of Section 2.3 in Wegman:
A social network is a mathematical structure made of nodes, which are generally taken to represent individuals or organizations. … Social network analysis (also called network theory) has emerged as a key technique and a topic of study in modern sociology, anthropology, social psychology and organizational theory.
And here’s Wikipedia:
A social network is a social structure between actors, mostly individuals or organizations… Social network analysis (also sometimes called network theory) has emerged as a key technique in modern sociology, anthropology, Social Psychology and organizational studies, as well as a popular topic of speculation and study.
It’s quite close. But the changes don’t even make sense. Right of the top, Wegman et al seem to misunderstand the difference between a structure and its representation (in this case as a graph). The second sentence has been shortened – but also mangled in such a way as to remove the original sense of identifying social network analysis as both a key social science technique and a topic of study on its own.
Continuing on, there are fewer and fewer changes (shown with the occasional added or removed word as bold or cross out):
Research in a number of academic fields have has demonstrated that social networks, operating on many levels, from families up to the level of nations, and play a critical role in determining the way problems are solved, organizations are run, and the degree to which individuals succeed in achieving their goals. The shape of the social network helps determine a network’s usefulness to its individuals …
And so on for several more sentences, pausing only to skip over the Wikipedia’s table of contents. And that’s just the first paragraph.
Then it’s on to paragraph two – and the most used (or abused) source for this section: Exploratory Social Network Analysis with Pajek, by Wouter de Nooy, Andrej Mrvar and Vladimir Batagelj. This book has the undoubted attraction of serving as both a “how to” guide for the Pajek software package used by Wegman et al to generate various diagrams, as well as a providing some overview of social network concepts. (Not to mention that it appears to be freely available as a full PDF and so easily copied).
De Nooy et al have this apparently irresistable nugget:
… Social network analysts assume that interpersonal ties matter, as do ties among organizations or countries, because they transmit behavior, attitudes, information, or goods…
That has been lightly massaged to read:
Social network analysis assumes that interpersonal ties matter, whether they exist among individuals, organizations or countries. Interpersonal connections matter because they are conduits for the transmission of information, goods, behavior and attitudes.
It’s not clear why the order of transmitted items has been changed or why Wegman et al felt compelled to unnecessarily complicate the phrasing. But certainly small changes like “analysts” into “analysis” do make possible plagiarism detection harder.
At paragraph three, Wegman et al lift a long series of key definitions from the classic 1994 text, Social Network Analysis: Methods and Applications, by Stanley Wasserman and Katherine Faust. These concepts include “actor”, “dyad” “relational tie” and so on.
For example, here is the Wasserman and Faust definition of “relational tie”:
Relational tie. Actors are linked to one another by social ties. …[T]he range and type of ties can be quite extensive. The defining feature of a tie is that it establishes a linkage between a pair of actors. Some of the more common examples of ties employed in network analysis are:
Evaluation of one person by another (for example expressed friendship, liking, or respect)
Transfers of material resources (for example business transactions, lending or borrowing things)
Association or affiliation (for example jointly attending a social event, or belonging to the same social club)
Behavioral interaction (talking together, sending messages)
Movement between places or statuses (migration, social or physical mobility)
Physical connection {a road. river, or bridge connecting two points}
Formal relations (for example authority)
Biological relationship (kinship or descent)
Wegman et al cut out a few words and replaced the bullet points with a long list:
Relational Tie: Social ties link actors to one another. The range and type of social ties can be quite extensive. A tie establishes a linkage between a pair of actors. Examples of ties include the evaluation of one person by another (such as expressed friendship, liking, respect), transfer of material resources (such as business transactions, lending or borrowing things), association or affiliation (such as jointly attending the same social event or belonging to the same social club), behavioral interaction (talking together, sending messages), movement between places or statues (migration, social or physical mobility), physical connection (a road, river, bridge connecting two points), formal relations such as authority and biological relationships such as kinship or descent.
So it goes for another two pages.
Finally, Wegman et al cover “computational aspects”; for that it’s back to de Nooy et al to cover such graph-related concepts as partitioning and clustering. This section includes a melange of general concepts such as brokerage, as well as their graphic analogues such as vertex centrality. As the material is lifted from the introductions to various book sections, the whole feel of this section is oddly unbalanced, as the discussion jumps from topic to topic with little apparent logic.
As the section lurches to a close, the apparent plagiarism becomes clearer and clearer as there are only desultory changes, or none at all. For example, at page 22 of Wegman et al we have:
The concepts of vertex centrality and network centralization are best understood by considering undirected communication networks. If social relations are channels that transmit information between people, central people are those people who either have quick access to information circulating in the network or who may control the circulation of information.
Except for the indicated addition of the third “people” and the removal of two words, that’s identical to the paragraph found at section 6.5 (p. 133) of de Nooy et al.
Indeed, easily ninety percent of the material was lifted and lightly edited, with the only significant original material was from a paper by John Rigsby on the concept of allegiance.
And all of this, remember, was done without any attribution whatsoever.
Said et al rework it
Now let’s look at the sources for the introduction of Said et al, a comparison that it turns out also provides clues as to how the material was lifted. Once again, you can follow along with a detailed side-by-side comparison, wherein the same three sources are shown as for Wegman et al.
Indeed, as mentioned previously, the Said et al introduction is largely a condensation of the five-page Wegman et al section. But it starts out a little differently:
A social network is an emerging tool frequently used on quantitative social science to understand how individuals or organizations are related. The basic mathematical structure for visualizing the social network is a graph. A graph is a pair (V ,E) where V is a set of nodes or vertices and E is a set of edges or links.
The first part is barely comprehensible English and so is probably from the authors; once again, there is rampant confusion between a structure and its representation. Presumably they mean that social network analyis, not “a social network”, is an “emerging tool”. And surely they also want to say it is a tool used in quantitative social science rather than “on” it (whatever that means).
Said et al quickly return to the safe ground of Wikipedia, however, with the same text as rendered in Wegman et al:
Social network analysis (also called network theory) has emerged as a key technique and a topic of study in modern sociology, anthropology, social psychology and organizational theory. The shape of the social network helps determine a network’s usefulness to its individuals. Smaller, tighter networks can be less useful to their members than networks with lots of loose connections (weak ties) to individuals outside the main network. More “open” networks, with many weak ties and social connections, are more likely to introduce new ideas and opportunities to their members than closed networks with many redundant ties. See Granovetter (1973).
In this case, however, the authors have cut out the five subsequent Wikipedia sentences that appeared in the Wegman et al version. Another change is the addition of a reference to Granovetter, the one and only citation in the whole introduction; it’s reasonable enough, but that is clearly not the true source.
Presumably in the interest of brevity, Said et al then skip to the definitions found in Wasserman and Faust. This time, the headings have been removed altogether.
Social network analysis is concerned with understanding the linkages among social entities and the implications of these linkages. The social entities are referred to as actors that are represented by the vertices of the graph.
Once again this is taken straight from Wasserman and Faust’s actor definition, with the additional information that actors are represented by graph vertices (the sentence should presumably read “which are represented”, rather than “that”).
The exact same passage on relational ties is also given as in Wegman et al, but again without the heading.
Well, it’s almost the same. Wasserman and Faust refer to a type of linkage they call “movement between places or statuses (migration, social or physical mobility)”. In Wegman et al, this became “movement between places or statues“! You read that right – “statues”.
In Said et al, this was corrected, sort of. It now reads “movement between places or states, which undoubtedly makes more sense, if not exactly what Wasserman and Faust meant.
Beyond the amusing “typo”, though, this is a very telling detail. It implies that Wegman et al were likely working from a scanned version of the introductory chapter of Wasserman and Faust, converted to text using OCR. And it also implies that Said et al simply started from the earlier Wegman et al version “correcting” and condensing as they went.
In the final passages of the introduction, much of the de Nooy et al material makes a reappearance. The sections on partitions and clustering have been reduced, though, and the concepts of cohesion, brokerage and affiliation has been omitted altogether. But the section on centrality have been retained, once again virtually identical to de Nooy et al passages discussed above.
And, once again there is no attribution to the actual sources used. Wasserman and Faust are cited, but only in a subsequent section of the paper, so this is not helpful to the cause of Said et al.
It is also worth noting an oft-quoted definition from Wasserman and Faust, used verbatim in both Wegman et al and Said et al (and a number of other sources).
A social network consists of a finite set or sets of actors and the relation or relations defined on them.
Now look at this Google Scholar search on that exact sentence. In each and every case, a direct citation of Wasserman and Faust is given, with the sentence usually rendered in quotes.
Every case, that is, except one – Said et al.
Before leaving this piece of shoddy scholarship, I should mention the acknowledgments.
The work of Dr. Yasmin Said was supported in part by the National Institutes on Alcohol Abuse and Alcoholism under grant 1 F32 AA015876-01A1. The work of Dr. Edward Wegman was supported in part by the Army Research Office under contract W911NF-04-1-0447. The work of Dr. Said and Dr. Wegman was also supported in part by the Army Research Laboratory under contract W911NF-07-1-0059…
To be sure, the distortions and problems with the lifted material fall somewhat short of those I identified in the case of Wegman et al’s treatment of paleoclimatology. Nevertheless, as usual in cases of wholesale appropriation, even a cursory examination of the social network material betrays a shocking lack of understanding of social network analysis, accompanied by a complete failure to tie the background material to any meaningful analysis. The analysis itself consists of speculation based on a simplistic matrix of the analysis target’s co-authors. That this incompetent and deceptive work appears to have been subsidized by the U.S. government is outrageous. But that fact may have a silver lining – for it provides additional impetus for a long overdue investigation.
Implications for peer review
Indeed, the work of Wegman and his young team has raised some interesting questions about peer review and social networks. But ironically those questions point back, not to the scientific community, but to Wegman and his own social network of proteges and colleagues.
A full exploration of peer review issues in both the Wegman report and Said et al will have to wait for a subsequent post. But clearly there are many problems with Wegman’s hastily convened ensemble of reviewers; for example, William Wieczorek was on Yasmin Said’s PhD advisory committee. And Said et al sailed through peer review in a mere six days from submission to acceptance.
In both cases, there appears to have been no peer review from experts with relevant expertise in social network analysis or paleoclimatology, which explains the complete failure to notice the paucity of citations in key background sections. That was an obvious “red flag” that should have given any reasonable reviewer pause. (There are also numerous problems with Wegman et al’s paleoclimate material, such as confusion between the Northern Hemisphere 20th century temperature rise described by Mann et al, and the global rise discussed in the NAS report).
I now have demonstrated that two key background sections in Wegman et al (as well as the introduction to the follow up paper from Said et al) are riddled with apparent plagiarism and other problems. On top of that, of course, there is every indication that the Barton investigation and the Wegman panel were nothing more than a politically motivated attack on climate science and scientists from the start.
The flimsy accusations of “climategate” have been addressed by no less than three investigations in the U.K., and found to be groundless by the two completed so far. What will it take for responsible media to finally focus on the real scandal here? And how can a supposedly responsible academic institution continue to ignore the obvious?
For starters, George Mason University must initiate a complete, independent investigation of the activities and scholarship of Edward Wegman and his proteges. Now.
deepclimate.org |
