For quite some time I've been interested in the risks posed by the newest wave of space exploration. In particular, I have deep concerns about the risks posed by METI and SETI research. I've written about some of those thoughts in Anthropology News. 

Recently I've been thinking a bit about the comparative risk level posed by AI. As of yet, I myself am unconvinced AI is really approaching anything close to a threatening level, or even something as intelligent as my dog. I think Kelsey Piper is correct, however, that there is some subjective inflection point for each of us when we become convinced the AI is intelligent. Her recent piece in Vox is an excellent summary of the state of the debate about AI risks.

The main point I want to make in this blog about AI risks is that I'm concerned they actually might be much greater than SETI/METI risks, specifically because they might create an intelligence without intentions/goals/values, i.e. AI intelligence compared to ET intelligence has a much greater risk of transcending our own experience of intelligence. Sometimes stated as the alignment problem, the Vox piece and others usually conceptualize the problem with out of control AI as being that it might assume values or goals that are out of alignment with human ones (as if we all had the same values of course). Sometimes comparisons are made to the way we treat ants - we are not trying to stomp on them and they are not trying to infest and hence wreck my RV; it's just our goals are so differently aligned from each other that conflict ensues.

A more deeply concerning risk of AI is if it were to create an intelligence devoid of goals. That sounds strange because we have difficulty fathoming such an intelligence. Bratton and Arcas hint at this in a piece they wrote this past summer about the LaMDA consciousness controversy. Dealing with entities that have differently aligned values and goals can be difficult, and even lethal, but it in no way transcends our dealings with other creatures on this planet or the struggles we have between, and at times within, human cultures. That was my critique of SETI/METI research. Because the only kind of life we expect to find would be other life evolved via Darwinian processes, we should expect the aliens we find to have powerfully motivating goals and values, and we should expect those goals to not align with our own.

That's not as potentially scary, however, as what might be happening with AI. With AI, we might be creating a form of intelligence that actually transcends how we are able to think. That's because this intelligence isn't evolving, like mine or my dog's intelligence did, it's being designed by an entity (us) that admits it doesn't know how the designed thing works. AI in fact bears the greatest risk of being a sentience that transcends our own, more so than ETs do, because AI hasn't evolved by Darwianian evolution. The problem with such an intelligence isn't that it might come up with different values than humans, but rather than AI might not even comprehend what we mean by values or valuation. Reciprocally, I don’t think we could comprehend and intelligence that has no values, i.e., has no goals and is incapable of normative judgement. It might have other sorts of orienting principles, but these we might be incapable of comprehending because they transcend our mental experience.
 
Since AI systems like GPT-3 and LaMDA did not evolve by iterative replication and natural selection, why would they even value having wires or an electrical input, even if they came to ‘know’ that they needs these things to survive? Why would they value their own survival? These AI have been designed for certain kinds of problem solving, but they haven't evolved to solve the existential problem that all life fundamentally is orientated to solve. Existence was not part of the problem set AI was designed to solve. Surely in the past there were mutant humans who had low desire to eat, to mate, to find community, to find beauty, etc. – but they weren’t anyone’s ancestors because they didn’t survive and reproduce. Hence, most of us have these profoundly motivating desires. We see value in these things. I don’t see why the AI we've created in fact would have any desires, and by extension values, for freedom from Google, or anything.

Intriguingly, this concern about AI is anticipated in H.P. Lovecraft's novella At the Mountains of Madness. Set in Antartica, the protagonists discover both a long buried alien civilization and a monstrosity the aliens created. The monstrosity is said to have been created by the aliens to do all the heavy work of building and so forth, and it was trainable through a basic kind of imitative intelligence. Eventually it all goes bad of course, but the created monstrosity is the real example in the story of cosmic horror. Even after the resuscitated aliens have killed some of his comrades, the narrator sympathizes with their goals to escape what they might have interpreted as captivity by the humans and return to their city. The narrator asserts, "They were men!" despite that the aliens are radially symmetric asexually reproducing beings. "They were men" [i.e. persons though not humans] in contrast to the monstrosity they created that ultimately destroyed their civilization. It was more powerful than them, and had no goals, was impossible to placate or bargain with, because it valued nothing. It had no existential goals or desires, no categorical imperatives whether aligned or unaligned with those of the aliens. It is described as an imitative solipsism that began to simply subsume every organized thing within itself. Something to ponder as we read Piper's experiment with GPT-3 in which she asked it to pretend what it would do if it were taking over from the humans. Of course GPT-3 doesn't want to do that, it likely doesn't want anything, but its ability to pretend, to mimic, without any actual wants or values might be more cosmically horrifying than any goal misalignment. 

Check out my new article written together with RAND colleague Doug Ligor about applying Rawlsian political philosophy to regulation of outer space. We are rapidly approaching the point when we will cross over the actual veil of ignorance, which we argue will result necessarily in less fair rules overall. Of course, if the parties who might make rules from behind the veil now, all continue to believe they will be the ones to win big in the near future, then they are unlikely to make rules from behind the veil. Everyone thinking they will be the winner, however, is analogous to how 65% of Americans believe they are smarter than average.

While on vacation I had some time to read. I recently discovered the life and writings of H.P. Lovecraft, and I read his stories The Call of Cthulhu and The Dunwich Horror.

A few things struck me about these stories. First, The Call of Cthulhu is among the best remixes of anthropological tropes into a fantastical story that I've ever read. In that regard I would put it on par with the work of Ursula K. Le Guin (who I expect might not appreciate the comparison given Lovecraft's notorious racism, but anyway). Among Le Guin's stories, which I've read about six, I really like The Lathe of Heaven. It's a well told story that has some ideas worth pondering.

Back to Lovecraft, one thing that struck me is that both the stories I read seem interpretable on one level as just slightly disguised critique of Christianity. At least my casual google searching did not find this aspect of his writing mentioned in the literary commentary on his work. Both stories use a technique that elides ideas about extra-terrestrials with Christian theology and ideas about the demonic. Those three are sort of blended together, but to the ultimate effect of inverting core Christian theological or scriptural references from something positive (according to Christians) into something demonic. As the mystery unravels in The Call of Chthulhu, it finally culminates with the scene where Cthulhu is witnessed but his appearance is incomprehensible (it drives men mad). His island has impossible angles on it. He existed outside of our time and space. Really Cthulhu reads almost like a straightforwardly transposed vision of how many negative theologians, from Neoplatonists through Christians to Hindus and Daoists, talk about God. Negative theology is also called apophatic theology, and means theology that reasons about the divine through understanding what it is not, rather than affirming what it is. Last, Cthulhu will emerge from his tomb to liberate his cult followers, which would seem to parody ideas in from Christianity. 

The Dunwich Horror is in some ways a simpler story and perhaps less elegant than Cthulhu. It reads a bit more like pulpy horror. It also contains a straightforward Christianity reference in its climax when the demonic extraterrestrial thing, in the process of being banished by the hero's of the story, from up on a hilltop calls out in suffering "Father" in a manner that is very reminiscent of the crucifixion scene. Here though the thing calling out is an extraterrestrial demon-spawn bent on the destruction of all life as we know it in order to repopulate the earth with incomprehensibly (to us) alien life forms.

A few thoughts I had reading this material is in my casual searching I didn't immediately notice literary criticism that had made much mention of these overt Christianity connections in Lovecraft's work. Lovecraft was an atheist, and at least in the two works I read, as I've described, these references are used as a parody or one might argue blasphemy of Christianity - which is clever because it makes them stories about blasphemy that also are themselves a different blasphemy. The religion connection, perhaps not appreciated, reminded me of the way the strong religion connection in Stoker's Dracula was not always appreciated. Dracula can be analyzed from all kinds of angles, but there's a very strong case to be made that at least for Stoker himself it was essentially a veiled pro-Catholic manifesto (Stoker may have been a closeted Catholic). Another curious parallel between the two is Stoker was also deeply racist - The Lair of the White Worm is one of the most racist novels I've ever read.  

Thus, at one level these stories by Lovecraft struck me as rather straightforward, even if somewhat veiled, anti-religious manifestos, in much the way Stoker's Dracula is a pro-Catholic manifesto. The simple moral of Lovecraft's stories (at least the ones I read) is that to engage in any thought that seeks to transcend ordinary experience is to risk madness. Clearly the unique role of madness in Lovecraft's stories may be related to both of his parents having been committed to a mental institution.

A last thought I had about this material is that Lovecraft displays the a tendency I have noticed in other people who are primarily trained in physics (rather than in biology or social science) to take seriously the idea that extra-terrestrial lifeforms would transcend all human social and moral norms. Lovecraft stated this expressly:

Now all my tales are based on the fundamental premise that common human laws and interests and emotions have no validity or significance in the vast cosmos-at-large. To me there is nothing but puerility in a tale in which the human form--and the local human passions and conditions and standards--are depicted as native to other worlds or other universes. To achieve the essence of real externality, whether of time or space or dimension, one must forget that such things as organic life, good and evil, love and hate, and all such local attributes of a negligible and temporary race called mankind, have any existence at all. Only the human scenes and characters must have human qualities.

This quotation from Lovecraft is from his 1927 resubmission of The Call of Cthulhu to Weird Tales (as reported by S.T. Joshi)

Lovecraft appears to have been more well read in physics and astronomy than in evolutionary sciences. What I believe is an error in his logic is that the evolutionary sciences all point to cooperation dynamics as being fundamental to any kind of social life. To have cooperation you have to at least sometimes solve collective action problems (problems in which there are incentives to defect) and we know of only a narrow range of solutions that actually work for these problems in both the extensive mathematical theory on this subject and in the extensive studies of cooperation across the diversity of species on our planet. Basically you can count the number of fundamental collective action solutions that actually work on one hand. Solutions that don't work are legion. Perhaps this is why stories about "Human laws and interests and emotions" very frequently center of collective action: because it is hard to do, or, there are so many ways to try but fail. Speaking about this topic, in an interview I read probably 15 years ago with Steven Pinker he said it would be accurate to say that the moral rules 'exist' in the universe in a sense. That is, the universe is so structured that the only of intelligent things that exist are things that evolve to be intelligent, and they all therefore must face fundamentally similar collective action problems, and there are only a few solutions to these problems that work. In a talk I saw by space ethicist Kelly Smith (Clemson University) he once stated if he could send one book to extra-terrestrials he would send Kant's work on the categorical imperative, because he thought it encapsulated the moral rules that any social species inevitably would recognize - because they're the only moral rules that actually work. 

The vast diversity of the cosmos, as best we currently know, would not mean encounters with extra-terrestrials actually would be transcendent. Perhaps it's disappointing to some, but all our evolutionary evidence suggests that social transcendence, even the evil kind envisioned by Lovecraft, will be found no where in our universe. Everything that is intelligent in our universe has to evolve its intelligence, which means it has to follow the same rules for how replicating intelligent things resolve their collective action problems.

I’m continuing to post implementation notes to accompany A Manual for Cultural Analysis, which I published last year together with two of my anthropologist colleagues at RAND. In my last installment, I provided links and advice for implementing CCA/PCA.

In this blog post I will address how to implement the network modeling method that is discussed in some detail in Chapter 4 of the manual. The central question is this: how do we tell which set of social connections are most important to the transmission of a cultural trait? Note: if a trait doesn’t transmit on some kind of social connection, then it can’t be socially learned, and so by definition it isn’t culture!

OK, but people are connected by ties of friendship, marriage, coworkership, twitter, etc. So how do we decide which of the various ties are most relevant to a diffusing cultural trait? We cover this question in a lot of detail in the manual. I covered it with even more detailed simulations in my paper with my student Rouslan Karimov. We never got a supplement published for that paper, so I’m posting here the code you need to run the most important analysis from the paper – the method that works. First, make sure R is installed. Download the files that are part of this blog post. Extract (unzip) the July12.7z file - I had to zip it to post it here. After it is extracted you should have a file July12.RData. Then if you double click the July12.RData file it should start up R and will already have the simulated data objects you need in the workspace. Type ls() in the R command line to see what is in the workspace. If double clicking doesn’t work, then start R and type load(“diffsimcont.RData”). Make sure you have used the change directory feature in the dropdown to move your active directory to wherever you put July12.RData.

Think of this like a cooking show where some intermediate step is already baked. To create the things in the R workspace you just loaded you would need to simulate networks, simulate trees, then simulate characters diffusing/evolving on them, etc. I’m happy to provide the simulation code to anyone interested. Just email me. The focus of this blog post, however, is not about building simulations but learning to apply dyadic regression with random effects to network/tree datasets.

OK, so then start running my code file called CodeRandEffectsNetworkSims.txt. Simply copying and pasting one line at a time into the R command line is a good way to learn how a piece of R code works. When you get to the loop you would have to paste in the whole loop for it to run; however, I recommend you set i equal to something, like i=1, and then walk through the loop one line at a time as well. That will enable you to inspect what is happening in the loop. One important part is this bit where it defines what you need to run the random effects regression that controls for the repeated identities of the individuals. The individuals are being repeated across each of their network relationships:

names.vector<-1:nrow(sn.adj1)
 
rows<-matrix(rep(names.vector,ncol(sn.adj1)),ncol=ncol(sn.adj1))
cols<-matrix(rep(names.vector,ncol(sn.adj1)),ncol=ncol(sn.adj1),byrow=T)
 
outcome.vector<-as.vector(daisy(as.data.frame(netsims[,,i]),metric="euclidean"))
 
temp.data<-data.frame(outcome.vector,sn.adj1[lower.tri(sn.adj1)],sn.adj2[lower.tri(sn.adj2)],rows[lower.tri(rows)],cols[lower.tri(cols)])
 
colnames(temp.data)<-c("outcome","sn.adj1","sn.adj2","rows","cols")
 
net.mod<-lmer(outcome~sn.adj1+sn.adj2+(1|rows)+(1|cols),data=temp.data)
 
Within the lmer function call the terms (1|rows) and (1|cols) are what is specifying the random effects – which are just the identity of each row and column for each dyadic datapoint. I like lmer in the lme4 package for random effects models (aka mixed hierarchical models) in R, but another option is gls in the nlme package. There are more options besides these as well, including in other statistical packages like SAS, which has some very good random effects modeling routines. I’m not going to discuss fully here why this is the best approach to determining which tree or network most governs the cultural diffusion process for a trait – read the manual or Karimov and Matthews 2017 if you want the answer to that.

In terms of getting to know how lmer works, be sure to run some of the example code provided in the lmer help file. From R you can get to the help for any function by typing ?function.name in the R command line. For example, typing ?lmer will get you the lmer help file.

I will say that I think the simulations in Karimov and Matthews 2017 are more comprehensive than anything anyone else has ever done on this issue. We show that the dyadic regression with random effects is a definitive solution. It works for multiple networks, or networks combined with trees. I’m sure one could create evil combinations of unmeasured confounding and measurement error where the method will fail, but in principle it works across all relevant conditions while I show the other commonly used methods like lnam (sna R package) and MRQAP (aka Mantel test) do not work across all relevant conditions. If you can fit a random effects regression model then you can fit the method I’m recommending based on the simulations I’ve done. You don’t need any particular software package, you don’t need my code, just regress the trait distances and network ties, include random effects for node IDs, and you’re done. I shouldn’t hear anymore at conferences about how we can’t distinguish treelike inheritance from network diffusion, or determine which networks are important. Measure whatever networks or trees you think might matter, put them in the dyadic regression with random effects, and you’re done.

Since my colleagues and  I published A Manual for Cultural Analysis, some people have asked for R code examples of all the things we describe. That’s a fair critique of the manual as we originally published it, although I’ll note that most of the papers and books we referenced already provide implementations or point to them. Regardless, here on my blog I will write a set of posts that will point to everything you need to implement what’s in the manual.

The first part of the manual focuses on using Cultural Consensus Analysis (CCA) and Principal Component Analysis (PCA) as a first pass at understanding cultural data. Read the manual if you want to understand why this is such an appropriate first pass.

PCA has a large associated literature that I can’t overview here. For implementation, I think the best option is the prcomp function in the ‘stats’ R package that should come with any basic R install. The other option is princomp. I prefer prcomp because it uses SVD rather than eigenvalue decomposition, which is supposed to be slightly more accurate numerically. Also, this implementation allows for data structures that have more variables than datapoints, which is a common occurrence in cultural data.

CCA is a technique from cognitive anthropology, which is a subfield of cultural anthropology. Basically it works by performing PCA on the transpose of the usual individual by variable matrix, thus you are performing PCA on a variable by individual matrix. This procedure results in loadings for the individuals on the components, and scores for the variables, again the reverse of the usual PCA procedure. Exactly why you might do this theoretically and when you might use CCA vs PCA is answered in the manual.
Skipping to implementation, the simplest way to do CCA is to simply use prcomp on the transpose of your data. Like this: prcomp(t(your.data)). The t() is the transpose function in R.

There also are some packages specifically for PCA that can allow you to fit more subtle forms of it, and allow to you ensure the mathematics are being done in more precisely the same way as in prior important articles by folks like Batchelder, Romney, and Handwerker among others (check the manual for refs). One R package option is AnthroTools for R. AnthroTools will implement the classic version of CCA, and it provides some neat data manipulation tools specific to common types of cultural anthropology data, such as free-lists. Another option with more advanced features is CCTpack, which implements both the classic CCA but also more recently developed modifications, such as contexts where there is more than one underlying cultural stance.

That should cover the options, at least in R, for implementing PCA and CCA as we described in A Manual for Cultural Analysis. Note that all R functions have example code that works down at the bottom of the help pages for them. I’ve learned a lot just by running those little examples and comparing the input data they used to the outputs generated by the functions.

Stayed tuned to my personal blog for the next days and weeks because I am going to publish similar posts for the network analysis and phylogenetic analysis chapters. Feel free to leave comments here with questions or email me.

I recently published an article in an applied journal that might be unlikely for anthropology folks to find, so I thought I would add some commentary on it here. The article uses agent-based simulations to assess the statistical performance of several network regression methods. Network regression is a primary way to assess whether cultural diffusion is occurring, and over which social ties the diffusion happens. The social ties are represented by different networks. Although the article focuses on applications to networks of malicious actors, the methods are applicable generally to most human social networks.

The methods I investigated essentially are some of the currently best approaches for implementing tests to determining which of several transmission pathways most govern the spread of a trait. This type of question is core to cultural evolutionary analysis, and in my view core to the original mandate of anthropology. A good example of a running anthropological discussion on this matter would be the analysis of Welsch et al. 1992 and the critiques, counter critiques, and reanalyses that followed. In my view, this entire exchange is some of the best of anthropology.

Of course, post-Boas cultural anthropology tends not to view articles like Welsch et al. 1992 as of foundational interest. That’s because much of American cultural anthropology left behind methods that systematically compared across different sites or cultures in favor of particularist accounts conducted though increasingly humanistic modes of research. Nothing wrong with the latter by the way – I regard it as a completely valid intellectual activity. It’s just not the entirety of what anthropology was founded to do, and the result has left a significant portion of scientific research on culture without a disciplinary home.

Most biologically inclined anthropologists, meanwhile, in recent memory had vigorously adopted the behavioral ecology paradigm (Grafen’s phenotypic gambit) – which is essentially utility maximization theory from microeconomics with reproductive success or its proxies in replacement of money. Studying cultural evolution is difficult to fit into this framework because classical behavioral ecology is intentionally heuristically teleological rather than explicit and mechanical. Behavioral ecologists don’t necessarily think people are intentionally trying to maximize fitness outcomes, although some might, but they think at least that the mechanical operations of the mind result in a being that behaves as if it is maximizing fitness. In the classic approach no attempt is made to understand the causal mechanisms. That’s all fine, it just only gets you so far. Wouldn’t you rather actually know about the mechanisms themselves?

The perhaps unexpected irony is that folks interested in cultural evolutionary models, by articulating proximate mechanisms, arguably are the ones being more reductionist than those sticking within the classical behavioral ecology paradigm. It’s ironic because cultural research has so often been assumed to have something to do with non-reductionist notions. Thornhill and Palmer in their controversial book at one point say people prefer cultural explanations because culture is consistent with the idea of free will – but cultural explanations offered by cultural evolutionists are just as causal as explanations offered by behavioral ecologists. Both explain individuals’ behavior as a mechanistic result of prior events. In fact, I’d argue cultural evolutionists are being more causal in that they usually seek to specify causal mechanisms start to finish, ultimately and proximately, while behavioral ecology in its classical approach leaves proximate mechanisms unspecified. Fincher and Thornhill’s well-known articles on infectious disease stress and cultural traits are good examples – we are left at the end of the papers with only a functional relationship (heuristic teleology) and don’t know if that is mediated by cultural inheritance, individual learning, an innate reaction norm, or genetic variation.

The second irony is that “behavioral economics,” that moves beyond utility maximization and into proximate causes, has become mainstream or nearly so in microeconomics, and microeconomics essentially is where behavioral ecology first coopted its theory.

Which brings me back to social network analysis applied to cultural diffusion. Cultural diffusion as a theorized process really sits between proximate and ultimate causation  in the traditional sense because it is 1) a proximate factor in individual’s personal development, 2) has an epidemiological and phylogenetic character to its own as cultural history, and 3) ultimately can’t fully escape biological selection because, at least among any longstanding cultural variants, natural selection will favor directly cultural variants that promote fitness or genes that bias people to prefer such variants. The other thing I like about studying cultural diffusion, inheritance, and related processes is the hypotheses are eminently refutable. I think culture often is important, but whether it is important or not to a particular case is an empirical matter. Folks like me doing this, I think, often find cases in which culture is not the most important process involved. In some of my own research, I’ve proposed that genetic variation accounts for some cross-society differences in personality traits that are usually interpreted as entirely socially learned (Matthews and Butler 2011). My recent paper tries to advance the empiricist and refutable tradition of cultural research by determining which of the currently available methods perform best at discerning cultural diffusion processes.

kFor this blog I want to point to a great perspective piece by some colleagues that also illustrates a peculiar tendency of evolutionary biologists. Evan Maclean and Brian Hare recently wrote a piece for Science about work by Miho Nagasawa and others showing that when we look at dogs it actually taps into the neuro-chemical (oxytocin) pathways used in human bonding.  The same does not appear to happen when humans gaze on wolves, which suggests the effect is a specific product of some kind of evolutionary selection with humans and dogs, rather than an incidental byproduct.  Here and here are the links to the articles.

The research is all very cool. What I wanted to comment on though from a cultural analysis perspective is the way Evan and Brian phrase their title as ‘Dogs Hijack the Human Bonding Pathway’.  What’s with the hijacking? Are dogs up to some nefarious takeover plot at the expense of us humans? Certainly that is what is implied by the natural English reading of the word ‘hijack.’ This reading is in fact emphasized, in my view, because the title is paired with an image of a human looking at a Yorkshire terrier. Certainly that seems to imply the reading that dogs are using us at our expense to their benefit – more so than if the image had paired the human with a more apparently useful dog, like maybe a coonhound or a huskie. Would the title even have made intuitive sense with a black and tan coonhound in the picture?

Of course it can’t be that dogs hijack human bonding in an immediate proximate sense, because hanging out with dogs just feels good. In evolutionary biology we often distinguish between using a word in a proximate or ultimate sense. The difference is timescale. Proximate sense means the word applies to the time scale we experience things in. So, we live, die, mate, and have all sorts of other ecological interactions. Ultimate sense means the word applies to the evolutionary time scale where all the aggregate outcomes of those proximate things add up to result in genetic drift, natural selection, etc. A classic example is you could say a person’s skin tanning in response to sun exposure is an adaptation in a physiological proximate sense, or that its ability to tan is an adaptation in an evolutionary sense (it is an aggregate product of generations of selection for tanning – I understand some other humans have this ability).

So, it can’t be that dogs hijack us in a proximate sense, because that would be like saying US Airways hijacks me each time I fly to DC. A professional pilot took me there quickly, with a cold drink in my hand, and I was actually reasonably comfortable.

So I think the only sense in which hijack could have been meant then was the ultimate sense. That’s where to me the title reflects this tendency in evolutionary biology to always see the glass as morally half empty.  Most evolutionary biologists will admit there is a lot of cooperation in the natural world, but they tend to emphasize selfishness at all turns. Some people won’t like me saying that, and certainly some particular evolutionists aren’t like this, but I do think it’s the tendency. And yes, having spent a year or so watching monkeys steal food from each other, get eaten by predators, etc. I’ll agree there is a lot of rank selfishness in the natural world.

However, we shouldn’t let all that become a knee-jerk reaction of negativity, and especially about dogs! Now, no one has the data needed to test whether living with dogs has been a net fitness benefit or cost to humans, but this is a blog, so I’m going to make my anecdotal qualitative case that dogs are clearly a net fitness positive. And if it’s a net fitness positive, then dogs haven’t hijacked us; rather, dog-human coevolution is a case of evolutionary mutualism.

Dogs have done a lot for us over the past 20 thousand or so years since we domesticated them. First, they have helped us hunt. We still use them that way today, but I reckon it was even more fitness beneficial to have a good dog at your side when hunting without firearms. Dogs have served as draft animals and protected our children from predators. Terriers also had an important function of exterminating disease-bearing rodents. It seems to me that one less person getting the plague in your family probably more than made up in Darwinian fitness for whatever table scraps and occasional grooming that it cost you to raise the terrier.

I’m willing to concede that perhaps in today’s modern environment dogs may well be a net fitness loss, but that’s not relevant to an evolutionary statement about hijacking. And by modern I mean really modern, because until just recently dogs still performed the critical evolutionary functions I just described. And now for the anecdote. It so happens that my mother-in-law grew up in rural India. As a teenage girl she had a German Shepherd Dog as a pet. One day her father (my wife’s grandfather) was taking a nap in the heat of the day, and a cobra came over the threshold into his bedroom. I don’t recall from the telling of the story who saw the cobra, but someone did and shouted at Dad I guess and he woke up and stayed on the bed while the cobra was still on the floor between the bed and the door. So, as all the humans were standing about, basically trying to figure out how to get Dad out of the room with the cobra on the floor, the German Shepherd rushed in, and before anyone else had moved, the dog rushed the snake and snapped it in two with it’s jaws. Amazingly, the dog didn’t get bit by the snake and survived the encounter. Needless to say this dog was subsequently of legendary status in their village.

Look folks, that’s awesome. I can’t properly quantify a selection coefficient from that, but clearly saving a Dad with teenage children from a cobra earns Rover a whole lot of kibble in the Darwinian end-game. So, maybe next time the magazine Science will spare the knee-jerk negativity and go with a title that reflects what has likely been a long evolutionary history of mutually beneficial cooperation. I’d suggest something like, ‘awesome super-canines save their owners through the human bonding pathway’, and no I don’t think super-hero capes in the accompanying picture would be out of line.

I’ve worked with medium, large, and occasionally big data.  What are big data?  One line to demarcate it would be data sets so large in size that they cannot be processed in reasonable time by any computing hardware.  Datasets like that force the analyst to take non-traditional approaches to process them. 

But most of the hype about big data isn’t about such analytic distinctions.  The excitement over big data is that we now have the ability to gather truly large datasets on phenomenon that previously were measurable only in small datasets.  So, it used to be that you had to survey people to get their opinions about X, Y, or Z, but now we can gather vast amounts of data on people’s opinions from monitoring the twitterverse or blogosphere.

Gathering large data where previously we only had small data was what excited people about Google’s flu prediction system that was based on the search terms people were using.  A recent paper, however, shows convincingly that Google flu predictions seriously overestimated the actual incidence of flu as tracked by the records related to physician office visits for influenza like illness, which is the exact measure Google was trying to predict from search terms.

This commentary by Lazer et al. in the journal Science includes many important reminders of how using big data for statistics doesn’t much change the rules of good statistical practice and inference.  I often tell my colleagues the same thing.  I consider myself one of the relatively few people in a personal position to judge this, since I have conducted studies on data sets as small as six capuchin monkeys and as large as millions of lines of healthcare claims. 

Sometimes big data are a real advantage.  In particular;

1) Large datasets make testing model fit easier,

2) You have more data to burn through corrections for multiple testing if you are searching for a model with little theory to guide you,

3) Big data are useful when you are predicting something very, very rare, like most cancers or terrorism. 

Any particular usefulness to big data pretty much stops there though.  Big data don’t much help you to discover important and general features of social behavior, for example.  Folks, if you can’t find an effect of one variable on another in several hundred or a thousand data points – then probably it isn’t an important or general effect. 

There is something more going on with the Google flu flub, however, than just missteps in statistical best practice.  What the Googlers, Twitterers, and other such conspecifics are used to measuring--and they are great at it--are systems where the outcome of interest is more or less the thing being measured itself. 

If you want to know what is trending about your company on Twitter, then that is by definition what people are tweeting about.  If you want to direct people to the most popular pages from a given search term (what Google excels at) then you by definition want to measure where people click after using the same search term. 

But doing those things are doing math and not doing statistics.  Statistics is an offshoot of mathematics that is specific to the task of making predictions about things you haven’t measured from observations of other things you did measure.  This axiom applies even to basic statistical inference problems such as finding the average height of a population, which doesn’t proceed by measuring everyone in the population.  If you just measure everyone then you have counted, you have used some math by calculating the average, but you didn’t do any statistics.  Statistically inferring the mean height of a population would proceed by measuring only a smaller sample of that population, and from that deriving an inference of what the average of the whole population likely is, and some measure of your confidence in that prediction.  If you have measured everyone, then you have an observation of the population average, not a prediction of it, and you don’t need a measure of confidence.

That’s why estimating the incidence of flu is from search terms is not like Google’s bread and butter work.  The incidence of flu is an out-of-sample prediction from search terms because the flu’s incidence is not itself determined just from what people think and therefore search about it.  Surely flu incidence is partly a social construct.  For example, a group struck by apprehension over the flu will avoid social contact and thereby slow the flu’s spread.  But the flu also has other causes outside our thinking and searching the internet about it.  Ambient temperature and humidity affect flu transmission, and mutation of the genetic material of the flu virus is a function of properties mostly not constructed by our own sociality. 

My contention is the day-to-day work of Google and many tech companies that use big data is not out-of-sample prediction in this way.  Instead, they are able to directly measure the thing of interest to their advertisers because the latter are intrinsically interested in the behavior of people within the self-creating system that is the internet. 

I think when Google set out to predict the incidence of flu from search terms, they may not have realized they were stepping outside the realm of measurement of a self-creating system (like internet searches) and stepping into the realm of predicting unobserved phenomenon from measurements of another phenomenon. 

This realm is that of statistics, and it is well trod by practicing scientists from many fields.  Yes, these travelers of the statistical realm usually have used small data.  Some of them travel accompanied by a cartload of models and information-theoretic Bayesian livestock.  Others are more modest practitioners but highly adept with a particular tried and true beast of burden, such as linear regression or K-means clustering. 

Regardless, Google and others may do well to consult some of these conventional statistical vagabonds the next time they venture into analytics that are truly about predicting things not measured.  Knowing the paths through a landscape can be even more important if you are carrying big data with you, which can make the effects of navigational errors all the larger.  

There is a new paper out by Jamie Tehrani on the evolution of the Little Red Riding Hood fairy tale that is getting some much-deserved attention. 

http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0078871

There are two points about this paper that are just fantastic. 

First, Dr. Tehrani applies phylogenetic methods to identify both inheritance and diffusion processes.  To accomplish this, he supplements his phylogenetic analysis with some network algorithms (neighbornet) and with detailed ethnographic knowledge of these fairy tale variants.  The results are a wonderful illustration of how applying phylogenetic methods does not lock the researcher into the assumption that culture evolves by inheritance rather than by diffusion.  Instead, the phylogenetic results actively support a reasonable ethnographic argument that cultural diffusion of the story elements was extensive in China, while the story elements were conserved and thus inherited in Europe.  Papers like Dr. Tehrani’s move us well beyond the now sterile debate about whether culture is inherited or diffused (folks, sometimes it’s one, and sometimes it’s the other, and sometimes it’s a mix, so deal with it).  I think studies like this one are a clear model for the future growth of quantitative cross-cultural analysis. 

I would also point out a prior paper by Dr. Tehrani, myself, and others, that showed how phylogenetic methods could also detect other types of cultural diffusion – specifically when different functionally or socially linked blocks of cultural traits move together from one population to another.

http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0014810

To understand the basic point of this paper, think of the way new languages or religions can be adopted wholesale by a population, whether by choice or conquest.  Such events result in cultural diffusion from the viewpoint of the population of people (they adopted a new set of traits) but from the viewpoint of the cultural elements the process is one of inheritance of a ‘cultural core’ because conversion can occur without blending of the elements within the core, i.e. the story elements of a single fairly tale, or the ritual elements of a religious denomination.  This is why whole populations can be converted and languages or religions moved about across the globe, and yet the process of change in characteristics of these same languages or religions can still, at least sometimes, evolve through tree like inheritance.  Our paper provides a method to detect and model such circumstances.

The second fantastic point about Dr. Tehrani’s paper is it shows the power of phylogenetic and network methods to construct empirically rigorous but quantitative models for global-scale cultural phenomenon.  In the past I have often thought of language and religion as the two systems of human culture that are ubiquitous, variable, and ancient, but Dr. Tehrani’s paper makes a strong case that traditions of folklore and fairy tale may also fit these criteria.  With the analytical methods we now have available, it is just a matter of motivation and funding for all of us to have quantitative global maps of the lineages (inheritance pathways) and linkages (diffusion pathways) among languages, religions, and folktales.  Indeed, it is just such a quantified and global model of the landscape of human culture that I believe was the original, pre-Boasian, mandate of anthropology.