Why is society the way it is? The problem of infinity DAGs

Consider two questions:
1. Does racial bias in law enforcement in the U.S. occur? (assuming the answer is yes)…
2. Why does racial bias in law enforcement in the U.S. occur?

Ezra Klein wrote a piece in December on the danger of controlling for large numbers of variables in analysis because we could end up ‘controlling out’ key parts of our effect of interest (no, he doesn’t appear to be an epidemiologist or even any type of researcher, but nevertheless seems to have a better understanding of confounding than many with those titles).

In DAG language:

As he aptly recognizes, researchers know this. As we’ve been taught, there are also ways of dealing with it: base your variables on substantive knowledge, do not adjust for mediators, and if you do adjust for mediators, know what you’re doing [see: mediation analysis].

Klein’s problem with over-controlling is philosophically grounded in question number 2 above. He suggests that controlling for effects of the exposure prevents us from knowing why phenomenon occur. Once you control for location of drug use, black people end up far less likely to be arrested for drug crimes than white people. This is because they are more likely to use drugs in urban settings, and police are more likely to make arrests there. So in controlling for location, we lose the ‘why.’

But there is a distinction between question 1 & 2.  The first is complicated enough: teasing out whether an association exists and its strength is undoubtedly ‘epidemiology.’ It’s also quantitative ‘sociology’ with some ‘economics,’ and probably any science. It involves describing the world as it is.

Consider the (highly simplified) reality of why people who are black are potentially more likely to be arrested (Y= e.g. Arrest):


There are lot of cumulative, intertwined reasons ‘why’ racial bias might exist in U.S. law enforcement.  The particular letter (i.e. variable) we choose to study is somewhat arbitrary (A on Y? B on Y? C on Y?…).  Say we look at the effect of C on Y.  There are ancestors (A and B…) and mediating effects (D and E….).  Such is the case no matter where our study sits on the causal path.  In other words, there are infinity letters behind and ahead of our letter of choice.

Figuring out why society is the way it is is entirely relative. When viewed as ‘yes’ or ‘no,’ (‘Have you ever been target of racism?’) we can measure these things. But the cut-point is arbitrary.  When viewed as a cumulative sum of experiences, the DAG possibilities approach infinity; the ‘why’ is less and less measurable (‘So, what’s like to be Black in America?’).

As Klein suggests, we shouldn’t over-control or adjust for mediators. But perhaps the problem with this is more to do with biasing our analysis away from some true effect (i.e. the effect of the letter we arbitrarily choose to study) than Ezra Klein’s suggestion that it prevents us from knowing why. Are epidemiological studies of social phenomenon meant to answer ‘why’? Can they? The rigour in their methods comes from their ability to figure out what is. We know black people are more frequently arrested for similar crimes to white people. Why? We can only adjust so much, calculate the effect along so many possible pathways, and collect so much data. And we probably still wouldn’t know.

Open-access publishing – the good, the bad, and the ugly

This post summarizes a seminar given by Dr. Madhukar Pai at the Lady Davis Institute on February 2nd, 2015 entitled “Open-access publishing – the good, the bad, and the ugly”. A copy of the lecture slides were kindly provided by Dr. Pai and are available here (PDF).

As students and researchers, most of us have tried to access a scientific article important to our work only to be confronted by the publisher’s pay-wall. This is even more frustrating if the work we are trying to access or use is our own. Unfortunately, the body of scientific literature upon which we rely is primarily published using this “pay-wall” or “pay-for-access” model. This week, Dr. Madhukar Pai discussed open-access publishing as a worthy alternative while giving practical insight into its shortcomings and the outright predatory practices that have followed its introduction. For those unable to attend the talk, this post summarizes some of the important points discussed.

In the conventional publishing paradigm, researchers sign away the rights to their articles to publishers who exclusively provide access to readers for a fee. Publishers achieve high profit margins with this model while universities struggle to afford the exorbitant cost of subscriptions. For those without institutional access, results from high-quality scientific research are prohibitively expensive, disproportionately affecting researchers in low or middle-income countries. In clinical medicine and epidemiology, the public-at-large are the population for whom the research is conducted and yet they have the least access to the scientific findings. This is particularly questionable given the amount of this research that is publicly funded. Recently, high-profile scientists and institutions have joined the voices of criticism against traditional scientific publishing calling for a more open alternative. Many public granting agencies (e.g., CIHR, NIH) and private donors (e.g., Bill and Melinda Gates Foundation) are also now demanding that results of funded research be publicly accessible.

“Open-access” (OA) journals have been touted as the new alternative to conventional publishing. It stands for “unrestricted access and unrestricted reuse”. Under paid access, authors could be restricted from making copies of their own research available to colleagues or students, and could be required to get permission and pay additional fees to use parts of their work in other publications. In principle, OA eliminates such restrictions creating a freer dissemination environment with accelerated discovery, improved education, and public inclusion. Practically, there are different “shades” on the OA publishing spectrum. For example, some journals provide immediate and unrestricted access, while others use a hybrid approach where access requires a fee for a limited period of time, after which it becomes free.

Despite the potential benefits, open-access has its own shortcomings which have become apparent as OA journals have become more widespread. OA publications suffer from uncertain stability, which has been highlighted by shut-down publications. Additionally, these journals profit from accepting articles rather than providing access, meaning that charges for submitting authors are often substantial.

If we believe that the role of journals is to act as gate-keepers, then a fundamental conflict of interest is being created with respect to article quality when journals are paid based on how many articles they can publish. In the world of “publish or perish”, there has been no shortage of eager and outright predatory entrepreneurs vying to profit by publishing anything and everything that gets submitted.

While there are a number of major respected OA publishers (e.g., PLOS, BioMed Central), the number of dubious and so-called “predatory” OA publishers are on the rise. Predatory publishers operate by accepting fees to publish whatever is submitted, with little or no review, in order to make a profit. The worst predators are bogus operations that rely on spam-like tactics (including unsolicited emails and invitations), usually headquartered overseas. With predatory publishers corrupting the landscape of open-access, it is becoming increasingly difficult to identify reputable ones. Scholarly Open Access is an online resource created and maintained by Jeffrey Beall which systematically identifies and lists dubious/predatory publishers and journals (“Beall’s List“).

Other than checking Beall’s list, here are a few red flags for spotting a predatory journal:

  • unsolicited emails with spelling and grammatical errors,
  • journal/conference is completely unrelated to your field,
  • e-mail domain is generic (e.g., gmail or hotmail),
  • promises of rapid publication, reduced fees, or discounts,
  • lofty titles that mimic established journals (“International Journal of …”),
  • editor or editorial board names not provided,
  • fictitious or missing publisher address.

Take-home messages from the talk

Open-access publishing is here and there are great reasons to support it as an alternative to traditional pay-wall access. Regardless of the access model, a strong peer-review system will remain critical for ensuring the quality of scientific publications. The sustainability and credibility of publishers are potential challenges for open-access, particularly given the rise in predatory publishers. It is important that we are vigilant about identifying and avoiding predatory publishers both in order to protect ourselves and to stop incentivizing their practices.

Sources and additional reading

What should I do with my life? PhDs & PostDocs         

Drs. Nicholas King, Ashley Naimi, and Serene Joseph all gave their thoughts on deciding next life steps last week. They described their experiences as PhD students and post-docs in a session for students in the department on Thursday, January 29.  They had a slew of advice to guide choices that lots of us are contemplating.

General advice to make decisions:
-Picture yourself in 5 years walking into your dream job. What are you doing?
*solving problems
*thinking, writing
*working with people
*working in an office
-Your vision of where you want to be is the best guide to deciding your next move.

-There is no direct path to achieving your dream job. Are you:
*committed to one subject
*interested in several things
*ready to switch your area of expertise
-All are paths that take varying lengths of time to get where you want to be, but none are wrong.

-No matter what you do/don’t do, there is a sacrifice.
*Doing a PhD: Sacrifice income, savings, benefits, and settling other parts of your life
*Not doing a PhD: Sacrifice certain leading positions or promotions because you might hit a glass ceiling

On PhDs:
-Academia is not the only end goal
-Don’t feel you need to ‘know everything’ on entry: it’s not expected that PhD students know how to ask scientific questions or design methodology. Learning this is the point of a PhD.
-If you come in with a solid idea, be open minded: you might miss out on some pretty cool opportunities to learn from professors and work on existing projects
-Don’t go for the ‘hot’ topics for that reason alone: It’s tempting to study current flavours of the month, but genuine lack of interest will become apparent when you compete against people who are actually passionate about a topic for future positions
-Do other things: It’s important to get your PhD done, but enriching to do side RA projects/TAships. You get exposed to others’ work and styles.
-Publish, but not too much: This is a catch-22. If you’re too focused on publishing, you’ll miss out on the unique PhD student opportunity to read, learn and develop your own ideas.  If you don’t publish enough, it is tough to get future positions.
-Through your PhD, keep a running list of research ideas that you don’t have time to look at. A postdoc might be the perfect chance to explore these.

On Post-Docs:
-Really only necessary if academia is your goal
-Start applying at least 1 year before you plan to defend your thesis, because this leaves time to find your own funding for your ‘perfect’ post-doc if necessary.
-Never put all your eggs in one basket: even if a position seems a done deal, it can fall through.
-Your PhD supervisor can help to arrange a post-doc, but build your own connections too.
-A post-doc is a transition between a PhD and an academic position: you learn to write papers fast, respond to reviewers, and, ultimately, be a ‘true’ researcher
-There are varying degrees of flexibility: some supervisors assign you work, others let you work on your own. Make sure your offer is the right fit.
-Get a post doc at a different place than your PhD.  It could be your last chance to pick up and move somewhere for 1-2 years
-It’s hard to get a post-doc in a different subject than your PhD, but not impossible.  It’s a good time to move into another subject area if you want.