Adjuncts at colleges and universities work hard, teach the same types of classes as tenure-track faculty, but are paid very little and have almost no benefits or job security.

In a blog post using some slightly dated data, Matt Bruenig takes a contrarian position.  Some of the key points are that many adjuncts want part-time status, do not want tenure track jobs (coupled with high research expectations) and many adjuncts have high-paying primary employment.  These adjuncts mostly teach a class or two on the side.  Katina Rogers provides some good counterpoints.

My own personal experience with adjuncts has been mixed at Washington University in St. Louis.  In the Political Science Department we hire very few adjuncts.  In the past we’ve had a few adjuncts to cover classes, and most of them were similar to what Matt Bruenig described.  People who had employment elsewhere and wanted to teach a class or two.

More recently we have been offering different types of “postdocs” which are generally two-year appointments with a two or possibly three course teaching load.  These postdocs have been mostly allocated to our department to cover faculty on leave and to help cover a new environmental studies major in the department.  But we don’t see these postdocs as a long-term substitute for tenure-track positions.

Most of the tenure-track faculty see these postdocs as good PR for our department.  We do our best to integrate these postdocs into the department, let them give practice job talks just like our own graduate students, and hope they land great jobs. Many of these postdocs have gone on to get very good tenure-track jobs and I hope they would say that they benefited from their experience at WashU.  Adjuncts don’t seem to have it so bad.

Last year I was assigned to review a “shop” housed in the humanities.  I use the word “shop” because there are many programs that are either housed outside of departments or bigger than departments.  This is often the case for university-wide writing courses or the teaching of foreign languages.

I was stunned at seeing this side of the university.  There were literally dozens of poorly paid instructors living the classic caricature of the plight of adjuncts.  Seasoned educators, mostly with Ph.D.s in hand, were paid shockingly low salaries with essentially no benefits or job security.  In most cases the instructors were told a few weeks ahead of time if they would be able to teach a section or two.

These instructors lacked ownership over the classes that they taught. For these shops the curriculum and often the syllabus are designed by someone else.  While the instructors have some mechanisms of providing input, these feedback mechanisms don’t seem to take into account the enormous power asymmetry between adjuncts and heads of these programs.  Imagine a newly minted Ph.D., uncertain about his or her classes in the future, complaining to a very senior faculty member or administrator about the content of the course they designed.

I quickly realized for every two or three postdocs in political science, economics, and anthropology, there are dozens of adjuncts teaching English, Spanish, French…

This program at WashU had unique strengths and weaknesses, but I there are some systemtic problems for these shops built around adjuncts at any university.

The budget reality is that these are very big programs done on the cheap.  Obviously the University could spend more and hire more tenure-track faculty. But if the university was going to spend more, where should it go?  More faculty?  How about merit-based scholarships to low-income students?  What about further student services, ranging from tutoring to health services?  These are all worthy ways to use money.

Rather than debate these hypotheticals, let’s just assume the university isn’t going to radically change their model of spending and will just shift hiring priorities.

Each year WashU has about 12-15 total hires in the arts and sciences.  Replacing all of these shops on campus with tenure track hires would probably take a few years of exclusively hiring in these departments.

Even if we allocated every single hiring line to one or two departments for five years it isn’t certain any department could implement hiring at this scale.  We generally think that our department (around 30 faculty members) would struggle to do our due diligence in anything more than three searches a year.  In reality, I think having two searches in a year is pretty taxing.

Equally important is the goal conflict at the university.  The university allocates hiring for a number of reasons, and covering undergraduate classes is one of them.  But goals of research, service to the university, training of Ph.D.s, and public outreach are all important.

Replacing these adjuncts with tenure track faculty would require a massive reallocation of resources.  At WashU this would mean a major reallocation of faculty hiring from the social and natural sciences to a small number of departments in the humanities.  New initiatives such as those on energy or economic development, and recently discussed resurrection of our Sociology department would all be shelved.

At WashU one of our top humanities departments is our German department.  Not only are they top notch researchers, they are probably a model department in providing service to the university.  But enrollments of German courses are quite low, and thus under this reallocation they would probably receive no new hires.

These are just some off the cuff thoughts, but I think what everyone can agree on is that these are huge programs with very poorly paid adjuncts.  But rectifying this requires either new money or a major reallocation of resources.

The obvious point is that there is goal conflict within any organization.  A major challenge for the university is when the demand for teaching doesn’t map onto other priorities of the university.  Adding more money to the university doesn’t solve this goal conflict.

How does the university address this problem? I have no idea.  But I can think of at least one simple budget neutral proposal to at least address the job insecurity of adjuncts.

Given the massive demand for certain types of courses, there is no reason why adjuncts couldn’t be provided long-term contracts. While demand for courses varies from year to year, it doesn’t vary that much.  The university could probably provide 5 year contracts for 75% of the adjuncts in a writing or language problem.  Accidentally paying one extra adjunct or two in a year isn’t a major cost.

So this is a pretty small proposal for a major problem.  But most of what I read about adjuncts had no resemblance to the role of adjuncts in the social and natural sciences at my university.  I’ve just learned about the problem.  That is at least a starting point to help solve it.

Looks like International Studies Quarterly published my September 2013 article again a second time in December 2013.  This is obviously an error by the publisher and I contacted the journal.  But I might have two versions floating out there for some time.

We had beers last night to celebrate a dissertation defense (and a tenure-track job!) and chatted about what this means.  Some suggestions:

• This could be a really lame natural experiment.  How does publishing in September versus December affect publishing?
• Marketing research shows that you have to bombard consumers repeatedly to be effectively.  I hope ISQ can publish this article every couple of months.
• My citations will be split across two articles.  This could affect my H-index in Google Scholar.  That was from Debbie Downer.
• I could claim that this is a replication study.  A perfect replication study.
• Years from now I might have some university administrator look into self-plagiarism allegations.  Yes, it is the same article.  But no, I did not submit it twice to the same journal and got it published in the same year.  That would take more guts than I have.  For the record, I did not submit this same article twice.

I have another long-run issue with Google Scholar.  My most cited paper is incorrectly indexed on Google Scholar.  Instead of listing me as the sole author, they have all of the acknowledgements as the authors.  I knew I should have acknowledged myself.

When I went up for tenure I emailed Google Scholar and got a response that was something like, “it looks like we made a mistake.  But we don’t fix citations based on individual requests.  Thanks for contacting us.”

I sent another email this morning.  We’ll see what happens.

To be honest, I’m not losing any sleep over these two things too seriously.  What can I do other than inform ISQ of their mistake and send an email to Google?  Plus I have a 2.5 year old and 6 month old.  I lose sleep in other ways.

But citations do matter.  Anyone have related stories to share?

A recent post by Kim Yi Dionne on “Women Authors on African Politics Syllabi” and a response by Tom Pepinksy got me to look at my own graduate courses.  Tom did some accounting and calculated the percentage of women authors on his South Asian Politics syllabus. Only 16.7% of the authors are women.  This is better than the average African Politics course.

I did a similar calculation on my graduate international political economy (IPE) class.

I’m the only faculty currently teaching IR as WashU, and even at our peak we were three faculty.  So I build new  IPE grad classes every year that can be taken by first year students and be of interest to more advanced students.  I always include some recently published or unpublished working papers and build two weeks around them.  In the first week I cover the “classic” works that these unpublished pieces cite.  Then we read the recent papers the following week.

I’m constantly redesigning this course and I often search for other IPE grad syllabi. I look for interesting debates, other topics to cover, and new books and articles.  These details will be relevant very shortly.

IPE is a field with fantastic women scholars across all ranks.  There are plenty of women authors in IPE to choose from, and I have many of their books and papers on my syllabus.  Or so I thought.

Looking at my most recent version of this syllabus, 25 out of 88 readings (28%) had at least one woman author.  That is lower than I thought, but not awful.  At least not compared to the next number.

When I count up all of the authors on my syllabus (many of the papers have three or four authors), the number of women authors drops to about 20% (28 out of 138).

Damn.  Really?  I pulled a few more syllabi, and quickly eyeballed them. They all look roughly the same.  Some of them worse.  I’m still shocked at this.

How did this happen? 

This might be obvious by now, but here it goes.

1.     When built my syllabus I searched for other syllabi on the web and used this as part of the pool of articles I would consider for my class.  Thus my syllabus isn’t independent of other IPE syllabi and I am replicating, or maybe even amplifying the gender bias in the field.

2.     Taking recently published or working papers and looking at their citations is also problematic.  If there is gender bias in citations (there is) then it will show up in my syllabus as well.

3.     Me.  And this is the most painful part to admit.  A graduate syllabus, at least for me, is about satisficing.  There are so many interesting books and articles out there that I am often overwhelmed by the choices.  In most cases I’m look for an interesting portfolio of papers that taken together really lead to a great discussion and hopefully help our graduate students identify research questions and methodologies to address these questions.

But there are so many different ways to build a syllabus.  I can’t really say why I choose one author or the other. But the proof is in the accounting.  Somehow I made the choice to only include about 20% women authors.

What to do about it?

I had a stint as the director of graduate studies along with other administrative roles at Washington University.  Change comes slowly and often requires the support of faculty, administrators, and staff.

How do I address gender bias in my syllabus?  I can change my syllabus.  Unilaterally.

There are lots of ways to do this. I could set a percentage, say 35% women authors, and see if I could build a graduate syllabus that is just as rigorous and interesting as my present one.  I’m pretty sure I can.

Or I could do something similar to the Rooney Rule.  The Rooney Rule was a rule instituted in the NFL requiring teams hiring a head coach to interview at least one minority candidate.

What I could do is pick a topic for one week of my course and think hard about the best papers in that area.  Then make sure to look a second time at what are the best papers authored by women in that field.

I did a quick thought experiment on two weeks of readings for the course I will be teaching next semester. Just off the top of my head I thought of a bunch of papers on the topic authored by women faculty.  I’m pretty sure I can improve both of these weeks and include more works by women scholars.

No matter how I do it, I can construct a syllabus that is at least as good as my current one, and include a substantially higher number of women authors on the syllabus with just a little effort.

Rarely I have I come across a problem that is so obvious and has such an easy solution.  I guess it just required someone like Kim to provide some motivation to do a very quick self-study.

I have clearly made mistakes with my past courses.  I can do much better.  And I will start by changing my syllabus for next semester.

In a previous blog post I documented the number of manuscript reviews I have reviewed over the past few years.  The American Political Science Association also just circulated a survey about peer review, forcing me to count my reviews again.  I feel pretty exhausted by the number of papers I have to review.

Let me do so quick guess-accounting.  Over the past 12 or so years I probably averaged about 20 reviews a year for political science, economics, and management journals.  Fewer in the first few years of my career, and peaking just under 40 in some recent years.  That is about 240 reviews.

If I do one review a day and work five days a week, this is almost one year of reviewing alone.  I’m not sure if I am more depressed or proud.  Nobody can say I am shirking on my duties.

Except Jim Johnson.  In a previous blog post I talked about a paper that went to five different journals before getting submitting. In the comments secton Jim pointed out something that should have been obvious to me. If each journal had at least three reviewers then this paper alone generated 15 reviews.

Let’s do some more quick accounting.  I’ve published roughly 25 papers, have a bunch of papers still working through the process, and a few that were killed by the review process.  Let’s say 40 papers, although this number is probably a bit low.  These papers probably have averaged going through three journals.

360 reviews.  I have received at least 360 reviews over my career.

Ok, some of these papers were co-authored.  The papers they reviewed should also count towards my 240.  But some of my co-authors were grad students that shouldn’t be expect to review.  I’ve also send in more book reviews than I sent in.  Actually, no, I think I received more reviews.  I definitely received more reviews on NSF grants than I gave.

Damn.  Really?  I’m probably in the hole for around 100 reviews.

In recent years I have reviewed a lot more.  Maybe I am catching up?  The problem is that I’m also sending out more papers.  How do things look this year?

I’m on parental leave and have declined a number of reviewer invitations that aren’t in my field (often in econ journals) so my number s for this year are actually pretty low.  Thus far I have reviewed only 17 papers.

I also have made at least eight submissions to peer reviewed journals.  My deficit is at least seven reviews.  Damn.

Things look even worse if I look at the accounts of individual journals.  I review a lot of papers for journals that I rarely submit papers to.  But this year alone I think I have sent two papers to APSR.   I have not reviewed six pieces for any journals this year.  Or any year.

Over the past couple of weeks I have been chipping away at a tenure review.  Tenure reviews are a ton of work, requiring you to read all of the publish and much of the unpublished work and to write a letter with multiple audiences (departments, Deans, and tenure review committees).

Both of my boys (4 months and 2 years) aren’t sleeping very well during the night and then I am on full time day duty with the 4 month old this fall.  That means every time the little guy takes a nap, I quickly flip open my computer.  I answer work emails and read parts of this tenure case.  (I’m blogging again because I finally finished this tenure letter).

My wife often tells me I shouldn’t do this stuff while on leave.  I partially agree, but I know if I turn down the review a tenure review committee might see this turn down as an excuse so I don’t have to write a bad letter.  There are some things that are hard to avoid, even while on parental leave.  I will be a good citizen and do my part.

Well, it turns out that this is my seventh tenure letter I have written.  I think this is exactly the same number of tenure letters WashU received for my promotion case to associate professor a few years ago.  I am even.  Except that I will be starting at George Washington University next year, and they are running my tenure case.  I think I am now 6-8 letters in the hole.   God damn.

Rather than trying to pull off some policy suggestion on how to reward service, limit submissions to journals, or more fairly share the burden of reviews across faculty, let me keep this personal.  This calendar year I have sent two papers to the American Political Science Review.  I received a total of seven very high quality reviews plus thoughtful comments from the editors. Seven scholars (and folks at the APSR) spent a lot of time reading and comments on my papers.  They might also have families to spend time with, graduate and undergraduate students to mentor, plus their own teaching and research.  For junior faculty, they are providing these reviews knowing that their fate (tenure) will have nothing to do with the hard work they are putting in to review my paper.

I didn’t like that both of these papers were rejected, but the reviewers did their job and my papers will be better because of it.  I don’t know who you are, but you deserve my thanks.  Thank you.

A few weeks ago I shared a story of the paper that was published after over five years in the pipeline.  This wasn’t my best or worst publication experience.  I told this story because it was being published online the day of my post.

I was shocked at the number of people who viewed the post.  Google Analytics (with no IP tracking) has it at almost 8,000 views.  I’ve heard from many faculty members that my experience isn’t uncommon.  The fact that so many people viewed my post probably means that most of us just don’t talk about these failures.

I have another story.  This one might be a little more unique.  Almost every faculty member I know has quit on at least one research project.  Often this is when the “market” (reviewers) send a clear signal that the paper won’t be published.  Anywhere.

I had one paper that was constantly caught between American, Comparative, and International Relations reviewers.  They all though it was ok.  None of them wanted to publish it.  This is a warning about sitting at the intersection of a couple of fields.  Your paper will sit.  Maybe forever.

But the story I am going to tell is different. It is also about quitting a research project after many years and many, many hours of work.  But it is also about research ethics. I don’t know if I made the ethical choice.

My previous blog post centered around a paper on politics and corporate taxes that was published at International Studies Quarterly.  This tax paper used confidential data from the US Bureau of Economic Analysis (BEA).  As I documented in my another post, the start-up costs are high.  And so are the marginal costs.  I had to get a security clearance, learn the data, merge and clean the data, and then run every single regression in Washington D.C.  But I finally published a paper from this data.

The original plan wasn’t to get a single paper out of this high cost project.  I had another piece that I wrote using this data.   I presented this paper at multiple conferences, used the preliminary results to submit an unsuccessful NSF grant, and (rejected) journal submissions.

Before I introduce my dilemma, here is some very quick background.  This paper was started when I was a junior faculty member at Washington University.  My department conducts annual reviews for junior faculty members.  Junior faculty submit their vita, working papers, and even some questions for the senior faculty members every year.

I was lucky enough to get out of the gates quickly and have a couple of papers that were published in grad school and in my first few years as an assistant professor.  I worked on turning my dissertation into a book while also thinking about follow-up projects ranging from economic voting, trade policy, to civil war.  This eclectic mix of topics sounds pretty silly right now, but as a second year faculty member I was simply following my interests.  Good thing we had an annual review.

The senior faculty gave me some very good advice.  First, when I go up for tenure, there should be a set of reviewers that know most of my work. Second, I should do my best to be known for something.  Not just a paper or an idea, but a body of work.

I decided to focus most of my work, but not all of it, on the study of politics and foreign direct investment.  My paper that was eventually was published at International Studies Quarterly was a new project on firms and taxation.

But I also had a very clear idea for an extension of my dissertation work.  My dissertation focused on how political institutions shape the risk environment for firms and the BEA has fantastic firm level data that I would use to test some of the theories from my dissertation in a much more vigorous way.

This post is already too long, so let me skip to the good part.  After years of working with the data at the BEA, presenting at conferences and ultimately going through the review process with a rejection or two, I quit the project.

In my paper I found that firms around the world behaved substantially differently under democratic regimes than non-democratic regimes.  Forget about the fact that I was looking at firms that had invested in a country and that there is a serious selection problem.  I told you to forget about that part.  Where you Reviewer #2?

My reviewers highlighted the obvious research design problems, but were willing to overlook some of these issues based on the novelty of the project. They also seemed to recognize the complexity of trying to model the selection decision of firms, given that I had 20,000 firms investing in 200 different countries, and I had to do all of my analysis on location in DC.  The reviewers were much, much more forgiving of the limitation of what I could do.  Except Reviewer #2.

My friends would ask me about this paper.  Some of them would call it the “underwear paper”.  The BEA requires researchers to conduct all of the analysis in a windowless, secure room and I joked during a conference presentation that I couldn’t bring anything into the room and had to run my regressions in my underwear.  Sorry for that mental picture.

A few commentators at conferences mentioned that I focused too much on the statistical significance of my results and this could just be an artifact of the sample size.   A methodologist colleague thought I shouldn’t have standard errors at all.  This is the population.  In all, the substantive significance of my results was less impressive than the stars in my regression tables.

Every five years the BEA collects a firm census and new data became available in the middle of this project.  This new data gave me the opportunity to test my results with another year of data.  It wasn’t really necessary, since most of the reviewers weren’t asking for more data.  But I had access to the data and I eventually wanted to use more years, so why not test the robustness of my results?  I took another flight to DC, spent a few days preparing the data and I finally ran my models.

My findings weren’t robust. Forget about the technical details.  I no longer could in good faith say that I believed my original findings.

I told myself that a less ethical junior faculty member would forget about the new data and just kept pushing the old paper.   But, I decided that I could no longer send out this paper for review.  In reality, I really don’t think my colleagues would try to publish something that they know is clearly wrong.   Most people would have done the same in my position.

Or maybe not.  Over the years I have been paying more and more attention to the studies of publication bias.  It’s often easy to imagine some immoral researcher running regressions until enough stars pop up.  Did I do the opposite?  Did I drop a project because it didn’t give a clean result I could push into a journal?

I honestly haven’t though about this paper too much over the years.  I moved on.  But what I can say is that publishing is hard.  Damn hard.  Even the best paper that gets accepted at the first journal is a slog.  Reviews come back and you need a combination of a thick skin, belief that your project is important, and some optimism that this paper will make it through the capricious review process.

I’ve developed a pretty thick skin over the years, and I try to pick projects that matter.  At least to me.  It is the optimism that is the hard part.  For every paper you have to finish up the last 5% and drag the work across the finish line.  It is the least fun part of the process and makes me so sick of a paper that I often don’t read it again after it is published.  But this last part is essential, and probably where many people get hung up.

With a problematic research design and now contrasting results, I couldn’t justify the financial cost of going to DC a few more times and to do conduct days of non-stop data analysis.  I now have two little kids and let my BEA access expire.  The travel time is too much for my family.

I knew that this would be a really tough paper to ever get published, and I decided that I would focus more on new projects.  Was I going to spend a few more thousand dollars and countless hours running regressions in my underwear in the hope of sorting this out?  Sorry again.

My story is one of staying true to only publishing what you are confident in, while at the same time contributing to publication bias in the discipline.  Actually, this paper was really never going to make it any way.  My counterfactual reviewers are to blame.  Are my hands clean?

Forget about ethics.  Let’s go back to my previous post.  I have one dropped project and one project that took over five years to get through the publication process.

The longer you are on the profession the more a second truth becomes self-evident. Publishing is hard.  Damn hard.  It is easy to look at a person’s CV and be intimidated by their success.  But don’t think that this comes easy for anyone.

I am an advocate of judging the written work for junior hires and the published work at tenure time.  I probably weigh letters, reputation, or even the job talk less than many others.  I say we should read the working papers and look at the published work.  But this doesn’t mean that we have to use labels like “deadwood associates” or mock junior faculty that don’t get a paper published in their first few years on the job.  Don’t assume failure is caused by a lack of effort.

Also, admitting how hard this process is shouldn’t demoralize graduate students just entering the profession.  The best published scholars in the discipline send out lots of papers, deal with rejection, revise, get rejected, revise.  They rethink projects, publication strategies, and go through some soul searching in the process.  You can learn from them.  Ask your advisor about this.  It is hard for everyone.  But it can be done.

I have been overwhelmed by the number of responses to my post on my experience in the review process.  Here are links to some related blog posts.  Drop me an email or find me on twitter (@natemjensen) if you have an experience you want to share.

Kim Yi Dionne with her take on the process and a link to an older post on a paper that took a few years.

Tom Pepinsky’s response to my post

Kean Birch worked out his own rejection CV

There are a few really insightful comments on the post itself.  See Adam Berinsky’s note on his highly cited paper.

One quick note.  I worry that graduate students reading these posts will be very disheartened with our discipline.  Yes, the publication process is capricious and it really, really, really matters.  But the review process isn’t everything.  Ok, it is about 90% pre-tenure.

A few years ago I was asked by a chair of another department to provide some service to the discipline.  The details aren’t important.  I said yes.  He said something like, “I am always shocked by the generosity of people in IPE.  We have a field with some really, really nice people.”  I think this crosses fields in political science, but I’m not sure it crosses disciplines in academia.  Don’t let the nastiness of the blogs and the capacious review process fool you into thinking our profession is filled with nasty and capricious people.

I am on leave with my three-month old son.  My only visits to the office to check my mail or gloat about a Packer win.  I am on a short-timer at WashU, taking a position at GWU.  Are my visits to the office awkward?

A few weeks ago I popped in to say hi to my department chair.  We had a fun chat, he held my son Stanley for a while, and he literally brushed it off when Stanley spit up on him. This is how my department and discipline has treated me.  There are problems, but after “quitting” my “boss” was the same nice person he was to me a year ago.

This may sounds like a rosey take from a tenured professor.  Probably, but let me mention the review process one more time.

Tom Pepinsky mentions some criticisms of the review process and my former colleague Lester Spence seems disappointed in me that I didn’t engage the structural problems in political science.  I will disappoint again, but I can provide a hint of optimism.  We have a discipline that is full of smart, hard working, and ultimately good people that care about their colleagues and students.  The institution of the peer review process is the problem, not the people behind it.  But institutions do matter.  A lot.

I’m shooting off another quick blog post while my two boys nap.

I’m a little surprised by the amount of positive feedback I have received on my blog post about the multi-year process of getting my paper published at International Studies Quarterly.  Lots of people on twitter have been very supportive.  Many scholars have very similar stories.

Nothing fancy here.  I just documented my experience with a single paper.  It wasn’t my worst experience, but far from my best.  I hated the outcome (years of waiting for publication) but I felt that the process was fair.  Editors and reviewers gave me a fair shake at every turn.  i eventually got the paper published in a very good journal.

I made the mistake of checking out political science rumors to see if there was any talk of my post.

Let me be honest.  I do read the rumor blog (I’ve mention this on my blog previously), but I am pretty embarrassed to admit it.  It is like tabloid filled with very little news and some really lame attempts at sensationalism.  I stopped taking it seriously when I was on a search at WashU many years ago and most of the facts posted about the search were wrong.  I started hating it when anonymous posters started attacking faculty and graduate students for their publication records and discussing their personal lives.  Believe it or not, I do not post there.

I’m always a little disappointed that my posts on research get a lot less attention than posts on the job market, job talks, or publication process.  But this is probably a function of both having a lot wider audience and having less completion.  There are tons of great posts out their on interesting research projects, but not a lot of posts on the discipline.

I’m probably not the best person to be providing insights on our discipline.  I have mostly fumbled through my career, doing my best to do good research, be a good teacher, and provding service to my university and the discipline.  I don’t have especially strong opinions on most aspects of our profession, but I do have years of experience on search committees, APSA and MPSA section organizers, author and reviewer.  So I might as well embrace my role of at least providing some details on my own experience.

So I will take a quick stab at filling in a few more details from my previous post.  I had a paper that I started many, many years ago that was just published in ISQ yesterday.  Some more facts about the paper.

• I am a tenured faculty member now, but I started this project midway on my tenure clock.  I thought that I would have three papers using this data and a grant prior to tenure.  This is my only paper that came out of that project.
• Someone on the rumor blog asked about how much the paper changed over time.  To be honest, I don’t really remember.  I’ve published around 25 papers, each of them going to an average of 3 journals.  I’m sure I have at least 7-8 papers that died in the review process and quite a few in the pipeline.  I’d say I’ve made 100+ submissions in a little over 10 years.  I don’t really keep good records on the previous submissions, so I don’t know the exact changes.  What I do know is that more core result was the same throughout the process, but my framing, literature review, and robustness tests changed substantially over time.  This paper definitely got better through the review process.
• I had a sense that this was a high risk project, but I didn’t think too much about it.  I definitely wouldn’t have guessed that I would get one paper out of this project after all of these years.  (Chris Blattman has anice post on risky field experiments that is relevant to this point.)
• This study uses confidential data.  I archieved that data at the Bureau of Economic Analysis and have full replication materials there. But only a handful of people have access to this.  Many journal and NSF reviewers were very hostile to using confidential data.  This was probably the most unique part of my experience.  I have mixed feelings on the use of confidential data.
• As far as I can remember, the core results didn’t change much and I was always using the same data.  But what was stunning to me is how much reviewer tone changed based on my framing of the paper.  A friend of mine says that most reviewers make up their minds in the first five pages of a manuscript.  I don’t know if that is true in general, but it certainly felt like it with this paper.
• I made a lot of mistakes with this project.  I didn’t have a very clear testable theory, and made the classic mistake of presenting regression after regression which actually muddled my contribution.  This was far from a perfect paper when it started the review process so I should be clear that much of the blame for this delay is on my shoulders.

This is what I can remember from this project.  I have a companion paper that I actually scrapped for a number of reasons.  When to quit a project might make a good blog post as well.

I’m not sure of what to make about the lack of blog posts about personal experiences in the profession.  There certainly are scholars willing to post their exeriences, but there seems to be a real undersupply of factual information.  It could be that many of us are embarrssed by these experiences.  It could also be that most of the attention that comes from these sorts of posts isn’t good.  Being famous for a blog post is probably one of the worst professional outcomes.

My kid nap window is closing and I have to get back to a bunch of work emails.  Here is a quick set of links on my previous posts of our discipline.

Brands and Fads in Academia

The Stability of  US News Graduate Student Rankings

My Experience with Big(ish) Data

Posts on academic job talks here, here and here

Failed Research Log

Reviewing Articles, Lots of Articles

The Academic Job Market here, here, here, here and here

NOTE ON COMMENTS: