Where do research ideas come from
The Role of Anecdotes
You can't just take people at their word and call it science.
Anecdotes can still be useful, though!
We often believe that spending time and money on new research is worthwhile because of positive anecdotal experiences.
Anecdotes can be used to create hypotheses, which you can then test by doing research. Researchers can get ideas from experiences or from anecdotal stories, then use the scientific method to formalize that idea into one that can be tested.
Thinking from First Principles
First principles means distilling ideas down to what you actually know and building up from there.
Figure out what your fundamental research question is at its deepest: what do you really want to know? Then, build up from that deepest level to figure out how you can eventually triangulate around that question. Consider the limitations of what is possible to measure, then use your resources to figure out what is feasible for you to measure. You may end up with a different line of preliminary research questions that you need to answer before you can get to your deeper research question. Start out on that path, adjusting as needed and rethinking the framework periodically or when new evidence arises.
The Role of Published Literature
Thinking about how to build on previous work is relatively easy!
First, you replicate it, then you might run an additional condition or two, run it in a different population or culture, model some additional variables as possible predictors or moderators, model some potential mediators, develop an intervention to change the outcomes, and so on.
Unfortunately, fewer than half of the findings in psychology replicate!
Acting as if the existing literature would replicate no longer makes sense.
The literature is riddled with non-replicable, p-hacked, and HARKed publications that can be practically indistinguishable from replicable, controlled, legitimate research. Discerning which research was legitimate and which research wasn't is impossible by reading alone. Some replications have been attempted so those can also provide potential starting points, but running your own replication is almost always a good idea (feasibility permitting).
The first step is often running a replication of your own to see if a given finding is reliable.
Academics are still expected to read the existing literature.
Frustrating as it may be to know that published studies often reflect dubious research, your peers in the field will still expect you to learn the current state of the art defined in publications. Academics won't accept wholesale rejection of the past as an excuse not to read.
Read, but skeptically.
Don't allow yourself to become intellectually bound or limited by the existing literature. Learn to hold ideas you read as underdetermined. Refrain from imagining that any particular paper or paradigm is "True" or "Complete". Put everything in the "Maybe" pile.
Every piece of psychology research provides some bit of evidence of varied quality.
Think of research as incremental evidence, not definitively fact or fiction.
Feel your confidence in evidence as nuanced, not wholesale accept or reject.
Published evidence usually applies to a much narrower question than discussion sections imply.
For example, if a paper reports some finding in a sample of undergraduate students age 18–25 on a specific questionnaire or behavioural measure, do not automatically assume that these results will translate to all humans aged 0–90 let alone all adults aged 18–65. Do not automatically assume that the questionnaire really measures the construct of interest: obtain the full original questionnaire and read it, then make up your own mind. Approach behavioural measures similarly: do not take the author at their word that the specific task they used really extrapolates to real-world scenarios. A lot of experimenter-defined tasks are quite contrived and many have very little apparent equivalence with the world outside the lab.
When it comes time to pursue your own research, think from first principles!
If your ideas rely on the accuracy of some research, run a replication to build a foundation you can trust.
Index
Return to Psychology The State of the Field
Continue to Picking Your Hill - A Personal Story
Jump to Undergraduate Psychology