Join hosts Richard Landers and Tara Behrend as they explore Chapter 4 of their research methods textbook with Dr. Brad Brummel, professor and director of the industrial-organizational psychology program at University of Houston. Brad shares his unconventional journey through diverse research populations, from engineers to restaurant workers, revealing how thoughtful sampling decisions can transform workplace psychology research. The conversation covers practical strategies for accessing difficult populations, the importance of respecting research participants, and why your sample matters more than your statistics.
Key Takeaways:
- Sample selection is as important as statistical analysis in research design
- Building relationships and showing genuine curiosity opens doors to unique populations
- “Me search” can be valuable when it provides unique insights rather than personal validation
- Online panels require careful consideration of participant respect and appropriate compensation
- Hard-to-reach populations like job applicants and harassment perpetrators need more research attention
- Real workplace experience enhances understanding of organizational behavior
- Interdisciplinary collaboration can lead to innovative funding opportunities
- Limitations should be reframed as “threats to generalizability” for clearer communication
Website: https://thegig.online/
Follow us on LinkedIn: https://www.linkedin.com/company/great-io/
Join our Discord here: https://discord.gg/WTzmBqvpyt
JoinThe GIG Email List: https://docs.google.com/forms/d/e/1FAIpQLSfVQ4hyF8MA4G9W-ERwVL8_e91a-MUMuhNvxhXmgkSFUDFatg/viewform?embedded=true%22
Transcript
[Richard Landers] (0:00 – 0:31)
Welcome everyone to great I’ll get together at number 30. My name is Richard. This is my co-host Tara Today, we’re exploring chapter 4 of our textbook research methods for iocyte and this chapter is all about sample So to help us understand how to make good sampling decisions on the show today We have dr.
Brad Brummel professor of psychology and director of the IO Psych program. Thank You Richard good to see you and Tara again Happy to be here. Well, so just to start us out Can you just maybe tell us a little bit about yourself? Why you might be a sampling genius?
[Brad Brummel] (0:32 – 1:16)
I don’t know about sampling genius, but I I have you know What my friends have called like among the most eclectic CVS they’ve seen so I’ve done work with engineers and journalists and Early in my career. We were making some of the first websites to get people to fill things out You know my dissertation Study had 10,000 people on it because it was attached to a choose-your-own-adventure dating study that my colleague had made and people were excited to fill it out to sort of in the heydays of the 2007 internet so I have chased many different samples and So I know something about trying to get hard samples, I guess
[Richard Landers] (1:16 – 1:23)
Sure, so like you see you mentioned engineers, maybe we can we can start there what what interests you about engineers? Why study them?
[Brad Brummel] (1:23 – 3:10)
Well, I mean, there’s a lot of Simple reasons and then bigger ones. I come from a family of engineers. So two of my sisters are civil engineers My dad is an agricultural engineer.
So he designs houses for turkeys and cows and blueberries But probably the shorter answer there is that the NSF cares about the development of engineers More than they care about the development of other people and so I Have had opportunities to do some cool projects with engineers because there was funding The other thing I think that influenced it was at the University of Tulsa. It was a very small school And so it was really necessary and easy to work across colleges and so I had a number of engineers who started with me and they had ideas about projects that they could work on and so They’re like, hey, let’s do this project with Ethics or with soft skills training and they seemed fun. So that really drew me to that area As well.
So does studying engineers help you understand your family? Maybe at least at least when they ask questions I can say like, oh I have I know something about that I guess You know You can tell I can tell some stories and then actually a lot of the Sort of leadership development coaching work that I’ve done have been with engineer types So, you know so the challenge of taking someone who’s a strong individual contributor in a technical role and then having them move to a manager or leader position Requires sort of a big mindset shift and so that experience with engineers and like sort of the deep respect for what they do Helps when we start to say now you have to do work quite differently It’s gonna be a lot harder to count how effective you are
[Richard Landers] (3:10 – 3:30)
So so this raises kind of an interesting question. So I you know a lot of IO I think focuses on this like developing general Theories of the worker or theories of the manager like how much do you customize what you focus on based on these kind of like? Population issues like how how unique are engineers really in this problem?
[Brad Brummel] (3:31 – 4:40)
Oh, it’s interesting. I think There are cultures where The way in which you perform or get status are quite different at the individual performer level And then when you move up you have to do different developmental trajectories So I’ve worked with an organization that was almost all women doing social services work and then an organization that was almost all men you know making heaters and designing cell towers and Sort of the things that you need to change in terms of your style when you move up to leadership are quite different so stereotypically the engineers actually have to listen to people’s stories and not just tell them what to do and conversely the social services people have to become a little bit more task oriented and realize that sort of the Friendships that they have aren’t the way to always just get work done.
And so a lot of the times it is a Modulation sort of back to what’s ever not easy Because you’ll tend to do the thing that you’re sort of That comes most naturally and you have to do both sides when it comes to leading both the task and the person side One thing I like about the way that you’re framing this Brad is that I think we have a lot of people who?
[Tara Behrend] (4:41 – 5:30)
study identities like they study gender or race at work, but that’s the research question and What you’re saying is that? Whatever the research question is you need to be thinking about these dynamics, but it’s not the research question itself It’s not like how does gender work in the workplace, but rather understanding the fact that this is a predominantly This is an organization predominantly made up of women changes the way that you ask your research questions from there The reason I think that’s really interesting is that it’s come up before when we’ve been talking about the book the topic of me search and How to balance like doing research is just about yourself and like your own experiences in the world, which is you know Not necessarily you don’t know everything And in balancing that with using what you know using your first-hand experience to ask richer and more interesting research questions about the world
[Brad Brummel] (5:30 – 6:35)
Yeah And I think one of the ways I think about that is you know You if you’re studying why you tick the way you pick right?
Maybe that should be better done with your coach or therapist or friends But if you’re saying I have unique insight into an area or access because of the life I lived then you’re likely to ask deeper more interesting questions And so I think it is more for me this me search that there’s problematic when you’re trying to justify you It’s somebody who matters who’s right who has status or figure yourself out? But the experiences you have that give you unique insights into the nuance of a position that give you deep respect for what people are doing Those are advantages And so I think that’s one of the lines where people run into and even even in the the class Discussion about that part of the book people are like well, like am I not supposed to study anything?
Am I supposed to be totally objective like separated we could swap each other out. I’m like no That’s not the point right because if you’re not curious about something, I don’t think you’ll do good research either So I kind of say like follow your curiosity, but don’t just like turn the entire world into a projection
[Richard Landers] (6:36 – 6:53)
So you’ve taken us a kind of an interesting journey here I will admit to have been a little nervous and when you just started with well NSF gives me money for this So that’s what I do But it sounds like it’s a much more complex problem for you And that’s more that’s a that’s a pretty big simplification over what you how you really approach it
[Brad Brummel] (6:53 – 8:06)
Yeah, well the difference that I had coming out of graduate school is my engineer friends when they hear a question They say we could get money for that. And so my questions didn’t start with you know Let’s go get money.
They started with so we had a engineer who did car crash reconstruction a Lawyer who did expert evidence testimony class and then I came in talking about role plays and they were like I bet it would be more meaningful to teach engineering ethics if they reconstructed a car crash and then took that Reconstruction and applied to work with an unethical lawyer who was trying to get them to change the results and then we’re like Oh, that’s gonna be sweet. And then the NSF agreed and so it wasn’t chasing money It was like oh we have an idea that we think actually would be appealing And so in the cases that I’ve done it it wasn’t about the money It was that oh the question seemed like it related and then sort of the engineers that I worked with who really have more of That payoff structure.
We’re like I bet that’s just something the NSF would care about So, I think it’s yeah, it’s not it wasn’t really money first for me ever probably I’m not even strategic enough on that front But it has enabled us to do some interesting things.
[Tara Behrend] (8:06 – 8:22)
So well, I mean it seems like in many cases what you can get money for is a shortcut or a proxy for what’s in the national interest or what’s like a Priority to someone other than you which is it’s helpful I think to start thinking about like is this question important like important to who?
[Brad Brummel] (8:23 – 9:12)
Yeah, I know and I think in that case It’s really it’s really been the case where a lot of the work that I’ve done in the engineering space You know some of it’s been grant-funded Almost none of its had any chance of being published in an a tier io journal because it was about interventions It was about developing educational things. We might have 30 people go through it in a year hugely labor-intensive it wouldn’t be strategic work from a Academic career perspective, but we thought the questions were interesting enough to build on so I think that’s another piece to where Almost no one’s going to give you money to do the work that’s in your head But they will have opportunities if you can align with a question that’s of interest To the night to the nation to a specific group of people
[Richard Landers] (9:12 – 9:24)
So it’s kind of a little bit to the side of sampling But it seems like this has led to a successful academic career I mean, you’ve made it right like this is the job everybody wants.
So yeah, I mean they’re
[Brad Brummel] (9:25 – 10:28)
They’re stuck with me Well, so this is a different thing and it might be a little bit off the side But when I went to the University of Tulsa, so I was at University of Tulsa for 15 years and they kind of said you don’t Have to do anything, but you have to do something and from the very beginning They just said we’re cool with you doing interdisciplinary research You know, you can work with the clinical people that we had there or the engineers or whoever But you need to be doing something So the beauty of an r2 school at the time was that they were like you need to be doing something But you can make a case that what you’re doing is so important and interesting And so it really was much less stressful I think to start at a school that didn’t quite have the expectations of some bigger ones and I’ve kind of found myself at a place.
That’s a little bit Um, I mean, it’s r1 and how it is, but I think some of the same ideas still apply You know, the the University of Houston does appreciate interdisciplinary research and we work pretty collaboratively Which is again one of the reasons why I took the job
[Richard Landers] (10:28 – 10:51)
To get back to sampling a little bit, you know, we talked a lot about engineers That I don’t know if that’s a uniquely available Population to you versus like other grad like grad students thinking about their own set of sampling problems Are there populations that you would point? People toward like that you think are understudied and in IO or even more broadly than that kind of interdisciplinary like space what’s important?
[Brad Brummel] (10:51 – 13:24)
Yeah, I mean, I think they’re currently mice my graduates my former graduate student I are really interested again in job applicants We miss job applicants because they’re in the applicant pool and then they leave at some point or another and they’re really hard to get Because why would they keep working with they’re talking about what happened after they were removed for the pool if they even heard it and and so I think a big issue we have is that we’ve outsourced almost all of the attraction part of Applications to these internet HR systems that don’t treat people with any respect or even like people Until they’ve been in the system for maybe six months and then people are upset when someone goes to them on an interview After they gave them no respect on their way into the program Take all these tests and we’ll decide if you’re good enough for us to even talk to you And so we don’t know much about what that’s doing to people Other than some stories about how depressing it is to be in the market getting nothing back Throwing things out into the nether and so, you know, that’s an area where I think that we’re missing out on Understanding exactly what all this technology is doing to the relationship between someone and an employer And I really believe that a lot of people are leaving with some possibly high level of disdain for employers Without ever without ever acting or actually interacting with anyone who works there because they forced you to go through this gateway.
That’s so impersonal Without any knowledge of what’s happening in the process. And so for me, that’s that’s a current strong fascination Like what are we doing to people there? And then my second biggest sort of sample of people that I’ve always wanted to get a hold of and this comes all the way back from my First years in graduate school are the actual sexual harassers So we study the victims, but we don’t get to study the people who are actually the perpetrators very often It’s like what were you thinking?
Right, like what led you to think that behaving this way was in any way gonna lead to I don’t know what outcome Right, so we just tend to assume that they’re like Jerks or something, but there’s got to be a story in people’s heads there And so that’s another hard-to-reach sample that I think is Important to understand as we try to solve some of these things for a more equal workforce so anyway job applicants who then disappear into the next thing and you know The people who are sexually harassing people are my two sort of white whales of sample Yeah, those are those are pretty different
[Richard Landers] (13:26 – 13:39)
As a kind of a Thought experiment then if you know a grad student came to you and wanted to study one of those groups I will let you pick which what would you what would you recommend that they do?
Like how would you reach those populations realistically?
[Brad Brummel] (13:40 – 15:30)
The way I’ve done it is I’ve just started to talking to people with access in these spaces So when we want to do something we go to the populations and start having discussions So I know that we wrote a paper on police officer selection and we just started talking to police departments and two of 15 said yeah, we can give you some data in a certain way. So Someone contacted me actually earlier this year who I won’t email back to you about doing a little more testing as part of job boards So if you have somebody who runs a job board then all of a sudden they have the stories of people who are going through these interviews and in these spaces and You can say hey like can we talk what are you doing? Are there avenues in and what I really say is if you’re trying to be an expert, you know Maybe you get the reputation for being an expert if you publish on that analysis or get something done But a lot of the things we do won’t pay off in terms of publications, but each thing you do develops expertise so you go where the samples are and you just start asking people questions because you’re curious and Maybe what you end up doing will look a little different but that curiosity actually builds expertise that then leads to different questions And so I really try to push my students to do a lot of interviews Place themselves in the area and that sort of gives them whether they do a multi method study or they do a qualitative piece talk to a lot of people or on the front lines of whatever you’re doing and sometimes that leads to a sample that you would have never gotten if they didn’t if you didn’t show up or if you showed up and I would like you to give me a sample which again isn’t really how social exchange works, right?
That’s you just trying to get something for free. So yeah, most of the samples I’ve gotten access to have come from just talking to people and sort of seeing you know, what?
[Tara Behrend] (15:31 – 15:44)
Opportunities show up what aligns for both parties Do you think that our field would look different if more people spent more time just talking to workers and like spending time Watching workers at work.
[Brad Brummel] (15:44 – 17:11)
Oh Absolutely, like I think you know, one of the best things that happened to me was actually having sort of a few real jobs Actually, I have a really, you know big sort of origin story where I was fired for my first job for in subordination And that job was at Subway. So They had a big week they got a bonus back The owner put a sign on the wall, but didn’t give the employees anything And so on the whiteboard in the back. I told the owner what he could do with that award Which got me fired, but the next time they had a bonus for a good week Everyone got a bonus.
So like I have both positive change and I’ve kind of been insubordinate and cared about workers for the rest of my life, so But but I think those things happen, you know, I think you know, some of our ideas are kind of They don’t they’re not as deep in resonance right when you think about counterproductive work behaviors It’s like, you know stealing or time theft or kind of these white-collar things But like, you know, they don’t talk about, you know doing construction and having someone drop a dead cat on your head like while you’re working because he was petting it and then realized it Had been dead for a while and he was like, what do I do with this thing? I have to share the joy, right?
And so like those are the things that happen in real workplaces and if all we do is stare at data You know, we’re not really unpacking some of those things Is this is that a story I should know of a dead cat?
[Richard Landers] (17:12 – 17:12)
No
[Brad Brummel] (17:12 – 17:20)
It was a personal story that happened to me when I was doing construction, so that was a personal story, okay, okay
[Richard Landers] (17:23 – 17:49)
Yeah So maybe maybe turning back to the the kind of practical advice side of it, yeah, so You so you teach methods Thank you for your service to our book but what I would Ask next though is in your experience in that methods course with or without the book What what is it that you think students or have noticed students struggle with most?
[Brad Brummel] (17:50 – 19:58)
Yeah, how do you approach this? Yeah, I think they struggle the most with trying to do methods, right? You know They’re like I’m gonna do this right method and you know the point which I think you did a really nice job of with the Is this point we try to make where like you can’t do it right or perfectly but you can definitely do it wrong and Each of those choices is a decision that you need to make and I was kind of I don’t know teasing them a little bit But I’m like every single one of these decisions is absolutely fair game For your committee to ask you about on your thesis or dissertation and the answer isn’t just someone said I should do it you know and And so that piece is really tough to let them know like that each choice has implications But it’s not necessarily right or wrong and you know I think your book was a really nice invitation to think that way and one of my I don’t know dreams for the field or one of the things I’ve really been encouraging people to do is like change the name of their limitation section from limitations to limitations to generalizability So you say hey I wrote this paper here’s some things I wouldn’t say from what I did So you’re actively saying as someone who went through it. I don’t think you can say this I don’t think you can say this and here’s some things Maybe you can say and that’s a lot less defensive and it’s a lot more sort of realistic when it comes to what you’re doing So so I think that piece all ties together.
It’s just a number of choices. You’ve made hopefully informed choices and Hopefully if you didn’t know that much about it yet something you just asked some other experts about So I really try to get them to talk to people about the pros and cons of every choice And I think in some of the other methods books. They like don’t give you any invitation to ask those questions With curiosity naively, you know, why do that?
And so I think that’s really the struggle the worry about doing it perfectly Removes this curiosity about why this or why that?
[Richard Landers] (19:58 – 20:48)
Yeah, I I really like that that framing I especially like the the sort of threats the generalizability because I often feel that in certain contexts I In study design, I like will purposefully Sacrifice something in exchange for something else, right? And then you always fight this struggle when writing a paper of how do I? say this in a way that a reviewer appreciates that and Doesn’t immediately discount the study because they have some rule encoded in their head for how things work Actually kind of along those themes How does how does this?
Subtlety translate into your work perhaps as a reviewer and how you understand the research you you read like do do you? expect Very tightly well-defined samples. What kind of write-ups are you looking for?
How do you approach that kind of work?
[Brad Brummel] (20:49 – 24:07)
Yeah, I mean, I’m probably not a very common reviewer So I’ve been on a path for a while where I was spending half my review on how the practical implications weren’t relevant from what? They said so people will give good advice in their practical implications that have nothing to do with the study They did right there this mediated moderator huge model and they’re like I think bosses should be nice to their employees and higher conscientious people and I’m like that’s cool But like it had nothing to do with the research you did we either knew it before or generic advice But I do find myself really curious about how people got samples so one thing that really turns me off as a reviewer if people like pretend that they collected the sample to answer the research question when It’s really clear that they asked the question because they had the sample and so it’s like no No, you didn’t do that.
And sometimes I’ll write it that way when they’re using like, you know historical nationwide sort of sample It’s like well, we know what the sample is You didn’t collect this sample for this and it even says in here. So why are you kind of writing it as if? And so that’s a piece that gets to me and and then the last paper I reviewed they were trying to make conclusions about the way Leaders in companies make decisions about pay right?
So they’re like, how do we do pay equality? What is the system like and then all of their studies were? Decision-making out allocations like judgment decision-making done by mturk workers in some cases getting paid 20 cents And I’m like, I don’t think these decisions have anything to do with what people are actually doing in the real world Like this is completely inappropriate for you thinking that you can generalize in this way I think their studies showed something about people’s gender biases and the decisions they would make but you know They tried to frame it within these very structured, you know, very thoughtful ways of doing real money allocations And then to do it they gave people like, you know, no money so so that shows up and I think another thing that I’ve I’ve always cared about but I’ve come to care about even more is we call people in our studies participants for a reason and Because we’re supposed to respect them as humans and I really think that the online panels a lot of these approaches don’t show respect for the people that we’re working with and We all kind of went to graduate school at the same time. I used to have to debrief people after I made them do something And it’s all gone out the window and we’re paying people in these online Samples and I think there’s a real big loss there where you don’t think about telling the people what they’ve just been through So, I don’t know I was really proud of a recent study We did we had this whole debrief where we asked people about how they what they experienced going through our study if they liked it You know We just asked them all these questions about what it was like to go through and we got some like really nice comments Just about like oh, this was actually interesting. Oh, like I actually felt like I mattered unlike these other studies I get paid to do and so if you you know, I you know, I was I think that’s a big piece of it Right. So if we lose this idea that these are participants in our research Then a lot of downstream questions are answered In different ways, so it’s more expensive.
[Richard Landers] (24:07 – 24:26)
But if you actually respect the people on the other end of whatever You sample differently well So would you say that there is a a right way to go about? Mechanical Turk type research then well, what would be your recommendations for doing that kind of online panel stuff?
[Brad Brummel] (24:26 – 26:11)
I mean the first thing I would say is you need to make sure that’s a legitimate way to get data for your research question I mean, that’s number one number two. You have to decide what you think about pay, right? So if you think about them as workers, what’s the least you’re able to pay them to have them do something like that?
Like what would you do it for right? So you’re not taking advantage of people. You’re really respecting their time And then I think you know, I really think we should really clue them in on why they’re doing what they’re doing, right?
You know, you’re not just doing this for three dollars or five dollars or thirty cents or whatever You’ve got away with you know, this is the kind of study you were involved in and here’s why you know Even to say hey if you want to get the study once it’s done I’ll email it to you like these other ways of actually doing a little bit of the extra work to let people know that they were participants And so I’ve done I’ve never used mturk for a study I’ve used some of the other panels And there are ways to use them You know more or less thoughtfully I have to give I have to give my former student Seth Osborne a lot of credit because we were doing some research on some Applicant reactions type work and we wanted a demographically diverse sample And so one of the things that he figured out how to do in prolific actually is to not take one study and make it like four And to get a diverse sample So in one sub sample, he said we won’t take any white people in one sub sample He said we won’t take any heterosexual people and this actually allowed us to have a full sample that had Way more diversity than you have in a population by having certain samples that you selected out the majority group And so, you know, I think it’s you know, it depends on if you think everyone’s the same So it’s really research question dependent.
[Richard Landers] (26:11 – 26:23)
I would just throw in there prolific is amazing now because you can Select any demographic you want to target and then set quota percentages associated with them to target and sub target any specific population You want in a way?
[Brad Brummel] (26:23 – 26:44)
That’s not really possible with like an mturk, but they are people who are still making money on a survey platform Often well at work. And so again, you know that is a very specific kind of person Though maybe not of the traditional ways. We divide people up.
So it just has you unique affordances and limitations for sure Mm-hmm.
[Richard Landers] (26:44 – 26:59)
Absolutely Let’s turn to what is your maybe favorite example from your own work in terms of Especially kind of sampling and design issues What are what are you most proud of?
[Brad Brummel] (27:00 – 29:35)
Yeah, I think I’m most proud of a study. It’s my most cited paper and One of the favorite things I had happened about it It was once used by a graduate student at another university as an example of bad research Which was like super exciting because what happened is we wrote it so clearly So all the ways in which we violated the norms of what you would expect in it are just right in there We didn’t have a big enough sample for multi-level modeling. So we just within group normed performance And so we did all these things that like this first-year student was like that’s wrong.
That’s wrong. That’s wrong And I’m like man, you should have called me up for that seminar. I would really enjoy it but what we did is we looked at mindfulness in performance of restaurant workers and We had their supervisors rate their performance.
And so Along with my colleague Eric Dane who’s at Wash U in the business school first we actually went and interviewed a bunch of restaurant workers about what is like to work in a restaurant and You know what happened sort of in this dynamic environment? What was it like to be in the weeds and sort of did this part of it? And then we said okay servers are the right kind of audience for this kind of thing And then we’re like, oh, how do we get access to?
restaurant workers and their supervisors who’d be willing to rate them and Basically a couple of my graduate students just wandered around on weekends and begged them to do it So they would like show up and they’re like, hey We can show up at 3 when the shift comes over like we’ll have these paper surveys for you to fill out You know, we’ll do these things and they had been in restaurant industries and so they just went around and begged supervisors to like let us do it and So it was just a lot of effort to go around and see who was willing to do it We ended up like seven restaurants so the sample is way smaller than we would hope for but we just went around and just sort of asked people to Participate in those spaces and then made it really easy for them to do so But you know, so I think what was nice about that is we actually looked for a job where the theories could play out And then we talked to people to make sure that this was right, you know, we had these assumption revolve into restaurants and then after we had that we You know went around and found people in these spaces who are willing to do this work So, yeah, I mean, I think that that really aligns with how we’re trying to do things So it wasn’t convenient at all really, you know to collect that data Especially for my students who are giving up weekends to wander around and then just cold I mean it was cold call.
They’d wander into a restaurant and be like, hey, I’m a student.
[Richard Landers] (29:35 – 30:02)
I’m trying to do this work You know, do you think maybe you can get people to fill out these surveys and some people said yes Do you do you face the or have you faced the tension of There being a sample that you really want and you think you might be able to get access to it But not as many people as you want. Like, how do you how do you resolve that? Like is there a point when you say, you know what it’s not worth it Or is the drive to we can really like help some people here and really help them out.
Is that enough? Yeah, how do you navigate it?
[Brad Brummel] (30:03 – 32:46)
I mean, it’s so tough from Advice perspective. I mean almost all the work I’m doing now is with students and sort of with students projects And so the question is like, well, what are we getting out of this? And you know, one of my favorite studies that I was actually on was a clinical student who got 85 transgender and gender non-conforming Individuals to fill out like the full MMPI Two and three and a whole bunch of other scales.
So like three hours of personality testing and This student William Bryant went around to I think like six or seven different states to equality centers built Relationships and would show up to collect the data from people in person over like two years And so he ended up running t-test to compare it But the gist of the study allowed people to see that if they just went with the normal cutoffs that you’re getting for a normal Population you’d be over diagnosing this population on a number of disorders because of the unique stress that they were under And I worry that in our field like if someone just did a similar study that was out there in a unique population Like the field would shrug and not think it was good work Right, like the work of actually going to where they are as the hard work of the study, you know Somehow wouldn’t be respected as important and that’s a part of the field that I really worry about Because we we don’t need another theory or another moderator.
We probably need some unique samples to see if it holds up In that sample like that’s the right way to do boundary conditions a lot of times and then the study wouldn’t you replicate the study? exactly, but just with a new population and You know if that’s not rewarded then we’re missing out on something important So that’s a long way around of saying I would Strongly support a student who wanted to take that approach to what they were doing And I’m actually okay with students especially in our field starting with the population like I have a student right now who wants to do our dissertation on construction workers and Specifically issues around alcoholism and suicidality in a population that we know it’s high and we think they’re going to be unique Because it’s highly masculine they have travel they have all these different things and we kind of You know, our top researchers don’t approach work that way that often so I know that’s a long way of kind of Unpacking some of those questions, but I really do think sample is the hard work of a project Should be respected a lot more than just a new method a different theory another path and a model You know the other ways we can contribute sure
[Richard Landers] (32:47 – 33:07)
well Maybe to to wrap us up then if if you were to give a your your number one Recommendation your your single best piece of advice to the the grad student that only remembers one thing from listening to this entire video here About sampling. Well, what would your piece of advice be?
[Brad Brummel] (33:09 – 36:16)
Oh Oh man, it’s the one piece of advice Your sample matters as much or more than your statistics That you run on what you do, so think about who you actually want to generalize your research to and Then go figure out how to make it happen and I don’t have a lot of respect for people honestly who are students and say it’s too hard to get this sample Because you know when I was in graduate school I was told that I needed a employee sample for my dissertation because I had an online sample I had two studies just with undergrads which were pretty easy to get at, Illinois And so I actually again in 2007 was wandering around Champaign-Urbana with an envelope of cash Giving people five dollars of cash to fill out my surveys and then ten dollars if they could get their supervisor to do it, too And so like, you know, I’d be like going and getting my hair cut and I’m like, hey, you’re an employee Like well, you thought a survey for five dollars And so I think like that sort of, you know beat the pavement go find people who are willing to do things approach Changes how you think about research in all the ways we talked about you see the person you’re talking to them So you have this respect for them, you know They might not let you walk around with $10 or whatever.
You might need to do gift cards or something these days But I do think there’s something to be said for begging someone to be in your study To kind of have the proper level of humility about what we’re doing Right, and so this neat curated online sample or this already collected organizational data Really does separate us psychologically and physically from the workers so I think I Don’t know tell you tell your advisor that they’re wrong when they think something is too hard or interacting with people is a problem It does it is. Sorry Just I was just having like sort of a flashback to grad school and you know, Chuck Hewlin one time told us the great thing about IO psychology is he’s like we study constructs not people so you don’t even have to really like people or even interact with them to do this work And he always He always said something with like a wry like smile and he’s like he’s like, of course He’s like that makes you a certain kind of person and then you go from there But he’d have these pieces and you know, I always think about that right? It’s like do we study people?
It’s like no We kind of study constructs and this leads to a lot of bad and goofy science Where we’re talking about constructs instead of people and then all of a sudden we get to the practical implications We start talking about people again, it’s like well you never study people actually you only study traits Grafted away from people and then you looked at how those things correlated in order to predict an outcome But you never actually studied the people You know not in the way most people think about it.
[Richard Landers] (36:16 – 36:20)
So Well that that is a wonderful final piece of piece of advice.
[Brad Brummel] (36:21 – 36:33)
Thank you so much for coming by Brad Absolutely, thanks so much for having me and thank you for writing that great book. I made teaching the class so much more fun accessible The students really related to the approach.
[Richard Landers] (36:34 – 36:47)
That’s it for another gig to stay in touch subscribe on YouTube Check out our website at the gig dot online join our LinkedIn group sign up for our email notification list and join our discord Thanks for joining us and see you next time for another great IO get-together