Session Transcripts

A live transcription team captured the SRCCON sessions that were most conducive to a written record—about half the sessions, in all.

Finding a better way to test skills while hiring

Session facilitator(s): Rachel Schallom

Day & Time: Friday, 10-11:15am

Room: Thomas Swain

RACHEL: All right, let’s go ahead and get started. So today we’re going to talk about what it’s like evaluating design and development talent, and kind of get into a little bit of how we can make a that a more human and kind process. So my name is Rachel. I was laid off for eight months, and I was industry-famous for a hot minute when I wrote about hiring and all of the people that treated me like garbage. So it’s been really fun to be like, “I love your piece,” and I’m like, I’m still not over that. I applied for 71 jobs. Had interviews at 22 organizations and I sent 11 idea proposals and not a single one of them hired. I now work at the Wall Street Journal.

So this kind of all started in 1998. There was a study that showed that work assessment tests are the best predicter of how someone will perform in a job. The study said that this was better than reference checks, unstructured interviews and years of work experience and when the study came out, it really changed how people hired. And my personal belief is that people thought that they were doing it correctly. You know, instead of going with your gut feeling, you were giving someone a test, and whether they passed or not, it seemed fairer and so this was kind of ushered in where we started testing people, and then things kind of went awry.

So we’re going to talk about three main groupings of how I’ve grouped them. So you have your idea-sharing proposal, and so this is like any form of sharing ideas, strategies or plans what you would do in this position. Usually it’s done on your own time and submitted via email. Sometimes it’s done on an interview on a white board. This is done throughout journalism. It’s done in a lot of management position, but it’s also done for freelancers, pitching ideas. You code or find a solution. Usually that’s done in your own time as well but sometimes it is in a programming exercise with current members and then finally we’ll talk about critiques, and this is where they give you some of their work and they tell you to tell us what’s wrong with it or how you would have done it differently. Or sometimes it’s a final product and sometimes it’s a GitHub repo. So those are the three main categories of how I’m grouping and using those phrases today.

So we’re going to go through and answer a couple of questions, and what I want to note is, we shouldn’t get into too much stickiness at the beginning but the at the end I do want to have a wider discussion. The session is being transcribed, so if at any point you want to say anything off the record, please say that, and once we get into the Q & A, we decide everything needs to be off the record, totally fine. So we’re going to start with why do we use testing proposals and what do we want to get out of them. So I’m going to write the ideas over here and you are going to shout them at me.

AUDIENCE: To see like how someone would perform in a real-life situation?

AUDIENCE: I think it can help sort of level the playing field in terms of when people have different backgrounds, you can sort of use it as a baseline to test skills across candidates.

AUDIENCE: Some level of consistency across interviewees, since conversations can go in many different directions.

AUDIENCE: And sort of see how somebody thinks about something.

AUDIENCE: You can see the process, how they solve the problem, in addition to the final product.

AUDIENCE: There’s like a sense of objectivity that the person who solved it best is the best person.

AUDIENCE: It offers an opportunity for someone to show creativity, particularly if they’re an introvert or someone who may not be good about talking about themselves, but they can show, instead of tell.

AUDIENCE: You can see examples that are more closely related to the work that’s actually going to happen, than having to abstract from like a story or an experience.

RACHEL: So one of the things that came up. I wrote a lot of like hot takes about this. Like, people have strong feelings about them on the internet and one of the things that came up which I have now just completely lost my train of thought. Oh, you could just see what they individually can do, whereas group, you know if, turn in a portfolio, you’re not sure which part of the project they contributed to. Anything else? OK. So that is a great start. So now I left the big board for what are the problems. How are these things problematic?

AUDIENCE: Most of them are incorrect assumptions that you are getting a more realistic understanding of that person in your setting.

AUDIENCE: Work is almost never one person doing one project.

AUDIENCE: It’s that on-the-spot that can make people not perform as well.

RACHEL: I read one blogpost that had a code on the spot and when he said Google, and he asked why, they asked, “oh, do you not have Google?”

AUDIENCE: Tests are written by humans.

AUDIENCE: So one problem and I think you wrote about this in your piece, but the idea of getting a proposal from someone at some level, depending on how much work you’re asking them to do, ends up being free work, and, you know, as someone who’s been on sort of both sides of that, you know, obviously the organization, once they get the proposal for you, some of them will actually run with that proposal, even if they don’t bring you in or talk to you, and that just feels wrong.

RACHEL: I like to assume – it helps me slope at night if I believe that people aren’t doing it maliciously, but like, you get a good idea and like maybe you forgot when it came from, and then you suddenly implement it. But it’s – I mean it is hard to let go of good ideas, whether you hire that person or not.

AUDIENCE: When you’re performing the test you’re often doing so without your typical tools and your preferred environment, and it impedes your productivity.

AUDIENCE: I think in general, too, this might be covered by some of those that have been completed already, but these are completed out of context, right? You don’t have a real idea of what the real job will be.

RACHEL: The people giving the test don’t have the same information you do. You haven’t been in the same meetings and etc. It’s almost impossible to meet that expectation. So here are some that I came up with through what I was reading. You end up getting people that work exactly like you do so if you are solving the problem to see how they think and you like the way they think, it’s probably similar to the way you think. You only get candidates who have time to study and complete the test, which goes along with the free work.

Measures how you do under pressure, more than the work that you produce. Discussing algorithm-based ideas, so people with different backgrounds, if you don’t come from a classical journalism or computer science background, it’s unfair to ask for real work for free. It creates one-way trust. This is the biggest thing for me. So the candidate has to trust that you won’t use the work if you’re not hired. For critiques, the designer doesn’t have the same information, and the challenge of proving yourself doesn’t work for all personality types. So this was really big in imposter syndrome, if you’re essentially marketing and selling yourself in this sort of test, you’re really only going to get so many people that are comfortable doing that.

Which all of this, it should be said out loud, obviously creates diversity-in-hiring issues. So what we’re going to do is, at your tables or in small groups, I would like each table to take one of these problems and you’re going to brainstorm how to fix the problem in a different way, like how to – how can we achieve what we want to achieve over here in a different way than using a test or proposal idea? So so if you guys want to take about two minutes to decide which problem you want to solve and then we’re going to share so each table gets a different problem.

AUDIENCE: We’re choosing from one on there, right?

RACHEL: So each table is going to pick one, like for example, free work, and we’re going to talk about how we get around that ….

[group activity]

RACHEL: Do you guys know which problem you want to solve?

AUDIENCE: No, we’re not there yet.

RACHEL: OK, two more minutes …

[group activity off the record] … … … … … … …

RACHEL: OK, one minute.

OK, let’s come together as a big group again, and I want each group to tell me a little bit about what you guys talked about and what other solutions or better ideas you came up with. Why don’t you guys start over here?

AUDIENCE: We addressed sort of the on-the-spot problem, as well as the working for free aspect. And I’m just actually going to start with like, there was a couple of miscellaneous things that sort of cross over, which is be very specific about the communication process, like, what’s the timing and expectations about, like, when are you going to hear back? That’s always broken, no matter how people do it, and also this from the management side, like people get into management positions and they’re never trained in hiring processes and stuff like that. That needs to happen way before we get to this point. And you know, cover expenses when you get people to like travel and come up and do stuff, so these are – shouldn’t have to say these things, but –

RACHEL: You do.

AUDIENCE: Yeah, so the, like, working on the spot, we have – like, give people a heads up that there is going to be in-person work happening, so just having an awareness of that is helpful, and in some cases, actually give them the homework as one way to prepare for it, so it gives them confidence about the project or sort of the problem space. Let’s see: Ask people if they need or offer supplies and tools, so if they show up they’ve never used a Mac in their life, that would be a barrier that has nothing to do with being able to do the problem. Don’t expect them to read your mind about what you wanted. We talked about our RFP process and that’s problematic in some ways, but there are some benefits there, so if the process of hiring was like that, you could have sort of like, you know, here’s a set of expectations and problems you would like to see solved, and then you can have a back and forth to clarify and then the person can actually present a proposal that’s more thoughtful, but have a more clear process that you can repeat over time and that sort of overlaps with the work for free problem, so, you know, pay people. So you can do, just like there are a couple of options, but for example, a properly paid internship with a real path to a full position, that’s one possibility. Having a contract for you, so if you do an RFP type process or anything like that, like, have some terms written down, like that you can’t use the work, or if they do, you have to pay extra, what’s the copyright, just have those written down in some sort of a of addresses the work-for-free problem. You can have a candidate like you’re OK, this person would probably be a good fit, they can be in a real situation for you and that’s both good for you and the person to assess. And they’re getting paid for doing real work. Did I miss anything?

AUDIENCE: No. Good job.

RACHEL: I saw that coming up a lot, kind of easing into a full-time job. It gets problematic when it’s like, should I quit my job for like this trial period? Or in journalism, am I moving across the country? But in a lot of cases it is working. I have a friend doing one right now. She seems to like it, but she also gets to live with her parents while she’s doing it.

AUDIENCE: We were giving an example like I got someone who I wanted to hire, but it looked like it was going to be a good fit, but she was going to move out of her state with her whole family and I said, OK, why don’t you work remotely for a month and then everyone’s more comfortable with the transition.

RACHEL: What do you guys think about what they said? Questions? Thoughts.

AUDIENCE: I realize this is not always applicable to journalism and newsroom jobs but we’re possibly offering a permanent remote position is a great way of lowering the stakes of someone uprooting themselves and moving somewhere. We’re a digital agency so we’re able to offer everyone a remote position and never tell anyone like, you have to uproot yourself and move to New York where our headquarters is. And you know, I think that’s like reduced the stakes of is this job going to ruin the whole person’s life if the job blows up in a month or something. If possible.

RACHEL: Why don’t you guys go next? Because I think you were also talking about free work?

AUDIENCE: Yeah, we were. Should I read these stickies here? OK, so we came up with a couple of ideas for how to make the – I’ll stand up, to make make the free work thing less problematic for the people taking care of the projects. So one is to establish a like a uniform criteria for taking the test or if there’s a specific reason for someone to take the test and not someone else. In order to avoid putting pressure on people who have full-time jobs or family responsibilities or any other kind of demands on their time, to proactively offer a time frame for when the test happens to accommodate schedules rather than waiting for someone to request accommodations, go ahead and say, let’s schedule a time that works for you? How can we make this work for your schedule, whatever your schedule may be? To just bite the bullet and pay people money for doing a test project. They’re doing work, they should be paid money. End of story.

And give a test that’s not economically actionable for the company. Like, the work that’s being done couldn’t conceivably be use in a real context so there’s no risk that like you’re giving them the test in order to like sort of “consultaview” them, take advantage of their work. I interview a lot of developers. It’s consultaview. And they’re like, bring them in for an interview, tell me more about that, and then they’re like, never mind, but thanks for the free info. And that’s not nice. and finally substituting a narrative evaluation or more like of a kind of logistical evaluation of how you would have solved the problem, rather than making someone like sit down and write code or write a story or design a thing in order to evaluate them.

RACHEL: Awesome. Any thoughts? Ideas?

AUDIENCE: I’m curious if anyone else has paid people for their evaluations. This is the first time I’ve actually heard of a company doing that I think it’s cool.

AUDIENCE: So we may people once but it’s not for a full-time job. It was like we were looking for someone to help us do some overflow work so the way that we did it for finalists that we like is gave them different things that we needed to to have done and just paid them how much they would have been paid if they got the job and paid them that much money and they completed the task basically. But we’ve never done that on a full-scale level for a development project or something.

AUDIENCE: We did one for work that was more hypothetical, like designing a page that doesn’t exist.

AUDIENCE: The folks at Base Camp talked about this a lot in their writing. So David Hansen, they have a work called view work and he has a Medium blog and he has lots of thoughts on this, he’s very opinionated.

AUDIENCE: … Has that sort of thing. At the end of it, either party can just kind of say, “that was fun,” and I have a former coworker who just got done doing that and I think she actually decided, yeah, OK, that’s fine, and I’ll do something else now, but yeah, that’s a kind of a low-pressure thing. It is a little scary, the fact that you will take work that in 6 months you don’t know if you’ll continue to have work, especially if you’re not normally a hired gun.

AUDIENCE: And that only works if you don’t currently have a job, right?

AUDIENCE: Or if you’re willing to walk away from it.

RACHEL: I think there’s a lot of positives with that idea. One of the things that actually strikes me is so much of starting a new job is get to go know people and like, how do they communicate? Do they never answer emails and the politics of who knows who and who actually has power of the room and I feel like a lot of the first six months is like that so I would be nervous if I was, like, doing a project to prove myself and also learning, like, when I get an email from someone, does it matter? Does it not matter? Those are some of the things that we don’t like to admit that goes on in onboarding, but it does. How about you guys? What problem were you solving?

AUDIENCE: Sure, time pressure, also, so clearly that was one that mattered for a lot of us. Now, one interesting thing. Up talked about alternative solutions to tests and proposals and I would say that we found that there was a unique role for tests and proposals, that they were very valuable to us. We attempted to discuss briefly, was there a way to get what we needed without doing it at all and we didn’t think that was the case. Because there’s something about seeing the raw work product from someone and what they actually produce and how they interact that wasn’t working. So our thing was how could we minimize all of the bad things, but maximize what we needed out of that process.

RACHEL: Cool.

AUDIENCE: So several of us.

HADLEY: Some experiences with sending project things home with people but in a way that respected their time. So maybe it was a shorter half hour or hour, and they had a week to complete it. Maybe an open conversation that said, or I don’t know that we did this, but recommendation would be asking what time pressures they may have, would they need to reschedule, and that seeming a low-pressure ask, not prove your commitment to this process by doing this in the shortest amount of time.

Also only the idea that you would give these to people who are really were finalists for a position and thus weren’t requesting people give up time for someone who didn’t have a solid shot at getting. Part of the expectation would be to make it clear that the person completing the test could ask any questions that they wanted to throughout the process, that it wasn’t a closed-book test but they could make use of the resources that were around them, but also that they could come back to the hiring managers and say what did you mean here or which direction did you expect them to go? And that clarity also around what thecriteria would be for evaluating, so it seemed like what your team said, the more clarity you could provide up front, about what time period you could expect from them, the fact that they could ask whatever questions they needed to ask, how they were being evaluated, that those would be ways to reduce not just the time pressure, but the pressure itself.

AUDIENCE: And we also talked about keeping it to the bare minimum skills that you would need, not trying to get lots and lots. Part of that is the time, too. To keep it to the bare minimum of skills that they would have to do.

RACHEL: I think what it comes down to what I’m hearing is like the intention behind the hiring, that by putting this thought into what you’re wanting and what you’re asking for, you’re also sending the message that, like, we thought about this, we care about you. And it also seems really fair, like if there’s a lot of – this is what we’re looking for, this is what we expect, I would assume that that was somewhat of a template that you were sending to other candidates and that we were probably all being evaluated in and about the same way.

I would say that that is kind of what stood out to me from what you guys were speaking because when I was asked to do them, it all seemed very informal, it would be like in a phone conversation why don’t you write up some of your ideas and send them to us, but actually saying you know, we’ve thought about this and this is how we plan to use your ideas or not use them, I think sending that intention is just such a kind thing to do. I mean you have to have the intention first, which I think a lot of people don’t, but I also think that sharing that and being communicative, because I do really also believe that the candidate is hiring, or is interviewing the company, as well, so I definitely interviewed with people that were terrible communicators and I was like, this is going to be my boss.: And so I think that intention works, it’s not – like it takes time out of your day and out of your company to do that, but I think it’s also really good for your company’s brand, because how you hire does speak volumes about your company culture and your brand.

AUDIENCE: So one thing we talked a little bit is I had given a test at a previous company I was and I was saying in the beginning we weren’t hiring a lot of people and we were doing it more of an ad hoc basis and then when things slowed down and became a little bit more manageable, we got it to everyone who got to a certain level and the difference between the caliber of person we got was different, right, because we gave it to everyone in the second instance versus the first instance, and there was one person in particular that I can remember that later, when I was giving it to everybody, I was like I should have given it to that person, because I would have known that they weren’t able to do X, Y, and Z, and that would have made a big difference in the way that I would have made this decision about hiring them or not. It may not have been the only thing that would have sort of kicked them out of the process at that point, but it clearly had reverberations that they knew how to do that I saw in their actual work. So I think being intentional and explaining that it’s part of the process and making sure that you aren’t just doing it sort of on an ad hoc basis is really important. Because it also helps you. You’re able to look at it in a fairer way. Like, you’re actually comparing like to like, as opposed to like, well, I had this feeling that this person knew how to do this thing. So.

AUDIENCE: A couple of comments on that. One thing that I – people should have a specific rubric for evaluation. Because looking at something and trying to decide what is important and trying to weight the various factors, is just you doing it out of your head, but if you say, looking at this, it’s rated at 5, super helpful. And then the second thing is do you give the test to everybody or just people who pass a certain bar? If you only give it to a subset of people, then you’re giving it to people who are good at talking and not necessarily doing the task you’re hiring them to do. Now, if you get 100 applications, you might want to do some kind of prescreen, so you throw out the obviously unqualified or spam or whatever, but get it go to more people rather than fewer people, will give those folks who, you know, might be shy, or not good at promoting themselves the opportunity to do the work and then you can evaluate and then decide to bring them in if they succeed on your test.

RACHEL: Yeah, and I think some of that goes back to that intentionality, if you’re thinking about what this person needs to have, you’re doing more thinking about the job that you’re hiring for, the atmosphere that you’re hiring them in. Like, we need them to be really organized or we need them… And you’re looking for specific categories. There is some hiring, I mean there’s a lot of hiring guides out there, but a lot of them go back to the idea of coming up with characteristics that you’re looking for in that person, and then you can design your interview process around, you know, what questions you’re asking, what tests you’re giving, if you’re trying to get at certain things. This is off the record …

AUDIENCE: Like to what Paul said about giving the test to many people, even though they might not make the criteria, you took a lot of tests, and do you feel that that was more the case, that they were more likely to give the tests to people who weren’t going to advance in the process, or that – I guess like I would like to just know your thoughts on that, because that was what I took away from your story, was that there was a lot of tests.

RACHEL: Mine were given kind of what this group had said about giving it to a select group of people so that you’re not using everyone’s time, or that was the impression that I was given. I was always given the impression that I was on a short list when I was doing this. I will say that as I went through the process more and experienced more things, I started to pick up on cues and there was actually one that I refused to do a proposal for, because during my phone interview, he had questioned my age and my gender three times, and.

AUDIENCE: That’s illegal!

RACHEL: Oh, yeah, pretty much everything I experienced was illegal. And I knew – I knew from his tone in the phone interview, you’re not going to hire me. And I went back and he said, why don’t you send me some ideas and I went back to him and I said, I know you’re not going to hire me, because you have now questioned my age and my gender three times and this is a waste of everyone’s time. I knew he was going to be uncomfortable hiring a young woman and he never would do T but usually I felt like I was on a short list. Or in some cases they told me I’m the only candidate they were talking to. Like I was personally recommended by somebody else, so it always felt like I could do the proposal that I’m sending in. Especially in the beginning. As I did more of them, I was like, well, who knows?

AUDIENCE: So we’re talking about skills-based and sort of getting a sense of someone’s background. But there are certain jobs that you can teach on the job and you just want someone who is willing to learn, but you also want to make sure this someone is like cool to work with every day, and is going to be like a good colleague, or know how to handle difficult situations. Like, those things are obviously harder to test people for or get a read from a 30-minute conversation with someone. So it’s like, what are some good ways to get a sense of if someone’s going to be – because I mean these are people that you spend the majority of your time with every day, like, sometimes more than your own family, so what are good ways to get a sense of if someone is going to be a good fit?

AUDIENCE: Actually call their references. I feel like a lot of times with the references and they’re like, yeah, no one ever called me and I’m like, well, that’s where you ask those kind of questions.

RACHEL: Out of all the interviews I did, the Wall Street Journal, which is where I work now, was the only one to call my references.

AUDIENCE: It tends to be a sort of a per functry after the decision is already made thing, to call the references. I’ve found it really helpful to talk to references and one of the questions that I’ve asked references on on these calls is, as a manager, what advice would you have for me in managing this person? Should we hire them? Because it’s one that sort of – I mean you expect in a reference if they’re listed is going to give a positive review, right, but there are a lot of things that you can pick up about a person’s work style and ways of interacting with colleagues by asking a question like that that’s a little more open ended and has resulted in some good – I mean I don’t know that it’s swayed hiring decision, but it’s definitely given me some good insights.

RACHEL: I gave a reference for someone who used to work for me and this person had a lot of potential. I felt like he was someone that wasn’t challenged in the right way in the team that he worked with me for, and it was kind of really chaotic, but I saw a lot of potential in him and so I openly talked with the manager and I was like, this person if you pair him with a great editor, someone who’s going to give some direction, like fuel that creativity, but guide, I really think that that is doable and that’s not something that you can see from his portfolio.

AUDIENCE: I was just going to say I’ve had the same experience. I rely super-heavy on references and I call more if I can find more, but I will ask questions that will test my, in some ways yield can you give me a person of how much a person has had to weigh in to edit or shape, that kind of thing. It’s been really helpful for me at least.

AUDIENCE: The only thing I would add, too, is in addition to references, I love asking the candidate these questions usually in the application form itself. So if it’s something that’s going to be important right away, just in written form, give me an example of a time in which you had to work with a lot of people in a project and sort of how you dealt with communicating with everyone, or whatever you’ve identified as something they’ve got to do it a lot. If they can’t do it, they give you a real crappy example. I find that it gives you a lot of insight.

Literally the phrase, tell me the time when blah, blah, blah blah.

AUDIENCE: I think it’s better to put that in a written form, as well, because being put on the spot with that question.

RACHEL: All of my best questions came in on the cab ride on the way home. So one of the things that I wanted to bring up is kind of in what you were asking is structured interviews which is the idea that you’re asking the same question, you’re intentionally picking questions, you’re intentionally identifying what you want to find out about them and then you’re asking the same questions of every candidate and you’re keeping track of the answers so whether that’s a written form and then you have to ask them, or you have identified in formal interviews, like, I would do the question – the way I did it was I had a list of questions that I had in the first interview, a list of questions that I was going to ask in the second phone interview and I tried to keep it as conversational as possible and then I put them in a spreadsheet where I could look at the different answers and structured interviews, going back to that study was the No. 2 way. Like, after test, if you intentionally asked questions, that was the second-best way to evaluate answers and Stacey Marie Ishmael has written a lot about structured interviews and one of the things that she wrote about that I took away is if someone is coming into your office for an interview to have them briefed about what you were going to talk about. Because I’ve definitely answered the same exact thing to all five people because they all asked the same questions. So if you have five different people. You could maybe say I would really love you to dig into organization. I would love you to dig into project management, which can also help the interviewer.

AUDIENCE: Yeah, I dropped a link about structured interviewing in etherpad. I wanted to jump back to references for a minute. One of the problems with references is a bit of selection bias, they’re going to give you people who will say nice things about them. Now, sometimes they screw up and they accidentally give you someone who’s going to be honest with you. But people, especially in the US, are super-afraid of litigation and so they won’t say anything not nice, even if the person was fired for, like, cause, and they’ll be like, oh, they were fine, really. So trying to find questions you can ask that are positive, like, would you hire this person to be your right hand man or woman? That would force them to at least say no to something. And trying to find other, like, nonlisted references, so if you know someone else who’s worked at the organization or worked with them before that they didn’t tell you about, then you know ask. This is not to gossip. It’s to ask very specific job questions.

AUDIENCE: That’s tricky, though, because there are confidentiality issues sometimes around people applying for jobs and you don’t want to be the one to tell someone else that someone’s applying for a job at your place, necessarily if you’re asking someone outside. Like that’s really tricky, particularly in some industries.

AUDIENCE: Sure.

RACHEL: I think one of the good things about references in journalism is it is such a small industry that everyone knows each other, and your name is like literally all you got. No matter how good your portfolio is, you know, kind of who you are as a person and your name and who you know is a really big deal in journalism. And so I do think there’s a bit of a fear, like that people tend to speak honestly and in my experience, people tend to people honestly in reference in journalism because they in no way want to refer someone who isn’t going to do a good job because their name is attached to that person. I’ve had people I love as humans and I like getting drinks with, but I have been timid in my reference, because you know, I’m not totally sold that they’re right for this position and my name is the person that referred them. So I do think – I do think we have that going for us, even if the idea of it, a small industry is kind of a bad thing in general. One thing I wanted to point out before we run out of time is the one thing I kept coming back to in my research was designing exercises where there is no perfect answer and neither the interviewee or the interviewer are experts in that area. So if you just did a giant investigation into maternal deaths, maybe don’t give a test about that, because you have far more information than they do. And I thought that was a really interesting idea. Kind of goes back to what someone said about doing something that the place can’t use, but I thought that was an interesting tip I hadn’t thought of.

AUDIENCE: For structured interviews, yes, create structured interviews, but also have a guide of the interviewer of what’s the purpose of every question, because it’s just that they’re giving yes or nos. It’s not a check box. It’s a launching point to check out the details. And so when I’m interviewing somebody I need to know why we’re asking those questions, what we’re getting at. So I have to say, why, what did I really, really get out of this interview?

RACHEL: And as a human, you can ask better follow-up questions, because interviews should be more like conversations and less Q & As, and so I totally agree, if you know why you’re asking something, it can help you further that conversation and ask follow-up questions.

AUDIENCE: And another thing, too, that we’ve realized through experience is when you’re – is to actually weight the skills that you’re assessing in terms of from like a hiring manager, because I almost think of that as almost a project manager through the process, that your interviewers are going to be teammates. But at the end of the day when the hiring manager goes to make a decision, being able to be transparent about why they chose one person or another. Because someone might say, well, wait, this person didn’t do that well in my section and you know, for whatever reason, but the decision might have been weighted more on the facts that they were stronger on these three areas, which were more important. They should have all the skills we’re assessing for, but that has come up for us and it has caused strain on teams. And they were like, but we didn’t think they were as strong in this area, and we’re like, but they’re a lot stronger in these areas. You can do a kickoff call with your leadership team and say ultimately this is what we’re going to work for.

RACHEL: Right, we do job descriptions for perfect people that don’t exist. And so you have your job posting and you have 10 bullet points, which ones are more important than others? And some of that is team-based. Like I have someone really strong in A, so, like, having someone really strong in B makes a good team. You have to decide what’s best for you. We are running out of time, but I just wanted to note one thing. What you said about this is someone that you’re spending a lot of time with, I think a lot of decisions – like, we have a gut feeling, like I feel good about this person and I don’t think we should dismay those completely, because a lot of journalism is gut. I have a gut feeling I should look into this, or but I will say if you’re interested in looking it up, there’s a lot of research and thought on the idea of culture fit and how it creates a lot of biases and diversity issues. I was – I went through an eight-hour, in-person interview that I had flown to a new city for and when I left, he asked when I wanted to start and five days later I hadn’t heard anything and I called him and he said I wasn’t a good culture fit. I mean there are people that are like annoying and you don’t want to work with them and that’s fine, but I think if you’re going to go with culture fit as reasoning for not hiring them, I would challenge you to challenge other people in your organization to articulate that. Because what I think is culture fit is I can just say that I don’t think they’re going to be a good fit and we can move on. And I don’t want to address that it’s because they’re a young woman or they made me feel uncomfortable. And if it’s because they’re super-annoying or something, maybe that’s fine. Maybe that works in your organization, but force people around you to articulate that, instead of just moving forward to the next candidate. So like I said, I put up – I wrote a really long letter about everything that happened to me when I was laid off if you want to read it. Otherwise, ask me questions. I’m also doing follow-ups, reporting on hiring for Source, so if you have something you want to talk about, please reach out. Thank you so much for coming.

[applause]