Categories:
Economics of Education
My goal: Get a reasonably large number of college (or even high school) graduates to go over their transcripts, and state, course-by-course, how often they take what they learned in school and apply it on the job.
Respondents should have at least some post-graduation job experience; I don’t want to administer this to e.g. graduating seniors.
My question: Logistically, what is the optimal way for me to do this?
READER COMMENTS
Jason Briggeman
Mar 26 2007 at 6:26pm
A web survey would almost surely be most cost-effective. You’ll just tell people in your recruitment advertising that they will need their college transcript in hand to participate. If you are worried that this biases the sample and you have the money to do so, use a panel of respondents so you can manage your sample (pursuing non-responders with followup emails, etc.) to control for bias.
Pete
Mar 26 2007 at 6:48pm
College reunions, maybe if you coordinate with the college in advance(I recommend not telling them that your thesis is they are useless ;-)). Maybe even better to try to do a deal with a college career center…
Headhunters and/or temp agencies – may require transcripts.
I think selection bias will be a problem.
Pete
Fabio Rojas
Mar 26 2007 at 8:18pm
It’s not exactly what you asked, but there already is a GSS question that asks how much does your job depend on what you learned in school. IIRC, the average answer was not much.
As far as transcripts go, the research design is easy:
1. pick a high school or two
2. look at the year book from 1987
3. use zaba search to find people, which is easy and free
4. call them and get permission to obtain transcripts, then you can do the mentioned study
The other research design is to pay a survey firm to create a nationally representative sample and then obtain their permission to get transcripts.
The cheap version of the study should fetch you about a 20-40% response rate. With some real incentives, maybe $50 per respondent, or if youpay the surveyors a lot, you could jack it up to 60% or so.
Given that you know characteristics of all persons in the sample, you can then do a 2 stage model – selection into the study and then prediction of how much is used.
Of course, the real danger is that you might spend a substantial amount of money and effort for “no one uses algebra.”
Dr. T
Mar 26 2007 at 8:23pm
I think your approach is biased by your linkage of education to on-the-job tasks.
I don’t use my high school American history course on the job, but I certainly benefit from the information I gained. For example, knowledge of history helps me understand arguments about personal freedoms and recent encroachments on the Bill of Rights.
An understanding of optics from my college physics course has little relevance to my job, but it helped me understand cameras and photography.
You are trying to show that we are over-educated, but an approach that ignores personal benefits of education will be seen as biased from the start.
Fabio Rojas
Mar 26 2007 at 8:35pm
A note about the high school design: Doing nationally representative samples of anything is fairly expensive. Especially, when you need to do three steps:
1. contact people
2. get *written* permission for transcripts
3. get them on the phone again
Each step means you will lose quite a few people, unless you have a large budget or a lot of time and patience to track people down.
Choosing a few high schools may be useful in this case because people might still live in the area and you can model selection into the study using year book data, and build a 2 stage model to compensate for selection bias. Also, you can use networks to increase your response rate. For example, you could ask people to recommend to their friends that they do the survey.
However, you can’t really model selection into the study with a representative sample because you don’t have *any* information about non-respondents, and you don’t have ways to boost response rate through friendship networks.
Thus, I would prefer to try to totally canvass a few well selected high school classes than try the national sample. At least with the selected high school design, you could argue that you chose “typical” high schools, you have a decent response rate and you have ability to model selection bias.
quadrupole
Mar 26 2007 at 9:04pm
That very much depends on what aspect of what I learned you are refering to.
I’m a physicist by training, but I’ve *never* used physics in my post graduate school job. What I have used, in spades, is the modeling and ‘systematizing’ skills I learned from studying physics. They give me a notable edge over my colleagues. So the question is, going over my transcripts, did I learn those in stat mech or quantum, mechanics or electricity and magnetism? My answer would be I learned them in *all* of those courses.
One of my most valuable courses in college in terms of future application was an Abstract Algebra course because that was where I learned to stare at something I had know clue about till my brain started making sense of it. How much do I use of my knowledge or groups, rings, and fields? None. But that ability to perceive deep order has been hugely valuable.
How does your study address these sorts of things?
Ben Casnocha
Mar 26 2007 at 9:41pm
I think there are two levels of using knowledge — one is the very practical (“how to change the oil in your car”) and the other is the kind of knowledge that’s useful but we can’t put our finger on it, exactly. At least that’s how I think about what I learned in high school.
Timothy
Mar 27 2007 at 9:38am
I have to agree with Ben, and I’d go a step further to say that’s how I think about most of what I learned in college. I have a degree in economics, but nobody ever asks me to calculate the cross-price elasticity of demand. Of course, if I were an actuary, I’d use everything I learned pretty regularly, I imagine.
Nathan Whitehead
Mar 27 2007 at 12:35pm
Just a suggestion – include more fine-grained ways for respondents to classify individual classes. For example, knowledge from a class might be “directly applied to current job”, “has applied at some times in current or previous jobs”, “directly applies to pursuits outside of employment”.
People might feel bad saying that the classes they took have no applicability to anything. For example, I took music theory in college. I still compose music today as a hobby, but not for a job or as my primary activity. I also took physics, which I never use for anything now (though I loved physics).
Zach Phillips
Mar 27 2007 at 12:52pm
I think that a combination Mailing list/Web Survey would be your best bet. First you could create a sign up form where you could pool email addresses, and those who qualified could simply give submit theirs or not.
Secondly, once you had your list of Emails, you could have a web survey created that you would email only the prospective applicants.
I don’t know what capabilites you have with web design, but I’m sure you could find a reader or a friend who could create the desired applications for you for free or pretty cheap. This way, you could incorporate a database package like Mysql or something else that would allow you to easily collect and display the results in a clear fashion.
Now what I think would be great is if you could create a comparison of how much general education classes (I.e. Maths,Englishes, etc) help on the job scene compared to core major classes.
Virginia
Mar 27 2007 at 1:28pm
I recommend researching Alumni clubs. Alumni are often asked for money to support their university, so there’s a ready mailing list.
Another possibility is at job fairs. You could set up a booth and talk to people who would be ready with resumes and perhaps even transcripts in hand.
To ask for someone to send you a transcript is putting a person out for about $30 to order one. I would be extremely reluctant to let go of the confidential information in it.
That takes a lot of fine print and legalese to cover your liabilities.
I suggest working up a simple form for people to take home and send to you in a postage paid envelope. This would be perfect to hand out at a job fair, along with a pen imprinted with your contact information. If there were five items on it, you might get it back by the end of the fair.
I would fill out a form that asked me about 5 to 10 of my best (or favorite, or career-related) classes in order of “worst” to “best.”
I would try to answer questions about my entire transcript, but as it has over 100 courses in it, I would take forever doing it.
Make a survey and give it to someone else to read. Parents are ready sources of useful critique, no matter what age you are.
Use the form as a script. This will help you simplify the form so people will bother filling it out.
Develop general categories (maths, chemistry, physics, humanities, IT, Business, teaching, social work, etc.) so that respondents can give you a general idea what category it belongs in.
A course title like “Human Interaction and Professional Growth” might be thought of as psychological, or even business, but it was an Education Department course when I took it.
If someone approached me in person to take a survey of my education, I’d more likely take it than not. This goes for job fairs, alumni reunions and all the other group meetings the others posted.
Virginia
Mar 27 2007 at 1:51pm
Another thought occured to me: learning is on several different levels, and the others were right to differentiate between different kinds of learning. There are six kinds of learning, according to the education classes I studied:
Knowledge
Understanding
Analysis
Assimilation
Synthesis
Evaluation
Lower-order learning like rote learning and reading comprehension are covered by the first two, but by the time high school hits, learning goes deeper into higher-order learning such as analysis, assimilation, synthesis and evaluation.
For example, people claim not to use algebra a day in their lives, but anyone going comparison-shopping is doing it even if they aren’t using X’s and Y’s. They don’t do the problems in the book, but they evaluate the virtues of this sale versus “what I need” versus the generic brand and “how often will I come back.” This is analysis.
I have no particular use for triple integral gradient equations, since all I have to do is twist my pencil till it stops rolling downhill. You can still use the equation to find the line that corresponds to my pencil and to the amount of water flowing through a pipe in a certain interval of time.
Discovering those kinds of relationships between the technical and my everyday world is assimilation and synthesis.
The most direct way I have used any one piece or all of my education is in making jokes.
Bobby Newill
Mar 27 2007 at 2:04pm
So far, it has been the math…trig, some calc., and a lot of algebra that are integral in my daily work.
Business writing runs a close second place.
I use the Financial Analysis for my own personal use regularly.
Quite a bit of the rest of it though…not so much as I would have expected.
Comments are closed.