Thinking about exams

Yes, a powerful teaching tool, AND how to do better?

Exams are on my mind

We just completed exam #2. This blog post has two parts: a) what exams look like in my Emory teacher world b) what this might teach me as a trainer, curious person in the world of GenAI.

TLDR;

At the bottom of the post, I have a copy/paste ChatGPT prompt for you to create multiple choice questions on a topic (e.g., upload a Wall Street analyst report and ask me 10 questions like an exam)

30 years ago, I was not a fan of exams

As a college student, they sometimes felt arbitrary, episodic, and a ball of stress. Thirty years later, I am a teacher who is responsible for creating a learning environment for 200+ students every semester. This is harder than it sounds.

Sure, it’s more than exams. Strategy students and I explore questions like this and have fun doing it. There is music, papers, and yes. . . exams

Exams, can’t live with them, can’t live without them

Exams or tests are such a strange artifact from formalized education. They definitely have their place. It’s unrealistic that a teacher could spend 2 hours with each student to explore exactly how well each of them understood and applied each topic. Yet, we also know that it’s just one of the metrics to assess learning.

Some students are great test-takers, while others aren’t great at some formats (multiple choice as an example). Basically, it’s a useful AND an imperfect measurement device. Yes, grades are still important. 100% agree – how can we say that grades don’t matter, when some recruiters deliberately screen for GPA scores?

Also – and let’s be adult here – it is a forcing mechanism. Sure, there’s research, but also a lot of common sense. Trust me, it does encourage / nudge / force studiousness. Students do study extra hard before exams.

1) Exams are one element

Unlike law school, exams are just one portion of the total grade. In my strategy class, it is 45% of the total. The rest is made up of class participation, pre-reading assignments, and team-based presentations. Of course this depends entirely on the topic and purpose of instruction. However, this is just one part.

Just like LSAT, GMAT, MCAT, NCLEX; standardized tests are one tool among many.

2) More than one exam (15% + 15% + 15%)

Exams can be quite idiosyncratic. Two professor teach the same class, yet the exams are different. So, students get into a rhythm by the #2 and #3 exams; they know what to expect. Similar format, similar length, similar question types, similar experience.

3) Tip: Reverse-engineer the test maker

Being able to reverse-engineer your teacher, audience, boss, investors, or spouse is a superpower. Namely, what is the main takeaways and what is important to know. This is a meta-skill that requires listening, asking good questions, and shedding your own biases and thinking about things from the customer’s perspective.

For example, I tell students. . this is how John makes the exam. I look at X, then Y, then B. If I said it multiple times in class, and it shows up in our “afterSession meeting minutes”, there is a good chance it could be on the exam.

4) Yes, it’s a part of the learning

I am a bit of a andragogy (pedagogy for adults) geek, but my favorite book on teaching is Ken Bain, What the Best College Teachers Do here. It may be out of print, so recommend buying it used for $3 (affiliate link). So much good stuff there, but specific to assessments, what a great quote here:

Testing and grading are not incidental acts that come at the end of teaching, but are powerful aspects of education that have an enormous influence on the entire enterprise of encouraging students to learn.

Exams are a part of learning. My undergraduate students study-hard and it shows. WAY harder than I did in college. It’s impressive, mature, rigorous, encouraging, and awesome. Exams are a chance to “show off”.

5) Exams are difficult to create

A fellow professor told me this truth a few years ago, “The easier the exam is to make, the harder it is to grade.” So true, so true. If you throw out vague-y questions, you will get a scattershot of answers which all are sorta-right, and cannot be graded. Instead:

• Only test on what we taught repeatedly; not a detail in the book that wasn’t emphasized;

• Prioritize the most important points; not arcane trivia that students will never use again

• Challenge the students to apply what they learned (cook); not just repeat definitions (identifying ingredients)

6) It should mirror the class content

To any new teacher this is my first piece of advice. Test the students on content you collectively discussed, debated, and covered in class. This is the baseline for fairness. This is common sense and yet, we miss this.

Things that I often say leading up to the exam:

“Please write this down, it might be on the exam.”

“If we mention it 3-4 times in class = hmmm, that might means it’s important.”

7) It should be useful

I attempt to test students on things that would be useful to them in 5 years. Testing them on wonky words that are never used in corporate America (e.g., resource mobility) is not the goal. I need them to know things that will definitely be mentioned, referred to in their early careers:

• economies of scale

• switching costs

• customer acquisition costs

• scenario planning

• forward integration

• transaction costs

• barriers to entry

• economic moats

Trust me, you want the kind of jobs where you are talking about the industry, competition, strategic positioning, winning new customers, gaining market share, driving massive willingness-to-pay, increasing margin, getting ROE.

8) Hopefully, it’s fair

Don’t make the questions sneaky, wordplay, or strangely arcane. Don’t dupe anyone. Use plain words, make the request clear. Allow the test taker to know “oh, okay, I know what they are testing me on.”

Done well, I believe there’s a continuity between the course – what and how we learn – and how they demonstrate it. The biggest compliment I can get on an exam was “it was fair” and “it was fun, it made me think.’

Caveat: Teachers are not perfect (uh, look at me); so the wording of the exams might not be perfect, they need to be tweaked, improved, normalized. Hint: As a prof, it’s useful to load in an exam question and let ChatGPT answer it, and test the edges, to see what other answers the machine might plausibly come up with. . .because that distribution might also look like your set of 66 students.

Learning: Every year about 10% of my students are exchange students (1 semester in the US), so they have a LOT to get used to. New university, new town, new friends, new professors, new language, new culture, new expectations. It’s too easy for me to default to well-known US companies (e.g., Chick-Fil-A, Trader Joe’s, IN & OUT burger, Glossier.com) which non-Americans might not know. . .

9) Hopefully, it’s not a waste of your time

This is generally true of anything in life: a movie, a book, a party, a meeting, a task, a blog post. Hopefully, this is time well spent. The questions are not route and insulting. There is a flow to the exam, so it feels like a conversation of competence, curiosity, and verve. It rewards those students who “know their $h/t” and allows less room for super test-taking-guessers. Hopefully, it’s worth of your time and NOT an undue reflection on the professor’s personality and quirks. (listen – I try, we all do)

10) Hopefully, it’s cooking – not memorizing ingredients

Focus on the SO WHAT?

For me, I am less interested in testing on the WHAT definitions of words because you can Google or ChatGPT that anytime. Turning you into a memory machine = sounds terrible. a) it’s not fun b) it’s less useful c) the computer will always be better than you at that d) your future boss will not ask you such simple questions.

The more interesting – stretch – part of exams is to think about the problem. So WHAT? Metaphorically, now that we’ve learn about carrots (nutrition, recipes, taste), what can we cook from it?

• What are the recipes that would most benefit from carrots?

• What are the limitations of carrots?

• What spices go well with carrots and why?

• How are carrots different from other veggies that go into the mix?

When possible, I bring up news articles from the WSJ or recent events which demonstrate what we are learning in class. For example, if Amazon backward integrates and buys a supplier. . .let’s talk about it. One of the greatest compliments from student (year-end review) looks like this: “Exams are hard, but worth it.”

11) ChatGPT-proof

This is more around testing logistics. I allow GenAI in the classroom, on all presentations etc. . . I don’t allow it on exams (perhaps for obvious reasons), but this is definitely a limitation. . exams need to be in-person, timed, monitored. In a world where 10 seconds can get you master-level recall and synthesis, I want to test you / your smarts / your logic / your opinion / your conviction / your holistic mind / your decision-making. NOT how well you use GenAI. . .(because heck, I use it well too. . . that’s not a true source of differentiation)

12) Kaizen = Keep improving

Yes, continuous improvement. After the exams, learning platforms like Canvas show you what % of students get the right answer, and which other WRONG choices they opted for. A super low percentage of correct answers is a problem with the teacher, test-maker, or wording. Sometimes, the teacher needs to eat humble-pie and just admit the test problem was bad. Yes, it happens. Grrr.

13) Review the exams

“Follow through” and review the exams with the students. If you get a 88/100, then the logical question is “where did those 12 points leak out?” We can have a separate philosophical question of why are we starting with 100 points and marking people off (I get it. . . not ideal, and one of the limitations of exams). For me, after everyone has taken the exams and they are graded, we go through the correct answers in class. People get a sense of where those 12 points go, and if needed, they can see their specific, personal exam.

14) Develop some thick skin

Students aren’t always happy with their grade, or the exam. They vocalize that and the experience for me varies from instructional (that’s a great question) to combative (I cannot negotiate with you for points).

Okay teacher-boy. What does this mean to me? Hmm, a few thoughts:

a) We are all teachers

Whether we are in customer service, parenting our children, selling professional services, or being helpful, we are often educating others. . . Helping others to understand the situation, see the patterns, make good decisions. Reinforce good choices, and gently nudging people to see why something might be the “wrong answer”

b) How do we know if we learned it?

Anyone married knows this is true, heh heh. We might hear wise words many times, but it doesn’t always get tested. Just hearing something doesn’t mean that you really GROK it. Exams have the uniquely clarifying element to it. . .if 60+ of your peers (and you) take the same exam, after all doing the same homework, and you get top decile (or bottom decile), that tells you something. Benchmarking is not everything, but it is something

c) We all need motivation

Exams are motivating. . . sure, they are also stressful, sometimes unpleasant, not convenient. Got it. They are also super motivating. Cal Newport (Georgetown professor) will tell you that we get crazy distracted and lazy with our white collar work. Processes are loose. “Cross-functional coordination” is a fancy way to delay. We lack accountability. MOOCs (e.g., Coursera) = low completion rate because it can get lonely learning by yourself. Motivation is a superpower.

D) Acclimate to testing

Why not test ourselves? As a 50+ year old, when is the last time you sat down and tested yourself on how comprehensively you understood something? (uh, 20 years?) It reminds me of the Newlywed Game, where couples would go on TV and compete with other couples, based on the premise of “how well do you know your spouse?” Hint: it’s hysterical how much we don’t know each other.

How to test your retention? Sure, you may have learned it once, but education is what remains after you forget all that you have learned. Can you explain it simply? Can you explain it to your cousin?

e) Use ChatGPT to make a Jeopardy for yourself

Why not test ourselves? I created this simple prompt as something you can load into ChatGPT along with any article, analyst report, blog post, of video, and test yourself:

I will paste an article.
Create 10 practical, real-world application multiple-choice questions from it.
Present one question at a time like a game show.

Rules:
• 4 answer choices (A–D)
• Focus on judgment, strategy, consequences, and real-world scenarios
• Avoid trivia or definitions
• Do not reveal the answer until I respond
• After I answer, reveal the correct answer and a 1–2 sentence explanation
• Then automatically move to the next question
• If my answer is wrong, provide a short coaching comment
• Tone = fun, energetic, “host” style

Format:
Question #1: [text]
A)
B)
C)
D)

Wait for my answer. Do not show the answer yet.

After I reply, show:
✅ Correct answer: ___
💡 Why: ___
🎯 Quick insight: ___ (optional)

Then display the next question in the same format.

Reply “Ready” and wait for me to paste the article.

Exams are a teaching tool

Teaching at Emory, collegiate and masters-level, is a privilege. It engages my interests and ikigai. The realization – the part that is newer to me now (as of 2025) is that exams are a part of the teaching experience. It’s a teaching tools just like the syllabus, projects, hand-raises in class, discussions after class, the article posts by students, the pro/con debates on each side of the classroom, the office hours, the fun questions, the learning. Yep, exams. . .