Post-Lab 3 Activity

You will answer these questions individually and submit by Tuesday, November 10 at 11:59pm ET (Reflection 5). You may discuss with classmates, but each student writes their own. Name the specific item and quote the specific guideline rule wherever you can. Answers should be written by human students! As usual, “raw” observations in incomplete sentences or with nonstandard grammar are welcome. Choose any 3 of Q1–6 and write about 3–5 sentences for each, plus your collaboration statement (Q7). We are generally looking for open-ended thoughts on how the activity went and any takeaways relevant to your project.

When you are finished, write up your collaboration statement (Q7).

1. Compare the task you specified with the task you labeled for. Which was/seems harder for you to annotate? Why?
2. Was the task you specified meant to be completed by human annotators or AI models? Would you write guidelines for different annotators differently?
3. How did your peer annotators do? Did anything surprise you? (feel free to compare with your Q9 labels)
4. After today, which part(s) of your annotation task would you trust a model to do? Which, if any, needs a person? Did anything from today change your mind?
5. How do/should annotators handle uncertainty and ambiguity? Is it your job to minimize uncertainty and ambiguity?
6. (Free response– write 3-5 sentences’ worth of observations and thoughts)
7. (You may submit text identical to your group’s for this question) Did you work with anyone? If so, state their name(s) and Andrew ID, and briefly describe how you worked together.

Note: If you missed class and wish to complete this Reflection (reminder that your lowest Reflection submission will be dropped!), annotate one other group’s items from the shared doc using only their guidelines, then answer the questions that apply. Reach out if you’re not sure which.