Given the current COVID-19 situation, I’m prioritising my research and interest in pedagogy, particularly remote learning and alternative assessment. I’m posting general versions of the work I’m doing for my department. My goal is to support the six scriptural languages for which I’m the coordinator. With this context in mind, however, there may be elements which are useful to anyone teaching ancient languages more generally.
This post suggests some ways in which language papers could be reconfigured to be taken remotely but with assessment of learning outcomes mapped on to the aims and objectives articulated in a given schedule of papers (using Oxford’s Theology papers as a framework). I have put together guidelines and mark scheme, which I might put up as separate posts as they’re quite long. This post covers:
1.) The kinds of tasks we ask students to do
2.) Why we do this and how else we could think about it
3.) Some of the resource implications
4.) Some sample questions putting ideas into practice.
All feedback would be welcome!
1. Tasks in the language exams
- Tasks in language exams tend to include:
- Vocab translation
- Translation of set texts into English
- Translation of passages into the target language
- Grammatical analysis questions
- Translation comparison
- Unseen translation into English
These tasks reflect the challenges in testing language understanding, which is different from translation, but of which translation may play a part. The aims and objectives of most (beginners’) scriptural language courses are broadly similar. The following sections consider the main tasks and the way they relate to the aims, and suggest some ways in which alternative forms of assessment could be used to meet the same aims under the current exceptional constraints and circumstances.
In order to test students in different ways, provision would be needed at a policy level to allow this, and at a pedagogical level to prepare students for this. If the new assessment tasks are well-prepared for, and genuinely do test the same skills and knowledge, albeit in different ways, then these problems are not insurmountable.
2.a Vocab testing
Acquiring a basic vocabulary is in many language course aims. With this in mind, any alternative assessment does need to include a way to demonstrate that vocabulary has been learned. This could be directly via a vocab question. This is presently how Greek, for example, does this. Other options are available, which may also assess understanding of a language’s lexical system, allowing for greater differentiation.
The vocab section of a exam tests students’ ability to memorise and regurgitate data. It is a ‘knowledge’ based question, rather than an ‘understanding’ one (using the assessment objective terms of the secondary school curriculum).
Learning ancient languages requires students to absorb a lot of data. This has a number of positive deployments:
- Reading specialists suggest that vocab knowledge needs to be at 90% of a text’s words for effortful reading, and 95% for fluent reading. One could argue, therefore, that learning vocab is a critical tool in fluency and speed of reading, and so assessing it is part of demonstrating mastery of reading as a process.[1]
- Learning vocab and paradigms helps students to build up a conceptual map of a language, which is important in terms of reading more fluently.
- Learning something about how Greek lexicology and morphology works helps students to deal with unfamiliar words and forms more successfully.
- Learning words will give students more of an ‘interior world’ in Greek, better equipping them to think about Greek meaningfully.
Assessing vocab may be a good way to ensure a certain kind of learning takes place.
Doing this remotely risks being unfair. The existing exam question would be open to cheating. It would disadvantage students because it is time-consuming to look up words, but the time disadvantage may be compensated for by the relatively high weighting of the question.
One could extract this task from a paper and make it a stand-alone short assessment, using an online quiz on a VLE. If one put strict time barriers on it, and asked about inflected forms rather than straight dictionary entries, it would be harder for students to look up answers. One could ask for word manipulation rather than straight translation.
Alternative tasks might include:
- Give students words from their set texts. Give them dictionary entries for those words from three different dictionaries. Ask them to evaluate how best to translate a word, including which dictionary entry they prefer, on the basis of their understanding of their use of the word in context.
- Give students words from their set texts. Ask them to write dictionary entries for these, with examples from their set texts to help explain and explore the range of the semantic field covered by that word.
- Give students half a dozen lexemes formed from a single lexical root and ask them to write a short narrative explaining how the words relate to each other.
- Give students a list of words and ask them to group them by semantic field, or morphological features.
2.b Translation into English
Translation is one way of measuring understanding. As comprehensible input and communicative teaching literature demonstrates, however, the interpretative nature of translation is different to the understanding conveyed by reading.[2] Translation is not a straightforward way, therefore, to assess understanding. Assessment which helps students to demonstrate their understanding of the language and of the translation process would be more in keeping with the aims and objectives of the language courses.
Asking students to translate a passage in a remote assessment risks allowing students to copy translations from other sources, including published volumes, each other, or their notes.
We can, however, assess translation through means other than one simple translation.
Alternative tasks might include:
- Use Google Translate to translate a short passage. Give this to students and ask them to comment on the problem of technology doing translation. This could be made more complex and interesting by using more than one technology (although the options are limited for scriptural languages).
- Ask students to write three different translations of the same passage, with an explanation of how they are tailoring them to different audiences.
- Translation comparison – students are graded on their ability to demonstrate an understanding of the target language, but also to discuss how the process of translation works and what interpretative impact different choices make (see Appendix 1).
- Write my own translations of a passage and ask students to mark them and write an account of why they gave that mark.
- Write a commented translation of a passage of set text (see Appendix 2).
2.c Translation into the target language
Prose composition, writing in a target language, helps to demonstrate understanding of the language by making students be creative in it. Standard questions are relatively formulaic. Some students have previously done this sort of task at school. Many find it very challenging. Although scriptural languages tend not to be susceptible to ‘Google Translate’ or similar software for exam cheating purposes, a straightforward translation exercise would still be open to potential problems.
Alternative tasks might include:
- Compose a simplified summary of a passage from a set text in the target language
- Give students a list of vocab and possibly also grammatical constructions. Ask them to compose sentences from them under various constraints. Marking can be done by how well you understand what they are trying to say, as language learning is about communication after all.
2.d Grammar questions
Understanding grammar allows students to understand the conceptual map of the language and enter better into the thinking processes of that language. Most of the kinds of questions we therefore ask tend to be of the ‘understanding’ assessment objective kind rather than the ‘knowledge’ side. With this in mind, most carefully worded questions could be maintained.
Question types to avoid:
- Give the principal parts of λαμβανω.
- Give an example of a preposition that governs the dative.
- How does New Testament Greek use the subjunctive?
Question types which might work could include things such as:
- Explain three different uses of the genitive in this passage
- How does this passage differ grammatically from the rules you have learned in your textbook?
- Explain the form of x (where x isn’t immediately searchable).
2.e Translation Comparison
This has been dealt with as an alternative to ‘straight’ translation. As a task, it requires students to first understand the text in the target language, then think through the translation issues involved in ‘turning’ the text into English, before assessing the aims and styles of other translations. Students can write very superficial answers if they lack insight into the Greek, but the better their understanding of Greek, the more informed and thoughtful their points will be. The differentiation in language testing therefore comes in the quality and accuracy of the response.
2.f Unseen translation
This is one way to assess students’ ability to think on their feet, and understand the language. In terms of how they will use their languages for further study, however:
- They are reading texts that have usually already been translated, so will always have a ‘crib’
- They will always have grammar books, commentaries, dictionaries and other resources to hand.
It is perhaps, for scriptural purposes, perhaps not the most useful way to test understanding in a way meaningful to the course aims and objectives. It may therefore be worth changing it for one of the other translation tasks, to achieve assessment of the aims and objectives based around understanding literature in the target language by other means.
3. Teaching and Resources
In order to achieve the kinds of assessment above, a certain amount of preparation and change of teaching direction will be needed. Three key issues:
- Resource packs for students would need to be created, given that libraries are now inaccessible. These could be prepared by subject tutors from available resources (if possible), or in conjunction with any librarians still in situ. A certain level of subject community engagement would yield enough material to suffice. Appropriate online resources can be collated and vetted by tutors, in order to guide students more carefully. Some work on navigating the internet for academic purposes may be needed in training students. I would be happy to put together a short training resource on this for students.
- Alternative assessment would need to be authorised by a university. This might include:
- Timed tasks to be done within a defined window (e.g. online quizzes)
- Take home tasks to be done over a defined period (e.g. a week).
- Remote ‘synchronous’ assessment in a more standard format, i.e. virtual exams.
(My suggestion is that a and b are probably the most accessible formats, and with careful planning and strict contractual agreement from students we could minimise the potential for malpractice.
- Assessment wherein students are assigned a variety of questions from a question bank, so that no student sits an identical assessment, and therefore collaboration is not possible.
- A mark scheme that erred on the generous side in order to take account of the impact of the change of assessment.
- Some practice for students will be needed. Teaching preparation would need to include setting up demonstrations and discussions, and writing sample questions for students to work on.
4. Alternative Language Exam Question Types
1. Possible questions assessing knowledge and understanding of vocabulary
- There are only three examples of παρερχομαι in Luke, all of which occur in chapters you have studied. Considering its form as a compound, and its use in context, discuss how you would best translate it in the three following examples. You should include discussion of the three dictionary entries given, suggesting which you would direct a new Greek learner to and why. (15%)[3]
11:42 ἀλλ᾽ οὐαὶ ὑμῖν τοῖς Φαρισαίοις, ὅτι ἀποδεκατοῦτε τὸ ἡδύοσμον καὶ τὸ πήγανον καὶ πᾶν λάχανον καὶ παρέρχεσθε τὴν κρίσιν καὶ τὴν ἀγάπην τοῦ θεοῦ· ταῦτα δὲ ἔδει ποιῆσαι κἀκεῖνα μὴ παρεῖναι.
15:29 ὁ δὲ ἀποκριθεὶς εἶπεν τῷ πατρὶ αὐτοῦ, Ἰδοὺ τοσαῦτα ἔτη δουλεύω σοι καὶ οὐδέποτε ἐντολήν σου παρῆλθον, καὶ ἐμοὶ οὐδέποτε ἔδωκας ἔριφον ἵνα μετὰ τῶν φίλων μου εὐφρανθῶ
18:37 ἀπήγγειλαν δὲ αὐτῷ ὅτι Ἰησοῦς ὁ Ναζωραῖος παρέρχεται.
Bauer: 1 to go past a reference point, go by, pass by 2 of time: to be no longer available for someth., pass 3 to come to an end and so no longer be there, pass away, disappear 4 to ignore someth. in the interest of other matters, pass by, transgress, neglect, disobey 5 to pass by without touching, pass 6 to pass through an area, go through 7 to stop at a place as one comes by, come to, come by, come here
Middle Liddell Scott Jones: I.Dep.:— to go by, beside or past, to pass by, pass, Od.; παρῆλθεν ὁ κίνδυνος ὥσπερ νέφος passed away, Dem.
2.of Time, to pass, Hdt.; ὁ παρελθὼν ἄροτος the past season, Soph.; π. ὁδοί wanderings now gone by, id=Soph.; ἐν τῷ παρελθόντι in time past, of old, Xen.; τὰ παρεληλυθότα past events, Dem.
II.to pass by, outstrip, Hom., Theogn., attic; τοὺς λόγους τὰ ἔργα παρέρχεται Dem.
2.to outwit, escape, elude, Il., Hdt., Eur.
III.to arrive at, π. εἰς . . Hes.
2.to pass in, ἐς τὴν αὐλήν Hdt.; π. ἔσω or εἴσω to go into a house, etc., Trag.; c. acc., π. δόμους Eur.
IV.to pass without heeding, τεὸν βωμόν Il.: to pass by, pass over, disregard, slight, θεούς Eur.
2.to overstep, transgress, τοὺς νόμους Dem.
V.to pass unnoticed, escape the notice of, τουτὶ παρῆλθέ με εἰπεῖν id=Dem.
VI.in attic to come forward to speak, π. εἰς τὸν δῆμον Thuc.; absol., παρελθὼν ἔλεξε τοιάδε id=Thuc.
Pocket Oxford Greek Dictionary: mid go by, beside, past or beyond; pass; escape notice; neglect, slight; surpass, overreach, delude; overtake, outrun; come to, pass to; come forward, make one’s appearance
- Write a dictionary entry for δουλεύω (5%). You might want to include:
- a range of possible translations, thinking carefully about the order you give
- examples of where the word is used in your set text
- similar words
- grammatical information about how to use it (e.g. cases governed, irregularities, colloquialisms)
3a. Discuss the relationship between the following words and word forms found in Luke (6%).
- ἀμαρτωλος
- ἁμαρτανω
- ἁμαρτια
3b. An alternative to ἁμαρτια, ἁμαρτημα, is not found in Luke. Suggest, on the basis of its lexical and morphological features, why you think this might be the case (3%).
3c. On the basis of the words you have learned so far, what would a ‘sinner’ be in Greek (1%)?
- Split the following words into up to five categories and explain your categorisation (10%):
κατα ἡ μητηρ το ἀρνιον ἡ θυγατηρ το προβατον
περι ἀνα ὁ ἀδελφος λαμβανω γινωσκω
ὁ πατηρ λυω ὁ κυων
2. Possible questions assessing knowledge and understanding of translation into English
- Luke 15:27a (ὁ δὲ εἶπεν αὐτῷ ὅτι Ὁ ἀδελφός σου ἥκει,) is translated by Google translate as the following:
Discuss the problems the computer has encountered in creating this translation in the context of the interpretative judgements about Greek you make as a human translator of the same text (20%).
- Write three translations of the following passage which are aimed at different audiences and therefore contrast with each other. You might like to consider: children, non-native speakers, religious clergy, atheists, scholars, lay preachers, formal vs functional versions, believers in a home context. You should write a short introduction explaining your approach and the major differences between your translations. Please write your translation on alternate lines (30%).[4]
15 1 Ἦσαν δὲ αὐτῷ ἐγγίζοντες πάντες οἱ τελῶναι καὶ οἱ ἁμαρτωλοὶ ἀκούειν αὐτοῦ. 2 καὶ διεγόγγυζον οἵ τε Φαρισαῖοι καὶ οἱ γραμματεῖς λέγοντες ὅτι Οὗτος ἁμαρτωλοὺς προσδέχεται καὶ συνεσθίει αὐτοῖς. 3 εἶπεν δὲ πρὸς αὐτοὺς τὴν παραβολὴν ταύτην λέγων, 4 Τίς ἄνθρωπος ἐξ ὑμῶν ἔχων ἑκατὸν πρόβατα καὶ ἀπολέσας ἐξ αὐτῶν ἓν οὐ καταλείπει τὰ ἐνενήκοντα ἐννέα ἐν τῇ ἐρήμῳ καὶ πορεύεται ἐπὶ τὸ ἀπολωλὸς ἕως εὕρῃ αὐτό; 5καὶ εὑρὼν ἐπιτίθησιν ἐπὶ τοὺς ὤμους αὐτοῦ χαίρων 6 καὶ ἐλθὼν εἰς τὸν οἶκον συγκαλεῖ τοὺς φίλους καὶ τοὺς γείτονας λέγων αὐτοῖς, Συγχάρητέ μοι, ὅτι εὗρον τὸ πρόβατόν μου τὸ ἀπολωλός. 7 λέγω ὑμῖν ὅτι οὕτως χαρὰ ἐν τῷ οὐρανῷ ἔσται ἐπὶ ἑνὶ ἁμαρτωλῷ μετανοοῦντι ἢ ἐπὶ ἐνενήκοντα ἐννέα δικαίοις οἵτινες οὐ χρείαν ἔχουσιν μετανοίας.
Luke 15:1-7
- Evaluate the following translations of the passage given in Greek (Luke 14:21-24), explaining which one you prefer and why (40%).
21 καὶ παραγενόμενος ὁ δοῦλος ἀπήγγειλεν τῷ κυρίῳ αὐτοῦ ταῦτα. τότε ὀργισθεὶς ὁ οἰκοδεσπότης εἶπεν τῷ δούλῳ αὐτοῦ, Ἔξελθε ταχέως εἰς τὰς πλατείας καὶ ῥύμας τῆς πόλεως καὶ τοὺς πτωχοὺς καὶ ἀναπείρους καὶ τυφλοὺς καὶ χωλοὺς εἰσάγαγε ὧδε. 22 καὶ εἶπεν ὁ δοῦλος, Κύριε, γέγονεν ὃ ἐπέταξας, καὶ ἔτι τόπος ἐστίν. 23 καὶ εἶπεν ὁ κύριος πρὸς τὸν δοῦλον, Ἔξελθε εἰς τὰς ὁδοὺς καὶ φραγμοὺς καὶ ἀνάγκασον εἰσελθεῖν, ἵνα γεμισθῇ μου ὁ οἶκος 24 λέγω γὰρ ὑμῖν ὅτι οὐδεὶς τῶν ἀνδρῶν ἐκείνων τῶν κεκλημένων γεύσεταί μου τοῦ δείπνου.
21 So the servant came and reported this to his master. Then the householder in anger said to his servant, ‘Go out quickly to the streets and lanes of the city, and bring in the poor and maimed and blind and lame.’ 22 And the servant said, ‘Sir, what you commanded has been done, and still there is room.’ 23 And the master said to the servant, ‘Go out to the highways and hedges, and compel people to come in, that my house may be filled. 24 For I tell you,[b] none of those men who were invited shall taste my banquet.’
RSV
21 “The servant came back and reported this to his master. Then the owner of the house became angry and ordered his servant, ‘Go out quickly into the streets and alleys of the town and bring in the poor, the crippled, the blind and the lame.’
22 “‘Sir,’ the servant said, ‘what you ordered has been done, but there is still room.’
23 “Then the master told his servant, ‘Go out to the roads and country lanes and compel them to come in, so that my house will be full. 24 I tell you, not one of those who were invited will get a taste of my banquet.’”
NIV
21 So the slave came back and reported this to his master. Then the master of the household was furious and said to his slave, ‘Go out quickly to the streets and alleys of the city, and bring in the poor, the crippled, the blind, and the lame.’ 22 Then the slave said, ‘Sir, what you instructed has been done, and there is still room.’ 23 So the master said to his slave, ‘Go out to the highways and country roads and urge people to come in, so that my house will be filled. 24 For I tell you, not one of those individuals who were invited will taste my banquet!’”
NET
- What would you change about this translation to correct / improve it (10%)?
8 Ἢ τίς γυνὴ δραχμὰς ἔχουσα δέκα ἐὰν ἀπολέσῃ δραχμὴν μίαν, οὐχὶ ἅπτει λύχνον καὶ σαροῖ τὴν οἰκίαν καὶ ζητεῖ ἐπιμελῶς ἕως οὗ εὕρῃ; 9 καὶ εὑροῦσα συγκαλεῖ τὰς φίλας καὶ γείτονας λέγουσα, Συγχάρητέ μοι, ὅτι εὗρον τὴν δραχμὴν ἣν ἀπώλεσα. 10 οὕτως, λέγω ὑμῖν, γίνεται χαρὰ ἐνώπιον τῶν ἀγγέλων τοῦ θεοῦ ἐπὶ ἑνὶ ἁμαρτωλῷ μετανοοῦντι.
Luke 15:8-10
There was a certain woman having ten drachmas if she had lost one drachma, doesn’t clutch a lamp and sweep the house and look carefully until she doesn’t find it? An on finding it she calls together her friends and neighbours saying ‘rejoice with me, that I have found the drachma which I lost.’ Thus, I say to you, becomes joy face to face with the angels of god on the one sinner changing his mind.
- Write a commented translation of the following passage of Luke.
38 Ἐν δὲ τῷ πορεύεσθαι αὐτοὺς αὐτὸς εἰσῆλθεν εἰς κώμην τινά· γυνὴ δέ τις ὀνόματι Μάρθα ὑπεδέξατο αὐτόν. 39 καὶ τῇδε ἦν ἀδελφὴ καλουμένη Μαριάμ, [ἣ] καὶ παρακαθεσθεῖσα πρὸς τοὺς πόδας τοῦ κυρίου ἤκουεν τὸν λόγον αὐτοῦ. 40 ἡ δὲ Μάρθα περιεσπᾶτο περὶ πολλὴν διακονίαν· ἐπιστᾶσα δὲ εἶπεν, Κύριε, οὐ μέλει σοι ὅτι ἡ ἀδελφή μου μόνην με κατέλιπεν διακονεῖν; εἰπὲ οὖν αὐτῇ ἵνα μοι συναντιλάβηται. 41 ἀποκριθεὶς δὲ εἶπεν αὐτῇ ὁ κύριος, Μάρθα Μάρθα, μεριμνᾷς καὶ θορυβάζῃ περὶ πολλά, 42 ἑνὸς δέ ἐστιν χρεία · Μαριὰμ γὰρ τὴν ἀγαθὴν μερίδα ἐξελέξατο ἥτις οὐκ ἀφαιρεθήσεται αὐτῆς.
Luke 10:38-42
3. Possible questions assessing knowledge and understanding of translation into Greek
- Write 30-50 words in Greek summarising or simplifying this passage from Luke 15 (15%).
25 Ἦν δὲ ὁ υἱὸς αὐτοῦ ὁ πρεσβύτερος ἐν ἀγρῷ· καὶ ὡς ἐρχόμενος ἤγγισεν τῇ οἰκίᾳ, ἤκουσεν συμφωνίας καὶ χορῶν, 26 καὶ προσκαλεσάμενος ἕνα τῶν παίδων ἐπυνθάνετο τί ἂν εἴη ταῦτα. 27 ὁ δὲ εἶπεν αὐτῷ ὅτι Ὁ ἀδελφός σου ἥκει, καὶ ἔθυσεν ὁ πατήρ σου τὸν μόσχον τὸν σιτευτόν, ὅτι ὑγιαίνοντα αὐτὸν ἀπέλαβεν. 28 ὠργίσθη δὲ καὶ οὐκ ἤθελεν εἰσελθεῖν, ὁ δὲ πατὴρ αὐτοῦ ἐξελθὼν παρεκάλει αὐτόν. 29 ὁ δὲ ἀποκριθεὶς εἶπεν τῷ πατρὶ αὐτοῦ, Ἰδοὺ τοσαῦτα ἔτη δουλεύω σοι καὶ οὐδέποτε ἐντολήν σου παρῆλθον, καὶ ἐμοὶ οὐδέποτε ἔδωκας ἔριφον ἵνα μετὰ τῶν φίλων μου εὐφρανθῶ 30 ὅτε δὲ ὁ υἱός σου οὗτος ὁ καταφαγών σου τὸν βίον μετὰ πορνῶν ἦλθεν, ἔθυσας αὐτῷ τὸν σιτευτὸν μόσχον. 31 ὁ δὲ εἶπεν αὐτῷ, Τέκνον, σὺ πάντοτε μετ᾽ ἐμοῦ εἶ, καὶ πάντα τὰ ἐμὰ σά ἐστιν 32 εὐφρανθῆναι δὲ καὶ χαρῆναι ἔδει, ὅτι ὁ ἀδελφός σου οὗτος νεκρὸς ἦν καὶ ἔζησεν, καὶ ἀπολωλὼς καὶ εὑρέθη.
2. Create three sentences in Greek using at least thirteen of the following words (15%). You will need to change their endings to reflect their use in your sentence. You may repeat και. You may also use words you know which are not in the list. You should include at least:
- One sentence entirely in a past tense
- One sentence using a subordinate clause
- One sentence with a change of person
καιρος κατα νομος ἑτερος γη και δυο ἐξω ἐκβαλλω ἀγαπη βαπτιζω ἐν ἱερον ἀνθρωπος ἀναβλεπω ὀχλος λογος κυριος παρα οἰκια ὁσος οὐ οὐπω ἀκουω τηρεω πολυς σημερον δια μετα
4. Possible questions assessing knowledge and understanding of Greek grammar
- From your reading of Luke, including passages on this paper, discuss three ways in which his Greek differs from the Greek you have learned in the textbook (10%).
- Explain the use of the form εἰπαν in the New Testament (3%).
- Discuss three ways in which the dative is used in the New Testament, with examples drawn from Luke (10%).
Notes
[1] A 90% threshold for the New Testament is 901 lexemes, compared with around 600 in Duff (the current source for the question). This is roughly equivalent to an A-Level vocab list in a relevant language. It would be helpful to move vocab learning towards this kind of text-centred mastery, for the sake of improving student reading, which should be the main goal of the course. One might want to adapt this list to include important but infrequent words, for example, or to limit it to a particular author. As a case study, there are 2346 lemmata in Luke. The 100 most frequent conclude with ἁμαρτωλος, used 18 times. There are 1107 lemmata used more than once, and 725 used more than twice. Both of these could be used as thresholds for assessing student knowledge of a text.
[2] See in particular the Journal of Classics Teaching issue 20 (39) Spring 2019 https://www.cambridge.org/core/journals/journal-of-classics-teaching/issue/33E8E71CE3BE6A4D28966E8A8210E140
[3] Percentages do not add up to 100 in this sample, but give an idea of the kind of weighting tasks could be given, with individual languages choosing what combination is needed to reach 100% in their language paper.
[4] This passage may be too long for the exercise, but is indicative of the idea.
