中国高校英语专业写作测试效度检验研究PDF|Epub|txt|kindle电子书版本网盘下载

中国高校英语专业写作测试效度检验研究PDF格式电子书版下载

注意：本站所有压缩包均有解压码： 点击下载压缩包解压工具

INTRODUCTION1

PART Ⅰ LITERATURE REVIEW5

CHAPTER 1 LITER ATURE REVIEW I—LANGUAGE TESTING AND VALIDITY EVIDENCE7

1.0 Introduclion7

1.1 A Short History of Language Testing7

1.2 Validity and Validation9

1.3 Aspects of Validity Evidence12

1.3.1 Theory-based Validity13

1.3.2 Context Validity15

1.3.3 Scoring Validity17

1.3.4 Consequential Validity18

1.3.5 Criterion-related Validity21

1.4 Summary22

CHAPTER 2 LLTERATURE REVIEW Ⅱ—PARAMETERS FOR A VALIDATION FRAMEWORK FOR AWT24

2.0 Introduction24

2.1 Test-taker Characteristics24

2.2 Theory-based and Context Validity Evidence39

2.2.1 The Definition of Academic Writing39

2.2.2 The Cognitive Process Models in Writing41

2.2.3 A Socio-cognitive Model of Writing:Process and Context59

2.2.4 Text Types and Text Construction in Academic Writing64

2.2.5 A Summary of Theoretical Models of Academic Writing74

2.2.6 The Specific Parameters of Theory-based Validity and Context Validity78

2.3 The Parameters of Sconng Validity Evidence89

2.4 The Parameters of Consequential Validity Evidence91

2.5 The Parameters of Criterion-related Validity Evidence93

2.6 A Summary Prototype Framework for Validating Academic Writing Tests95

2.7 The Situation of Academic Writing Tests at Tertiary Level in China97

2.7.1 General Requirements for Academic Writing97

2.7.2 Specifications for Academic Writing in TEM4 and TEM898

2.7.3 Additional Syllabus Requirements for the University Graduation Dissertation98

2.8 General Research Question and Research Sub-questions99

2.9 Implications of the Research100

2.10 Summary100

PART Ⅱ PILOT STUDIES103

CHAPTER 3 THE DEVELOPMENTAND PILOTING OF INSTRUMENTS AND PROCEDURES FOR APPLYING THE VALIDATION FRAMEWORK TO TEM8E AND GRADUATION DISSERTATIONS105

3.0 Introduction105

3.1 Pilot Study One:The Development and Piloting of the Instrumentsand Procedures on Test-taker Characteristics106

3.1.1 Sampling Participants106

3.1.2 Establishment of Variables in Test-taker Characteristics108

3.1.3 Development and Piloting of Instruments108

3.2 Pilot Study Two:The Development and Piloting of the Instruments and Procedures for the Theory-based Validation of TEM8E and the Dissertation122

3.2.1 Participants122

3.2.2 Establishment of Variables in Theory-based Validity Evidence122

3.2.3 Development and Piloting of Instruments123

3.3 Pilot Study Three:The Development and Piloting of the Instruments and Procedures for the Context Validation of TEM8E and the Dissertation153

3.3.1 Participants154

3.3.2 Establishment of Variables in Context Validity Evidence154

3.3.3 Development and Piloting of Instruments155

3.4 Pilot Study Four:The Development and Piloting of the lnstruments and Procedures for the Scoring Validation of TEM8E and the Dissertation159

3.4.1 Participants160

3.4.2 Establishment of Variables in Scoring Validity Evidence160

3.4.3 Development and Piloting of Instruments160

3.5 Pilot Study Five:The Development and Piloting of the lnstruments and Procedures for the Consequential Validation of TEM8E and the Dissertation162

3.5.1 Participants162

3.5.2 Establishment of Variables in Consequential Validity Evidence162

3.5.3 Development and Piloting of Instruments163

3.6 Pilot Study Six:The Development and Piloting of the Instruments and Procedures for the Criterion-related Validation of TEM8E and the Dissertation166

3.7 Summary168

PART Ⅲ MAIN STUDY173

CHAPTER 4 DISCUSSION OF THE RESULTS CONCERNED WITH THE EVIFENCE OF TEST-TAKER CHARACTERISTICS177

4.0 Introduction177

4.1 Sampling Participants177

4.2 Discussion of the Results from Questionnaire A1(QA1)182

4.2.1 Discussion of Learner Background Variables183

4.2.2 Discussion of Motivation201

4.2.3 Discussion of Orientation to the Language Study Context204

4.2.4 Discussion of Learning Strategies210

4.2.5 Discussion of Learning Approaches213

4.3 Discussion of the Results Obtained from Questionnaire A2(QA2)216

4.3.1 Discussion of Academic Writing Leaming Strategies216

4.3.2 Discussion of Use/Process Strategies in Academic Writing220

4.4 Summary224

CHAPTER 5 DISCUSSION OF THE RESULTS CONCERNED WITH THEORY-BASED VALIDITY EVIDENCE228

5.0 Introduction228

5.1 Participants229

5.2 Discussion of the Results from Questionnaire B1(QB1)229

5.2.1 Statistical Analysis of the Results from Factor Analysis of QB1230

5.2.2 Quantitative Analysis of the Results from Regression and ANOVA on QB1234

5.2.3 Brief Qualitative Analysis of Academic Writing Process Feedback from QB1236

5.2.4 Discussion of the Results from SEM on QB1 and QA2239

5.3 Discussion of the Results from the Think Aloud Protocol Experiment248

5.4 Discussion of the Results from Questionnaire B2(QB2)256

5.4.1 Discussion of the Results from Factor Analysis of QB2257

5.4.2 Discussion of the Comparison of the Responses to QB1 with those to QB2260

5.5 Discussion of the Results from Focus Groups(FG)264

5.5.1 Participants and Administration of FG264

5.5.2 Discussion of the Feedback from FG265

5.6 Discussion of the Feedback from Diary B272

5.6.1 Participants and Administration of Diary B272

5.6.2 Discussion of the Feedback from the Diary273

5.7 Summary276

CHAPTER 6 DISCUSSION OF THE RESULTS CONCERNED WITH CONTEXT VALIDITY EVIDENCE284

6.0 Introduction284

6.1 Participants284

6.2 Discussion of the Results from Questionnaire C1a (QC1a)285

6.3 Discussion of the Results from Questionnaire C1b(QC1b)288

6.4 Discussion of the Results from Questionnaire C2a(QC2a)289

6.5 Discussion of the Results from Questionnaire C2b (QC2b)291

6.6 Discussion of the Results from Interview C1293

6.7 Discussion of the Results from Interview C2293

6.8 Summary294

CHAPTER 7 DISCUSSION OF THE RESULTS CONCERNED WITH SCORING VALIDITY-,CONSEQUENTIAL VALIDITY-,AND CRITERION-RELATED VALIDITY EVIDENCE298

7.0 Introduction298

7.1 Scoring Validity Evidence299

7.1.1 Participants299

7.1.2 Discussion of the Results from Questionnaire D1(QD1)299

7.1.3 Discussion of the Results from Questionnaire D2(QD2)300

7.1.4 Summary of 7.1302

7.2 Consequential Validity Evidence303

7.2.1 Participants304

7.2.2 Discussion of the Results from Questionnaire E1a(QE1a)304

7.2.3 Discussion of the Results from Questionnaire E1b(QE1b)307

7.2.4 Discussion of the Results from Questionnaire E2a(QE2a)309

7.2.5 Discussion of the Results from Questionnaire E2b(QE2b)312

7.2.6 Summary of 7.2314

7.3 Criterion-related Validity Evidence316

7.3.1 Participants316

7.3.2 Discussion of the Results from the Correlations of the two Assessments317

7.4 Summary of Chapter 7319

CHAPTER 8 CONCLUSIONS AND IMPLICATIONS323

8.0 Introduction323

8.1 Research Questions Revisited323

8.1.1 Test-taker Characteristics324

8.1.2 Theory-based Validity326

8.1.3 Context Validity328

8.1.4 Scoring Validity330

8.1.5 Consequential Validity330

8.1.6 Criterion-related Validity331

8.2 Limitations of the Study332

8.3 Implications333

8.3.1 Implications for Test Developers in General334

8.3.2 Implications for Exit Assessment at Universities334

8.3.3 Implications for University Students335

8.4 Future Research Directions335

8.5 Concluding Remarks336

REFERENCES337

APPENDICES358

List of Tables2

CHAPTER2

Table 2.1 Definitions of learning strategies33

CHAPTER3

Table 3.1 Some demographical characteristics of the seniors at YNU in the pilot study107

Table 3.2 The main demographical characteristics of 11 participants in TA experiment127

Table 3.3 Transcription conventions (adapted from Bracewell and Breuleux 1994:59)128

Table 3.4 Think-aloud essay marking scheme142

Table 3.5 Cronbach alpha coefficients for marking essays in piloting TA143

Table 3.6 Frequency of score distribution of theessay in piloting TA144

Table 3.7 Pearson correlations between TEM8E and the dissertation at YNU in 2003166

Table 3.8 Spearman rho between the ranks of TEM8E and the dissertation at YNU in 2003167

Table 3.9.1 Instruments developed and piloted for test-taker characteristics168

Table 3.9.2 Instruments developed and piloted for theory-based validity169

Table 3.9.3 Instnunents developed and piloted for context validity170

Table 3.9.4 Instruments and/or procedures developed and/or piloted forscoring,consequential and criterion-related validity171

CHAPTER4

Table 4.1 The demographic overview of student-participants in the main study(QA1.1.02,03,04,05,11)178

Table 4.2 Data collection list in the Main Study on test-taker characteristics225

CHAPTER5

Tables 5.1-31 appear inAppendix 5.1

Table 5.32 Data collection list in the Main Study on theory-based validity evidence277

CHAPTER6

Tables 6.1-4 appear in Appendices 6.1-6.4

Table 6.5 Data collection list in the Main Study on context validity evidence294

CHAPTER7

Tables 7.1-11 appear in Appendices 7.1-7.11

Table 7.12 Data collection list in the Main Study on scoring-,consequential-,and criterion-related validity evidence320

List of Figures2

CHAPTER2

Figure 2.1 The external characteristics of test-takers (O'Sullivan 2000;O'Sullivan and Weer 2002)27

Figure 2.2 Structure of the writing model(Flower and Hayes 1981:370)42

Figure 2.3 Structure of the knowledge-telling model(Bereiter and Scardamalia 1987:8)50

Figure 2.4 Structure of the knowledge-transforming model(Bereiter and Scardamalia 1987:12)52

Figure 2.5 Model of writing as communicative languageuse(Grabe and Kaplan 1996:226)61

Figure 2.6 Elements of text structure(Grabe and Kaplan 1996:63)70

Figure 2.7 A model of text construction(Grabe and Kaplan 1996:81)72

Figure 2.8 A summary model of academic writing78

Figure 2.9 Theory-based validity and context validity(adapted from Weir 2005:47)79

Figure 2.10 Components of language knowledge(adapted from Bachman 1990:87;Bachman and Palmer 1996:68)81

Figure 2.11 Sub-framework of scoring validity(Weir 2005:47)90

Figure 2.12 Sub-framework of consequential validity(adapted from Weir 2005:47)92

Figure 2.13 Sub-framework of criterion-related validity(Weir 2005:47)94

Figure 2.14 A socio-cognitive framework for validating writing tests(adapted from Weir 2005:47)96

CHAPTER3

Figure 3.1 Q-Q Plot of essay scores from 11 participants in the TApilot144

CHAPTER4

Figure 4.1 Distribution of participants:Gender by University(QA1.1.01)180

Figure 4.2 Distribution of participants:Ageby University(QA1.1.03)180

Figure 4.3 Distribution of participants:Region by University(QA1.1.04)181

Figure 4.4 Distribution of participants:Type of secondary school by University(QA1.1.05)181

Figure 4.5 Distribution of participants:Beginning time of studying English by University(QA1.1.11)182

Figure 4.6 Linear and curvilinear regression on QA1.3.08 'If we had no tests,I think I would actually learn more'in the variable of test-related anxiety in orientation206

Figure 4.7 Linear and curvilinear regression on QA1.3.09 'During importanttests,I often feel so nervous that I often forget what Ireally know'in the variable of test-related anxiety in orientation206

Figure4.8 Linear and curvilinear regression on QA1.3.10'Even when I am well prepared for a test,I feel very worried aboutit'in the variable of test-related anxiety in orientation207

CHAPTER5

Figure 5.1 Scree plot for component number on QB1 in Factor analysis231

Figure 5.2 The model for causal relationships between academic writing strategies,sub-processes and score gains243

Figure 5.3 Scree plot for component number on QB2 in Factor analysis258