A Critical Review of Language Research

A Critical Review of Language Research

“Correlations between active skill and passive skill test scores”

By: Hendra/NIM: 060954

A.    Background

Research is scientific way to get data in certain objectives and benefits. Scientific way means that research must be based on rationality, empiric, and systematic. (Sugiyono: 2006). Rationality is logic and reasonable. Empiric is that the method perceived by human brain, so human can observe it. Systematic is the use of logical process. The data also must be valid, reliable, and objective. Valid is accuracy between the real data and the collected data. Reliable is consistency of data in certain time. Objective is interpersonal agreement. Someone will find the accountable result through research and take benefit from it   

All researches should possess two necessary characteristics (Young: 2003). They are substance and form. Substance refers to the content of the research; it is mostly dependent on the researcher’s field of specialization. On the other hand, form refers to how the research is written, it includes language and format. Particular attention should be given to correct grammar, capitalization, punctuation, sentence construction, proper indention and spacing. 

B.     The Title of Research

In research, the title must be specific, concise and complete. And the title of this research is:

“Correlations between active skill and passive skill test scores”

Note: Active skills are speaking and writing and passive skills are listening and Reading

 

C.    The Researcher :

Michihiro Hirai. Director of Hitachi Institute of Foreign Languages

D.    The purpose of Research

The clear purpose of research is the key of success (Syamsudin and Damaianti: 2006). It is a foundation for every stage in research from the beginning to the gaining findings.  

This research intends to as follows:

1.      Compare the scores of a company-internal interview test and the TOEIC.

2.      Compare the scores of the Business Language Testing Service (BULATS) Writing Test, which is part of a four-test suite developed by the University of Cambridge Local Examinations Syndicate (UCLES), and the TOEIC test.

3.      Present how these sets of skills correlate with each other.

4.      Shed light on how best to interpret the correlation data in a business context.

E.     Research Methodology.

Research methodology is a way of problem solving which is done by planning and accurate to find facts and conclusion in order to understand, describe, predict and control the condition (Syamsudin and Damaianti: 2006). Research method is controlled by conceptual and procedural thoughts. Conceptual thought is original ideas and procedural thought is the beginning from observation, experiments, and general statements. It is one of the most important parts of a research (Young, CF: 2003). It consists of set rules and procedures that require consistent application.  It determines the tool and techniques by which a researcher can systematically arrives at the solution of his/her statement of the problem and his/her other minor problem. 

In this research, writer has some methods:

1.      Participants

The author collected the test scores of 475 students enrolled in an intensive, total immersion business English program (divided into Intermediate and Advanced levels, which students self-select) at the Hitachi Institute of Foreign Languages (HIFL) in Yokohama between October 1999 and September 2001. Each student took the interview test at the beginning and the end of the course. Each student's most recent TOEIC score before the start of the program was recorded, and all Intermediate level students took the TOEIC test at the end of the program.

2.      Material/Instrument

The author employed the following three tests:

• A company-internal interview test, which, based on a series of questions, evaluates the test taker's aural comprehension, grammatical accuracy, vocabulary, pronunciation, and fluency. This test lasts 10 to 20 minutes, with 20 to 40 items covering about a dozen topics.
• The BULATS Writing Test, which evaluates the test taker's ability to produce short, well-organized business-related memos/letters/e-mail with linguistic accuracy and appropriateness. The test lasts 45 minutes and consists of two tasks.

·         The TOEIC, which is designed to evaluate the test taker's ability to read and aurally comprehend "real-life, business-type" English. The listening and reading parts have 100 items each, and the total test time is 120 minutes.

3.      Procedure

To investigate the correlation between writing skill and receptive skills, the British Council and the author jointly administered the BULATS Writing Test to a total of 102 Hitachi employees, 90 of whom were students of either the Intermediate or Advanced course between September and November 2001. Twelve people, who were not students at HIFL at the time, took the test in April 2001. The correlation was calculated between the BULATS Writing Test scores and either the actual TOEIC scores on exiting the program (Intermediate students) or the most recent TOEIC scores (all others).

F.      Analysis of data and Result

This aspect of the research paper presents the core of the study (Young, CF: 2003). It gives a clear description of what the research is all about the information gathered, tabulated, interpreted, and analyzed.

When the scores of the Intermediate course students and the Advanced course students were combined, the interview scores were found to correlate fairly well with the TOEIC Total and Listening scores, as shown in Figures 1 and 2. The correlation coefficients of 0.78 for the Total and 0.73 for the Listening score were in line with the correlation coefficient of 0.74 between the direct speaking measure and the TOEIC Total.

When the two groups were examined individually, the correlation coefficient between the interview score and the TOEIC Total score dropped significantly (0.49 for the Intermediate course students and 0.65 for the Advanced course students), as shown in Figures 3 and 4. Figure 3 also reveals a distribution pattern for the Intermediate course students that are markedly different from that of the entire sample.

Fig. 1. Correlation between Hitachi Interview scores and TOEIC composite scores

Fig. 2. Correlation between Hitachi Interview scores and TOEIC (L) scores

Fig. 3. Correlation between Hitachi Interview scores and TOEIC (T) scores
among intermediate students before starting an intensive English program

Fig. 4 Correlation between Hitachi Interview scores and TOEIC (T) scores among advanced students before starting an intensive English program

BULATS Writing scores were found to correlate a little more loosely than overall interview scores with the TOEIC Total and Reading scores, as shown in Figures 5 and 6. The correlation coefficient was found to be 0.66 for the Total and 0.59 for the Reading score, both considerably lower than the correlation coefficient of 0.83.

Fig. 5. Correlation between BULATS Writing levels and TOEIC (T) scores

Fig. 6. Correlation between BULATS Writing scores and TOEIC Reading scores

Over a wide range of TOEIC scores, interview and TOEIC scores were found to correlate relatively with a correlation coefficient of 0.78 (TOEIC Total) or 0.73 (TOEIC Listening). When two groups with different proficiency levels were taken separately, the correlation coefficient dropped significantly. For example, the Intermediate course students' interview scores showed a correlation coefficient of 0.49, especially because they included employees with relatively high TOEIC scores who lacked experience speaking English. The BULATS Writing Level and the TOEIC scores were found to correlate more loosely than overall interview and TOEIC scores, with a correlation coefficient of 0.66. The author attributes this discrepancy to the difference in nature between the two writing tests. The author further suggests that TOEIC scores be interpreted cautiously in the organization's business context. To assess business people's writing skill, an exam designed with a typical business environment in mind such as the BULATS Writing Test is recommended.

G.    Conclusion

Having analyzed this research, then it can be concluded such as follows:

1.      The research is a correlational research, which involves collecting data in order to determine whether, and to what degree, a relationship exists between two or more quantifiable variables.

2.      It used Pearson Product Moment Correlation by computing correlation coefficient and computing correlation from raw scores. It used it because the data is interval data.

3.      The Pearson correlation allows us to establish the strength of relationships of continuous variables. One of the easiest ways to visualize the relationship between two variables is to plot the values of one variable against the values of the other.

References:

AR. Syamsuddin. And Damaianti, Vismaia S.(2006) Metode Penelitian Pendidikan Bahasa. Bandung: SPS UPI dan PT Remaja Rosdakarya.

Gay, L.R (1987).Educational research. Florida: Merrill Publishing Company.

Hatch, Evelyn and Lazaraton. Anne. (1991). Research Manual. Boston, Massachusetts. Heinle & Heinle Publishers

Sugiyono.(2006). Statistika untuk Penelitian. Bandung: Alfabeta

Young, C. Felina. (2003). Fundamentals of Research Writing made easy. Davao City, Philippines: Khanser Publishing House

Young, C. Felina. (2003). Statistic made simple. Davao City, Philippines: Khanser Publishing House