Always Learning

Report on the English Speaking Test Forum 2006

The English Speaking Test Forum 2006 - What is English that can be used for business? - was held in Teikyo University's Itabashi Campus on Monday, 30th October.

About the Forum

At the forum, distinguished companies and organizations that conduct
English Testing namely GTEC/TOEFL/TOEIC/T-SST/Versant
(formerly PhonePass)
came together to discuss and to make presentations on the theme of
'How to assess English Speaking Skills'.
The forum is reported in details on the AllAbout website.
Please refer to the URL below.

’Voice of EJ’, a supplementary volume of ‘English Journal’ featured
an article about the forum on its back cover of the 8th December issue.


- The first part -
Despite the forum starting at 2:30pm on a Monday,
the turnout was approximately 170 people. The large number came close to filling all the seats, and proves many people are indeed interested in the talks on English that is applicable in the business scene and the English speaking abilities of Japanese people.

- The second part -
Participants were able to network in the second part of the forum while talking to the testing companies and organizations that set up booths for parallel presentations to discuss on topics they could not have covered in the first part.


About the Forum

You can retrieve your test results by entering the 8-digit number that is on
your test paper into the “score report” field on the top right of
the main page of Versant's website.

Questions and Answers with Versant's presenter

Mr.Masanori Suzuki,participating in theforum from the US

Profile: Masanori Suzuki

What are your impressions after participating
in today’s forum?
It was very worthwhile.
It is very rare to have the opportunity to speak to not only both university and corporate representatives at the same time but also to such a large number of people about Versant. I would be very happy if the participants were able to have a 'correct' understanding of the Versant test

What kind of test is Versant exactly?
Versant is a test that is designed to measure how well a person can use the core skills in spoken English. By core skills I mean pronunciation,basic vocabulary and grammar. How well a person can use these core skills in real-time listening and speaking activities will have a great influence in making communication a success or a failure.

There were presentations on other tests at the forum.
What kind of characteristics does Versant have in comparison to other tests?
I would say ‘Automated Scoring’. Versant uses a voice recognition system and automated scoring system to mark the tests automatically. Moreover, the test is delivered automatically, either by telephone or computer, so the test is completely automatic, from beginning to end. As a result of this automatic delivery and scoring system, the test can be taken 24 hours a day, 365 days a year, anytime, anywhere. The test results are usually available within minutes, so it is a very practical test.

Versant also has another advantage, because it does not have room for subjective decisions of an interviewer or rater. Versant tests a person’s oral answers according to sentence mastery, vocabulary, pronunciation and fluency, and is conducted very objectively.

Some of the tests presented today were specifically for Japanese learners of English. However, Versant is marketed and used not only in Japan, but also all over the world.
To show you a few examples, Versant was used in the recruitment exams for volunteer interpreters at the 2002 FIFA World Cup Korea/Japan (in Korea) as well as for JETRO Global Common attendants at the Aichi Expo in 2005.
The American Embassy in Japan also accepts Versant as one of certifications to prove job applicants’ English abilities.

You mentioned that Versant is characterized
by its core skills focused evaluation.
Please tell us the importance of evaluating these core skills.
Whether it is a business negotiation, daily conversation or a conversation over the telephone, things progress in real-time in listening and speaking activities, unlike reading books or email correspondence. If a person cannot comprehend and respond immediately to what they hear, it is possible that they cannot get an important message across at a crucial moment. At the very least, it is not possible to have interactive communications.

A person has to not only recognize the vocabulary and syntax used in
the conversation, but also to comprehend them in order to understand
and respond in real time. Moreover, he or she must also be able to select vocabulary and syntax immediately and pronounce them properly
in order to respond accurately.

For these psycholinguistics reasons, it is very important to evaluate
how well a person can use these core skills in real time.

What kinds of people take Versant?
Although Versant is aimed at all English learners who are above
high school level, university students and working people make up
the largest number of test takers. For example, various universities in the US use Versant to test international post-graduate students who wish to become a Teaching Assistant to make sure they have sufficient oral English abilities.
In Japan, Versant is used to test the effectiveness of overseas language training, by testing trainees before and after they go overseas. Companies use Versant for the placement of their in-house corporate English classes as well as to evaluate the training’s effectiveness by getting trainees to take the tests before and after the training.

Versant is scored automatically.
How reliable is this system?
Please also tell us about the system’s validity.
The reliability coefficient of Versant’s overall score is 0.97, so the test is very reliable. People often think Versant is less reliable than interview type exams that are marked by raters because the voice recognition system and the automatic scoring system mark test takers automatically.
However, this is not the case.

Moreover, there are concerns that if a large group of people is taking
an interview test at one time, the interviewer or rater’s ‘fatigue’ will influence the scoring or the interview itself. With Versant, highly consistent scoring is possible even for a large number of test takers.

In regards to Versant’s validity, let me speak briefly about construct validity.
Versant evaluates core skills, and native speakers of English are expected to have these core skills. Versant should then be very easy for native speakers of English, and they should achieve very good results.

However, for learners of English, the difficulty will differ according to their abilities, and the test results will also be varied. This trend can be clearly seen in the score distribution of both native speakers of English and learners of English.
Versant tests what it says it tests, so I think it is a very valid testing system.

Does Versant correlate to other tests?
Yes. Versant tests how well a person uses core skills in real time,
but interestingly, we have found that it has very high correlations
to other English speaking tests.
For example, Versant has a correlation coefficient of 0.88 with
TSE (Test of Spoken English), which is run by ETS.

Furthermore, in the study conducted by TOEFL prior to the introduction of
a speaking section to its test (Enright, 2002), it was reported that
it had a correlation coefficient of 0.84 with Versant. Versant also had
a correlation coefficient of 0.75 with the interview test of ILR
(Interagency Language Roundtable). A person who achieves high scores
with Versant has strong core skills,
so they achieve good results in other speaking tests as well.

At the forum you only had 15 minutes to make
your presentation.
Do you have anything you would like to say that
you could not have said in those 15 minutes?
Speaking tests until now were conducted with interviewers and raters and scored by them. However, the advancement of computer technology including voice recognition technology has made it possible to conduct highly reliable automated tests as well. Nevertheless, it is still hard for people to understand what an automated test is exactly, and whether the results of the automated scoring are reliable or not, even when Versant's correlation with other tests and other facts are presented before them. I urge everyone to give Versant a try, and experience for themselves the 'benefits' such as speed and objectivity as well as 'reliability'.
Thank you very much for your time.