Design, Research and Methodology.
It is one of the primary aims of the dissertation to understand best practice throughout the delivery and assessment of NVQs in Catering industry in Wales. The methods of collecting data were both formal and informal coming from candidates, students, lecturers, trainers/assessors, college managers, training managers, and external verifiers.
A mixture of data collection techniques were used to attempt to overcome the subjective nature of the questions and the wide range of experiences available to NVQ candidates and assessors. This is because individual establishments operate in very different market places and the individual owners and managers apply standards as they see fitting to their business situation. For example a small hotel in a coastal resort would differ considerably in their operation and training practices to say MacDonalds with an international training programme. Colleges also choose to offer different types of Realistic Work Environments (RWE) to suit the perceived needs of the Industry and the preferences of and the resources available to the management and staff at that location. To overcome these problems in establishing a reliable data base the methodology adopted attempted to balance the differences by establishing underlying themes.
Identification of suitable population from contacts with 6 colleges and training industrial providers. A population of suitable colleges and thirty-three catering establishments was established through contacts and discussions with FE colleges and industry and training contacts.
The research strategy of the dissertation aims to use the most appropriate methodology to address the specific objectives stated in the introduction:
Figure 1 shows the focus of the research study
Focus of the research into delivery and assessment approaches
Review of literature Main issues arising Why NVQs?
in delivery and what is the historical background?
assessment of NVQs What are the major criticisms?
What are the perceived strengths of NVQ?
Questionnaire survey A. Population of To establish data and attitudes
candidates in colleges and perceptions of candidates
and with in the different routes
B Population college To establish data and
Lecturers and managers attitudes and perceptions
Observations Selected sample of To examine perceived
Lectures, Section Heads, differences between routes
Training providers and and verify data gathered
Assessors in questionnaires
The questionnaires were designed and distributed to six colleges to thirty-three catering establishments ranging from small pubs to large five star hotels.
Two questionnaires were constructed which sought to gather reliable and valid data, one for candidates and students and the other for managers, lecturers and trainers or assessors. In both cases the questions were designed to be as clear as possible and used open questions to identify reasons for the choice of and satisfaction with the route.
" The basic rules of good questionnaire design are to keep it short keep it simple and make sure that it is understood." (Belsen W. 1962)
The methodology utilised for data collection involving a broad approach was a postal questionnaire of two pages; this was sometimes conducted face to face. The questionnaire was designed to encourage qualitative responses to enable views relevant to the dissertation to be extracted. The data gathered was examined quantitatively using a database and the qualitative data evaluated and analysed. Research has shown that responses to questionnaires are typically varied and are there is often a low response rate. The writer attempted to overcome these problems by asking for co-operation from the organisations prior to sending out, by repeated chasing of those organisations who did not respond promptly, and by face to face interviews in one college and in some eighteen industrial establishments. Problems were encountered in that responses were often vague and sometimes appeared to have been the result of discussion between a group of candidates at one location or that "guidance" had been given by tutor, trainer or employer. The writer subsequently rang up the tutor to clarify the responses. The reason for this guidance was that the case where learning difficulty candidates on Level 1 courses and who had difficulty in expressing themselves in detail or articulating themselves without help.
" Some common causes of respondents failing to fully understand the interviewers questions are due to the use of unfamiliar words."(Morton-Williams 1978)
In retrospect the word 'route' needed to be clarified. When the writer realised this was causing some difficulty a covering letter was sent out to the establishments requesting that they clarify what was meant by the word. Some learning difficulty candidates in particular needed guidance when completing the questionnaires to answer some questions with complete understanding. Furthermore the response from some organisation to returning the questionnaires was slow despite telephone agreement and discussion with the relevant managers. As a result it was necessary to chase them Costs were involved in sending out the questionnaire and in preparatory phone calls to get agreement on distribution.
The questionnaire for managers, lecturers and trainers asked similar questions but looked for a more in depth response. This was conducted mainly on a face to face or on the telephone with some postal responses. It was expected that the views of this sample would be more polarised and informed and the validity of certain responses could be tested against the responses from the candidate questionnaire and the review of literature. The reliability of these responses was considered in relation to the probable belief of each group in the superiority of their own route. Future trends were also examined in this questionnaire in an attempt to ascertain if the respondents had any common views on likely developments and changes in NVQ assessment and delivery. The response were varied
Data was then collected and collated. A spreadsheet database was set up to collect the data and to enable easier interpretation and visual representation of certain statistics. The data was then entered into the database and sorted to establish patterns and relationships in the responses.
The data provided by the questionnaires required further exploration to clarify and validate the findings. For example the differences in information on the key skills delivered was often caused by candidates not fully comprehending which qualifications they were taking. In order to clarify this organisations were telephoned and responsible parties asked to verify the data. Certain subjective comments made by candidates needed to be examined in more detail in order to clarify them. For example a number of respondents in industry had problems with off job training the nature of these was not always clear so verbal interviews were conducted to attempt to establish the reasons.
Literature was sought from various sources and with varying degrees of success.Libraries provided a good deal of information on the historical background to NVQ and the key issues in the development of them as a training approach. Various libraries were used to search for information mainly Newport Library and Caerleon College. Most information was found by using the computerised reference systems through topics and authors. The lack of subject specific literature required broader searches to find more generic background information. Journals were sought through a different computerised system.
Quantitative data was obtained by establishing the main sources of data available in this field mainly through HTF and government sources
HTF were faxed and asked to forward up to date reports and relevant articles, also quantitative and qualitative reports were obtained form this source
HMI reports issued to colleges were obtained
The Welsh Office provides some reports in related areas.
HCTC Ltd provided some statistics and general literature including standards
The Internet provided some substantive information but most of the available data was in abstract form and referred to booklets designed to assist candidates or on standards, which were already available to the writer. The QCA site provided data on NVQ uptake.
Key Facts and Figures 1999 HTF 1999 was used to provide factual data on NVQ qualifications uptake and completion
The writer carried out observations in order to determine the major differences that exist between the routes.
"The basic idea is that by watching the behaviour of specific respondents in controlled situations insights into their reactions can be achieved" (Goldner 1975)
A major advantage of observational study lies in their ability to help describe routines and patterns of behaviour. The emphasis of observations was to clarify the approaches of colleges and industry training providers and to validate their perception of the differences and in particular the strengths and weaknesses of both routes to NVQ delivery and assessment. Observations of delivery and assessment of NVQs were carried out at the three colleges. Industry observations were carried out at 18 different locations from small restaurants to large Hotels. The writer was aware of the potential disadvantage of observation in that they can be subjective. So observations were selected on the basis of the questionnaire responses where the writer wished to clarify and elaborate. The writer decided carry out observation in half of the establishments surveyed on a proportional basis. The opportunity was taken during observations to carry out non-directive individual interviews as these were seen by the writer as an opportunity to probe in depth the respondents attitudes and opinions. The observations were used to give greater depth to the study and to enhance the reliability and validity of the study.
Some observations and opportunities to carry out semi structure interviews were carried out by the writer in the course of her work, when the writer came across individuals or situations during the completion of the study.