Evaluating a Design: The Written Answer

This is not the only answer you could write, but it focuses clearly on weaknesses, comes up with possible solutions, and then discusses any further problems the solution creates.

The first possible weakness with my design is validity. Semi-structured interviews can create a problem of interviewer bias, as the interviewer can influence the respondent, particularly in the unstructured parts of the interview. One way of minimising this would be to add a second method, such as a questionnaire, with a similar sample to check the validity of the responses. This would be using triangulation to check validity. However, this would cause other problems. The advantage of the unstructured sections, that the respondent could talk about what is meaningful to them, would be lost if there were fixed questions in a questionnaire.

A second problem with my design is reliability. In a semi-structured interview, parts of the interview are reliable, where all respondents are asked the same questions. But the unstructured parts are very difficult to repeat, as respondets are not asked the same questions, and may talk about very different issues. This could be solved by making the interview more structured, so that all respondents were asked the same questions. However, this would involve a "trade-off" with validity - removing the unstructured parts of the interview would make the interview more reliable, but less valid, as respondents could not talk about issues that were meaningful to them.

A third problem with my design is representativeness/generalisability. I have chosen four schools, and this may be too few to include all different sorts of young people. Because my sample is quite small, it is not representative of all young people, and I have also chosen to concentrate on one age group to make it easier to gain access. I could make the sample larger by including more schools and colleges, or by including more than 10 pupils in each school. However, this would also make it more time-consuming to carry out the interviews and code the findings. I could also try to include younger and older young people, as they may be involved in different kinds of music-making. For example, younger children might be more likely to join clubs or groups, but less likely to form bands with other young people. Again, this would make the research more time-consuming, and my method is not completely reliable.

There are also two practical problems with my design. I have already said that it is time-consuming to carry out this research. Making the method more structured would make it quicker to code, but this would involve some sacrifice in validity. I could also have problems of access, if a gatekeeper (the Head) refused to allow access to a school. I could approach other schools instead. However, this would make the sample less representative, as the schools were chosen for the type of pupils they contained.

The last problem I could have is whether I have operationalised the concept of informal music-making correctly. Have I included the main forms of informal music-making in which young people are involved? To test this out, I could use a pilot study to see whether my questions make sense to young people, and whether semi-structured interviews are the best method. If I found that my respondents did not have much to say in the unstructured parts of the interview, I might want to change the method.

My design is interpretivist. I have chosen to make validity my main concern. Positivists would not be happy with my design, as they have different criteria for evaluating research. They would prefer to use a method which is more reliable and a larger sample, as part of a more quantitative approach to studying this topic. However, I would have to use methods such as questionnaires or structured interviews to achieve this. Interpretivists would argue that this would not be a valid method of finding out what informal music-making means to young people. There is no perfect design, and the objections of positivists to this design cannot be overcome without stepping outside the brief I was given.