Saturday, January 31, 2009

Week-4 Readings

What distinguishes quantitative from qualitative designs? What is the difference between validity and reliability, and what is meant by the term "probability" and "significance"?

Quantitative vs. Qualitative Research Designs

Both the quantitative and qualitative research designs are empirical research designs used in social sciences and recently adopted in technical and business communication research projects. As empirical research designs, they undertake research on certain inconsistencies or problems through inductive processes. However, these two methods, as the names suggest, are significantly different in terms of the processes they use and the nature of the conclusion they draw. The nomenclatures “quantitative” and “qualitative,” unlike what we initially tend to predict, are quite misleading. The essential difference of these methods is not the use of quantity or numbers. Both the methods use numbers or quantities, though in varying degrees. But, what distinguishes them is that quantitative research is experimental and qualitative research is descriptive.
As quantitative research design is experimental, it divides the variables into two equal groups, one control and the other experimental (sometimes also called independent and dependent). Then the researcher applies certain treatment to the experimental group and observes the changes in it compared to the control group due to the treatment applied on it. So, the second important element of quantitative research is comparison. Next important aspect of it is its emphasis on cause and effect relationship. It tries to establish that the change has occurred as a result of the treatment applied and, therefore, the treatment is the cause for the effect the experiment displays. The other two features are random sampling (randomization) of the subjects and the generalizability of the conclusion of this research, though to be used with some caution. One major drawback of this method is the isolation of the variables from the other circumstances, which makes this method unnatural. Its result may not be applicable to real situations.
Qualitative research design is basically about “process and description” of a certain phenomenon or situation. It is a thus a descriptive research. The researcher observes a specific situation and “identifies key variables in a given situation” (Goubril, p. 584). Unlike, quantitative research, it does not divide subjects into two groups, rather studies some situation and the environment around it or the subjects to find solution to specific research problem. Unlike in quantitative research method, research subjects are not randomly chosen, rather the researcher tries to choose the most representative one(s), and even if sampling is done, it is purposeful sampling. In this research, no treatment is applied; the subjects are observed and studied in the natural(aistic) situation. Due to the absence of randomization of subjects and treatment, the researcher is required to provide complete description of the situation, the modality of data collection, the position and the role of the researcher, management and recording of data, and the strategies of data analysis in qualitative research method. Equally important is the difference in these two methods of the participation of the researcher as an active observer in qualitative research and a detached analyst in quantitative research. One important notion in qualitative research is triangulation (comparison of the information gathered at different times, analysis of the data by different researchers, and the use of different perspectives to analyze data), which is used to establish validity of qualitative research. The benefit of this method is the reflection of real situation, whereas its drawback is the less possibility of generalization in comparison to the quantitative research.
Now, for example, let us assume that most of the teachers, students, and the employees do not make optimum use of Blackboard system of Clemson University. If a quantitative research is to be done, the researcher should begin with a certain hypothesis, for instance, the reason behind this underuse of Blackboard system is the lack of training. Then the researcher first makes random sampling (in this case randomly choosing faculty, students, and employees), divides them into independent variables (group) and dependent variables (group) and gives training to the dependent/experimental group and sees the result. If the participants of training start making more use of the system than the other group, the hypothesis works (but probability measurement and other factors have to be considered).
If one is to conduct a qualitative research of the same phenomenon, he/she needs to observe few representative students and faculties (or employees) in their workplace, take interviews, and conduct surveys to explore into the nature of the problem. Here, unlike in the previous example, comparison, treatment, or sampling (?) are not used. Through observation, interviews, and surveys, the researcher tries to describe the phenomenon and identify the nature of the problem.

Validity and Reliability

These two concepts are central to empirical research design. They are concerned with measurement. Though they are related, there are certain differences between them. Validity is concerned with accuracy: how much accurately the research methods measure what they intend to measure. So, its focus is on the truthfulness of the result. Reliability, on the other hand, concerns the consistency of the result in different times. It refers to the issue of how much does the result of the test remains the same if the same “measures were applied and reapplied under precise replication of conditions” (Williams, p. 22). For instance, we can see how much the “speak test” conducted to measure the level of oral language competency of the foreign students measures their real capacity. Or similarly, how much does a TOEFL test measure the language skills. If it exactly reflects the language ability of the candidate or the subject, the test has good validity and if there is very little correspondence, it does not have good validity. In the above example, if the same test is administered again in the same situation, if it gives similar result, it is reliable. But here, the influence of the practice or the familiarity of the test has to be kept null.
Hence, validity concerns the truthfulness of the result of the measurement; reliability concerns the consistency. Williams distinguishes these concepts as follows: “While there are many and more complex approaches to assessing validity and reliability than this illustration provides, the ideas that evaluation of validity requires some type of outside standard, and that evaluation of reliability requires some way of comparing a measure with itself, remain basic considerations” (p.23). In other words, to check the validity, the result is measured against the external factor where as to check the reliability, one test is compared to the other similar test.

Probability and Significance

Probability and significance are related to statistical analysis. Probability is used literally to mean the probable or possible occurrence of certain event or phenomenon. Most common example is that of tossing a coin and predicting the probability of turning head or tail. If it is tossed once, then the probability of turning the head is 0.5 and the same is the case with the tail. The hundred percent probability (certainty) is taken as 1 and other values of probabilities remain between 0 to 1. The notion of probability is mostly used when conducting quantitative research and testing hypothesis. There are two kinds of hypothesis, research hypothesis and null hypothesis. And research hypothesis is regarded “valid” or “proven” when the probability of its alternative hypothesis (null hypothesis) is below certain preset level (generally 0.05). So, the probability of the research hypothesis being true or closer to true is high when the probability of null hypothesis being possible is below, e.g., 0.05. This level can vary in different cases.
“Significance” is quite a related term to probability. To reject the null hypothesis, certain level of probability has to be made a criterion. For instance, as stated in the previous paragraph, assume that the probability level of 0.05 is taken as sufficient for rejecting null hypothesis and verifying research hypothesis, this level is called “rejection region or the significance level” (Williams, p. 61). Williams defines it as “a level of probability set by the researcher as grounds for the rejection of the null hypothesis” (Williams, p. 61).

Thursday, January 22, 2009

Kenneavy’s Modes and Week-3 Readings

Kenneavy’s division of discourses into four modes is both quite useful and equally problematic. Her division helps us to see the organization of most discourses we confront. The discourses we read are either narrative, descriptive, classificatory, or evaluative. For instance, if we read a story about Obama’s success, it is a narrative; his personality, descriptive; Obama as a democratic president, classificatory; and an editorial about his good and bad qualities, evaluative. However, the more we try to see the real writings from this perspective, the more we become baffled by the difficulty to categorize them into any of the modes.
The writings in our reading list pose us a difficult question about whether questioning Kenneavy’s modes or the modes used by actual writings. We cannot directly apply Kenneavy to the writings. The dominant mode of Hackos pieces or that of Garrett is obvious. These writings follow what Kenneavy calls the classificatory mode as they define and classify the concepts and ideas they present without evaluating any prior concepts or narrating the history of those concepts (user experience or task analysis). But again, are they not descriptive? They are descriptive as they take certain individual examples and describe the features or characteristics of them to clarify the concepts. Nevertheless, they primarily define and explain the general notion about user experience and task analysis.
However, as we try to see Miller’s and Plato’s essays from the perspective of Kenneady’s model, we are puzzled. Miller’s essay is quite straightforward; however, its mode is neither dominantly classificatory nor evaluative. It is surely not predominantly descriptive nor is it narrative. We may argue that her article is classificatory as classification is about definition and classification and her article too answers the question (requiring definition) “What is Practical?” and distinguishes low form of the “practical” from its high form. She is redefining the idea of the “practical” in technical communication. Similarly, equally dominant mode in the essay is evaluative as she evaluates the view of some technical writing scholars who define it as instrumental. Nevertheless, I don’t believe that classification and evaluation capture the dominant mode of the essay.
One problem with me is that I have not read Kenneavy’s book dealing with the division of discourses in terms of aims and audiences. Kenneavy claims that argumentation cannot be a mode as it is an aim of a discourse. Then, what about evaluation? Kenneavy her (him) self says that evaluative discourse may narrate, but we should see its “ulterior purpose,” which is “only to show how good or bad the action or performance was.” Kenneavy is bringing aim while she is discussing the modes. Therefore, I believe that the traditional category of “argumentation” is still valid and necessary to talk of the large number of writings produced in the academia. Almost all of the academic writings (research writings, from dissertations to journal articles) fall into this category. Argumentation is a mode, a way of presenting ideas or opinions; it does have certain structure or organization: a problem statement and hypothesizing, review, substantiating of the hypothesis through reasons and evidences, (addressing counterargument), and summing up.
Plato’s is another writing that challenges this division. Kenneavy says modes deal with what the discourse is about: being or becoming. What is “Phaedrus” about? Is it about change? Yes it is. Socrates changes the notion of Phaedrus about love and rhetoric. So, it is narration. But at the same time, does it have description and classification, or even evaluation? Yes, it does integrate all the modes. We can easily argue that its dominant mode is classification rather than narration: Plato is classifying and later defining love and rhetoric: bad rhetoric corresponding emotional love; neutral rhetoric corresponding non-lover; and noble rhetoric corresponding noble lover.
It seems that the dominant mode of discourse in all of the writings in the reading list is classification, either defining or classifying or comparing and contrasting or combining them. Then a question arises: among them who are successful and who are not so? This is a very complex question. What makes a mode of classification successful? Pure classification? Certainly not in all of the cases. I believe it all depends on various other factors from the audience to the context.

Monday, January 12, 2009

Welcome to my blog