Sierra Sloan and Dr. William Baker, Management Communications

The business world requires efficiency and accuracy to be profitable, and poorly written business communications can waste time and resources. Thus, any research that seeks to enhance the effectiveness of writing has the potential to yield significant personal and organizational improvement. While most writing-analysis studies examine the text directly using various textual-analysis methods, this study fills the gaps in academic research by analyzing text through the verbalized mental processing of the audience, or verbal protocol analysis (VPA), resulting in richer and more accurate conclusions. In particular, this study uses VPA to analyze the effect of typical message enhancements used in business writing: headings, agendas, direct vs. indirect structure, and visuals.

The VPA approach asks subjects to “think aloud” while completing a task, giving insights into concurrent mental processes instead of retrospective analyses, which are considered incomplete and often inaccurate. Our study has implemented this research practice by creating sets of practically identical documents—one which includes the enhancement (headings, visuals, etc.) and the other which is completely identical, except that it does not include the enhancement. One version of the document was then given to a test subject, who read through the document while speaking aloud into a recording device about any thoughts or impressions. This method allowed us to isolate the effects of the message enhancement by comparing transcriptions between readers who had the enhancements and those who did not, giving more accurate data that text analysis would permit.

Analyzing the transcriptions of the verbal protocols was largely qualitative, though attempts at quantitative analysis are still in progress. Each transcription was searched for examples of confusion, differences in text-processing techniques, positive or negative evaluations of the documents, reader analysis independent of the documents, or reader comments about the topic, structure, and (of course) message enhancements. Comparing the results from these analyses between document pairs revealed interesting conclusions as trends emerged between documents with enhancements and those without. Equally interesting were the differences found between the message enhancements; headings, structure, agendas, and visuals each had unique results:

Headings. Perhaps the enhancement that revealed starkest contrast was headings, where the documents were identical except for four headings which label the different paragraphs. Readers given headings responded very positively. They tended to jump down the document to read headings early, predicting what the document would discuss and commenting on the clarity of the overall organization. Those without headings had an opposite response; their comments revealed confusion, frustration, skimming, and overall dissatisfaction with the document.

Structure. The direct structure document begins by presenting the concluding argument, followed by the supporting arguments; the indirect structure document begins instead with the arguments, moving the conclusion to the last paragraph. The major difference in reader response was their resistance to the conclusion—when presented with the concluding argument upfront (as with the direct structure), the readers tended to be antagonistic through all supporting arguments, ending with a negative view of the solution. If the supporting arguments are presented before the conclusion, then the reader’s ultimate verdict is of agreement or neutrality—all due to a simple movement of a paragraph from the beginning to the end.

Agenda. Business communications experts consistently promote using an agenda, or a brief outline of arguments, in the first paragraph, and then restating the agenda in the conclusion. The differences between the agenda and no-agenda documents were more subtle than in other areas. Adding an agenda did not appear to add additional understanding or support of the document ideas; however, the repetition inherent in the agenda structure did cause readers to reflect more often on previous arguments, resulting in more individual analysis than shown by readers who were not presented with an agenda.

Visuals. The simple introduction of three visuals (a table and two graphs) as message enhancements resulted in major differences from the same document without the visuals. Though the visuals were occasionally confusing to readers and caused frustration, ultimately the readers who had visuals had a better understanding of the subject. Readers without visuals tended to skim through the document, skipping over important points that the visuals would have emphasized.

Though many of these results supported hypotheses about the importance of using message enhancement techniques, others simply raised more questions, such as why visuals tended to increase frustration in readers. These questions leave area for future research and improvement upon this research model, particularly regarding the documents. As the first study of its kind to specifically focus on written business communications, it clearly has room for document perfection to achieve more comprehensive results. Another question raised by the results, for example, was why did including an agenda have such an insignificant direct impact on understanding? The nature of an agenda may provide the answer—in a simple one-page, five-paragraph document, an agenda may have less of an effect than it would in a more complex piece. In future studies, varying the parameters for comparison could answer these and other questions.

The results of this study are being compiled and analyzed for an Honors thesis to be published in May. Additionally, the author and faculty sponsor hope to present at the Association for Business Communication conference in October in Montreal, Quebec. We also plan to submit the results to an academic journal in business communications