Question

(Text Analysis) The availability of computers with string-manipulation capabilities has resulted in some rather interesting approaches to analyzing the writings of great authors. Much attention has been focused on whether William Shakespeare ever lived. Some scholars believe there is substantial evidence indicating that Christopher Marlowe or other authors actually penned the masterpieces attributed to Shakespeare. Researchers have used computers to find similarities in the writings of these two authors. This exercise examines three methods for analyzing texts with a computer. Note that thousands of texts, including Shakespeare, are available online at www.gutenberg.org. a. Write a program that reads several lines of text from the keyboard and prints a table indicating the number of occurrences of each letter of the alphabet in the text. For example, the phrase To be, or not to be: that is the question: contains one "a," two "b's," no "c's," etc. b. Write a program that reads several lines of text and prints a table indicating the number of one-letter words, two-letter words, threeletter words, etc., appearing in the text. For example, the phrase Whether 'tis nobler in the mind to suffer contains the following word lengths and occurrences:c. Write a program that reads several lines of text and prints a table indicating the number of occurrences of each different word in the text. The first version of your program should include the words in the table in the same order in which they appear in the text. For example, the lines To be, or not to be: that is the question: Whether 'tis nobler in the mind to suffer contain the words "to" three times, the word "be" two times, the word "or" once, etc. A more interesting (and useful) printout should then be attempted in which the words are sorted alphabetically.

Short answer

Implement separate programs to count letter occurrences, word lengths, and unique words, then sort words alphabetically.

Step-by-step solution

Count Occurrences of Each Letter

Write a program that reads text input from a user, then iterate character by character to count each letter's occurrences. Use a dictionary where keys are alphabet letters and values are the counts of occurrences for each letter. Loop through each character, and if it's a letter, increment its count in the dictionary. Finally, display the count for each letter.

Count Word Lengths

Create a program to read the input text and split it into words. For each word, measure its length in letters. Use a dictionary where keys represent word lengths (e.g., 'one letter', 'two letters') and values are the counts of such words. Increment the count in the dictionary for each word's length. Print the result, showing how many words there are for each length.

Count Unique Words

Develop a program that processes the text input to extract each word. Use a dictionary where the keys are unique words and the values are their occurrence counts. Split the text by spaces and punctuation, convert each word to lowercase to ensure case insensitivity, and update the dictionary accordingly. List the words based on their appearance order, not alphabetically, in your initial printout.

Sort Words Alphabetically

Take the dictionary from Step 3 and sort the words alphabetically. Use Python's `sorted()` function to sort the dictionary keys (words) and then print the words along with their occurrence counts in sorted order. This will allow users to see frequency counts while easily finding words in the sorted table.

Key concepts

String Manipulation

String manipulation refers to the process of changing, analyzing, or inspecting strings – sequences of characters. In text analysis programming, we often deal with tasks like transforming strings to lowercase to ensure uniformity, removing unwanted characters such as punctuation, or splitting strings into smaller parts, such as words or letters. These operations are foundational for analyzing textual data because they prepare the data for more complex processing tasks.

Common string manipulation techniques include:

• **Changing Cases**: Conversion of text to upper or lower case to ensure consistency during comparison.
• **Trimming**: Removal of white spaces from the beginning and end, which can help avoid errors in text analysis.
• **Splitting**: Division of a string into parts using delimiters like spaces or commas, which is especially useful for counting words or evaluating text structure.
• **Replacement**: Substituting certain text segments with others, such as removing punctuation or correcting spelling errors.

Using these techniques ensures accurate processing of text data, essential for tasks like letter frequency analysis or word counting.

Letter Frequency Analysis

Letter Frequency Analysis involves counting how often each letter appears within a given text, which can provide critical insights into the text's characteristics. For example, certain authors might have unique linguistic signatures that can be identified through this type of analysis.

The process usually follows these steps:

• **Initialization**: Create a dictionary with letters of the alphabet as keys and initialized counts as values.
• **Iteration**: Loop through each character in the text. If the character is a letter, increase its count in the dictionary.
• **Normalization**: Convert all text to the same case (usually lowercase) to ensure that the frequency count is case-insensitive.
• **Result Display**: Print the frequency of each letter, which can be used for further statistical or pattern analysis.

This type of analysis is foundational in cryptography and historical research, aiding in tasks like deciphering codes or investigating authorship questions.

Word Count Programming

Word count programming automates the process of counting words in text, which is a frequently required data point in many text analysis scenarios. By counting the occurrences of word lengths, we can better understand the complexity of the language used.

The general process involves:

• **Reading and Splitting**: Start by reading the input text and splitting it into a list of words.
• **Length Counting**: For each word, calculate the number of letters it contains. Use a dictionary where keys represent the word lengths, and their corresponding values indicate how many times those lengths occur.
• **Outputting Results**: Display the word length counts, which provides insight into the writing style, such as whether an author favors longer, more complex words or shorter, simpler ones.

Word count programming is used in numerous applications, from simple content length verification in writing software to complex linguistic research.

Unique Word Counting

Unique Word Counting focuses on identifying and tallying distinct words in text, which helps examine text complexity and diversity. This form of analysis considers both the variety of words and their frequency to offer insights into linguistic richness.

Here's a typical approach to counting unique words:

• **Data Cleansing**: Initially, process the text to remove punctuation and convert each word to lowercase, ensuring uniformity.
• **Extraction and Counting**: Split the text into words and, as you iterate through them, add each unique word to a dictionary with its occurrence count.
• **Initial and Sorted Output**: First, print words in the order they appear to offer an unaltered overview of word usage. Additionally, sort these words alphabetically to provide an organized and easy-to-reference list of word frequencies.

Unique word counting is essential in both computational linguistics and text analytics, facilitating applications like keyword extraction, author profiling, and sentiment analysis.