The data for this project was acquired from three sources.
requests library from URL)tweepy library)Development for the second and third methods of data gathering was undertaken using the Spyder IDE. The resulting final code was then copied over to the Jupyter Notebook where the cleaning and analyzing functions took place.
Eleven data quality issues were identified, and six tidiness issues. Quality issues concern problems of content such as when the data is incomplete, inaccurate, or unuseable in its current form. Tidiness issues concern the structure of the data in/and among tables such that each object under study is represented by a single row within a table and each attribute of interest for the objects under study is represented by a single column.
Many things were involved in cleaning up the identified data quality and tidiness issues that don't warrant reporting. One of the more challenging and interesting ones, however, concerned the "We Rate Dogs" terminology.
"We Rate Dogs" has invented its own terminology for describing dogs: 'doggo', 'floofer', 'pupper', and 'puppo'. The original twitter archive file included columns with these headings, thus treating each of them as a variable. However, under the 'doggo' column, for example, the values were either 'doggo' or 'None'. Similarly with the other three. Besides the fact that, the value 'None' is not the Pythonic way of recording a null value, it raised the question whether these dog descriptions should be treated as separate variables, or whether they should be considered as different values of a single variable -- a dog_designation variable. At issue is whether these designations are mutually exclusive. Unfortunately, some of the terms are ambiguous. (See "The Dogtionary" below.) So, whether they are mutually exclusive is ambiguous. On one hand, the words 'puppo', 'pupper', and 'doggo' are often used to designate mutually exclusive stages in the development of a dog. One might call this the developmental use of these terms such that a 'puppo' is a really young dog, a 'pupper' an adolescent, and a 'doggo' an adult. However, the term 'doggo' is not always used that way. The term 'doggo' sometimes has a generic usage, covering any dog no matter what age. With the generic usage of 'doggo' in play, something can be both a pupper and a doggo -- a 'pupper-doggo' (12 cases). Similarly, a dog can be a 'puppo-doggo' (1 case) or a 'fluffer-doggo' (1 case) when 'doggo' is used in the generic sense. To complicate matters further, some of these cases of dual designation were due to the tweet's picture containing more than one dog -- often an older "doggo" with a younger "pupper". In any case, the decision was made to combine these four designations into a single dog_designation variable and allow for hyphenated values (such as 'pupper-doggo') for those few cases where it occurs.
Notice that the definitions are often not mutually exclusive even though some suggest different (i.e., mutually exclusive) dog developmental stages. Ultimately, these terms are not meant for precise categorization but for fun and smiles.