---
title: "Data Visualization and Descriptive Statistics"
date: "2024-01-24"
output:
  html_document: 
    toc: true
    toc_depth: 1
---

From now on, we do things inside a project to keep a clear and coherent workspace. Go to File --\> New Project (choose either a New directory or an existing one). For instance, you can choose the folder you downloaded for this session from ILIAS. You can have an overview of the folder in the output pane.

```{r setup, include=FALSE}

knitr::opts_chunk$set(echo = TRUE)

```

## Set working directory

```{r include=FALSE}

# set working directory to source file location. If you are already in an R project, you do not need this.

setwd(dirname(rstudioapi::getSourceEditorContext()$path))

getwd()

```

## Load libraries

We use the `ggplot2` package today. However, it is included in tidyverse, so we do not need to load it separately. But if you do not use tidyverse, you should install and load the library.

```{r}

library(tidyverse)

```

## Story

As you remember, you left academia and decided to invest your time and money in a dog shelter. You received the dog records from the previous owner in a very poor format.

You did several rounds of data cleaning. Then based on the cleaned data, you created content for your website, and also you created a linguistic corpus called Furry Friends Corpus.

Furthermore, since your dogs are getting more popular, you have received more funding and you managed to hire an assistant to do further analysis on your dog's lifestyle in order to improve their well-being. This is why the new dataset has lot more columns, including data about their inherent activity level, behavior, color, diet type, etc.

**Your goal for today:** Now, to help your websites visitors, you have decided to add more visual info such as plots to the website.

## Read in the data

Let's bring in the dog and corpus data into R.

```{r}

# UD corpus created by udpipe
dog_ud <- read_tsv("data/dog_ud.tsv")

# Extended dog data
dog_df <- read_tsv("data/enhanced_dog_dataset.tsv")

```

## Data Visualization

Content of this session is mostly from the book "Modern Data Visualization with R" [link](https://rkabacoff.github.io/datavis/IntroGGPLOT.html), in some cases copy-pasted.

## GGPLOT

Ggplot is a data visualization package in R. The idea behind it is to describe plots as **a collection of independent components**: data, geom, aesthetics, scales, and so on. Therefore, these components are built on top of each other and they are chained together with the symbol `+`.

**Components of a ggplot**

-   Data: The actual data (dataframe) you want to plot from.

-   Aesthetics: Mapping variables to visual properties (e.g., x-axis and y-axis on the plot).

-   Geometries (geoms): The type of plot you're making., e.g., barplot, scatter plot, etc.

-   Additional layers and customization

![](img/ggplot2_syntax.PNG){width="295"}

There are different ways of looking at the data. For instance, you can look at the distributions of data from a single variable (**univariate** graphs), or the relation between two variables (**bivariate** graphs), or the relation among several variables (**multivariate** graphs).\

Furthermore, our variables might be **categorical** (such as gender) or **numeric/quantitative/continuous** (such as age or salary).

### Univariate graphs

Univariate graphs plot the distribution of data from a single variable. For instance, if we have a "smoking" column with values "smoker" and "non-smoker", it plots how many of the observations in this column are smoker and how many are non-smoker. The variable we are plotting can be categorical (e.g., race, sex, political affiliation) or quantitative (e.g., age, weight, income).

**Story**: Since you're super proud of the diversity in your shelter, like the different breeds, colors, sizes, and all, you want to show it off somehow. That's why you've decided to **make a graph that shows how many dogs of each color you have.** You can use a bar chart for this purpose. However, before that, let's use the `table()` function just to get an idea of what we have.

```{r table}

table(dog_df$dog_color)

```

#### Creating a Bar chart

The first function in building a graph is the `ggplot()` function.

It specifies the data frame to be used and the **mapping** of the variables to the visual properties of the graph.

The mappings are placed within the `aes` function, which stands for aesthetics.

The first thing we have to do is to map our desired data to a ggplot map. In this case, it would be the data from column "color" of the dog_df dataframe.

It is a univariate plot, because we are plotting the distribution of a single variable. Therefore, we only map the x column to the graph.

```{r ggsyntax}

ggplot(data = dog_df,
       mapping = aes(x = dog_color)) ## we use aes for mapping. Here you see the full syntax [mapping = aes()], but you can omit "mapping" and keep only the aes() function.

```

Why is the graph empty?

We specified that the "dog_color" variable should be mapped to the x-axis, but we haven't yet specified what we type of plot we want to have.

**GEOMS**

Geoms are the geometric objects (points, lines, bars, etc.) that can be placed on a graph. They are added using functions that start with `geom_` such as `geom_bar`, `geom_point`, etc. For now, we want to put bar on the x-axis.

IMPORTANT: In ggplot2 graphs, functions are chained together using the plus `+` sign to build a final plot. Therefore, we can chain the `ggplot` function to `geom_bar`

```{r simple-barplot}

ggplot(dog_df, aes(x = dog_color)) + 
  geom_bar()

```

**Story**: The graph has useful information, but it is not attractive enough for the website. You decide to make some changes to it to make it more applealing and informative to the visitors.

There are a couple of things we can change in this graph. For instance:

-   changing the color and transparency of the bars.
-   changing the color of the bar borders.
-   Changing the graph title and axis labels.
-   Changing width of Bars
-   Changing the orientation of the bar (horizontal vs. vertical)

Check this comprehensive reference: [link](https://ggplot2.tidyverse.org/reference/)

[colorLink](https://sape.inf.usi.ch/quick-reference/ggplot2/colour)

We start by changing the color (`fill`), transparency (`alpha`), the border color (`color`), and width (`width`) of the graphs. These changes will be made to the bars, therefore, there will be added as parameters to the bar_geom function.

```{r purple-barplot}

ggplot(dog_df, aes(x = dog_color)) + 
  geom_bar(fill = "purple",  alpha = .3, color = "black", width = .3)

# The bars are filled with purple color (fill = "purple"), semi-transparent (alpha = .3), with black borders (color = "black"), and have a width of 0.3 (width = .3).

```

As you can see, our plot has an ugly gray background. We can set different themes for our plot, for instance `theme_minimal()`. Themes are a set of visual design elements for a plot. They define the overall appearance of the plot, like background color, grid lines, and font styles.

```{r barplot-theme}

ggplot(dog_df, aes(x = dog_color)) + 
  geom_bar(fill = "purple",  alpha = .3, color = "black", width = .3) + 
  theme_minimal()

```

We can also change the labels of the x and y axis, and give a title to our graph. We can use the function `lab()` to change the main title, subtitle, axis labels, caption, etc.

```{r}

ggplot(dog_df, aes(x = dog_color)) + 
  geom_bar(fill = "purple", alpha = .3, color = "black", width = .3) +
  labs(x = "Color of the dogs",
       y = "Frequency", 
       title = "Color distribution of our dogs", 
       subtitle = "Bar plot", 
       caption = "This bar plot shows the color distribution of the dogs in our shelter. As can be seen, brown dogs are the most common ones in the shelter.")


# It is always always a good practice to add good and informative captions to your graphs. Don't be afraid of making them long.

```

Let's look at another categorical variable with more categories.

**Story**: While you are working on the visualization of your website, you also work on the linguistic paper you are writing with your colleague Maria about the Furry Friends Corpus. For that, you are interested to know the distribution of different part of speech tags in the dog descriptions provided in the corpus. For that, we can look at the distribution of part of speech tags in the "upos" column in the "dog_ud" dataframe. Let's plot it first:

```{r}

ggplot(dog_ud, aes(x = upos)) + 
  geom_bar(fill = "lightseagreen")

```

Here, we got the count of POS tags from the upos column (x = upos) and plotted them.

Another way to plot: we first create a frequency dataframe (pos_freq) in which we store the frequency of each postag in a new column. Then, we map the POS tags from the upos column to the x-axis and the counts from the count column to the y-axis. Let's do it (also we can practice a bit of group-by, summarise this way.)

```{r}

# In what follows, we group the data by the variable upos and count the number of rows in each upos

pos_freq <- dog_ud %>% 
  group_by(upos) %>%  
  summarise(., count= n()) %>% 
  ungroup()

```

Now, we can plot the data from the pos_freq dataframe. This time, for the mapping, we map the upos info to the x-axis, and the count info to the y-axis. Then, we tell to our plot that we got the counts from a column and it does not need to count them. The option `stat="identity"` placed in `geom_bar`tells the plotting function not to calculate counts, because they are supplied directly.

```{r}

ggplot(pos_freq, 
       aes(x = upos, y = count)) + 
  geom_bar(stat="identity", fill = "lightcoral", width = 0.5) + 
  labs(x = "POS tags", 
       y = "Frequency", 
       title  = "POS tag distributions in the Furry Friends corpus")

```

It looks good, but not too good. To improve the readability, we may want to:

1.  Order the values in an ascending order. For this, we can use the `reorder` function to sort the categories by the frequency (in ascending order).

```{r}

ggplot(pos_freq, 
       aes(x = reorder(upos, count), y = count)) + 
  geom_bar(stat="identity", fill = "lightcoral", width = 0.5)


```

Any idea how to make it descending?

2.  Change the orientation of the graph (from vertical to horizontal). We use the function `coord_flip` for this purpose.

```{r}

ggplot(pos_freq, 
       aes(x = reorder(upos, count), y = count)) + 
  geom_bar(stat="identity", fill = "lightcoral", width = 0.5) +
  coord_flip()

```

3.  Add the actual counts to the bar plots. By `geom_text()` we can add text directly to the plot.

```{r}

ggplot(pos_freq, aes(x = reorder(upos, count), y = count)) +
  geom_bar(stat = "identity", fill = "lightcoral") +
  geom_text(aes (label = count), vjust = -0.3) # This means that the value of the labels are coming from the count column. 

```

And to make every category look more distinctive, let's give each category a different color. Also, we should not forget to change the labels.

```{r}

ggplot(pos_freq, aes(x = reorder(upos, count), y = count, fill = upos)) +
  geom_bar(stat = "identity") +
  geom_text(aes (label = count), vjust = -0.3) +
  labs(x = "POS tags", y = "Frequency", title = "POS tag frequency in Furry Friends Corpus" )

# fill = "upos": This sets the fill color of the bars based on the upos categories. Each unique upos value will be represented by a different color.

```

#### Saving Graphs

**Story**: As mentioned earlier, we want to use the created POS tag plot in our paper which is written in a word document. Therefore, we need to save our plot first as an image (e.g., png).

Graphs can be saved via the RStudio interface (less preferred) or through code (much more preferred).

To save the graph via R Studio, go to the Plots panels --\> Export --\> save as image. There, you can choose the right height and width for the image.

To save it via code, you can use the `ggsave` function.

```{r}

ggplot(pos_freq, aes(x = reorder(upos, count), y = count, fill = upos)) +
  geom_bar(stat = "identity") +
  geom_text(aes (label = count), vjust = -0.3) +
  labs(x = "POS tags", y = "Frequency", title = "POS tag frequency in Furry Friends Corpus" )


ggsave(filename = "plot/test_pos_plot.png")

ggsave(filename = "plot/pos_plot.png",
       width = 3000, 
       height = 1200, units = "px")

```

Note: by default, ggsave saves the last plot you have created. However, you can specify a different plot by using the plot argument.

Any ggplot2 graph can be saved as an object. Then you can use the ggsave function to save the graph to disk. It is a good practice to save your plot as an object and then pass its name to ggsave.

```{r}

posPlot <- ggplot(pos_freq, aes(x = reorder(upos, count), y = count, fill = upos)) +
  geom_bar(stat = "identity") +
  geom_text(aes (label = count), vjust = -0.3) +
  labs(x = "POS tags", y = "Frequency", title = "POS tag frequency in Furry Friends Corpus" )

ggsave(posPlot, filename = "plot/pos_plot.png",
       width = 3000, 
       height = 1200, units = "px")

```

#### Creating a Histogram

Histograms are the most common approach to visualizing a quantitative (numeric) variable. In a histogram, the values of a variable are typically divided up into adjacent, equal width ranges (called bins), and the number of observations in each bin is plotted with a vertical bar. For instance, if our dogs' age is from 0 (new born) to 12 years old, then we might decide to group them into 4 bins: 0-3, 3-6, 6-9, 9-12.

**Story**: You are deeply committed to the well-being of your dogs and ensure they receive regular daily exercise. You believe that by showcasing this practice to a broader audience, you can achieve two goals: (1) inspire other shelters to increase the frequency of exercise for their dogs, and (2) attract families who prioritize the well-being of their pets. Consequently, you have decided to plot the extent of your dogs' exercise on a histogram and display it on your website.

```{r}

# plot the excercise time distribution using a histogram

ggplot(dog_df, aes(x = daily_exercise_time_min)) + 
  geom_histogram(fill = "blue", color = "black") +
  labs(title = "Histogram of Daily Exercise Time",
       x = "Daily Exercise Time (minutes)",
       y = "Count of Dogs") 

```

Two parameters are important in histograms: number of bins (`bins`) and range of bins (`binwidth`). Bins controls the number of bins into which the numeric variable is divided (i.e., the number of bars in the plot). The default is 30, but it is helpful to try smaller and larger numbers to get a better impression of the shape of the distribution.

```{r}

ggplot(dog_df, aes(x = daily_exercise_time_min)) + 
  geom_histogram(fill = "blue", bins = 10, color = "black") +
  labs(title = "Histogram of Daily Exercise Time",
       x = "Daily Exercise Time (minutes)",
       y = "Count of Dogs")


```

Alternatively, you can specify the binwidth, the width of the bins represented by the bars.

```{r}

ggplot(dog_df, aes(x = daily_exercise_time_min)) +
  geom_histogram(fill = "blue", binwidth = 10, color = "black") + 
  labs(title = "Histogram of Daily Exercise Time",
       subtitle = "number of bins = 10",
       x = "Daily Exercise Time (minutes)",
       y = "Count of Dogs")

```

The histogram shows a somewhat irregular distribution of daily exercise times (not a normal distribution which would be symmetrical, and not a skewed one which would have a tail on one side.)

The most common exercise time seems to fall between 25 to 50 minutes, as this bin has the highest count of dogs.

There is considerable variability in exercise times as the times range from less than 25 minutes to over 100 minutes. The bins representing the shortest (\<25 minutes) and longest (\>100 minutes) exercise times are less populated.

Before we move on, another thing you can change is the scale shown on the x-axis and y-axis using **scales**. Scales control how variables are mapped to the visual characteristics of the plot. Scale functions (which start with scale\_) allow you to modify this mapping.

```{r}
ggplot(dog_df, aes(x = daily_exercise_time_min)) +
  geom_histogram(fill = "blue", binwidth = 10, color = "black") + 
  labs(title = "Histogram of Daily Exercise Time",
       subtitle = "number of bins = 10",
       x = "Daily Exercise Time (minutes)",
       y = "Count of Dogs") +
  scale_x_continuous(breaks = seq(0,130,10)) 

# scale_x_continuous(breaks = seq(0,130,10)) tells the graph that the values on the x-axis are continuous. They are a sequence of numbers from 0 to 130 and we have breaks every 10 units. This means you will see marks on the x-axis at 0, 10, 20, 30, 40, 50, 60, etc. 

## Can you change the scale for the y axis?

```

### Bivariate Graphs

Bivariate graphs display the relationship between two variables. So, they are good for answering the question "what is the relationship between A and B". The type of the graphs depends on the measurement level of each variable (categorical or quantitative).

#### Categorical vs. Categorical variables

When plotting the relationship between two categorical variables, stacked, grouped, or segmented bar charts are typically used.

**Stacked bar chart**

**Story**: You are very concerned about the respective adopters for your dogs. You think if you illustrate the connection between the dogs' behavioral patterns and their activity levels, it will aid future owners in finding a compatible pet by providing insights into the dogs' general behavior and energy needs. For this reason, you decide to plot the correlation between dog's behavior and their level of activity. You can use a bar plot in a stacked mode for this purpose.

```{r}
# stacked bar chart

table(dog_df$activity_level)
table(dog_df$behavior)

ggplot(dog_df, aes(x = behavior , fill = activity_level)) + 
  geom_bar(position = "stack") 

```

The graph is a stacked bar chart that displays the relationship between the behavior and activity level of the dogs. There are five categories of behavior. Each behavior category has three associated activity levels: high, moderate, and low, represented by different colors (high in red, low in green, and moderate in blue) in each bar.

We can for instance see in this example that playful dogs belong only to the category of dogs with high or moderate activity. This is a quite expected outcome. Also, we see that shy dogs have dominantly low activity.

**Grouped bar chart** Grouped bar charts place bars for the second categorical variable side-by-side. To create a grouped bar plot use the position = "dodge" option.

```{r}

ggplot(dog_df, aes(x = behavior, fill = activity_level)) + 
  geom_bar(position = "dodge")  

```

This is a grouped bar chart. The activity levels are color-coded and represented as "high" (red), "low" (green), and "moderate" (blue). Each behavior category has three bars corresponding to these activity levels, showing the count for each.

#### Categorical vs. Quantitative variables

This is one of the most common types of correlation, and there are many different types of graphs to plot such a correlation. Box plot is a popular option. A boxplot displays the 25th percentile, median, and 75th percentile of a distribution. The whiskers (vertical lines) captures the spread of the bulk of the data, excluding the potential outliers.

![](img/boxPlot.JPG){width="300"}

**Story**: We want to explore the relationship between a dog's activity level and its daily exercise time. This analysis would be beneficial for understanding how the inherent activity level of dogs (categorized as low, moderate, or high) correlates with the actual amount of exercise they get each day. This could inform on whether dogs with higher activity levels are indeed getting more exercise. If not, we should find a solution.

We can use a box plot to visualize the distribution of daily exercise time (daily_exercise_time_min) across different activity levels (activity_level). We use `geom_boxplot` for that.

```{r}

ggplot(dog_df, aes(x = activity_level, y = daily_exercise_time_min, fill = activity_level)) +
  geom_boxplot(alpha = .6) +
  labs(title = "Box Plot of Daily Exercise Time by Activity Level",
       x = "Activity Level",
       y = "Daily Exercise Time (minutes)") +
  theme_minimal() 


```

The bottom and top of the boxes represent the first (Q1) and third (Q3) quartiles, respectively, indicating that the middle 50% of the data falls within this range.

![](img/Quartiles-and-quantiles-What-are-quartiles.jpg){width="220"}

The horizontal line inside the box indicates the median, which is the middle value of the data. The whiskers extend from the box to the smallest and largest values within 1.5 times the interquartile range (IQR) from the Q1 and Q3 [(Q3 - Q1) \* 1.5]. Points outside this range are considered outliers and are represented as dots.

Dogs with a high activity level have the widest IQR range. The median exercise time is around 60 minutes. The IQR for the other two boxes is narrower, showing less variability in the exercise time they get. For the low activity, we see this crazy outlier (shown by a dot), meaning that one of the dogs from the low activity level gets radically more exercise. I think this is the owner's favorite and the owner takes it with her everywhere :-D

Furthermore, it seems the moderate and the low group have similar median. This could be a concern, as the moderate group perhaps needs more exercise than the low activity group.

#### Quantitative vs. Quantitative variables

The simplest display of two quantitative continuous variables is a scatterplot, with each variable represented on an axis.

**Story**: You have different assistants in the shelter and some take dogs outside for a walk more than others. You have noticed some differences in the sleeping pattern of your dogs and you are not sure where it comes from. However, one of your speculations is that the amount of daily exercise has a positive impact on the hours the dogs sleep, i.e., you expect the dogs who exercise more also sleep more at night (since they are tired.) We plot these trends using a scatterplot (`geom_point`). It displays points at the coordinates determined by the values of the x and y variables (in this case, sleep hours and daily exercise time). Each point on the plot represents one dog from the dataset.

```{r}

ggplot(dog_df, aes(x = daily_exercise_time_min, y = sleep_amount_hours)) +
  geom_point() +
  labs(title = "Scatter Plot of Dog Sleeping Hours vs. Daily Exercise Time",
       x = "Daily Exercise Time (minutes)",
       y = "Sleep Hours (hours)") +
  theme_minimal()

```

It is often useful to summarize the relationship displayed in the scatterplot, using a best fit line. Many types of lines are supported, including linear, polynomial, and nonparametric.

```{r}

ggplot(dog_df, aes(x = daily_exercise_time_min, y = sleep_amount_hours)) +
  geom_point() +
  labs(title = "Scatter Plot of Dog Sleeping Hours vs. Daily Exercise Time",
       x = "Daily Exercise Time (minutes)",
       y = "Sleep Hours (hours)") +
  theme_minimal() +
  geom_smooth(method = "lm")

# geom_smooth(method = "lm") adds a linear regression line to the scatter plot, showing the average linear relationship between daily exercise time and sleep hours in dogs. The line added to the plot represents the best-fit linear relationship between these two variables.

## Default value is loess (Locally Estimated Scatterplot Smoothing): it creates a smooth line through a scatter plot by fitting simple models to localized subsets of the data. It is non-parametric, meaning it does not not assume a specific functional form or shape for the relationship between variables. 

```

### Multivariate Graphs

Sometimes, we are interested to study the relationship between several variables. Multivariate graphs display the relationships among three or more variables. There are two common methods for accommodating multiple variables: grouping and faceting.

#### Grouping

In grouping, the values of the first two variables are mapped to the x and y axes. Then additional variables are mapped to other visual characteristics such as color, shape, size, line type, and transparency. Grouping allows you to plot the data for multiple groups in a single graph.

For grouping, we can introduce the other variables by the change in their color, shape, etc.

```{r}

ggplot(dog_df, aes(x = daily_exercise_time_min, 
                   y = sleep_amount_hours,
                   shape = activity_level)) +
  geom_point() +
  labs(title = "Dog Sleeping Hours by Daily Exercise Time and Activity Level",
       x = "Daily Exercise Time (minutes)",
       y = "Sleep Hours (hours)") +
  theme_minimal() 

## instead of color, you can use shape or size. Also, you may have more variables for which you want to use other visual cues.

```

#### Faceting

Grouping allows you to plot multiple variables in a single graph, using visual characteristics such as color, shape, and size. In faceting, a graph consists of several separate plots, one for each level of a third variable.

```{r}

ggplot(dog_df, aes(x = dog_color, fill = dog_color)) + 
  geom_bar() +
  facet_wrap(~sex, ncol = 2) +
  labs(title = "Bar plot of Dogs' Color by Sex",
       x = "Dogs' Color",
       y = "Count of Dogs") 

# For multiple categorical variables
#facet_grid(sex ~ activity_level)

```

The facet_wrap function creates a separate graph for each level of sex variable. Therefore, you will have one graph for male and one graph for female dogs. Each plot shows the distribution of dog colors for either gender.

The `ncol` option controls the number of columns. If you put it on 1, then it means that it will show both plots in the same column. If you set it to 2, they will be shown in two adjacent columns.

### Plot themes

You can have different plot themes, such as barbie (or many others actually).

```{r}

#remotes::install_github("MatthewBJane/ThemePark")

## https://github.com/MatthewBJane/ThemePark

library(ThemePark)

```

```{r}

ggplot(dog_df, aes(x = daily_exercise_time_min, y = sleep_amount_hours)) +
  geom_point(color = "lightseagreen") +
  labs(title = "Scatter Plot of Dog Sleeping Hours vs. Daily Exercise Time",
       x = "Daily Exercise Time (minutes)",
       y = "Sleep Hours (hours)") +
  geom_smooth(method = "lm", color = "purple")+
  theme_friends()

```

#### Advice/Best practices

[link](https://rkabacoff.github.io/datavis/Advice.html)

## STORY:

You are very pleased by the plots you added to your website, and you have already noticed their effect. You have this feeling that people who visit your shelter for adopting your lovely dogs are more informed and ask more specific inquiries about your dogs. However, to provide a more comprehensive overview (also because some weird people like numeric values more than visual graphs), you decide to also provide comprehensive descriptive statistics about your dogs.

## Descriptive statistics

Descriptive statistics summarize and describe the main features of a dataset. Two major components are:

-   *Measures of Central Tendency:* values that represent the center point or typical values of the dataset.

-   **Mean**: The average of the data, calculated by adding up all the values and dividing by the number of values.

-   **Median**: The middle value in the data set when it is ordered from smallest to largest. If there is an even number of values, the median is the average of the two middle values.

-   **Mode**: The most frequently occurring value in the dataset. There can be more than one mode in a dataset.

-   *Measures of Spread:* Values that describe the amount of variation or dispersion in the dataset.

-   **Range**: The difference between the highest and lowest values in the dataset.

-   **Interquartile Range (IQR)**: Measures the spread of the middle 50% of the data.

-   **Variance**: The average of the squared differences from the Mean.

-   **Standard Deviation (SD)**: A measure of the amount of variation or dispersion of a set of values. It is the square root of the variance and it measures the dispersion from the mean of your data. Interpreting SD is more straightforward than interpreting variance (the unit of the measurement of variance is different from the original unit in your dataset because of squaring. SD brings back the unit to its original version).

### Measuring these values in R

You can use the functions `mean`, `median`, `sd` to easily calculate these values for your numeric columns.

```{r}

mean_excercise = mean(dog_df$daily_exercise_time_min)
print(str_c("Mean excercice time in minutes: ", round(mean_excercise, digits = 2)))

sd_excercise = sd(dog_df$daily_exercise_time_min)

sd_excercise

#This standard deviation indicates the extent of variation around the mean exercise time. A standard deviation of 28.5 minutes suggests that the individual dog's daily exercise times tend to vary by about 28.5 minutes from the mean exercise time of 53.22 minutes. Also, while a significant portion of the data will be within one standard deviation of the mean, there will certainly be values that lie outside this range (more than one standard deviation away from the mean).


median_excercise = median(dog_df$daily_exercise_time_min)
# This implies that 50% of your dogs receive more than 52 min of sport.

```

### Using group-by and summarise

You can use a combination of group-by and summarise for creating summary statistics of your grouped data.

```{r}

summary_dogs <- dog_df %>% 
  group_by(sex) %>%
  summarise(mean_excercise = mean(daily_exercise_time_min),
            mean_sleep = mean(sleep_amount_hours),
            max_excercise = max(daily_exercise_time_min)) %>% 
  ungroup()

summary_dogs


```

### using the summarytools package

summarytools provides a coherent set of functions centered on data exploration and simple reporting. With this package, you can also mention in which format you want your data to be outputted (plain-text, markdown, html). [link](https://cran.r-project.org/web/packages/summarytools/vignettes/introduction.html).

#### Function descr()

descr() generates comprehensive descriptive / univariate statistics, i.e. common central tendency statistics and measures of dispersion. It accepts single columns as well as data frames; in the latter case, all non-numerical columns are ignored.

```{r}
#install.packages("summarytools")
library(summarytools)

# Getting summary statistics from all numeric columns in the dataframe
df_descr <- descr(dog_df) %>% 
  as.data.frame()

df_descr

# Getting summary statistics for specific columns

exercise_descr <- descr(dog_df$daily_exercise_time_min, style = "rmarkdown")

## NOTE FOR LATEX USERS: If you use latex, you can create nice tables directly from R using the xtable package.
# install.packages("xtable")
# xtable(exercise_descr)
library(xtable)
xtable(exercise_descr)

```

This function output many different types of descriptive statistics, which you do not necessarily need them all. You can specifically specify the ones you are interested in:

```{r}

descr(dog_df$daily_exercise_time_min,
      stats     = c("mean", "sd", "min", "max"),
      transpose = TRUE,
      headings  = FALSE)

```

Alternatively, you can simply say that you want the "common" statistics.

```{r}

descr(dog_df$daily_exercise_time_min,
      stats     = "common",
      transpose = FALSE,
      headings  = FALSE)

```

#### Function freq()

The freq() function generates frequency tables with counts, proportions, as well as missing data information. Therefore, it is basically used for categorical columns to see the frequency distribution of different values in a column.

```{r}

behavior_freq <- freq(dog_df$behavior, plain.ascii = TRUE, style = "simple")

behavior_freq

behavior_df <- as.data.frame(behavior_freq)

```

The thing is that there is too much info in the table, and you do not need all of them (for instance, you do not really need the cumulative values.) You can simplify the output.

```{r}

behavior_freq_simple <- freq(dog_df$behavior, 
                             plain.ascii = TRUE, 
                             style = "simple",
                             report.nas = FALSE, 
                             totals     = FALSE, 
                             cumul      = FALSE, 
                             headings   = FALSE)


behavior_freq_simple

```

Also, sometimes, for instance when you want to report the distribution of let's say POS tags, you may have too many values and you are not necessarily interested in all those values. You can group the values with lower frequency together in an "other" group.

```{r}

table(dog_ud$upos)

freq(dog_ud$upos, 
     order    = "freq",
     rows     = 1:10,
     headings = FALSE,
     cumul      = FALSE)

```

#### Function ctable()

This one is the coolest function from this package. ctable() generates cross-tabulations (joint frequencies) for pairs of categorical variables. You could achive the same by group-by and summarise.

```{r}
ctable(x = dog_df$sex, 
       y = dog_df$activity_level, 
       prop = "n")   # Show raw distributions

#The table can show different proportions. If you set it to "n", it shows the raw distributions. "c" shows column-wise distributions, "r" shows row-wise distributions, and "t" gives the total distributions [this one is a bit confusing].
```

```{r}

ctable(x = dog_df$sex, 
       y = dog_df$activity_level, 
       prop = "c")   # Show column proportions

```

According to the above table, 11 dogs have high activity level, of which 54.5% are female and 45.5% are male.

```{r}

ctable(x = dog_df$sex, 
       y = dog_df$activity_level, 
       prop = "r")   # Show row proportions


```

According to this table, 13 dogs are female: 6 (46.2%) have high, 3 have low and 4 have moderate activity levels.

```{r}
with(dog_df, 
     print(ctable(x = behavior, 
                  y = activity_level, 
                  prop     = 'r',
                  totals   = FALSE, 
                  headings = FALSE),
           method = "browser")
)

```
