Using the stlcsb Package

Branson Fox and Christopher Prener

2019-02-07

Source: vignettes/stlcsb.Rmd

Overview

The stlcsb package is meant to provide easy access to the City of St. Louis Citizen’s Service Bureau data in R

Installation

We recommend that users install sf before proceeding with the installation of stlcsb. Windows users should be able to install sf without significant issues, but macOS and Linux users will need to install several open source spatial libraries to get sf itself up and running. The easiest approach for macOS users is to install the GDAL 2.0 Complete framework from Kyng Chaos.

For Linux users, steps will vary based on the flavor being used. Our configuration file for Travis CI and its associated bash script should be useful in determining the necessary components to install. Linux users will need to make sure the dependency lwgeom is also installed correctly.

Once sf is installed, the easiest way to get stlcsb is to install it from CRAN:

Alternatively, the development version of stlcsb can be accessed from GitHub with remotes:

Tips

  • All functions support non-standard evaluation, meaning you can use either quoted or unquoted inputs for arguments.
  • Use RStudio for its helpful auto completion ability. Type csb_ to see all of the functions available in this package, or cat_ to see all of the category data objects.
  • All functions return data as a tibble, which is essentially identical to a data.frame, except for its print behavior in the console and in notebooks.

Downloading CSB Data

Before using the other functions, you will need to acquire the CSB data. The csb_get_data function will download the most recent version of the data, and return it as a single tibble. By default, variable names are clean-up automatically and two variables that largely containing NA values are removed (PROBCITY and PROBZIP). Variables are also reordered in a more logical fashion.

The tidy = FALSE argument for csb_get_data() will return a tibble with both variables as well as the original variable names in the order they are disseminated by the City:

It is also possible to only import a single year or a range of years:

csb18 <- csb_get_data(years = 2018)
csb16_18 <- csb_get_data(years = 2016:2018)

Date of Last Update

These data are updated on a weekly basis, so it may be useful to know what version you are working with. The csb_last_update() function will return the date that the data were last modified as a string:

Categorization

The most important challenge with these data are the lack of intelligible categorization of Problem codes. These categories were created by hand - we selected like problem codes and created meta-categories that represented an overarching concept.

Each category is available in a named character vector that begins with cat_. They are used internally and are also exported for reference and extension:

Applying Categories

There are several functions available to represent data categorically. csb_categorize() can be used to categorize data based on its problem code:

csb_filter() can be used to filter the data based on these same categories. Category takes single or vector arguments:

> csbSub <- csb_filter(csb, var = problemcode, category = c(cat_construction, cat_debris))
>
> nrow(csb)
[1] 1160418
> nrow(csbSub)
[1] 140920

Finally, csb_vacant() is used for appending a logical vector indicating vacancy related problem codes. It is important to know that vacant exists inclusive of other categories, or in other words, a problem code will vacancy related in addition to its association with a more general category.

> csb <- csb_vacant(csb, var = problemcode, newVar = vacant)
> head(csb$vacant)
[1] FALSE FALSE FALSE FALSE FALSE FALSE

Removing Canceled Calls

csb_canceled() is used for the removal of CSB requests that were canceled. By default it deletes the column specified (in this case datecancelled) since it will contain only NA values after the data are subset.

> csbValid <- csb_canceled(csb, var = datecancelled)
>
> nrow(csb)
[1] 1160418
> nrow(csbValid)
[1] 1129055

Spatial Features

The stlcsb package also provides helpful functions for dealing with spatial data. The csb_missingXY() function is used to identify observations with incomplete or invalid spatial data:

> csb <- csb_missingXY(csb, varX = srx, varY = sry, newVar = missingXY)
> head(csb$missingXY)
[1] FALSE FALSE FALSE FALSE FALSE FALSE

The csb_projectXY is used to create a simple features object, which can then be manipulated using the sf package or plotted using any of the mapping packages for R. It is critical missing spatial data be removed prior to executing this function:

> csb %>%
+   dplyr::filter(missingXY == FALSE) %>%
+   csb_projectXY(varX = srx, varY = sry) -> csb_sf
>
> class(csb_sf)
[1] "sf"         "tbl_df"     "tbl"        "data.frame"

Time and Date

The package offers two functions for handling time and date. The csb_date_filter()function is used to subset observations for a specified time period. It is flexible in syntax, accepting 3 letter abbreviations, full month names, or a numeric argument:

csb_date_filter(csb, datetimeinit, day = 1)
csb_date_filter(csb, datetimeinit, day = 1:15, month = 1)
csb_date_filter(csb, datetimeinit, month = "January", year = 09)
csb_date_filter(csb, datetimeinit, month = c("jan", "feb", "Mar", "Apr"), year = 2009)
csb_date_filter(csb, datetimeinit, day = 1:15, month = 1:6, year = 08:13)

The csb_date_parse() is used to parse part of, or whole dates from observations. It too can delete the original date variable. Arguments are the names of the variables to be appended to the data.

csb_date_parse(csb, var = datetimeinit, day = dayInit)
csb_date_parse(csb, var = datetimeinit, day = dayInit, month = monthInit)
csb_date_parse(csb, var = datetimeinit, month = monthInit)
csb_date_parse(csb, var = datetimeinit, month = monthInit, year = yearInit)
csb_date_parse(csb, var = datetimeinit, day = dayInit, month = monthInit, year = yearInit, drop = TRUE)

Full Example: Mapping Potential Vacant Properties in Summer 2017

Data Cleaning Workflow

The following is an example of a full workflow for importing the data, filtering for vacancy indicating problem codes, filtering by time, removing observations with missing spatial data, and projecting it to an sf object. Notice that the pipe operator %>% from magrittr is used. This passes the output of the prior function to the first argument of the next function, reducing typing and making code easier to read.

csb <- csb_get_data(year = 2017)

csb %>%
  csb_filter(var = problemcode, category = cat_vacant) %>%
  csb_date_filter(var = datetimeinit, month = c("Jun", "Jul", "Aug")) %>%
  csb_missingXY(varX = srx, varY = sry, newVar = missingXY) %>%
  dplyr::filter(missingXY == FALSE) %>%
  csb_projectXY(varX = srx, varY = sry) -> vacant_sf

Mapping CSB Data

Once the data have been projected, they can be previewed with a package like mapview:

These data can also be mapped using ggplot2 once they have been projected:

Getting Help

  • If you are new to R itself, welcome! Hadley Wickham’s R for Data Science is an excellent way to get started with data manipulation in the tidyverse, which stlcsb is designed to integrate seamlessly with.
  • If you are new to spatial analysis in R, we strongly encourage you check out the excellent new Geocomputation in R by Robin Lovelace, Jakub Nowosad, and Jannes Muenchow.
  • If you have questions about using stlcsb, you are encouraged to use the RStudio Community forums. Please create a reprex before posting. Feel free to tag Chris (@chris.prener) in any posts about stlcsb.
  • If you think you’ve found a bug, please create a reprex and then open an issue on GitHub.