How to Wrangle JSON Data in R with jsonlite, purr and dplyr – Part II

See Part I to learn how to load JSON as nested named lists.

Look inside JSON lists

str(strikes, max.level = 1)

## List of 1
##  $ data:List of 2440

This tells us we have a component named “data”. Let’s look at that a little more closely:

str(strikes$data, max.level = 1, list.len = 10)

## List of 2440
##  $ :List of 40
##  $ :List of 40
##  $ :List of 40
##  $ :List of 40
##  $ :List of 40
##  $ :List of 40
##  $ :List of 40
##  $ :List of 40
##  $ :List of 40
##  $ :List of 40
##   [list output truncated]

This suggests we have homogenous lists of 40 elements each (an assumption we’ll check shortly).

Let’s look at one of those lists:

str(strikes$data[[1]])

## List of 40
##  $ ticker          : chr "SPY"
##  $ tradeDate       : chr "2020-05-19"
##  $ expirDate       : chr "2020-05-29"
##  $ dte             : int 11
##  $ strike          : int 140
##  $ stockPrice      : num 293
##  $ callVolume      : int 0
##  $ callOpenInterest: int 0
##  $ callBidSize     : int 20
##  $ callAskSize     : int 23
##  $ putVolume       : int 0
##  $ putOpenInterest : int 2312
##  $ putBidSize      : int 0
##  $ putAskSize      : int 7117
##  $ callBidPrice    : num 152
##  $ callValue       : num 153
##  $ callAskPrice    : num 153
##  $ putBidPrice     : int 0
##  $ putValue        : num 1.12e-25
##  $ putAskPrice     : num 0.01
##  $ callBidIv       : int 0
##  $ callMidIv       : num 0.98
##  $ callAskIv       : num 1.96
##  $ smvVol          : num 0.476
##  $ putBidIv        : int 0
##  $ putMidIv        : num 0.709
##  $ putAskIv        : num 1.42
##  $ residualRate    : num -0.00652
##  $ delta           : int 1
##  $ gamma           : num 9.45e-16
##  $ theta           : num -0.00288
##  $ vega            : num 2e-11
##  $ rho             : num 0.0384
##  $ phi             : num -0.0802
##  $ driftlessTheta  : num -6.07e-09
##  $ extSmvVol       : num 0.478
##  $ extCallValue    : num 153
##  $ extPutValue     : num 1.77e-25
##  $ spotPrice       : num 293
##  $ updatedAt       : chr "2020-05-19 20:02:33"

All these elements look like they can be easily handled. For instance, I don’t see any more deeply nested lists, weird missing values, or anything else that looks difficult.

So now I’ll pull out the interesting bit:

strikes <- strikes[["data"]]

How many observations do we have?

length(strikes)

## [1] 2440

Are all strike sublists identically named?

This is where we’ll check that our sublists are indeed homogeneously named, as we assumed above:

strikes %>%
  map(names) %>%  # this applies the base R function names to each sublist, and returns a list of lists with the output
  unique() %>%
  length() == 1

## [1] TRUE

Visit Robot Wealth website to learn how to make a dataframe, and to download the complete set of scripts: https://robotwealth.com/how-to-wrangle-json-data-in-r-with-jsonlite-purr-and-dplyr/.

Past performance is not indicative of future results.

Any stock, options or futures symbols displayed are for illustrative purposes only and are not intended to portray recommendations.