A Research Workflow
We are interested in the way research within a particular field is communicated. Given this, we want to use CORE to examine trends in publishing for a set of key terms. To avoid being completely swamped by results we will pick a topic that is a relatively small subdiscipline, and, to ensure that we understand some of the broader patterns, we will pick a discipline that we have some knowledge about. We will also try to find a discipline that is linked to an exisitng rOpenSci package, so why not choose paleoecology, and, more specifically, palynology.
Palynology is the study of organic-walled microfossils. Of these microfossils, the most well known is pollen fossilized in lake or ocean sediments. Using relative abundances of pollen can give us an indication of the composition of ancient fossils, and, handily enough, there is an R package, neotoma that allows us to explore the pollen data.
Finding articles
So, lets see how many articles there are in the CORE holdings that contain the term palynology:
The result is a list with three components: ['Status', 'totalHits', 'data']. In our case, the result for status is "OK" (as opposed to not found), the totalHits are 13175 and then there are 10 articles returned. The total number of articles is much higher than the number of articles returned because by default the limit flag for core_search is set to 10.
We can constrain our search, to look at the change in results over time by looking only for articles within certain specified time limits. For this we can use the core_advanced_search:
# Define a data.frame with a row per year bin.
query <- data.frame(
all_of_the_words = "palynology",
year_from = as.character(seq(1950, 2016)),
year_to = as.character(seq(1951, 2017)),
stringsAsFactors = FALSE)
paly_bins <- core_advanced_search(query = query, key = api_key)
plot(seq(1951, 2017), paly_bins[[2]],
xlab = "Published Year", ylab = "Articles", type = 'l')
But this mapping of a single term likely reflects both an increase in the use of the term “palynology” (coined in 1947), but also in the number of records within the CORE repository. As such we need to use some sort of control, so let’s take a very common word, for example “and”:
query <- data.frame(
all_of_the_words = "and",
year_from = as.character(seq(1950, 2016)),
year_to = as.character(seq(1951, 2017)),
stringsAsFactors = FALSE)
control_bins <- core_advanced_search(query = query, key = api_key)
plot(seq(1951, 2017), control_bins[[2]] / max(control_bins[[2]]),
xlab = "Published Year", ylab = "Articles", lty = 2, col = 'red', type = 'l')
lines(seq(1951, 2017), paly_bins[[2]] / max(paly_bins[[2]]))
Given that the number of papers for both "palynology" & "and" are well matched, it seems like this curve is largely a function of available publications. Given that "and" should represent the majority of all publications, let’s look at a set of search strings. Palynology is a branch of paleoecology. Europeans often spell paleoecology as palaeoecology. Let’s look at all three terms as a function of th "and" results. We’ll wrap it into a function to streamline things:
ann_query <- function(string, control = NULL) {
query <- data.frame(
all_of_the_words = string,
year_from = as.character(seq(1950, 2016)),
year_to = as.character(seq(1951, 2017)),
stringsAsFactors = FALSE)
hit_bins <- core_advanced_search(query = query, key = api_key)
if (is.null(control)) control = hit_bins
return(data.frame(years = seq(1951, 2017),
hits = hit_bins[[2]],
transformed = hit_bins[[2]] / control[[2]],
string = string,
stringsAsFactors = FALSE))
}
string_sets <- do.call(rbind.data.frame,
lapply(c("and", "palynology", "paleoecology", "palaeoecology"),
ann_query, control = control_bins))
string_sets <- string_sets[!string_sets$string %in% 'and',]
plot(data = string_sets,
transformed ~ years,
col = factor(string_sets$string),
pch = 19, cex = 0.5)
It’s interesting to note the overall changes in the rates of publication. In particular, the contined growth of the use of palaeoecology vs paleoecology. Given the results we’ve pulled, it’s possible to do further analysis of the publications, but this provides a simple overview of some of the capabilities of the package, and ways to generate simple analytics for search strings from the publically available papers within CORE.