Divide and Recombine for Large, Complex Data


[Up] [Top]

Documentation for package ‘datadr’ version 0.8.6.1

Help Pages

A B C D F G H K L M N P R S T U misc

datadr-package datadr

-- A --

addData Add Key-Value Pairs to a Data Connection
addSplitAttrs Functions used in divide()
addTransform Add a Transformation Function to a Distributed Data Object
adult "Census Income" Dataset
applyTransform Apply transformation function(s)
as.data.frame.ddf Turn 'ddf' Object into Data Frame
as.list.ddo Turn 'ddo' / 'ddf' Object into a list

-- B --

bsv Construct Between Subset Variable (BSV)
bsvInfo Accessor Functions

-- C --

calculateMoments Functions to Compute Summary Statistics in MapReduce
charFileHash Character File Hash Function
combCollect "Collect" Recombination
combDdf "DDF" Recombination
combDdo "DDO" Recombination
combineMoments Functions to Compute Summary Statistics in MapReduce
combineMultipleMoments Functions to Compute Summary Statistics in MapReduce
combMean Mean Recombination
combMeanCoef Mean Coefficient Recombination
combRbind "rbind" Recombination
condDiv Conditioning Variable Division
convert Convert 'ddo' / 'ddf' Objects
counters Accessor Functions

-- D --

datadr datadr
ddf Instantiate a Distributed Data Frame ('ddf')
ddf-accessors Accessor methods for 'ddf' objects
ddo Instantiate a Distributed Data Object ('ddo')
ddo-ddf-accessors Accessor Functions
ddo-ddf-attributes Managing attributes of 'ddo' or 'ddf' objects
dfSplit Functions used in divide()
digestFileHash Digest File Hash Function
divide Divide a Distributed Data Object
divide-internals Functions used in divide()
drAggregate Division-Agnostic Aggregation
drBLB Bag of Little Bootstraps Transformation Method
drFilter Filter a 'ddo' or 'ddf' Object
drGetGlobals Get Global Variables and Package Dependencies
drGLM GLM Transformation Method
drHexbin HexBin Aggregation for Distributed Data Frames
drJoin Join Data Sources by Key
drLapply Apply a function to all key-value pairs of a ddo/ddf object
drLM LM Transformation Method
drPersist Persist a Transformed 'ddo' or 'ddf' Object
drQuantile Sample Quantiles for 'ddf' Objects
drRead.csv Data Input
drRead.csv2 Data Input
drRead.delim Data Input
drRead.delim2 Data Input
drRead.table Data Input
drSample Take a Sample of Key-Value Pairs Take a sample of key-value Pairs
drSubset Subsetting Distributed Data Frames

-- F --

flatten "Flatten" a ddf Subset

-- G --

getAttribute Managing attributes of 'ddo' or 'ddf' objects
getAttributes Managing attributes of 'ddo' or 'ddf' objects
getAttributes.ddf Managing attributes of 'ddo' or 'ddf' objects
getAttributes.ddo Managing attributes of 'ddo' or 'ddf' objects
getBsv Construct Between Subset Variable (BSV)
getBsvs Construct Between Subset Variable (BSV)
getCondCuts Get names of the conditioning variable cuts
getKeys Accessor Functions
getSplitVar Extract "Split" Variable(s)
getSplitVars Extract "Split" Variable(s)

-- H --

hasAttributes Managing attributes of 'ddo' or 'ddf' objects
hasAttributes.ddf Managing attributes of 'ddo' or 'ddf' objects
hasExtractableKV Accessor Functions
hdfsConn Connect to Data Source on HDFS

-- K --

kvApply Apply Function to Key-Value Pair
kvExample Accessor Functions
kvPair Specify a Key-Value Pair
kvPairs Specify a Collection of Key-Value Pairs

-- L --

length.ddo Accessor Functions
localDiskConn Connect to Data Source on Local Disk
localDiskControl Specify Control Parameters for MapReduce on a Local Disk Connection

-- M --

makeExtractable Take a ddo/ddf HDFS data object and turn it into a mapfile
moments2statistics Functions to Compute Summary Statistics in MapReduce
mr-summary-stats Functions to Compute Summary Statistics in MapReduce
mrExec Execute a MapReduce Job

-- N --

names.ddf Accessor methods for 'ddf' objects
NCOL Accessor methods for 'ddf' objects
ncol Accessor methods for 'ddf' objects
NCOL-method Accessor methods for 'ddf' objects
ncol-method Accessor methods for 'ddf' objects
NROW Accessor methods for 'ddf' objects
nrow Accessor methods for 'ddf' objects
NROW-method Accessor methods for 'ddf' objects
nrow-method Accessor methods for 'ddf' objects

-- P --

print.ddo Print a "ddo" or "ddf" Object
print.kvPair Print a key-value pair
print.kvValue Print value of a key-value pair

-- R --

readHDFStextFile Experimental HDFS text reader helper function
readTextFileByChunk Experimental sequential text reader helper function
recombine Recombine
removeData Remove Key-Value Pairs from a Data Connection
rhipeControl Specify Control Parameters for RHIPE Job
rrDiv Random Replicate Division

-- S --

setAttributes Managing attributes of 'ddo' or 'ddf' objects
setAttributes.ddf Managing attributes of 'ddo' or 'ddf' objects
setAttributes.ddo Managing attributes of 'ddo' or 'ddf' objects
setupTransformEnv Set up transformation environment
splitRowDistn Accessor methods for 'ddf' objects
splitSizeDistn Accessor Functions
summary.ddf Accessor methods for 'ddf' objects
summary.ddo Accessor methods for 'ddf' objects

-- T --

tabulateMap Functions to Compute Summary Statistics in MapReduce
tabulateReduce Functions to Compute Summary Statistics in MapReduce
to_ddf Convert dplyr grouped_df to ddf

-- U --

updateAttributes Update Attributes of a 'ddo' or 'ddf' Object

-- misc --

%>% Pipe data