Grid Search Algorithm with a Zoom

Yukai Yang

2019-01-02

zoomgrid version 1.0.0 (Red Grid)

The package implements provides the grid search algorithm with a zoom. The grid search algorithm with a zoom aims to help solving difficult optimization problem where there are many local optimizers inside the domain of the target function. It offers suitable initial or starting value for the following optimization procedure, provided that the global optimum exists in the neighbourhood of the initial or starting value. The grid search algorithm with a zoom saves time tremendously in cases with high-dimenstional arguments.

and the corresponding paper

Modelling Nonlinear Vector Economic Time Series

See section 1.5.4.

Example

After installing the package, you need to load (attach better say) it by running the code

library(zoomgrid)

You can take a look at all the available functions and data in the package

ls( grep("zoomgrid", search()) ) 
#> [1] "build_grid"        "grid_search"       "grid_search_check"

Motivation

Consider the two-dimensional Rastrigin function is a non-convex function which is widely used for testing the performances of some optimization algorithms.

where \(x_i \in [-5.12, 5.12]\) and \(A = 10\). It has many local minimum and its global minimum is at (0, 0) with the minimum value 0. Rastrigin function

Graph source: Rastrigin function @ WIKIPEDIA.

We give the function in R:

# Rastrigin function
ndim = 2 # number of dimension
nA = 10 # parameter A
# vx in [-5.12, 5.12]

# minimizer = rep(0, ndim)
# minimum = 0
Rastrigin <- function(vx) return(nA * ndim + sum(vx*vx - nA * cos(2*pi*vx)))

Then let us try the optimization algorithms available in the optim function.

# set seed and initialize the initial or starting value
set.seed(1)
par = runif(ndim, -5.12, 5.12)
cat("start from", par)
#> start from -2.401191 -1.309451

# results from different optimization algorithms
tmp1 = optim(par = par, Rastrigin, method='Nelder-Mead')
tmp2 = optim(par = par, Rastrigin, method='BFGS')
tmp3 = optim(par = par, Rastrigin, method='L-BFGS-B')
tmp4 = optim(par = par, Rastrigin, method='SANN')

tmp1$par; tmp1$value
#> [1] -1.9899136 -0.9949483
#> [1] 4.97479
tmp2$par; tmp2$value
#> [1] -0.9949586  0.9949586
#> [1] 1.989918
tmp3$par; tmp3$value
#> [1] -1.989912e+00  2.913338e-09
#> [1] 3.979831
tmp4$par; tmp4$value
#> [1] 0.97915333 0.01486102
#> [1] 1.088185

None of them are satisfactory…

Build the grid

We need to build grid first for the grid search. For details, see

?build_grid

We build the grid by running

# build the grid
bin = c(from=-5.12, to=5.12, by=.1)
grid = build_grid(bin,bin)