Design Doc: support optim() and optimize()

Owner: @fritzo

Objective

Compile R code that uses optim() and optimize(), such that the compiled function directly uses the underlying C interface.

Outline of processing flow ideas

We start with a call like (R syntax): ans <- optim(par, fn, gr, ..., method = 'Nelder-Mead', lower, upper, control, hessian)

Ideas will follow sequence of processing steps.

1. Do we need keyword processing? In general, keyword processing is necessary when we need to insert new setup code. It is sometimes convenient for simple code transformations even if we don't need new setup code, but in general I prefer to have such transformations at later steps (like size processing, which is misnamed because we do more than size processing there).

In the current prototype, keyword processing creates something like the following

{
  vPtrFor_innerNF <- voidPtr(innerNF,nimbleFunction)
  vPtrFor_yProvided <- voidPtr(ARG2_yProvided_,double(1))
  vPtrFor_2p5 <- voidPtr(Interm_6,double(=0,default=2.5))
  bareBonesOptim(initPar=ARG1_xInit_,optFun=OPTIMREADY_nfObjective,voidNimFunPtr=vPtrFor_innerNF,numAdditionalArgs=3,y=vPtrFor_yProvided,z=vPtrFor_2p5,w=vPtrFor_2p5)
}

It also creates a line of newSetupCode (I think to create an object named OPTIMREADY_nfObjective whose role is really to trigger some other steps later).

Some observations about this:

• We originally thought the "method" would have to be baked in at compile time and become a different keyword (e.g. bareBonesOptim. We can evaluate if that is necessary. We can probably have run-time flexibility for method.

• Lifting of arguments to temporary variables (voidPtrs) is typically done in size processing, not keyword processing.

• voidPtr is an example of an internal part of the DSL. It's valid, but we don't document it because there's nothing to do with it as a programmer.

• The prototype only works if the objective function is the run function of a nimbleFunction class (i.e. with setup code). We probably, ideally, want optim to work with 3-4 cases: an RCfunction (aka simple nimbleFunction, without setup code), a member function of the same class in which optim is being used, or a method of another class (potentially nested: method of a nimbleFunction contained in another nimbleFunction, optim(par, fn = A$B$foo)). The prototype works only because (using our example) keyword processing for nf2Gen$run can expect that innerNF is already populated in the symbolTable and we can look up its run symbol. But for some of the desired cases, we won't be able to find another nimbleFunction's symbolTable until size processing. E.g. if we call optim(par, fn = foo) and foo is an RCfunction, we would normally not know anything about foo during keyword processing, but we would know (or be able to find out) about it during size processing. Summary: I bet we'll want to move much (all?) of what keyword processing does in the prototype to size processing.

Tasks to be done:

1. Add a new keyword optim(-) that just aliases the identity function (optim(x) = x).

1a. Make optim become nimOptim.

1b. Try adding optim where needed to various steps. E.g. size processing. One thing not stated there is to add an entry to sizeCalls for "nimOptim".

1c. (Skip eigenization steps for now).

1d. (Can skip genCpp steps because default will be to emit nimOptim as the function name. Later if needed we can make an entry in cppOutputCalls in genCpp_generateCpp.R )

2. Define two nimbleList types: one for control, one for output of optim()

3. Make optim return its nimbleList type.

4. Let optim take first two arguments (par, fn), while assuming method = 'Nelder-Mead' (say), with a choice of what fn should be (RCfunction, or a local method, or another nimbleFunction's method). Have the size processor reconstruct steps done in keyword processing right now:

4a. create voidPtrs (but you could add them directly to the symbolTable instead of constructing code);

4b. We have a "neededTypes" system. For a nimbleFunction, neededTypes is a list of objects (e.g. nfProcessing) that represent types that will be needed for successful compilation of the current code. For an RCfunProc, instead of "NeededTypes" it is called "neededRCfuns". This should be renamed neededTypes (also for purposes of nimbleLists), and we can probably use it to record types needed for optim.