rorg — Code evaluation in org-mode, with an emphasis on R

We (optionally) incorporate the text output as text in the target document

We either link to the graphics or (html/latex) include them inline.

? We link to other file output

We bear this in mind

Using some version of the code block ideas that Eric and Austin have worked on. (In addition, an aim of org-R was to allow Org users who are not R users to specify R code implicitly, using native org syntax. I'd like to maintain that, but it's not central to this project.)

Let's use an asterisk to indicate content which includes the result of code evaluation, rather than the code itself. Clearly we have a requirement for the following transformation:

org → org*

Let's say this transformation is effected by a function `org-eval-buffer'. This transformation is necessary when the target format is org (say you want to update the values in an org table, or generate a plot and create an org link to it), and it can also be used as the first step by which to reach html and latex:

org → org* → html

org → org* → latex

Thus in principle we can reach our 3 target formats with `org-eval-buffer', `org-export-as-latex' and `org-export-as-html'.

An extra transformation that we might want is

org → latex

I.e. export to latex without evaluation of code, in such a way that R code can subsequently be evaluated using Sweave(driver=RweaveLatex), which is what the R community is used to. This would provide a `bail out' avenue where users can escape org mode and enter a workflow in which the latex/noweb file is treated as source.

• How do we implement `org-eval-buffer'?
  

  AIUI The following can all be viewed as implementations of org-eval-buffer for R code:

  • org-eval-light
    This is the beginnings of a general evaluation mechanism, that could evaluate python, ruby, shell, perl, in addition to R. The header says it's based on org-eval, what is org-eval??
  • org-R
    This accomplishes org → org* in elisp by visiting code blocks and evaluating code using ESS.
  • RweaveOrg
    This accomplishes org → org* using R via

    Sweave("file-with-unevaluated-code.org", driver=RweaveOrg, syntax=SweaveSyntaxOrg)
    

  • org-exp-blocks.el
    Like org-R, this achieves org → org* in elisp by visiting code blocks and using ESS to evaluate R code.

Here we have to consider text/numeric output, and graphical output. And also the stage at which evaluation occurs

• org → org*
  
  • Text / numerical output
    In the case of org → org*, I would argue that, where appropriate, it should be stored in org tables. Thus an advantage our project would have over Sweave is that tabular output is automatically conveqrted to native tables on export to HTML and latex.
  • Graphical output
    We place an org link to the file. This is done already by org-R-apply, and by RweaveOrg.
• latex → latex*
  This is done by Sweave(driver=RweaveLatex) and so is out of our hands

• naive
  Naive implementation would be to use (org-export-table "tmp.csv") and (ess-execute "read.csv('tmp.csv')").
• org-R
  org-R passes data to R from two sources: org tables, or csv files. Org tables are first exported to a temporary csv file using org-R-export-to-csv.
• org-exp-blocks
  org-exp-blocks uses org-interblock-R-command-to-string to send commands to an R process running in a comint buffer through ESS. org-exp-blocks has no support for dumping table data to R process, or vice versa.
• RweaveOrg
  NA

When R code evaluation generates vectors and 2-dimensional arrays, this should be formatted appropriately in org buffers (orgtbl-mode) as well as in export targets (html, latex)

Agreed, if we can convert the vector data to lists then we can use the many orgtbl-to-* functions to convert the list to whatever output format we desire. See `orgtbl-to-orgtbl, `orgtbl-to-latex', `orgtbl-to-html', `orgtbl-to-csv', etc…

• Implementations
  
  • org-R
    org-R converts R output (vectors, or matrices / 2d-arrays) to an org table and stores it in the org buffer, or in a separate org file (csv output would also be perfectly possible).
  • org-exp-blocks
    
  • RweaveOrg
    

R can generate graphical output on a screen graphics device (e.g. X11, quartz), and in various standard image file formats (png, jpg, ps, pdf, etc). When graphical output is generated by evaluation of R code in Org, at least the following two things are desirable:

1. output to screen for immediate viewing is possible
2. graphical output to file is linked to appropriately from the org file This should have the automatic consequence that it is included appropriately in subsequent export targets (html, latex).

• Implementations
  
  • org-R
    org-R does (1) if no output file is specified and (2) otherwise
  • org-exp-blocks
    org-exp-blocks tries to do 2, but I don't think that part was every really working
  • RweaveOrg
    

I (Eric) propose that we use the syntax of source code blocks as they currently exist in org-mode with the addition of evaluation, header-arguments, exportation, single-line-blocks, and references-to-table-data.

1. evaluation: These blocks can be evaluated through \C-c\C-c with a slight addition to the code already present and working in org-eval-light.el. All we should need to add for R support would be an appropriate entry in org-eval-light-interpreters with a corresponding evaluation function. For an example usinga org-eval-light see * src block evaluation w/org-eval-light.
2. header-arguments: These can be implemented along the lines of Austin's header arguments in org-sweave.el.
3. exportation: Should be as similar as possible to that done by Sweave, and hopefully can re-use some of the code currently present in org-exp-blocks.el.
4. single-line-blocks: It seems that it is useful to be able to place a single line of R code on a line by itself. Should we add syntax for this similar to Dan's #+R: lines? I would lean towards something here that can be re-used for any type of source code in the same manner as the #+begin_src R blocks, maybe #+src_R?
5. references-to-table-data: I get this impression that this is vital to the efficient use of R code in an org file, so we should come up with a way to reference table data from a single-line-block or from an R source-code block. It looks like Dan has already done this in org-R.el.

What do you think? Does this accomplish everything we want to be able to do with embedded R source code blocks?

• src block evaluation w/org-eval-light
  here's an example using org-eval-light.el

  first load the org-eval-light.el file

  <elisp:(load (expand-file-name "org-eval-light.el" (expand-file-name "existing_tools" (file-name-directory buffer-file-name))))>

  then press \C-c\C-c inside of the following src code snippet. The results should appear in a comment immediately following the source code block. It shouldn't be too hard to add R support to this function through the `org-eval-light-interpreters' variable.

  (Dan: The following causes error on export to HTML hence spaces inserted at bol)

  #+begin_src shell date #+end_src

Org has an extremely useful method of editing source code and examples in their native modes. In the case of R code, we want to be able to use the full functionality of ESS mode, including interactive evaluation of code.

Source code blocks look like the following and allow for the special editing of code inside of the block through `org-edit-special'.

,## hit C-c ' within this block to enter a temporary buffer in r-mode.

,## while in the temporary buffer, hit C-c C-c on this comment to
,## evaluate this block
a <- 3
a

,## hit C-c ' to exit the temporary buffer

dblocks are useful because org-mode will automatically call `org-dblock-write:dblock-type' where dblock-type is the string following the #+BEGIN: portion of the line.

dblocks look like the following and allow for evaluation of the code inside of the block by calling \C-c\C-c on the header of the block.

In developing RweaveOrg, Austin created org-sweave.el. This allows for the kind of blocks shown in testing.Rorg. These blocks have the advantage of accepting options to the Sweave preprocessor following the #+BEGIN_R declaration.