|
|
@@ -114,7 +114,7 @@ table, allowing the test suite to be run be evaluation of the table
|
|
|
and the results to be collected in the same table.
|
|
|
|
|
|
|
|
|
-* Tasks [22/42]
|
|
|
+* Tasks [25/42]
|
|
|
** TODO add a function to jump to a source-block by name
|
|
|
I've had an initial stab at that in org-babel-find-named-block
|
|
|
(library-of-babel branch).
|
|
|
@@ -122,1250 +122,1252 @@ and the results to be collected in the same table.
|
|
|
At the same time I introduced org-babel-named-src-block-regexp, to
|
|
|
match src-blocks with srcname.
|
|
|
|
|
|
-** TODO Create objects in top level (global) environment [4/5]
|
|
|
-*sessions*
|
|
|
-
|
|
|
-*** initial requirement statement [DED]
|
|
|
- At the moment, objects created by computations performed in the
|
|
|
- code block are evaluated in the scope of the
|
|
|
- code-block-function-body and therefore disappear when the code
|
|
|
- block is evaluated {unless you employ some extra trickery like
|
|
|
- assign('name', object, env=globalenv()) }. I think it will be
|
|
|
- desirable to also allow for a style wherein objects that are
|
|
|
- created in one code block persist in the R global environment and
|
|
|
- can be re-used in a separate block.
|
|
|
-
|
|
|
- This is what Sweave does, and while I'm not saying we have to be
|
|
|
- the same as Sweave, it wouldn't be hard for us to provide the same
|
|
|
- behaviour in this case; if we don't, we risk undeservedly being
|
|
|
- written off as an oddity by some.
|
|
|
-
|
|
|
- IOW one aspect of org-babel is that of a sort of functional
|
|
|
- meta-programming language. This is crazy, in a very good
|
|
|
- way. Nevertheless, wrt R I think there's going to be a lot of value
|
|
|
- in providing for a working style in which the objects are stored in
|
|
|
- the R session, rather than elisp/org buffer. This will be a very
|
|
|
- familiar working style to lots of people.
|
|
|
+** TODO support for working with =*Org Edit Src Example*= buffers [1/4]
|
|
|
+*** TODO set buffer-local-process variables appropriately [DED]
|
|
|
+ I think something like this would be great. You've probably
|
|
|
+already thought of this, but just to note it down: it would be really
|
|
|
+nice if org-babel's notion of a buffer's 'session/process' played
|
|
|
+nicely with ESS's notion of the buffer's session/process. ESS keeps
|
|
|
+the current process name for a buffer in a buffer-local variable
|
|
|
+ess-local-process-name. So one thing we will probably want to do is
|
|
|
+make sure that the *Org Edit Src Example* buffer sets that variable
|
|
|
+appropriately. [DED]
|
|
|
|
|
|
- There are no doubt a number of different ways of accomplishing
|
|
|
- this, the simplest being a hack like adding
|
|
|
+I had not thought of that, but I agree whole heartedly. [Eric]
|
|
|
|
|
|
-#+begin_src R
|
|
|
-for(objname in ls())
|
|
|
- assign(objname, get(objname), envir=globalenv())
|
|
|
-#+end_src
|
|
|
+Once this is done every variable should be able to dump regions into
|
|
|
+their inferior-process buffer using major-mode functions.
|
|
|
|
|
|
-to the source code block function body. (Maybe wrap it in an on.exit() call).
|
|
|
+*** TODO some possible requests/proposed changes for Carsten [2/3]
|
|
|
+ While I remember, some possible requests/proposed changes for Carsten
|
|
|
+ come to mind in that regard:
|
|
|
|
|
|
-However this may deserve to be thought about more carefully, perhaps
|
|
|
-with a view to having a uniform approach across languages. E.g. shell
|
|
|
-code blocks have the same semantics at the moment (no persistence of
|
|
|
-variables across code blocks), because the body is evaluated in a new
|
|
|
-bash shell process rather than a running shell. And I guess the same
|
|
|
-is true for python. However, in both these cases, you could imagine
|
|
|
-implementing the alternative in which the body is evaluated in a
|
|
|
-persistent interactive session. It's just that it's particularly
|
|
|
-natural for R, seeing as both ESS and org-babel evaluate commands in a
|
|
|
-single persistent R session.
|
|
|
+**** DONE Remap C-x C-s to save the source to the org buffer?
|
|
|
+ I've done this personally and I find it essential. I'm using
|
|
|
+#+begin_src emacs-lisp
|
|
|
+(defun org-edit-src-save ()
|
|
|
+ "Update the parent org buffer with the edited source code, save
|
|
|
+the parent org-buffer, and return to the source code edit
|
|
|
+buffer."
|
|
|
+ (interactive)
|
|
|
+ (let ((p (point)))
|
|
|
+ (org-edit-src-exit)
|
|
|
+ (save-buffer)
|
|
|
+ (org-edit-src-code)
|
|
|
+ (goto-char p)))
|
|
|
|
|
|
-*** sessions [Eric]
|
|
|
+(define-key org-exit-edit-mode-map "\C-x\C-s" 'org-edit-src-save)
|
|
|
+#+end_src
|
|
|
+ which seems to work.
|
|
|
|
|
|
-Thanks for bringing this up. I think you are absolutely correct that we
|
|
|
-should provide support for a persistent environment (maybe called a
|
|
|
-*session*) in which to evaluate code blocks. I think the current setup
|
|
|
-demonstrates my personal bias for a functional style of programming
|
|
|
-which is certainly not ideal in all contexts.
|
|
|
+I think this is great, but I think it should be implemented in the
|
|
|
+org-mode core
|
|
|
+**** TODO Rename buffer and minor mode?
|
|
|
+ Something shorter than *Org Edit Src Example* for the buffer
|
|
|
+ name. org-babel is bringing org's source code interaction to a
|
|
|
+ level of maturity where the 'example' is no longer
|
|
|
+ appropriate. And if further keybindings are going to be added to
|
|
|
+ the minor mode then maybe org-edit-src-mode is a better name than
|
|
|
+ org-exit-edit-mode.
|
|
|
|
|
|
-While the R function you mention does look like an elegant solution, I
|
|
|
-think we should choose an implementation that would be the same across
|
|
|
-all source code types. Specifically I think we should allow the user to
|
|
|
-specify an optional *session* as a header variable (when not present we
|
|
|
-assume a default session for each language). The session name could be
|
|
|
-used to name a comint buffer (like the *R* buffer) in which all
|
|
|
-evaluation would take place (within which variables would retain their
|
|
|
-values --at least once I remove some of the functional method wrappings
|
|
|
-currently in place-- ).
|
|
|
+ Maybe we should name the buffer with a combination of the source
|
|
|
+ code and the session. I think that makes sense.
|
|
|
|
|
|
-This would allow multiple environments to be used in the same buffer,
|
|
|
-and once this setup was implemented we should be able to fairly easily
|
|
|
-implement commands for jumping between source code blocks and the
|
|
|
-related session buffers, as well as for dumping the last N commands from
|
|
|
-a session into a new or existing source code block.
|
|
|
+ [ES] Are you also suggesting a new org-edit-src minor mode?
|
|
|
+ [DED] org-exit-edit-mode is a minor mode that already exists:
|
|
|
|
|
|
-Please let me know if you foresee any problems with this proposed setup,
|
|
|
-or if you think any parts might be confusing for people coming from
|
|
|
-Sweave. I'll hopefully find some time to work on this later in the
|
|
|
-week.
|
|
|
+ Minor mode installing a single key binding, "C-c '" to exit special edit.
|
|
|
|
|
|
-*** can functional and interpreted/interactive models coexist?
|
|
|
+ org-edit-src-save now has a binding in that mode, so I guess all
|
|
|
+ I'm saying at this stage is that it's a bit of a misnomer. But
|
|
|
+ perhaps we will also have more functionality to add to that minor
|
|
|
+ mode, making it even more of a misnomer. Perhaps something like
|
|
|
+ org-src-mode would be better.
|
|
|
|
|
|
-Even though both of these use the same =*R*= buffer the value of =a=
|
|
|
-is not preserved because it is assigned inside of a functional
|
|
|
-wrapper.
|
|
|
+**** DEFERRED a hook called when the src edit buffer is created
|
|
|
+This should be implemented in the org-mode core
|
|
|
+
|
|
|
|
|
|
-#+srcname: task-R-sessions
|
|
|
-#+begin_src R
|
|
|
-a <- 9
|
|
|
-b <- 21
|
|
|
-a + b
|
|
|
-#+end_src
|
|
|
+*** DEFERRED send code to inferior process
|
|
|
+Another thought on this topic: I think we will want users to send
|
|
|
+chunks of code to the interpreter from within the *Org Edit Src*
|
|
|
+buffer, and I think that's what you have in mind already. In ESS that
|
|
|
+is done using the ess-eval-* functions. [DED]
|
|
|
|
|
|
-#+srcname: task-R-same-session
|
|
|
-#+begin_src R
|
|
|
-a
|
|
|
-#+end_src
|
|
|
+I think we can leave this up to the major-mode in the source code
|
|
|
+buffer, as almost every source-code major mode will have functions for
|
|
|
+doing things like sending regions to the inferior process. If
|
|
|
+anything we might need to set the value of the buffer local inferior
|
|
|
+process variable. [Eric]
|
|
|
|
|
|
-This functional wrapper was implemented in order to efficiently return
|
|
|
-the results of the execution of the entire source code block. However
|
|
|
-it inhibits the evaluation of source code blocks in the top level,
|
|
|
-which would allow for persistence of variable assignment across
|
|
|
-evaluations. How can we allow *both* evaluation in the top level, and
|
|
|
-efficient capture of the return value of an entire source code block
|
|
|
-in a language independent manner?
|
|
|
+*** TODO optionally evaluate header references when we switch to =*Org Edit Src*= buffer
|
|
|
+That seems to imply that the header references need to be evaluated
|
|
|
+and transformed into the target language object when we hit C-c ' to
|
|
|
+enter the *Org Edit Src* buffer [DED]
|
|
|
|
|
|
-Possible solutions...
|
|
|
-1) we can't so we will have to implement two types of evaluation
|
|
|
- depending on which is appropriate (functional or imperative)
|
|
|
-2) we remove the functional wrapper and parse the source code block
|
|
|
- into it's top level statements (most often but not always on line
|
|
|
- breaks) so that we can isolate the final segment which is our
|
|
|
- return value.
|
|
|
-3) we add some sort of "#+return" line to the code block
|
|
|
-4) we take advantage of each languages support for meta-programming
|
|
|
- through =eval= type functions, and use said to evaluate the entire
|
|
|
- blocks in such a way that their environment can be combined with the
|
|
|
- global environment, and their results are still captured.
|
|
|
-5) I believe that most modern languages which support interactive
|
|
|
- sessions have support for a =last_result= type function, which
|
|
|
- returns the result of the last input without re-calculation. If
|
|
|
- widely enough present this would be the ideal solution to a
|
|
|
- combination of functional and imperative styles.
|
|
|
+Good point, I heartily agree that this should be supported [Eric]
|
|
|
|
|
|
-None of these solutions seem very desirable, but for now I don't see
|
|
|
-what else would be possible.
|
|
|
+(or at least before the first time we attempt to evaluate code in that
|
|
|
+buffer -- I suppose there might be an argument for lazy evaluation, in
|
|
|
+case someone hits C-c ' but is "just looking" and not actually
|
|
|
+evaluating anything.) Of course if evaluating the reference is
|
|
|
+computationally intensive then the user might have to wait before they
|
|
|
+get the *Org Edit Src* buffer. [DED]
|
|
|
|
|
|
-Of these options I was leaning towards (1) and (4) but now believe
|
|
|
-that if it is possible option (5) will be ideal.
|
|
|
+I fear that it may be hard to anticipate when the references will be
|
|
|
+needed, some major-modes do on-the-fly evaluation while the buffer is
|
|
|
+being edited. I think that we should either do this before the buffer
|
|
|
+is opened or not at all, specifically I think we should resolve
|
|
|
+references if the user calls C-c ' with a prefix argument. Does that
|
|
|
+sound reasonable? [Eric]
|
|
|
|
|
|
-**** (1) both functional and imperative evaluation
|
|
|
-Pros
|
|
|
-- can take advantage of built in functions for sending regions to the
|
|
|
- inferior process
|
|
|
-- retains the proven tested and working functional wrappers
|
|
|
+Yes [Dan]
|
|
|
|
|
|
-Cons
|
|
|
-- introduces the complication of keeping track of which type of
|
|
|
- evaluation is best suited to a particular context
|
|
|
-- the current functional wrappers may require some changes in order to
|
|
|
- include the existing global context
|
|
|
+** TODO resolve references to other org buffers/files
|
|
|
+ This would allow source blocks to call upon tables, source-blocks,
|
|
|
+ and results in other org buffers/files.
|
|
|
+
|
|
|
+ See...
|
|
|
+ - [[file:lisp/org-babel-ref.el::TODO%20allow%20searching%20for%20names%20in%20other%20buffers][org-babel-ref.el:searching-in-other-buffers]]
|
|
|
+ - [[file:lisp/org-babel.el::defun%20org-babel%20find%20named%20result%20name][org-babel.el#org-babel-find-named-result]]
|
|
|
+** TODO resolve references to other non-org files
|
|
|
+ - tabular data in .csv, .tsv etc format
|
|
|
+ - files of interpreted code: anything stopping us giving such files
|
|
|
+ similar status to a source code block?
|
|
|
+ - Would be nice to allow org and non-org files to be remote
|
|
|
+** TODO figure out how to handle errors during evaluation
|
|
|
+ R has a try function, with error handling, along the lines of
|
|
|
+ python. I bet ruby does too. Maybe more of an issue for functional
|
|
|
+ style; in my proposed scripting style the error just gets dumped to
|
|
|
+ the org buffer and the user is thus alerted.
|
|
|
+** TODO figure out how to handle graphic output
|
|
|
+This is listed under [[* graphical output][graphical output]] in out objectives.
|
|
|
|
|
|
-**** (4) exploit language meta-programming constructs to explicitly evaluate code
|
|
|
-Pros
|
|
|
-- only one type of evaluation
|
|
|
+This should take advantage of the =:results file= option, and
|
|
|
+languages which almost always produce graphical output should set
|
|
|
+=:results file= to true by default. That would handle placing these
|
|
|
+results in the buffer. Then if there is a combination of =silent= and
|
|
|
+=file= =:results= headers we could drop the results to a temp buffer
|
|
|
+and pop open that buffer...
|
|
|
|
|
|
-Cons
|
|
|
-- some languages may not have sufficient meta-programming constructs
|
|
|
+** TODO Finalise behaviour regarding vector/scalar output
|
|
|
+*** DONE Stop spaces causing vector output
|
|
|
+This simple example of multilingual chaining produces vector output if
|
|
|
+there are spaces in the message and scalar otherwise.
|
|
|
|
|
|
-**** (5) exploit some =last_value= functionality if present
|
|
|
+[Not any more]
|
|
|
|
|
|
-Need to ensure that most languages have such a function, those without
|
|
|
-will simply have to implement their own similar solution...
|
|
|
+#+begin_src R :var msg=msg-from-python
|
|
|
+paste(msg, "und R", sep=" ")
|
|
|
+#+end_src
|
|
|
|
|
|
-| language | =last_value= function |
|
|
|
-|------------+-----------------------------|
|
|
|
-| R | .Last.value |
|
|
|
-| ruby | _ |
|
|
|
-| python | _ |
|
|
|
-| shell | see [[* last command for shells][last command for shells]] |
|
|
|
-| emacs-lisp | see [[* emacs-lisp will be a special case][special-case]] |
|
|
|
+#+resname:
|
|
|
+: org-babel speaks elisp y python und R
|
|
|
|
|
|
-#+srcname: task-last-value
|
|
|
-#+begin_src ruby
|
|
|
-82 + 18
|
|
|
+#+srcname: msg-from-python
|
|
|
+#+begin_src python :var msg=msg-from-elisp
|
|
|
+msg + " y python"
|
|
|
#+end_src
|
|
|
|
|
|
-***** last command for shells
|
|
|
-Do this using the =tee= shell command, and continually pipe the output
|
|
|
-to a file.
|
|
|
-
|
|
|
-Got this idea from the following [[http://linux.derkeiler.com/Mailing-Lists/Fedora/2004-01/0898.html][email-thread]].
|
|
|
-
|
|
|
-suggested from mailing list
|
|
|
-
|
|
|
-#+srcname: bash-save-last-output-to-file
|
|
|
-#+begin_src sh
|
|
|
-while read line
|
|
|
-do
|
|
|
- bash -c "$line" | tee /tmp/last.out1
|
|
|
- mv /tmp/last.out1 /tmp/last.out
|
|
|
-done
|
|
|
-#+end_src
|
|
|
-
|
|
|
-another proposed solution from the above thread
|
|
|
-
|
|
|
-#+srcname: bash-save-in-variable
|
|
|
-#+begin_src sh
|
|
|
-#!/bin/bash
|
|
|
-# so - Save Output. Saves output of command in OUT shell variable.
|
|
|
-OUT=`$*`
|
|
|
-echo $OUT
|
|
|
+#+srcname: msg-from-elisp
|
|
|
+#+begin_src emacs-lisp :var msg="org-babel speaks"
|
|
|
+(concat msg " elisp")
|
|
|
#+end_src
|
|
|
|
|
|
-and another
|
|
|
-
|
|
|
-#+begin_quote
|
|
|
-.inputrc:
|
|
|
-"^[k": accept-line
|
|
|
-"^M": " | tee /tmp/h_lastcmd.out ^[k"
|
|
|
+** TODO re-implement helper functions from org-R
|
|
|
+*** Initial statement [Eric]
|
|
|
+ Much of the power of org-R seems to be in it's helper functions for
|
|
|
+ the quick graphing of tables. Should we try to re-implement these
|
|
|
+ functions on top of org-babel?
|
|
|
|
|
|
-.bash_profile:
|
|
|
-export __=/tmp/h_lastcmd.out
|
|
|
+ I'm thinking this may be useful both to add features to org-babel-R and
|
|
|
+ also to potentially suggest extensions of the framework. For example
|
|
|
+ one that comes to mind is the ability to treat a source-code block
|
|
|
+ like a function which accepts arguments and returns results. Actually
|
|
|
+ this can be it's own TODO (see [[* source blocks as functions][source blocks as functions]]).
|
|
|
+*** Objectives [Dan]
|
|
|
+ - We want to provide convenient off-the-shelf actions
|
|
|
+ (e.g. plotting data) that make use of our new code evaluation
|
|
|
+ environment but do not require any actual coding.
|
|
|
+*** Initial Design proposal [Dan]
|
|
|
+ - *Input data* will be specified using the same mechanism as :var
|
|
|
+ references, thus the input data may come from a table, or
|
|
|
+ another source block, and it is initially available as an elisp
|
|
|
+ data structure.
|
|
|
+ - We introduce a new #+ line, e.g. #+BABELDO. C-c C-c on that
|
|
|
+ line will apply an *action* to the referenced data.
|
|
|
+ - *Actions correspond to source blocks*: our library of available
|
|
|
+ actions will be a library of org-babel source blocks. Thus the
|
|
|
+ code for executing an action, and the code for dealing with the
|
|
|
+ output of the action will be the same code as for executing
|
|
|
+ source blocks in general
|
|
|
+ - Optionally, the user can have the relevant source block inserted
|
|
|
+ into the org buffer after the (say) #+BABELDO line. This will
|
|
|
+ allow the user to fine tune the action by modifying the code
|
|
|
+ (especially useful for plots).
|
|
|
+ - So maybe a #+BABELDO line will have header args
|
|
|
+ - :data (a reference to a table or source code block)
|
|
|
+ - :action (or should that be :srcname?) which will be something
|
|
|
+ like :action pie-chart, referring to a source block which will
|
|
|
+ be executed with the :data referent passed in using a :var arg.
|
|
|
+ - :showcode or something controlling whether to show the code
|
|
|
+
|
|
|
+*** Modification to design
|
|
|
+ I'm implementing this, at least initially, as a new interpreter
|
|
|
+ named 'babel', which has an empty body. 'babel' blocks take
|
|
|
+ a :srcname header arg, and look for the source-code block with
|
|
|
+ that name. They then execute the referenced block, after first
|
|
|
+ appending their own header args on to the target block's header
|
|
|
+ args.
|
|
|
|
|
|
-If you try it, Alt-k will stand for the old Enter; use "command $__" to
|
|
|
-access the last output.
|
|
|
+ If the target block is in the library of babel (a.o.t. e.g. the
|
|
|
+ current buffer), then the code in the block will refer to the
|
|
|
+ input data with a name dictated by convention (e.g. __data__
|
|
|
+ (something which is syntactically legal in all languages...). Thus
|
|
|
+ the babel block will use a :var __data__ = whatever header arg to
|
|
|
+ reference the data to be plotted.
|
|
|
|
|
|
-Best,
|
|
|
+** TODO share org-babel
|
|
|
+how should we share org-babel?
|
|
|
|
|
|
---
|
|
|
+- post to org-mode and ess mailing lists
|
|
|
+- create a org-babel page on worg
|
|
|
+- create a short screencast demonstrating org-babel in action
|
|
|
|
|
|
-Herculano de Lima Einloft Neto
|
|
|
-#+end_quote
|
|
|
+*** examples
|
|
|
+we need to think up some good examples
|
|
|
|
|
|
-***** emacs-lisp will be a special case
|
|
|
-While it is possible for emacs-lisp to be run in a console type
|
|
|
-environment (see the =elim= function) it is *not* possible to run
|
|
|
-emacs-lisp in a different *session*. Meaning any variable set top
|
|
|
-level of the console environment will be set *everywhere* inside
|
|
|
-emacs. For this reason I think that it doesn't make any sense to
|
|
|
-worry about session support for emacs-lisp.
|
|
|
+**** interactive tutorials
|
|
|
+This could be a place to use [[* org-babel assertions][org-babel assertions]].
|
|
|
|
|
|
-*** Further thoughts on 'scripting' vs. functional approaches
|
|
|
+for example the first step of a tutorial could assert that the version
|
|
|
+of the software-package (or whatever) is equal to some value, then
|
|
|
+source-code blocks could be used with confidence (and executed
|
|
|
+directly from) the rest of the tutorial.
|
|
|
|
|
|
- These are just thoughts, I don't know how sure I am about this.
|
|
|
- And again, perhaps I'm not saying anything very radical, just that
|
|
|
- it would be nice to have some options supporting things like
|
|
|
- receiving text output in the org buffer.
|
|
|
+**** answering a text-book question w/code example
|
|
|
+org-babel is an ideal environment enabling both the development and
|
|
|
+demonstrationg of the code snippets required as answers to many
|
|
|
+text-book questions.
|
|
|
|
|
|
- I can see that you've already gone some way down the road towards
|
|
|
- the 'last value' approach, so sorry if my comments come rather
|
|
|
- late. I am concerned that we are not giving sufficient attention
|
|
|
- to stdout / the text that is returned by the interpreters. In
|
|
|
- contrast, many of our potential users will be accustomed to a
|
|
|
- 'scripting' approach, where they are outputting text at various
|
|
|
- points in the code block, not just at the end. I am leaning
|
|
|
- towards thinking that we should have 2 modes of evaluation:
|
|
|
- 'script' mode, and 'functional' mode.
|
|
|
+**** something using tables
|
|
|
+maybe something along the lines of calculations from collected grades
|
|
|
|
|
|
- In script mode, evaluation of a code block would result in *all*
|
|
|
- text output from that code block appearing as output in the org
|
|
|
- buffer, presumably as an #+begin_example...#+end_example. There
|
|
|
- could be an :echo option controlling whether the input commands
|
|
|
- also appear in the output. [This is like Sweave].
|
|
|
+**** file sizes
|
|
|
+Maybe something like the following which outputs sizes of directories
|
|
|
+under the home directory, and then instead of the trivial =emacs-lisp=
|
|
|
+block we could use an R block to create a nice pie chart of the
|
|
|
+results.
|
|
|
|
|
|
- In functional mode, the *result* of the code block is available as
|
|
|
- an elisp object, and may appear in the org buffer as an org
|
|
|
- table/string, via the mechanisms you have developed already.
|
|
|
+#+srcname: sizes
|
|
|
+#+begin_src bash :results replace
|
|
|
+du -sc ~/*
|
|
|
+#+end_src
|
|
|
|
|
|
- One thing I'm wondering about is whether, in script mode, there
|
|
|
- simply should not be a return value. Perhaps this is not so
|
|
|
- different from what exists: script mode would be new, and what
|
|
|
- exists currently would be functional mode.
|
|
|
+#+begin_src emacs-lisp :var sizes=sizes :results replace
|
|
|
+(mapcar #'car sizes)
|
|
|
+#+end_src
|
|
|
|
|
|
- I think it's likely that, while code evaluation will be exciting
|
|
|
- to people, a large majority of our users in a large majority of
|
|
|
- their usage will not attempt to actually use the return value from
|
|
|
- a source code block in any meaningful way. In that case, it seems
|
|
|
- rather restrictive to only allow them to see output from the end
|
|
|
- of the code block.
|
|
|
+** TODO command line execution
|
|
|
+Allow source code blocks to be called form the command line. This
|
|
|
+will be easy using the =sbe= function in [[file:lisp/org-babel-table.el][org-babel-table.el]].
|
|
|
|
|
|
- Instead I think the most accessible way to introduce org-babel to
|
|
|
- people, at least while they are learning it, is as an immensely
|
|
|
- powerful environment in which to embed their 'scripts', which now
|
|
|
- also allows them to 'run' their 'scripts'. Especially as such
|
|
|
- people are likely to be the least capable of the user-base, a
|
|
|
- possible design-rule would be to make the scripting style of usage
|
|
|
- easy (default?), perhaps requiring a special option to enable a
|
|
|
- functional style. Those who will use the functional style won't
|
|
|
- have a problem understanding what's going on, whereas the 'skript
|
|
|
- kiddies' might not even know the syntax for defining a function in
|
|
|
- their language of choice. And of course we can allow the user to
|
|
|
- set a variable in their .emacs controlling the preference, so that
|
|
|
- functional users are not inconveniennced by having to provide
|
|
|
- header args the whole time.
|
|
|
+This will rely upon [[* resolve references to other buffers][resolve references to other buffers]].
|
|
|
|
|
|
- Please don't get the impression that I am down-valuing the
|
|
|
- functional style of org-babel. I am constantly horrified at the
|
|
|
- messy 'scripts' that my colleagues produce in perl or R or
|
|
|
- whatever! Nevertheless that seems to be how a lot of people work.
|
|
|
-
|
|
|
- I think you were leaning towards the last-value approach because
|
|
|
- it offered the possibility of unified code supporting both the
|
|
|
- single evaluation environment and the functional style. If you
|
|
|
- agree with any of the above then perhaps it will impact upon this
|
|
|
- and mean that the code in the two branches has to differ a bit. In
|
|
|
- that case, functional mode could perhaps after all evaluate each
|
|
|
- code block in its own environment, thus (re)approaching 'true'
|
|
|
- functional programming (side-effects are hard to achieve).
|
|
|
+** TODO inline source code blocks [3/5]
|
|
|
+ Like the =\R{ code }= blocks
|
|
|
|
|
|
-#+begin_src sh
|
|
|
-ls > files
|
|
|
-echo "There are `wc -l files` files in this directory"
|
|
|
+ not sure what the format should be, maybe just something simple
|
|
|
+ like =src_lang[]{}= where lang is the name of the source code
|
|
|
+ language to be evaluated, =[]= is optional and contains any header
|
|
|
+ arguments and ={}= contains the code.
|
|
|
|
|
|
-#+end_src
|
|
|
+ (see [[* (sandbox) inline source blocks][the-sandbox]])
|
|
|
|
|
|
-*** even more thoughts on evaluation, results, models and options
|
|
|
+*** DONE evaluation with \C-c\C-c
|
|
|
+Putting aside the header argument issue for now we can just run these
|
|
|
+with the following default header arguments
|
|
|
+- =:results= :: silent
|
|
|
+- =:exports= :: results
|
|
|
|
|
|
-Thanks Dan, These comments are invaluable.
|
|
|
+*** DONE inline exportation
|
|
|
+Need to add an interblock hook (or some such) through org-exp-blocks
|
|
|
+*** DONE header arguments
|
|
|
+We should make it possible to use header arguments.
|
|
|
|
|
|
-What do you think about this as a new list of priorities/requirements
|
|
|
-for the execution of source-code blocks.
|
|
|
+*** TODO fontification
|
|
|
+we should color these blocks differently
|
|
|
|
|
|
-- Sessions
|
|
|
- 1) we want the evaluation of the source code block to take place in a
|
|
|
- session which can persist state (variables, current directory,
|
|
|
- etc...).
|
|
|
- 2) source code blocks can specify their session with a header argument
|
|
|
- 3) each session should correspond to an Emacs comint buffer so that the
|
|
|
- user can drop into the session and experiment with live code
|
|
|
- evaluation.
|
|
|
-- Results
|
|
|
- 1) each source-code block generates some form of results which (as
|
|
|
- we have already implemented) is transfered into emacs-lisp
|
|
|
- after which it can be inserted into the org-mode buffer, or
|
|
|
- used by other source-code blocks
|
|
|
- 2) when the results are translated into emacs-lisp, forced to be
|
|
|
- interpreted as a scalar (dumping their raw values into the
|
|
|
- org-mode buffer), as a vector (which is often desirable with R
|
|
|
- code blocks), or interpreted on the fly (the default option).
|
|
|
- Note that this is very nearly currently implemented through the
|
|
|
- [[* DONE results-type header (vector/file)][results-type-header]].
|
|
|
- 3) there should be *two* means of collecting results from the
|
|
|
- execution of a source code block. *Either* the value of the
|
|
|
- last statement of the source code block, or the collection of
|
|
|
- all that has been passed to STDOUT during the evaluation.
|
|
|
+*** TODO refine html exportation
|
|
|
+should use a span class, and should show original source in tool-tip
|
|
|
|
|
|
-**** header argument or return line (*header argument*)
|
|
|
+** TODO formulate general rules for handling vectors and tables / matrices with names
|
|
|
+ This is non-trivial, but may be worth doing, in particular to
|
|
|
+ develop a nice framework for sending data to/from R.
|
|
|
+*** Notes
|
|
|
+ In R, indexing vector elements, and rows and columns, using
|
|
|
+ strings rather than integers is an important part of the
|
|
|
+ language.
|
|
|
+ - elements of a vector may have names
|
|
|
+ - matrices and data.frames may have "column names" and "row names"
|
|
|
+ which can be used for indexing
|
|
|
+ - In a data frame, row names *must* be unique
|
|
|
+Examples
|
|
|
+#+begin_example
|
|
|
+> # a named vector
|
|
|
+> vec <- c(a=1, b=2)
|
|
|
+> vec["b"]
|
|
|
+b
|
|
|
+2
|
|
|
+> mat <- matrix(1:4, nrow=2, ncol=2, dimnames=list(c("r1","r2"), c("c1","c2")))
|
|
|
+> mat
|
|
|
+ c1 c2
|
|
|
+r1 1 3
|
|
|
+r2 2 4
|
|
|
+> # The names are separate from the data: they do not interfere with operations on the data
|
|
|
+> mat * 3
|
|
|
+ c1 c2
|
|
|
+r1 3 9
|
|
|
+r2 6 12
|
|
|
+> mat["r1","c2"]
|
|
|
+[1] 3
|
|
|
+> df <- data.frame(var1=1:26, var2=26:1, row.names=letters)
|
|
|
+> df$var2
|
|
|
+ [1] 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
|
|
|
+> df["g",]
|
|
|
+ var1 var2
|
|
|
+g 7 20
|
|
|
+#+end_example
|
|
|
|
|
|
- Rather than using a header argument to specify how the return value
|
|
|
- should be passed back, I'm leaning towards the use of a =#+RETURN=
|
|
|
- line inside the block. If such a line *is not present* then we
|
|
|
- default to using STDOUT to collect results, but if such a line *is
|
|
|
- present* then we use it's value as the results of the block. I
|
|
|
- think this will allow for the most elegant specification between
|
|
|
- functional and script execution. This also cleans up some issues
|
|
|
- of implementation and finding which statement is the last
|
|
|
- statement.
|
|
|
+ So it's tempting to try to provide support for this in org-babel. For example
|
|
|
+ - allow R to refer to columns of a :var reference by their names
|
|
|
+ - When appropriate, results from R appear in the org buffer with "named
|
|
|
+ columns (and rows)"
|
|
|
|
|
|
- Having given this more thought, I think a header argument is
|
|
|
- preferable. The =#+return:= line adds new complicating syntax for
|
|
|
- something that does little more than we would accomplish through
|
|
|
- the addition of a header argument. The only benefit being that we
|
|
|
- know where the final statement starts, which is not an issue in
|
|
|
- those languages which contain 'last value' operators.
|
|
|
+ However none (?) of the other languages we are currently supporting
|
|
|
+ really have a native matrix type, let alone "column names" or "row
|
|
|
+ names". Names are used in e.g. python and perl to refer to entries
|
|
|
+ in dicts / hashes.
|
|
|
|
|
|
- new header =:results= arguments
|
|
|
- - script :: explicitly states that we want to use STDOUT to
|
|
|
- initialize our results
|
|
|
- - return_last :: stdout is ignored instead the *value* of the final
|
|
|
- statement in the block is returned
|
|
|
- - echo :: means echo the contents of the source-code block along
|
|
|
- with the results (this implies the *script* =:results=
|
|
|
- argument as well)
|
|
|
+ It currently seems to me that support for this in org-babel would
|
|
|
+ require setting rules about when org tables are considered to have
|
|
|
+ named columns/fields, and ensuring that (a) languages with a notion
|
|
|
+ of named columns/fields use them appropriately and (b) languages
|
|
|
+ with no such notion do not treat then as data.
|
|
|
|
|
|
-*** DONE rework evaluation lang-by-lang [4/4]
|
|
|
+ - Org allows something that *looks* like column names to be separated
|
|
|
+ by a hline
|
|
|
+ - Org also allows a row to *function* as column names when special
|
|
|
+ markers are placed in the first column. An hline is unnecessary
|
|
|
+ (indeed hlines are purely cosmetic in org [correct?]
|
|
|
+ - Org does not have a notion of "row names" [correct?]
|
|
|
+
|
|
|
+ The full org table functionality exeplified [[http://orgmode.org/manual/Advanced-features.html#Advanced-features][here]] has features that
|
|
|
+ we would not support in e.g. R (like names for the row below).
|
|
|
+
|
|
|
+*** Initial statement: allow tables with hline to be passed as args into R
|
|
|
+ This doesn't seem to work at the moment (example below). It would
|
|
|
+ also be nice to have a natural way for the column names of the org
|
|
|
+ table to become the column names of the R data frame, and to have
|
|
|
+ the option to specify that the first column is to be used as row
|
|
|
+ names in R (these must be unique). But this might require a bit of
|
|
|
+ thinking about.
|
|
|
|
|
|
-This should include...
|
|
|
-- functional results working with the comint buffer
|
|
|
-- results headers
|
|
|
- - script :: return the output of STDOUT
|
|
|
- - write a macro which runs the first redirection, executes the
|
|
|
- body, then runs the second redirection
|
|
|
- - last :: return the value of the last statement
|
|
|
- -
|
|
|
|
|
|
-- sessions in comint buffers
|
|
|
+#+TBLNAME: egtable
|
|
|
+| col1 | col2 | col3 |
|
|
|
+|------+---------+------|
|
|
|
+| 1 | 2 | 3 |
|
|
|
+| 4 | schulte | 6 |
|
|
|
|
|
|
-**** DONE Ruby [4/4]
|
|
|
-- [X] functional results working with comint
|
|
|
-- [X] script results
|
|
|
-- [X] ensure scalar/vector results args are taken into consideration
|
|
|
-- [X] ensure callable by other source block
|
|
|
+#+TBLNAME: egtable2
|
|
|
+| 1 | 2 | 3 |
|
|
|
+| 4 | schulte | 6 |
|
|
|
|
|
|
-#+srcname: ruby-use-last-output
|
|
|
-#+begin_src ruby :results replace
|
|
|
-a = 2
|
|
|
-b = 4
|
|
|
-c = a + b
|
|
|
-[a, b, c, 78]
|
|
|
+#+begin_src R var tabel=egtable
|
|
|
+tabel
|
|
|
#+end_src
|
|
|
|
|
|
-#+resname: ruby-use-last-output
|
|
|
-| 2 | 4 | 6 | 78 |
|
|
|
+#+resname:
|
|
|
+| "col1" | "col2" | "col3" |
|
|
|
+|--------+-----------+--------|
|
|
|
+| 1 | 2 | 3 |
|
|
|
+| 4 | "schulte" | 6 |
|
|
|
|
|
|
-#+srcname: task-call-use-last-output
|
|
|
-#+begin_src ruby :var last=ruby-use-last-output :results replace
|
|
|
-last.flatten.size + 1
|
|
|
-#+end_src
|
|
|
|
|
|
-#+resname: task-call-use-last-output
|
|
|
-: 5
|
|
|
+Another example is in the [[*operations%20in%20on%20tables][grades example]].
|
|
|
|
|
|
-***** ruby sessions
|
|
|
+** PROPOSED add =:none= session argument (for purely functional execution)
|
|
|
+This would allow source blocks to be run in their own new process
|
|
|
|
|
|
-#+srcname: first-ruby-session-task
|
|
|
-#+begin_src ruby :session schulte :results silent
|
|
|
-schulte = 27
|
|
|
-#+end_src
|
|
|
+These blocks could then also be run in the background.
|
|
|
|
|
|
-#+srcname: second-ruby-session-task
|
|
|
-#+begin_src ruby :session schulte :results silent
|
|
|
-schulte + 3
|
|
|
-#+end_src
|
|
|
+** PROPOSED Are we happy with current behaviour regarding vector/scalar output?
|
|
|
+This simple example of multilingual chaining produces vector output if
|
|
|
+there are spaces in the message and scalar otherwise.
|
|
|
|
|
|
-#+srcname: without-the-right-session
|
|
|
-#+begin_src ruby :results silent
|
|
|
-schulte
|
|
|
+#+begin_src R :var msg=msg-from-python
|
|
|
+paste(msg, "und_R", sep="_")
|
|
|
#+end_src
|
|
|
|
|
|
-**** DONE R [4/4]
|
|
|
-
|
|
|
-- [X] functional results working with comint
|
|
|
-- [X] script results
|
|
|
-- [X] ensure scalar/vector results args are taken into consideration
|
|
|
-- [X] ensure callable by other source block
|
|
|
-
|
|
|
-To redirect output to a file, you can use the =sink()= command.
|
|
|
-
|
|
|
-#+srcname: task_R_B
|
|
|
-#+begin_src R :results value vector silent
|
|
|
-a <- 9
|
|
|
-b <- 10
|
|
|
-b - a
|
|
|
-a + b
|
|
|
+#+srcname: msg-from-python
|
|
|
+#+begin_src python :var msg=msg-from-elisp
|
|
|
+msg + "_y_python"
|
|
|
#+end_src
|
|
|
|
|
|
-#+srcname: task-R-use-other-output
|
|
|
-#+begin_src R :var twoentyseven=task_R_B() :results replace value
|
|
|
-83
|
|
|
-twoentyseven + 9
|
|
|
+#+srcname: msg-from-elisp
|
|
|
+#+begin_src emacs-lisp :var msg="org-babel_speaks"
|
|
|
+(concat msg "_elisp")
|
|
|
#+end_src
|
|
|
|
|
|
-#+resname: task-R-use-other-output
|
|
|
-: 28
|
|
|
+** PROPOSED conversion between org-babel and noweb (e.g. .Rnw) format
|
|
|
+ I haven't thought about this properly. Just noting it down. What
|
|
|
+ Sweave uses is called "R noweb" (.Rnw).
|
|
|
+
|
|
|
+ I found a good description of noweb in the following article (see
|
|
|
+ the [[http://www.cs.tufts.edu/~nr/pubs/lpsimp.pdf][pdf]]).
|
|
|
+
|
|
|
+ I think there are two parts to noweb, the construction of
|
|
|
+ documentation and the extraction of source-code (with notangle).
|
|
|
|
|
|
-**** DONE Python [4/4]
|
|
|
-- [X] functional results working with comint
|
|
|
-- [X] script results
|
|
|
-- [X] ensure scalar/vector results args are taken into consideration
|
|
|
-- [X] ensure callable by other source block
|
|
|
+ *documentation*: org-mode handles all of our documentation needs in
|
|
|
+ a manner that I believe is superior to noweb.
|
|
|
+
|
|
|
+ *source extraction* At this point I don't see anyone writing large
|
|
|
+ applications with 100% of the source code contained in org-babel
|
|
|
+ files, rather I see org-babel files containing things like
|
|
|
+ - notes with active code chunks
|
|
|
+ - interactive tutorials
|
|
|
+ - requirements documents with code running test suites
|
|
|
+ - and of course experimental reports with the code to run the
|
|
|
+ experiment, and perform analysis
|
|
|
|
|
|
-#+srcname: task-new-eval-for-python
|
|
|
-#+begin_src python :results silent output scalar
|
|
|
-8
|
|
|
-9
|
|
|
-10
|
|
|
-#+end_src
|
|
|
+ Basically I think the scope of the programs written in org-babel
|
|
|
+ (at least initially) will be small enough that it wont require the
|
|
|
+ addition of a tangle type program to extract all of the source code
|
|
|
+ into a running application.
|
|
|
+
|
|
|
+ On the other hand, since we already have named blocks of source
|
|
|
+ code which reference other blocks on which they rely, this
|
|
|
+ shouldn't be too hard to implement either on our own, or possibly
|
|
|
+ relying on something like noweb/notangle.
|
|
|
|
|
|
-#+srcname: task-use-new-eval
|
|
|
-#+begin_src python :var tasking=task-new-eval-for-python() :results replace
|
|
|
-tasking + 2
|
|
|
-#+end_src
|
|
|
+** PROPOSED support for passing paths to files between source blocks
|
|
|
+Maybe this should be it's own result type (in addition to scalars and
|
|
|
+vectors). The reason being that some source-code blocks (for example
|
|
|
+ditaa or anything that results in the creation of a file) may want to
|
|
|
+pass a file path back to org-mode which could then be inserted into
|
|
|
+the org-mode buffer as a link to the file...
|
|
|
|
|
|
-#+resname: task-use-new-eval
|
|
|
-: 12
|
|
|
+This would allow for display of images upon export providing
|
|
|
+functionality similar to =org-exp-blocks= only in a more general
|
|
|
+manner.
|
|
|
|
|
|
-**** DONE Shells [4/4]
|
|
|
-- [X] functional results working with comint
|
|
|
-- [X] script results
|
|
|
-- [X] ensure scalar/vector results args are taken into consideration
|
|
|
-- [X] ensure callable by other source block
|
|
|
+** DEFERRED use textConnection to pass tsv to R?
|
|
|
+ When passing args from the org buffer to R, the following route is
|
|
|
+ used: arg in buffer -> elisp -> tsv on file -> data frame in R. I
|
|
|
+ think it would be possible to avoid having to write to file by
|
|
|
+ constructing an R expression in org-babel-R-assign-elisp, something
|
|
|
+ like this
|
|
|
|
|
|
-#+srcname: task-shell-new-evaluation
|
|
|
-#+begin_src sh :results silent value scalar
|
|
|
-echo 'eric'
|
|
|
-date
|
|
|
+#+begin_src emacs-lisp
|
|
|
+(org-babel-R-input-command
|
|
|
+ (format "%s <- read.table(textConnection(\"%s\"), sep=\"\\t\", as.is=TRUE)"
|
|
|
+ name (orgtbl-to-tsv value '(:sep "\t" :fmt org-babel-R-quote-tsv-field))))
|
|
|
#+end_src
|
|
|
|
|
|
-#+srcname: task-call-other-shell
|
|
|
-#+begin_src sh :var other=task-shell-new-evaluation() :results replace scalar
|
|
|
-echo $other ' is the old date'
|
|
|
-#+end_src
|
|
|
+ I haven't tried to implement this yet as it's basically just
|
|
|
+ fiddling with something that works. The only reason for it I can
|
|
|
+ think of would be efficiency and I haven't tested that.
|
|
|
|
|
|
-#+resname: task-call-other-shell
|
|
|
-: $ Fri Jun 12 13:08:37 PDT 2009 is the old date
|
|
|
+ This Didn't work after an initial test. I still think this is a
|
|
|
+ good idea (I also think we should try to do something similar when
|
|
|
+ writing out results frmo R to elisp) however as it wouldn't result
|
|
|
+ in any functional changes I'm bumping it down to deferred for
|
|
|
+ now. [Eric]
|
|
|
|
|
|
-*** DONE implement a *session* header argument [4/4]
|
|
|
-=:session= header argument to override the default *session* buffer
|
|
|
+for quick tests
|
|
|
|
|
|
-**** DONE ruby
|
|
|
+#+tblname: quick-test
|
|
|
+| 1 | 2 | 3 |
|
|
|
|
|
|
-#+srcname: task-ruby-named-session
|
|
|
-#+begin_src ruby :session schulte :results replace
|
|
|
-schulte = :in_schulte
|
|
|
+#+srcname: quick-test-src-blk
|
|
|
+#+begin_src R :var vec=quick-test
|
|
|
+mean(mean(vec))
|
|
|
#+end_src
|
|
|
|
|
|
-#+resname: task-ruby-named-session
|
|
|
-: :in_schulte
|
|
|
+: 2
|
|
|
|
|
|
-#+srcname: another-in-schulte
|
|
|
-#+begin_src ruby :session schulte
|
|
|
-schulte
|
|
|
-#+end_src
|
|
|
+** DEFERRED re-implement R evaluation using ess-command or ess-execute
|
|
|
+ I don't have any complaints with the current R evaluation code or
|
|
|
+ behaviour, but I think it would be good to use the ESS functions
|
|
|
+ from a political point of view. Plus of course it has the normal
|
|
|
+ benefits of an API (insulates us from any underlying changes etc). [DED]
|
|
|
|
|
|
-#+resname: another-in-schulte
|
|
|
-: :in_schulte
|
|
|
-: :in_schulte
|
|
|
-: :in_schulte
|
|
|
+ I'll look into this. I believe that I looked at and rejected these
|
|
|
+ functions initially but now I can't remember why. I agree with
|
|
|
+ your overall point about using API's where available. I will take
|
|
|
+ a look back at these and either switch to using the ess commands,
|
|
|
+ or at least articulate under this TODO the reasons for using our
|
|
|
+ custom R-interaction commands. [Eric]
|
|
|
|
|
|
-**** DONE python
|
|
|
+ ess-execute
|
|
|
|
|
|
-#+srcname: python-session-task
|
|
|
-#+begin_src python :session what :results silent
|
|
|
-what = 98
|
|
|
-#+end_src
|
|
|
+ Lets just replace =org-babel-R-input-command= with =ess-execute=.
|
|
|
|
|
|
-#+srcname: python-get-from-session
|
|
|
-#+begin_src python :session what :results replace
|
|
|
-what
|
|
|
-#+end_src
|
|
|
+ I tried this, and although it works in some situations, I find that
|
|
|
+ =ess-command= will often just hang indefinitely without returning
|
|
|
+ results. Also =ess-execute= will occasionally hang, and pops up
|
|
|
+ the buffer containing the results of the command's execution, which
|
|
|
+ is undesirable. For now these functions can not be used. Maybe
|
|
|
+ someone more familiar with the ESS code can recommend proper usage
|
|
|
+ of =ess-command= or some other lower-level function which could be
|
|
|
+ used in place of [[file:lisp/org-babel-R.el::defun%20org-babel%20R%20input%20command%20command][org-babel-R-input-command]].
|
|
|
|
|
|
-#+resname: python-get-from-session
|
|
|
-: 98
|
|
|
+*** ess functions
|
|
|
+
|
|
|
+#+begin_quote ess-command
|
|
|
+(ess-command COM &optional BUF SLEEP NO-PROMPT-CHECK)
|
|
|
|
|
|
-**** DONE shell
|
|
|
+Send the ESS process command COM and delete the output
|
|
|
+from the ESS process buffer. If an optional second argument BUF exists
|
|
|
+save the output in that buffer. BUF is erased before use.
|
|
|
+COM should have a terminating newline.
|
|
|
+Guarantees that the value of .Last.value will be preserved.
|
|
|
+When optional third arg SLEEP is non-nil, `(sleep-for (* a SLEEP))'
|
|
|
+will be used in a few places where `a' is proportional to `ess-cmd-delay'.
|
|
|
+#+end_quote
|
|
|
|
|
|
-#+srcname: task-shell-sessions
|
|
|
-#+begin_src sh :session what
|
|
|
-WHAT='patton'
|
|
|
-#+end_src
|
|
|
+#+begin_quote ess-execute
|
|
|
+(ess-execute COMMAND &optional INVERT BUFF MESSAGE)
|
|
|
|
|
|
-#+srcname: task-shell-sessions-what
|
|
|
-#+begin_src sh :session what :results replace
|
|
|
-echo $WHAT
|
|
|
-#+end_src
|
|
|
+Send a command to the ESS process.
|
|
|
+A newline is automatically added to COMMAND. Prefix arg (or second arg
|
|
|
+INVERT) means invert the meaning of
|
|
|
+`ess-execute-in-process-buffer'. If INVERT is 'buffer, output is
|
|
|
+forced to go to the process buffer. If the output is going to a
|
|
|
+buffer, name it *BUFF*. This buffer is erased before use. Optional
|
|
|
+fourth arg MESSAGE is text to print at the top of the buffer (defaults
|
|
|
+to the command if BUFF is not given.)
|
|
|
+#+end_quote
|
|
|
|
|
|
-#+resname: task-shell-sessions-what
|
|
|
-: patton
|
|
|
+*** out current setup
|
|
|
|
|
|
-**** DONE R
|
|
|
+ 1) The body of the R source code block is wrapped in a function
|
|
|
+ 2) The function is called inside of a =write.table= function call
|
|
|
+ writing the results to a table
|
|
|
+ 3) The table is read using =org-table-import=
|
|
|
|
|
|
-#+srcname: task-R-session
|
|
|
-#+begin_src R :session what :results replace
|
|
|
-a <- 9
|
|
|
-b <- 8
|
|
|
-a + b
|
|
|
+** DEFERRED Rework Interaction with Running Processes [2/5]
|
|
|
+*** DONE robust to errors interrupting execution
|
|
|
+
|
|
|
+#+srcname: long-runner-ruby
|
|
|
+#+begin_src ruby :results silent
|
|
|
+ sleep(10)
|
|
|
+ :patton_is_an_grumpy
|
|
|
#+end_src
|
|
|
|
|
|
-#+resname: task-R-session
|
|
|
-: 17
|
|
|
+*** DEFERRED use =C-g= keyboard-quit to push processing into the background
|
|
|
+This may be possible using the `run-with-timer' command.
|
|
|
|
|
|
-#+srcname: another-task-R-session
|
|
|
-#+begin_src R :session what :results replace
|
|
|
-a + b
|
|
|
+I have no idea how this could work...
|
|
|
+
|
|
|
+#+srcname: long-runner-ruby
|
|
|
+#+begin_src ruby :results silent
|
|
|
+ sleep(10)
|
|
|
+ :patton_is_an_grumpy
|
|
|
#+end_src
|
|
|
|
|
|
-*** DONE function to bring up inferior-process buffer [4/4]
|
|
|
+*** TODO ability to select which of multiple sessions is being used
|
|
|
+ Increasingly it is looking like we're going to want to run all
|
|
|
+ source code blocks in comint buffer (sessions). Which will have
|
|
|
+ the benefits of
|
|
|
+ 1) allowing background execution
|
|
|
+ 2) maintaining state between source-blocks
|
|
|
+ - allowing inline blocks w/o header arguments
|
|
|
|
|
|
-This should be callable from inside of a source-code block in an
|
|
|
-org-mode buffer. It should evaluate the header arguments, then bring
|
|
|
-up the inf-proc buffer using =pop-to-buffer=.
|
|
|
+**** R sessions
|
|
|
+ (like ess-switch-process in .R buffers)
|
|
|
+
|
|
|
+ Maybe this could be packaged into a header argument, something
|
|
|
+ like =:R_session= which could accept either the name of the
|
|
|
+ session to use, or the string =prompt=, in which case we could use
|
|
|
+ the =ess-switch-process= command to select a new process.
|
|
|
+
|
|
|
+*** TODO evaluation of shell code as background process?
|
|
|
+ After C-c C-c on an R code block, the process may appear to
|
|
|
+ block, but C-g can be used to reclaim control of the .org buffer,
|
|
|
+ without interrupting the R evalution. However I believe this is not
|
|
|
+ true of bash/sh evaluation. [Haven't tried other languages] Perhaps
|
|
|
+ a solution is just to background the individual shell commands.
|
|
|
|
|
|
-For lack of a better place, lets add this to the
|
|
|
-`org-shiftmetadown-hook' hook.
|
|
|
+ The other languages (aside from emacs lisp) are run through the
|
|
|
+ shell, so if we find a shell solution it should work for them as
|
|
|
+ well.
|
|
|
+
|
|
|
+ Adding an ampersand seems to be a supported way to run commands in
|
|
|
+ the background (see [[http://www.emacswiki.org/emacs/ExecuteExternalCommand#toc4][external-commands]]). Although a more extensible
|
|
|
+ solution may involve the use of the [[elisp:(progn (describe-function 'call-process-region) nil)][call-process-region]] function.
|
|
|
+
|
|
|
+ Going to try this out in a new file [[file:lisp/org-babel-proc.el][org-babel-proc.el]]. This should
|
|
|
+ contain functions for asynchronously running generic shell commands
|
|
|
+ in the background, and then returning their input.
|
|
|
|
|
|
-**** DONE ruby
|
|
|
+**** partial update of org-mode buffer
|
|
|
+ The sleekest solution to this may be using a comint buffer, and
|
|
|
+ then defining a filter function which would incrementally interpret
|
|
|
+ the results as they are returned, including insertion into the
|
|
|
+ org-mode buffer. This may actually cause more problems than it is
|
|
|
+ worth, what with the complexities of identifying the types of
|
|
|
+ incrementally returned results, and the need for maintenance of a
|
|
|
+ process marker in the org buffer.
|
|
|
|
|
|
-#+srcname: task-ruby-pop-to-session
|
|
|
-#+begin_src ruby :var num=9 :var another="something else"
|
|
|
-num.times{|n| puts another}
|
|
|
-#+end_src
|
|
|
+**** 'working' spinner
|
|
|
+ It may be nice and not too difficult to place a spinner on/near the
|
|
|
+ evaluating source code block
|
|
|
|
|
|
-**** DONE python
|
|
|
+*** TODO conversion of output from interactive shell, R (and python) sessions to org-babel buffers
|
|
|
+ [DED] This would be a nice feature I think. Although an org-babel
|
|
|
+ purist would say that it's working the wrong way round... After
|
|
|
+ some interactive work in a *R* buffer, you save the buffer, maybe
|
|
|
+ edit out some lines, and then convert it to org-babel format for
|
|
|
+ posterity. Same for a shell session either in a *shell* buffer, or
|
|
|
+ pasted from another terminal emulator. And python of course.
|
|
|
|
|
|
-#+srcname: task-python-pop-to-session
|
|
|
-#+begin_src python :var num=9 :var another="something else"
|
|
|
-another * num
|
|
|
-#+end_src
|
|
|
-**** DONE R
|
|
|
+** DEFERRED improve the source-block snippet
|
|
|
+any real improvement seems somewhat beyond the ability of yasnippet
|
|
|
+for now.
|
|
|
|
|
|
-#+srcname: task-R-pop-to-session
|
|
|
-#+begin_src R :var a=9 :var b=8
|
|
|
-a * b
|
|
|
-#+end_src
|
|
|
+[[file:~/src/emacs-starter-kit/src/snippets/text-mode/rst-mode/chap::name%20Chapter%20title][file:~/src/emacs-starter-kit/src/snippets/text-mode/rst-mode/chap::name Chapter title]]
|
|
|
+#+begin_example
|
|
|
+,#name : Chapter title
|
|
|
+,# --
|
|
|
+${1:Chapter}
|
|
|
+${1:$(make-string (string-width text) ?\=)}
|
|
|
|
|
|
-**** DONE shell
|
|
|
+$0
|
|
|
+#+end_example
|
|
|
|
|
|
-#+srcname: task-shell-pop-sessions
|
|
|
-#+begin_src sh :var NAME="eric"
|
|
|
-echo $NAME
|
|
|
-#+end_src
|
|
|
+[[file:snippets/org-mode/sb][sb -- snippet]]
|
|
|
|
|
|
-*** TODO function to dump last N lines from inf-proc buffer into the current source block
|
|
|
+waiting for guidance from those more familiar with yasnippets
|
|
|
|
|
|
-Callable with a prefix argument to specify how many lines should be
|
|
|
-dumped into the source-code buffer.
|
|
|
+** DONE fully purge org-babel-R of direct comint interaction
|
|
|
+try to remove all code under the [[file:lisp/org-babel-R.el::functions%20for%20evaluation%20of%20R%20code][;; functions for evaluation of R code]] line
|
|
|
|
|
|
-*** REJECTED comint notes
|
|
|
+** DONE Create objects in top level (global) environment [5/5]
|
|
|
+*sessions*
|
|
|
|
|
|
-Implementing comint integration in [[file:lisp/org-babel-comint.el][org-babel-comint.el]].
|
|
|
+*** initial requirement statement [DED]
|
|
|
+ At the moment, objects created by computations performed in the
|
|
|
+ code block are evaluated in the scope of the
|
|
|
+ code-block-function-body and therefore disappear when the code
|
|
|
+ block is evaluated {unless you employ some extra trickery like
|
|
|
+ assign('name', object, env=globalenv()) }. I think it will be
|
|
|
+ desirable to also allow for a style wherein objects that are
|
|
|
+ created in one code block persist in the R global environment and
|
|
|
+ can be re-used in a separate block.
|
|
|
|
|
|
-Need to have...
|
|
|
-- handling of outputs
|
|
|
- - split raw output from process by prompts
|
|
|
- - a ring of the outputs, buffer-local, `org-babel-comint-output-ring'
|
|
|
- - a switch for dumping all outputs to a buffer
|
|
|
-- inputting commands
|
|
|
+ This is what Sweave does, and while I'm not saying we have to be
|
|
|
+ the same as Sweave, it wouldn't be hard for us to provide the same
|
|
|
+ behaviour in this case; if we don't, we risk undeservedly being
|
|
|
+ written off as an oddity by some.
|
|
|
|
|
|
-Lets drop all this language specific stuff, and just use
|
|
|
-org-babel-comint to split up our outputs, and return either the last
|
|
|
-value of an execution or the combination of values from the
|
|
|
-executions.
|
|
|
+ IOW one aspect of org-babel is that of a sort of functional
|
|
|
+ meta-programming language. This is crazy, in a very good
|
|
|
+ way. Nevertheless, wrt R I think there's going to be a lot of value
|
|
|
+ in providing for a working style in which the objects are stored in
|
|
|
+ the R session, rather than elisp/org buffer. This will be a very
|
|
|
+ familiar working style to lots of people.
|
|
|
|
|
|
-**** comint filter functions
|
|
|
-: ;; comint-input-filter-functions hook process-in-a-buffer
|
|
|
-: ;; comint-output-filter-functions hook function modes.
|
|
|
-: ;; comint-preoutput-filter-functions hook
|
|
|
-: ;; comint-input-filter function ...
|
|
|
+ There are no doubt a number of different ways of accomplishing
|
|
|
+ this, the simplest being a hack like adding
|
|
|
|
|
|
-#+srcname: obc-filter-ruby
|
|
|
-#+begin_src ruby :results last
|
|
|
-1
|
|
|
-2
|
|
|
-3
|
|
|
-4
|
|
|
-5
|
|
|
+#+begin_src R
|
|
|
+for(objname in ls())
|
|
|
+ assign(objname, get(objname), envir=globalenv())
|
|
|
#+end_src
|
|
|
|
|
|
-** TODO add =:none= session argument (for purely functional execution)
|
|
|
-This would allow source blocks to be run in their own new process
|
|
|
+to the source code block function body. (Maybe wrap it in an on.exit() call).
|
|
|
|
|
|
-These blocks could then also be run in the background.
|
|
|
+However this may deserve to be thought about more carefully, perhaps
|
|
|
+with a view to having a uniform approach across languages. E.g. shell
|
|
|
+code blocks have the same semantics at the moment (no persistence of
|
|
|
+variables across code blocks), because the body is evaluated in a new
|
|
|
+bash shell process rather than a running shell. And I guess the same
|
|
|
+is true for python. However, in both these cases, you could imagine
|
|
|
+implementing the alternative in which the body is evaluated in a
|
|
|
+persistent interactive session. It's just that it's particularly
|
|
|
+natural for R, seeing as both ESS and org-babel evaluate commands in a
|
|
|
+single persistent R session.
|
|
|
|
|
|
-** TODO support for working with =*Org Edit Src Example*= buffers [1/4]
|
|
|
-*** TODO set buffer-local-process variables appropriately [DED]
|
|
|
- I think something like this would be great. You've probably
|
|
|
-already thought of this, but just to note it down: it would be really
|
|
|
-nice if org-babel's notion of a buffer's 'session/process' played
|
|
|
-nicely with ESS's notion of the buffer's session/process. ESS keeps
|
|
|
-the current process name for a buffer in a buffer-local variable
|
|
|
-ess-local-process-name. So one thing we will probably want to do is
|
|
|
-make sure that the *Org Edit Src Example* buffer sets that variable
|
|
|
-appropriately. [DED]
|
|
|
+*** sessions [Eric]
|
|
|
|
|
|
-I had not thought of that, but I agree whole heartedly. [Eric]
|
|
|
+Thanks for bringing this up. I think you are absolutely correct that we
|
|
|
+should provide support for a persistent environment (maybe called a
|
|
|
+*session*) in which to evaluate code blocks. I think the current setup
|
|
|
+demonstrates my personal bias for a functional style of programming
|
|
|
+which is certainly not ideal in all contexts.
|
|
|
|
|
|
-Once this is done every variable should be able to dump regions into
|
|
|
-their inferior-process buffer using major-mode functions.
|
|
|
+While the R function you mention does look like an elegant solution, I
|
|
|
+think we should choose an implementation that would be the same across
|
|
|
+all source code types. Specifically I think we should allow the user to
|
|
|
+specify an optional *session* as a header variable (when not present we
|
|
|
+assume a default session for each language). The session name could be
|
|
|
+used to name a comint buffer (like the *R* buffer) in which all
|
|
|
+evaluation would take place (within which variables would retain their
|
|
|
+values --at least once I remove some of the functional method wrappings
|
|
|
+currently in place-- ).
|
|
|
|
|
|
-*** TODO some possible requests/proposed changes for Carsten [2/3]
|
|
|
- While I remember, some possible requests/proposed changes for Carsten
|
|
|
- come to mind in that regard:
|
|
|
+This would allow multiple environments to be used in the same buffer,
|
|
|
+and once this setup was implemented we should be able to fairly easily
|
|
|
+implement commands for jumping between source code blocks and the
|
|
|
+related session buffers, as well as for dumping the last N commands from
|
|
|
+a session into a new or existing source code block.
|
|
|
|
|
|
-**** DONE Remap C-x C-s to save the source to the org buffer?
|
|
|
- I've done this personally and I find it essential. I'm using
|
|
|
-#+begin_src emacs-lisp
|
|
|
-(defun org-edit-src-save ()
|
|
|
- "Update the parent org buffer with the edited source code, save
|
|
|
-the parent org-buffer, and return to the source code edit
|
|
|
-buffer."
|
|
|
- (interactive)
|
|
|
- (let ((p (point)))
|
|
|
- (org-edit-src-exit)
|
|
|
- (save-buffer)
|
|
|
- (org-edit-src-code)
|
|
|
- (goto-char p)))
|
|
|
+Please let me know if you foresee any problems with this proposed setup,
|
|
|
+or if you think any parts might be confusing for people coming from
|
|
|
+Sweave. I'll hopefully find some time to work on this later in the
|
|
|
+week.
|
|
|
|
|
|
-(define-key org-exit-edit-mode-map "\C-x\C-s" 'org-edit-src-save)
|
|
|
-#+end_src
|
|
|
- which seems to work.
|
|
|
+*** can functional and interpreted/interactive models coexist?
|
|
|
|
|
|
-I think this is great, but I think it should be implemented in the
|
|
|
-org-mode core
|
|
|
-**** TODO Rename buffer and minor mode?
|
|
|
- Something shorter than *Org Edit Src Example* for the buffer
|
|
|
- name. org-babel is bringing org's source code interaction to a
|
|
|
- level of maturity where the 'example' is no longer
|
|
|
- appropriate. And if further keybindings are going to be added to
|
|
|
- the minor mode then maybe org-edit-src-mode is a better name than
|
|
|
- org-exit-edit-mode.
|
|
|
+Even though both of these use the same =*R*= buffer the value of =a=
|
|
|
+is not preserved because it is assigned inside of a functional
|
|
|
+wrapper.
|
|
|
|
|
|
- Maybe we should name the buffer with a combination of the source
|
|
|
- code and the session. I think that makes sense.
|
|
|
+#+srcname: task-R-sessions
|
|
|
+#+begin_src R
|
|
|
+a <- 9
|
|
|
+b <- 21
|
|
|
+a + b
|
|
|
+#+end_src
|
|
|
|
|
|
- [ES] Are you also suggesting a new org-edit-src minor mode?
|
|
|
- [DED] org-exit-edit-mode is a minor mode that already exists:
|
|
|
+#+srcname: task-R-same-session
|
|
|
+#+begin_src R
|
|
|
+a
|
|
|
+#+end_src
|
|
|
|
|
|
- Minor mode installing a single key binding, "C-c '" to exit special edit.
|
|
|
+This functional wrapper was implemented in order to efficiently return
|
|
|
+the results of the execution of the entire source code block. However
|
|
|
+it inhibits the evaluation of source code blocks in the top level,
|
|
|
+which would allow for persistence of variable assignment across
|
|
|
+evaluations. How can we allow *both* evaluation in the top level, and
|
|
|
+efficient capture of the return value of an entire source code block
|
|
|
+in a language independent manner?
|
|
|
|
|
|
- org-edit-src-save now has a binding in that mode, so I guess all
|
|
|
- I'm saying at this stage is that it's a bit of a misnomer. But
|
|
|
- perhaps we will also have more functionality to add to that minor
|
|
|
- mode, making it even more of a misnomer. Perhaps something like
|
|
|
- org-src-mode would be better.
|
|
|
+Possible solutions...
|
|
|
+1) we can't so we will have to implement two types of evaluation
|
|
|
+ depending on which is appropriate (functional or imperative)
|
|
|
+2) we remove the functional wrapper and parse the source code block
|
|
|
+ into it's top level statements (most often but not always on line
|
|
|
+ breaks) so that we can isolate the final segment which is our
|
|
|
+ return value.
|
|
|
+3) we add some sort of "#+return" line to the code block
|
|
|
+4) we take advantage of each languages support for meta-programming
|
|
|
+ through =eval= type functions, and use said to evaluate the entire
|
|
|
+ blocks in such a way that their environment can be combined with the
|
|
|
+ global environment, and their results are still captured.
|
|
|
+5) I believe that most modern languages which support interactive
|
|
|
+ sessions have support for a =last_result= type function, which
|
|
|
+ returns the result of the last input without re-calculation. If
|
|
|
+ widely enough present this would be the ideal solution to a
|
|
|
+ combination of functional and imperative styles.
|
|
|
|
|
|
-**** DEFERRED a hook called when the src edit buffer is created
|
|
|
-This should be implemented in the org-mode core
|
|
|
-
|
|
|
+None of these solutions seem very desirable, but for now I don't see
|
|
|
+what else would be possible.
|
|
|
|
|
|
-*** DEFERRED send code to inferior process
|
|
|
-Another thought on this topic: I think we will want users to send
|
|
|
-chunks of code to the interpreter from within the *Org Edit Src*
|
|
|
-buffer, and I think that's what you have in mind already. In ESS that
|
|
|
-is done using the ess-eval-* functions. [DED]
|
|
|
+Of these options I was leaning towards (1) and (4) but now believe
|
|
|
+that if it is possible option (5) will be ideal.
|
|
|
|
|
|
-I think we can leave this up to the major-mode in the source code
|
|
|
-buffer, as almost every source-code major mode will have functions for
|
|
|
-doing things like sending regions to the inferior process. If
|
|
|
-anything we might need to set the value of the buffer local inferior
|
|
|
-process variable. [Eric]
|
|
|
+**** (1) both functional and imperative evaluation
|
|
|
+Pros
|
|
|
+- can take advantage of built in functions for sending regions to the
|
|
|
+ inferior process
|
|
|
+- retains the proven tested and working functional wrappers
|
|
|
|
|
|
-*** TODO optionally evaluate header references when we switch to =*Org Edit Src*= buffer
|
|
|
-That seems to imply that the header references need to be evaluated
|
|
|
-and transformed into the target language object when we hit C-c ' to
|
|
|
-enter the *Org Edit Src* buffer [DED]
|
|
|
+Cons
|
|
|
+- introduces the complication of keeping track of which type of
|
|
|
+ evaluation is best suited to a particular context
|
|
|
+- the current functional wrappers may require some changes in order to
|
|
|
+ include the existing global context
|
|
|
|
|
|
-Good point, I heartily agree that this should be supported [Eric]
|
|
|
+**** (4) exploit language meta-programming constructs to explicitly evaluate code
|
|
|
+Pros
|
|
|
+- only one type of evaluation
|
|
|
|
|
|
-(or at least before the first time we attempt to evaluate code in that
|
|
|
-buffer -- I suppose there might be an argument for lazy evaluation, in
|
|
|
-case someone hits C-c ' but is "just looking" and not actually
|
|
|
-evaluating anything.) Of course if evaluating the reference is
|
|
|
-computationally intensive then the user might have to wait before they
|
|
|
-get the *Org Edit Src* buffer. [DED]
|
|
|
+Cons
|
|
|
+- some languages may not have sufficient meta-programming constructs
|
|
|
|
|
|
-I fear that it may be hard to anticipate when the references will be
|
|
|
-needed, some major-modes do on-the-fly evaluation while the buffer is
|
|
|
-being edited. I think that we should either do this before the buffer
|
|
|
-is opened or not at all, specifically I think we should resolve
|
|
|
-references if the user calls C-c ' with a prefix argument. Does that
|
|
|
-sound reasonable? [Eric]
|
|
|
+**** (5) exploit some =last_value= functionality if present
|
|
|
|
|
|
-Yes [Dan]
|
|
|
+Need to ensure that most languages have such a function, those without
|
|
|
+will simply have to implement their own similar solution...
|
|
|
|
|
|
-** TODO fully purge org-babel-R of direct comint interaction
|
|
|
-try to remove all code under the [[file:lisp/org-babel-R.el::functions%20for%20evaluation%20of%20R%20code][;; functions for evaluation of R code]] line
|
|
|
+| language | =last_value= function |
|
|
|
+|------------+-----------------------------|
|
|
|
+| R | .Last.value |
|
|
|
+| ruby | _ |
|
|
|
+| python | _ |
|
|
|
+| shell | see [[* last command for shells][last command for shells]] |
|
|
|
+| emacs-lisp | see [[* emacs-lisp will be a special case][special-case]] |
|
|
|
+
|
|
|
+#+srcname: task-last-value
|
|
|
+#+begin_src ruby
|
|
|
+82 + 18
|
|
|
+#+end_src
|
|
|
|
|
|
-** TODO improve the source-block snippet
|
|
|
+***** last command for shells
|
|
|
+Do this using the =tee= shell command, and continually pipe the output
|
|
|
+to a file.
|
|
|
|
|
|
-[[file:~/src/emacs-starter-kit/src/snippets/text-mode/rst-mode/chap::name%20Chapter%20title][file:~/src/emacs-starter-kit/src/snippets/text-mode/rst-mode/chap::name Chapter title]]
|
|
|
-#+begin_example
|
|
|
-,#name : Chapter title
|
|
|
-,# --
|
|
|
-${1:Chapter}
|
|
|
-${1:$(make-string (string-width text) ?\=)}
|
|
|
+Got this idea from the following [[http://linux.derkeiler.com/Mailing-Lists/Fedora/2004-01/0898.html][email-thread]].
|
|
|
|
|
|
-$0
|
|
|
-#+end_example
|
|
|
+suggested from mailing list
|
|
|
|
|
|
-[[file:snippets/org-mode/sb][sb -- snippet]]
|
|
|
+#+srcname: bash-save-last-output-to-file
|
|
|
+#+begin_src sh
|
|
|
+while read line
|
|
|
+do
|
|
|
+ bash -c "$line" | tee /tmp/last.out1
|
|
|
+ mv /tmp/last.out1 /tmp/last.out
|
|
|
+done
|
|
|
+#+end_src
|
|
|
|
|
|
-waiting for guidance from those more familiar with yasnippets
|
|
|
+another proposed solution from the above thread
|
|
|
|
|
|
-** TODO resolve references to other org buffers/files
|
|
|
- This would allow source blocks to call upon tables, source-blocks,
|
|
|
- and results in other org buffers/files.
|
|
|
-
|
|
|
- See...
|
|
|
- - [[file:lisp/org-babel-ref.el::TODO%20allow%20searching%20for%20names%20in%20other%20buffers][org-babel-ref.el:searching-in-other-buffers]]
|
|
|
- - [[file:lisp/org-babel.el::defun%20org-babel%20find%20named%20result%20name][org-babel.el#org-babel-find-named-result]]
|
|
|
-** TODO resolve references to other non-org files
|
|
|
- - tabular data in .csv, .tsv etc format
|
|
|
- - files of interpreted code: anything stopping us giving such files
|
|
|
- similar status to a source code block?
|
|
|
- - Would be nice to allow org and non-org files to be remote
|
|
|
-** TODO figure out how to handle errors during evaluation
|
|
|
- R has a try function, with error handling, along the lines of
|
|
|
- python. I bet ruby does too. Maybe more of an issue for functional
|
|
|
- style; in my proposed scripting style the error just gets dumped to
|
|
|
- the org buffer and the user is thus alerted.
|
|
|
-** TODO figure out how to handle graphic output
|
|
|
-This is listed under [[* graphical output][graphical output]] in out objectives.
|
|
|
+#+srcname: bash-save-in-variable
|
|
|
+#+begin_src sh
|
|
|
+#!/bin/bash
|
|
|
+# so - Save Output. Saves output of command in OUT shell variable.
|
|
|
+OUT=`$*`
|
|
|
+echo $OUT
|
|
|
+#+end_src
|
|
|
|
|
|
-This should take advantage of the =:results file= option, and
|
|
|
-languages which almost always produce graphical output should set
|
|
|
-=:results file= to true by default. That would handle placing these
|
|
|
-results in the buffer. Then if there is a combination of =silent= and
|
|
|
-=file= =:results= headers we could drop the results to a temp buffer
|
|
|
-and pop open that buffer...
|
|
|
+and another
|
|
|
|
|
|
-** TODO Finalise behaviour regarding vector/scalar output
|
|
|
-*** DONE Stop spaces causing vector output
|
|
|
-This simple example of multilingual chaining produces vector output if
|
|
|
-there are spaces in the message and scalar otherwise.
|
|
|
+#+begin_quote
|
|
|
+.inputrc:
|
|
|
+"^[k": accept-line
|
|
|
+"^M": " | tee /tmp/h_lastcmd.out ^[k"
|
|
|
|
|
|
-[Not any more]
|
|
|
+.bash_profile:
|
|
|
+export __=/tmp/h_lastcmd.out
|
|
|
|
|
|
-#+begin_src R :var msg=msg-from-python
|
|
|
-paste(msg, "und R", sep=" ")
|
|
|
-#+end_src
|
|
|
+If you try it, Alt-k will stand for the old Enter; use "command $__" to
|
|
|
+access the last output.
|
|
|
|
|
|
-#+resname:
|
|
|
-: org-babel speaks elisp y python und R
|
|
|
+Best,
|
|
|
|
|
|
-#+srcname: msg-from-python
|
|
|
-#+begin_src python :var msg=msg-from-elisp
|
|
|
-msg + " y python"
|
|
|
-#+end_src
|
|
|
+--
|
|
|
|
|
|
-#+srcname: msg-from-elisp
|
|
|
-#+begin_src emacs-lisp :var msg="org-babel speaks"
|
|
|
-(concat msg " elisp")
|
|
|
-#+end_src
|
|
|
+Herculano de Lima Einloft Neto
|
|
|
+#+end_quote
|
|
|
|
|
|
-** TODO re-implement helper functions from org-R
|
|
|
-*** Initial statement [Eric]
|
|
|
- Much of the power of org-R seems to be in it's helper functions for
|
|
|
- the quick graphing of tables. Should we try to re-implement these
|
|
|
- functions on top of org-babel?
|
|
|
+***** emacs-lisp will be a special case
|
|
|
+While it is possible for emacs-lisp to be run in a console type
|
|
|
+environment (see the =elim= function) it is *not* possible to run
|
|
|
+emacs-lisp in a different *session*. Meaning any variable set top
|
|
|
+level of the console environment will be set *everywhere* inside
|
|
|
+emacs. For this reason I think that it doesn't make any sense to
|
|
|
+worry about session support for emacs-lisp.
|
|
|
|
|
|
- I'm thinking this may be useful both to add features to org-babel-R and
|
|
|
- also to potentially suggest extensions of the framework. For example
|
|
|
- one that comes to mind is the ability to treat a source-code block
|
|
|
- like a function which accepts arguments and returns results. Actually
|
|
|
- this can be it's own TODO (see [[* source blocks as functions][source blocks as functions]]).
|
|
|
-*** Objectives [Dan]
|
|
|
- - We want to provide convenient off-the-shelf actions
|
|
|
- (e.g. plotting data) that make use of our new code evaluation
|
|
|
- environment but do not require any actual coding.
|
|
|
-*** Initial Design proposal [Dan]
|
|
|
- - *Input data* will be specified using the same mechanism as :var
|
|
|
- references, thus the input data may come from a table, or
|
|
|
- another source block, and it is initially available as an elisp
|
|
|
- data structure.
|
|
|
- - We introduce a new #+ line, e.g. #+BABELDO. C-c C-c on that
|
|
|
- line will apply an *action* to the referenced data.
|
|
|
- - *Actions correspond to source blocks*: our library of available
|
|
|
- actions will be a library of org-babel source blocks. Thus the
|
|
|
- code for executing an action, and the code for dealing with the
|
|
|
- output of the action will be the same code as for executing
|
|
|
- source blocks in general
|
|
|
- - Optionally, the user can have the relevant source block inserted
|
|
|
- into the org buffer after the (say) #+BABELDO line. This will
|
|
|
- allow the user to fine tune the action by modifying the code
|
|
|
- (especially useful for plots).
|
|
|
- - So maybe a #+BABELDO line will have header args
|
|
|
- - :data (a reference to a table or source code block)
|
|
|
- - :action (or should that be :srcname?) which will be something
|
|
|
- like :action pie-chart, referring to a source block which will
|
|
|
- be executed with the :data referent passed in using a :var arg.
|
|
|
- - :showcode or something controlling whether to show the code
|
|
|
-
|
|
|
-*** Modification to design
|
|
|
- I'm implementing this, at least initially, as a new interpreter
|
|
|
- named 'babel', which has an empty body. 'babel' blocks take
|
|
|
- a :srcname header arg, and look for the source-code block with
|
|
|
- that name. They then execute the referenced block, after first
|
|
|
- appending their own header args on to the target block's header
|
|
|
- args.
|
|
|
+*** Further thoughts on 'scripting' vs. functional approaches
|
|
|
|
|
|
- If the target block is in the library of babel (a.o.t. e.g. the
|
|
|
- current buffer), then the code in the block will refer to the
|
|
|
- input data with a name dictated by convention (e.g. __data__
|
|
|
- (something which is syntactically legal in all languages...). Thus
|
|
|
- the babel block will use a :var __data__ = whatever header arg to
|
|
|
- reference the data to be plotted.
|
|
|
+ These are just thoughts, I don't know how sure I am about this.
|
|
|
+ And again, perhaps I'm not saying anything very radical, just that
|
|
|
+ it would be nice to have some options supporting things like
|
|
|
+ receiving text output in the org buffer.
|
|
|
|
|
|
-** TODO share org-babel
|
|
|
-how should we share org-babel?
|
|
|
+ I can see that you've already gone some way down the road towards
|
|
|
+ the 'last value' approach, so sorry if my comments come rather
|
|
|
+ late. I am concerned that we are not giving sufficient attention
|
|
|
+ to stdout / the text that is returned by the interpreters. In
|
|
|
+ contrast, many of our potential users will be accustomed to a
|
|
|
+ 'scripting' approach, where they are outputting text at various
|
|
|
+ points in the code block, not just at the end. I am leaning
|
|
|
+ towards thinking that we should have 2 modes of evaluation:
|
|
|
+ 'script' mode, and 'functional' mode.
|
|
|
|
|
|
-- post to org-mode and ess mailing lists
|
|
|
-- create a org-babel page on worg
|
|
|
-- create a short screencast demonstrating org-babel in action
|
|
|
+ In script mode, evaluation of a code block would result in *all*
|
|
|
+ text output from that code block appearing as output in the org
|
|
|
+ buffer, presumably as an #+begin_example...#+end_example. There
|
|
|
+ could be an :echo option controlling whether the input commands
|
|
|
+ also appear in the output. [This is like Sweave].
|
|
|
|
|
|
-*** examples
|
|
|
-we need to think up some good examples
|
|
|
+ In functional mode, the *result* of the code block is available as
|
|
|
+ an elisp object, and may appear in the org buffer as an org
|
|
|
+ table/string, via the mechanisms you have developed already.
|
|
|
|
|
|
-**** interactive tutorials
|
|
|
-This could be a place to use [[* org-babel assertions][org-babel assertions]].
|
|
|
+ One thing I'm wondering about is whether, in script mode, there
|
|
|
+ simply should not be a return value. Perhaps this is not so
|
|
|
+ different from what exists: script mode would be new, and what
|
|
|
+ exists currently would be functional mode.
|
|
|
|
|
|
-for example the first step of a tutorial could assert that the version
|
|
|
-of the software-package (or whatever) is equal to some value, then
|
|
|
-source-code blocks could be used with confidence (and executed
|
|
|
-directly from) the rest of the tutorial.
|
|
|
+ I think it's likely that, while code evaluation will be exciting
|
|
|
+ to people, a large majority of our users in a large majority of
|
|
|
+ their usage will not attempt to actually use the return value from
|
|
|
+ a source code block in any meaningful way. In that case, it seems
|
|
|
+ rather restrictive to only allow them to see output from the end
|
|
|
+ of the code block.
|
|
|
|
|
|
-**** answering a text-book question w/code example
|
|
|
-org-babel is an ideal environment enabling both the development and
|
|
|
-demonstrationg of the code snippets required as answers to many
|
|
|
-text-book questions.
|
|
|
+ Instead I think the most accessible way to introduce org-babel to
|
|
|
+ people, at least while they are learning it, is as an immensely
|
|
|
+ powerful environment in which to embed their 'scripts', which now
|
|
|
+ also allows them to 'run' their 'scripts'. Especially as such
|
|
|
+ people are likely to be the least capable of the user-base, a
|
|
|
+ possible design-rule would be to make the scripting style of usage
|
|
|
+ easy (default?), perhaps requiring a special option to enable a
|
|
|
+ functional style. Those who will use the functional style won't
|
|
|
+ have a problem understanding what's going on, whereas the 'skript
|
|
|
+ kiddies' might not even know the syntax for defining a function in
|
|
|
+ their language of choice. And of course we can allow the user to
|
|
|
+ set a variable in their .emacs controlling the preference, so that
|
|
|
+ functional users are not inconveniennced by having to provide
|
|
|
+ header args the whole time.
|
|
|
|
|
|
-**** something using tables
|
|
|
-maybe something along the lines of calculations from collected grades
|
|
|
+ Please don't get the impression that I am down-valuing the
|
|
|
+ functional style of org-babel. I am constantly horrified at the
|
|
|
+ messy 'scripts' that my colleagues produce in perl or R or
|
|
|
+ whatever! Nevertheless that seems to be how a lot of people work.
|
|
|
+
|
|
|
+ I think you were leaning towards the last-value approach because
|
|
|
+ it offered the possibility of unified code supporting both the
|
|
|
+ single evaluation environment and the functional style. If you
|
|
|
+ agree with any of the above then perhaps it will impact upon this
|
|
|
+ and mean that the code in the two branches has to differ a bit. In
|
|
|
+ that case, functional mode could perhaps after all evaluate each
|
|
|
+ code block in its own environment, thus (re)approaching 'true'
|
|
|
+ functional programming (side-effects are hard to achieve).
|
|
|
|
|
|
-**** file sizes
|
|
|
-Maybe something like the following which outputs sizes of directories
|
|
|
-under the home directory, and then instead of the trivial =emacs-lisp=
|
|
|
-block we could use an R block to create a nice pie chart of the
|
|
|
-results.
|
|
|
+#+begin_src sh
|
|
|
+ls > files
|
|
|
+echo "There are `wc -l files` files in this directory"
|
|
|
|
|
|
-#+srcname: sizes
|
|
|
-#+begin_src bash :results replace
|
|
|
-du -sc ~/*
|
|
|
#+end_src
|
|
|
|
|
|
-#+begin_src emacs-lisp :var sizes=sizes :results replace
|
|
|
-(mapcar #'car sizes)
|
|
|
-#+end_src
|
|
|
+*** even more thoughts on evaluation, results, models and options
|
|
|
|
|
|
-** TODO command line execution
|
|
|
-Allow source code blocks to be called form the command line. This
|
|
|
-will be easy using the =sbe= function in [[file:lisp/org-babel-table.el][org-babel-table.el]].
|
|
|
+Thanks Dan, These comments are invaluable.
|
|
|
|
|
|
-This will rely upon [[* resolve references to other buffers][resolve references to other buffers]].
|
|
|
+What do you think about this as a new list of priorities/requirements
|
|
|
+for the execution of source-code blocks.
|
|
|
+
|
|
|
+- Sessions
|
|
|
+ 1) we want the evaluation of the source code block to take place in a
|
|
|
+ session which can persist state (variables, current directory,
|
|
|
+ etc...).
|
|
|
+ 2) source code blocks can specify their session with a header argument
|
|
|
+ 3) each session should correspond to an Emacs comint buffer so that the
|
|
|
+ user can drop into the session and experiment with live code
|
|
|
+ evaluation.
|
|
|
+- Results
|
|
|
+ 1) each source-code block generates some form of results which (as
|
|
|
+ we have already implemented) is transfered into emacs-lisp
|
|
|
+ after which it can be inserted into the org-mode buffer, or
|
|
|
+ used by other source-code blocks
|
|
|
+ 2) when the results are translated into emacs-lisp, forced to be
|
|
|
+ interpreted as a scalar (dumping their raw values into the
|
|
|
+ org-mode buffer), as a vector (which is often desirable with R
|
|
|
+ code blocks), or interpreted on the fly (the default option).
|
|
|
+ Note that this is very nearly currently implemented through the
|
|
|
+ [[* DONE results-type header (vector/file)][results-type-header]].
|
|
|
+ 3) there should be *two* means of collecting results from the
|
|
|
+ execution of a source code block. *Either* the value of the
|
|
|
+ last statement of the source code block, or the collection of
|
|
|
+ all that has been passed to STDOUT during the evaluation.
|
|
|
|
|
|
-** TODO inline source code blocks [3/5]
|
|
|
- Like the =\R{ code }= blocks
|
|
|
+**** header argument or return line (*header argument*)
|
|
|
|
|
|
- not sure what the format should be, maybe just something simple
|
|
|
- like =src_lang[]{}= where lang is the name of the source code
|
|
|
- language to be evaluated, =[]= is optional and contains any header
|
|
|
- arguments and ={}= contains the code.
|
|
|
+ Rather than using a header argument to specify how the return value
|
|
|
+ should be passed back, I'm leaning towards the use of a =#+RETURN=
|
|
|
+ line inside the block. If such a line *is not present* then we
|
|
|
+ default to using STDOUT to collect results, but if such a line *is
|
|
|
+ present* then we use it's value as the results of the block. I
|
|
|
+ think this will allow for the most elegant specification between
|
|
|
+ functional and script execution. This also cleans up some issues
|
|
|
+ of implementation and finding which statement is the last
|
|
|
+ statement.
|
|
|
|
|
|
- (see [[* (sandbox) inline source blocks][the-sandbox]])
|
|
|
+ Having given this more thought, I think a header argument is
|
|
|
+ preferable. The =#+return:= line adds new complicating syntax for
|
|
|
+ something that does little more than we would accomplish through
|
|
|
+ the addition of a header argument. The only benefit being that we
|
|
|
+ know where the final statement starts, which is not an issue in
|
|
|
+ those languages which contain 'last value' operators.
|
|
|
|
|
|
-*** DONE evaluation with \C-c\C-c
|
|
|
-Putting aside the header argument issue for now we can just run these
|
|
|
-with the following default header arguments
|
|
|
-- =:results= :: silent
|
|
|
-- =:exports= :: results
|
|
|
+ new header =:results= arguments
|
|
|
+ - script :: explicitly states that we want to use STDOUT to
|
|
|
+ initialize our results
|
|
|
+ - return_last :: stdout is ignored instead the *value* of the final
|
|
|
+ statement in the block is returned
|
|
|
+ - echo :: means echo the contents of the source-code block along
|
|
|
+ with the results (this implies the *script* =:results=
|
|
|
+ argument as well)
|
|
|
|
|
|
-*** DONE inline exportation
|
|
|
-Need to add an interblock hook (or some such) through org-exp-blocks
|
|
|
-*** DONE header arguments
|
|
|
-We should make it possible to use header arguments.
|
|
|
+*** DONE rework evaluation lang-by-lang [4/4]
|
|
|
|
|
|
-*** TODO fontification
|
|
|
-we should color these blocks differently
|
|
|
+This should include...
|
|
|
+- functional results working with the comint buffer
|
|
|
+- results headers
|
|
|
+ - script :: return the output of STDOUT
|
|
|
+ - write a macro which runs the first redirection, executes the
|
|
|
+ body, then runs the second redirection
|
|
|
+ - last :: return the value of the last statement
|
|
|
+ -
|
|
|
|
|
|
-*** TODO refine html exportation
|
|
|
-should use a span class, and should show original source in tool-tip
|
|
|
+- sessions in comint buffers
|
|
|
|
|
|
-** TODO formulate general rules for handling vectors and tables / matrices with names
|
|
|
- This is non-trivial, but may be worth doing, in particular to
|
|
|
- develop a nice framework for sending data to/from R.
|
|
|
-*** Notes
|
|
|
- In R, indexing vector elements, and rows and columns, using
|
|
|
- strings rather than integers is an important part of the
|
|
|
- language.
|
|
|
- - elements of a vector may have names
|
|
|
- - matrices and data.frames may have "column names" and "row names"
|
|
|
- which can be used for indexing
|
|
|
- - In a data frame, row names *must* be unique
|
|
|
-Examples
|
|
|
-#+begin_example
|
|
|
-> # a named vector
|
|
|
-> vec <- c(a=1, b=2)
|
|
|
-> vec["b"]
|
|
|
-b
|
|
|
-2
|
|
|
-> mat <- matrix(1:4, nrow=2, ncol=2, dimnames=list(c("r1","r2"), c("c1","c2")))
|
|
|
-> mat
|
|
|
- c1 c2
|
|
|
-r1 1 3
|
|
|
-r2 2 4
|
|
|
-> # The names are separate from the data: they do not interfere with operations on the data
|
|
|
-> mat * 3
|
|
|
- c1 c2
|
|
|
-r1 3 9
|
|
|
-r2 6 12
|
|
|
-> mat["r1","c2"]
|
|
|
-[1] 3
|
|
|
-> df <- data.frame(var1=1:26, var2=26:1, row.names=letters)
|
|
|
-> df$var2
|
|
|
- [1] 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
|
|
|
-> df["g",]
|
|
|
- var1 var2
|
|
|
-g 7 20
|
|
|
-#+end_example
|
|
|
+**** DONE Ruby [4/4]
|
|
|
+- [X] functional results working with comint
|
|
|
+- [X] script results
|
|
|
+- [X] ensure scalar/vector results args are taken into consideration
|
|
|
+- [X] ensure callable by other source block
|
|
|
|
|
|
- So it's tempting to try to provide support for this in org-babel. For example
|
|
|
- - allow R to refer to columns of a :var reference by their names
|
|
|
- - When appropriate, results from R appear in the org buffer with "named
|
|
|
- columns (and rows)"
|
|
|
+#+srcname: ruby-use-last-output
|
|
|
+#+begin_src ruby :results replace
|
|
|
+a = 2
|
|
|
+b = 4
|
|
|
+c = a + b
|
|
|
+[a, b, c, 78]
|
|
|
+#+end_src
|
|
|
|
|
|
- However none (?) of the other languages we are currently supporting
|
|
|
- really have a native matrix type, let alone "column names" or "row
|
|
|
- names". Names are used in e.g. python and perl to refer to entries
|
|
|
- in dicts / hashes.
|
|
|
+#+resname: ruby-use-last-output
|
|
|
+| 2 | 4 | 6 | 78 |
|
|
|
|
|
|
- It currently seems to me that support for this in org-babel would
|
|
|
- require setting rules about when org tables are considered to have
|
|
|
- named columns/fields, and ensuring that (a) languages with a notion
|
|
|
- of named columns/fields use them appropriately and (b) languages
|
|
|
- with no such notion do not treat then as data.
|
|
|
+#+srcname: task-call-use-last-output
|
|
|
+#+begin_src ruby :var last=ruby-use-last-output :results replace
|
|
|
+last.flatten.size + 1
|
|
|
+#+end_src
|
|
|
|
|
|
- - Org allows something that *looks* like column names to be separated
|
|
|
- by a hline
|
|
|
- - Org also allows a row to *function* as column names when special
|
|
|
- markers are placed in the first column. An hline is unnecessary
|
|
|
- (indeed hlines are purely cosmetic in org [correct?]
|
|
|
- - Org does not have a notion of "row names" [correct?]
|
|
|
-
|
|
|
- The full org table functionality exeplified [[http://orgmode.org/manual/Advanced-features.html#Advanced-features][here]] has features that
|
|
|
- we would not support in e.g. R (like names for the row below).
|
|
|
-
|
|
|
-*** Initial statement: allow tables with hline to be passed as args into R
|
|
|
- This doesn't seem to work at the moment (example below). It would
|
|
|
- also be nice to have a natural way for the column names of the org
|
|
|
- table to become the column names of the R data frame, and to have
|
|
|
- the option to specify that the first column is to be used as row
|
|
|
- names in R (these must be unique). But this might require a bit of
|
|
|
- thinking about.
|
|
|
+#+resname: task-call-use-last-output
|
|
|
+: 5
|
|
|
|
|
|
+***** ruby sessions
|
|
|
|
|
|
-#+TBLNAME: egtable
|
|
|
-| col1 | col2 | col3 |
|
|
|
-|------+---------+------|
|
|
|
-| 1 | 2 | 3 |
|
|
|
-| 4 | schulte | 6 |
|
|
|
+#+srcname: first-ruby-session-task
|
|
|
+#+begin_src ruby :session schulte :results silent
|
|
|
+schulte = 27
|
|
|
+#+end_src
|
|
|
|
|
|
-#+TBLNAME: egtable2
|
|
|
-| 1 | 2 | 3 |
|
|
|
-| 4 | schulte | 6 |
|
|
|
+#+srcname: second-ruby-session-task
|
|
|
+#+begin_src ruby :session schulte :results silent
|
|
|
+schulte + 3
|
|
|
+#+end_src
|
|
|
|
|
|
-#+begin_src R var tabel=egtable
|
|
|
-tabel
|
|
|
+#+srcname: without-the-right-session
|
|
|
+#+begin_src ruby :results silent
|
|
|
+schulte
|
|
|
#+end_src
|
|
|
|
|
|
-#+resname:
|
|
|
-| "col1" | "col2" | "col3" |
|
|
|
-|--------+-----------+--------|
|
|
|
-| 1 | 2 | 3 |
|
|
|
-| 4 | "schulte" | 6 |
|
|
|
+**** DONE R [4/4]
|
|
|
|
|
|
+- [X] functional results working with comint
|
|
|
+- [X] script results
|
|
|
+- [X] ensure scalar/vector results args are taken into consideration
|
|
|
+- [X] ensure callable by other source block
|
|
|
|
|
|
-Another example is in the [[*operations%20in%20on%20tables][grades example]].
|
|
|
+To redirect output to a file, you can use the =sink()= command.
|
|
|
|
|
|
-** PROPOSED Are we happy with current behaviour regarding vector/scalar output?
|
|
|
-This simple example of multilingual chaining produces vector output if
|
|
|
-there are spaces in the message and scalar otherwise.
|
|
|
+#+srcname: task_R_B
|
|
|
+#+begin_src R :results value vector silent
|
|
|
+a <- 9
|
|
|
+b <- 10
|
|
|
+b - a
|
|
|
+a + b
|
|
|
+#+end_src
|
|
|
|
|
|
-#+begin_src R :var msg=msg-from-python
|
|
|
-paste(msg, "und_R", sep="_")
|
|
|
+#+srcname: task-R-use-other-output
|
|
|
+#+begin_src R :var twoentyseven=task_R_B() :results replace value
|
|
|
+83
|
|
|
+twoentyseven + 9
|
|
|
#+end_src
|
|
|
|
|
|
-#+srcname: msg-from-python
|
|
|
-#+begin_src python :var msg=msg-from-elisp
|
|
|
-msg + "_y_python"
|
|
|
+#+resname: task-R-use-other-output
|
|
|
+: 28
|
|
|
+
|
|
|
+**** DONE Python [4/4]
|
|
|
+- [X] functional results working with comint
|
|
|
+- [X] script results
|
|
|
+- [X] ensure scalar/vector results args are taken into consideration
|
|
|
+- [X] ensure callable by other source block
|
|
|
+
|
|
|
+#+srcname: task-new-eval-for-python
|
|
|
+#+begin_src python :results silent output scalar
|
|
|
+8
|
|
|
+9
|
|
|
+10
|
|
|
#+end_src
|
|
|
|
|
|
-#+srcname: msg-from-elisp
|
|
|
-#+begin_src emacs-lisp :var msg="org-babel_speaks"
|
|
|
-(concat msg "_elisp")
|
|
|
+#+srcname: task-use-new-eval
|
|
|
+#+begin_src python :var tasking=task-new-eval-for-python() :results replace
|
|
|
+tasking + 2
|
|
|
#+end_src
|
|
|
|
|
|
-** PROPOSED conversion between org-babel and noweb (e.g. .Rnw) format
|
|
|
- I haven't thought about this properly. Just noting it down. What
|
|
|
- Sweave uses is called "R noweb" (.Rnw).
|
|
|
-
|
|
|
- I found a good description of noweb in the following article (see
|
|
|
- the [[http://www.cs.tufts.edu/~nr/pubs/lpsimp.pdf][pdf]]).
|
|
|
-
|
|
|
- I think there are two parts to noweb, the construction of
|
|
|
- documentation and the extraction of source-code (with notangle).
|
|
|
+#+resname: task-use-new-eval
|
|
|
+: 12
|
|
|
|
|
|
- *documentation*: org-mode handles all of our documentation needs in
|
|
|
- a manner that I believe is superior to noweb.
|
|
|
-
|
|
|
- *source extraction* At this point I don't see anyone writing large
|
|
|
- applications with 100% of the source code contained in org-babel
|
|
|
- files, rather I see org-babel files containing things like
|
|
|
- - notes with active code chunks
|
|
|
- - interactive tutorials
|
|
|
- - requirements documents with code running test suites
|
|
|
- - and of course experimental reports with the code to run the
|
|
|
- experiment, and perform analysis
|
|
|
+**** DONE Shells [4/4]
|
|
|
+- [X] functional results working with comint
|
|
|
+- [X] script results
|
|
|
+- [X] ensure scalar/vector results args are taken into consideration
|
|
|
+- [X] ensure callable by other source block
|
|
|
|
|
|
- Basically I think the scope of the programs written in org-babel
|
|
|
- (at least initially) will be small enough that it wont require the
|
|
|
- addition of a tangle type program to extract all of the source code
|
|
|
- into a running application.
|
|
|
+#+srcname: task-shell-new-evaluation
|
|
|
+#+begin_src sh :results silent value scalar
|
|
|
+echo 'eric'
|
|
|
+date
|
|
|
+#+end_src
|
|
|
|
|
|
- On the other hand, since we already have named blocks of source
|
|
|
- code which reference other blocks on which they rely, this
|
|
|
- shouldn't be too hard to implement either on our own, or possibly
|
|
|
- relying on something like noweb/notangle.
|
|
|
+#+srcname: task-call-other-shell
|
|
|
+#+begin_src sh :var other=task-shell-new-evaluation() :results replace scalar
|
|
|
+echo $other ' is the old date'
|
|
|
+#+end_src
|
|
|
|
|
|
-** PROPOSED support for passing paths to files between source blocks
|
|
|
-Maybe this should be it's own result type (in addition to scalars and
|
|
|
-vectors). The reason being that some source-code blocks (for example
|
|
|
-ditaa or anything that results in the creation of a file) may want to
|
|
|
-pass a file path back to org-mode which could then be inserted into
|
|
|
-the org-mode buffer as a link to the file...
|
|
|
+#+resname: task-call-other-shell
|
|
|
+: $ Fri Jun 12 13:08:37 PDT 2009 is the old date
|
|
|
|
|
|
-This would allow for display of images upon export providing
|
|
|
-functionality similar to =org-exp-blocks= only in a more general
|
|
|
-manner.
|
|
|
+*** DONE implement a *session* header argument [4/4]
|
|
|
+=:session= header argument to override the default *session* buffer
|
|
|
|
|
|
-** DEFERRED use textConnection to pass tsv to R?
|
|
|
- When passing args from the org buffer to R, the following route is
|
|
|
- used: arg in buffer -> elisp -> tsv on file -> data frame in R. I
|
|
|
- think it would be possible to avoid having to write to file by
|
|
|
- constructing an R expression in org-babel-R-assign-elisp, something
|
|
|
- like this
|
|
|
+**** DONE ruby
|
|
|
|
|
|
-#+begin_src emacs-lisp
|
|
|
-(org-babel-R-input-command
|
|
|
- (format "%s <- read.table(textConnection(\"%s\"), sep=\"\\t\", as.is=TRUE)"
|
|
|
- name (orgtbl-to-tsv value '(:sep "\t" :fmt org-babel-R-quote-tsv-field))))
|
|
|
+#+srcname: task-ruby-named-session
|
|
|
+#+begin_src ruby :session schulte :results replace
|
|
|
+schulte = :in_schulte
|
|
|
#+end_src
|
|
|
|
|
|
- I haven't tried to implement this yet as it's basically just
|
|
|
- fiddling with something that works. The only reason for it I can
|
|
|
- think of would be efficiency and I haven't tested that.
|
|
|
+#+resname: task-ruby-named-session
|
|
|
+: :in_schulte
|
|
|
|
|
|
- This Didn't work after an initial test. I still think this is a
|
|
|
- good idea (I also think we should try to do something similar when
|
|
|
- writing out results frmo R to elisp) however as it wouldn't result
|
|
|
- in any functional changes I'm bumping it down to deferred for
|
|
|
- now. [Eric]
|
|
|
+#+srcname: another-in-schulte
|
|
|
+#+begin_src ruby :session schulte
|
|
|
+schulte
|
|
|
+#+end_src
|
|
|
|
|
|
-for quick tests
|
|
|
+#+resname: another-in-schulte
|
|
|
+: :in_schulte
|
|
|
+: :in_schulte
|
|
|
+: :in_schulte
|
|
|
|
|
|
-#+tblname: quick-test
|
|
|
-| 1 | 2 | 3 |
|
|
|
+**** DONE python
|
|
|
|
|
|
-#+srcname: quick-test-src-blk
|
|
|
-#+begin_src R :var vec=quick-test
|
|
|
-mean(mean(vec))
|
|
|
+#+srcname: python-session-task
|
|
|
+#+begin_src python :session what :results silent
|
|
|
+what = 98
|
|
|
#+end_src
|
|
|
|
|
|
-: 2
|
|
|
+#+srcname: python-get-from-session
|
|
|
+#+begin_src python :session what :results replace
|
|
|
+what
|
|
|
+#+end_src
|
|
|
|
|
|
-** DEFERRED re-implement R evaluation using ess-command or ess-execute
|
|
|
- I don't have any complaints with the current R evaluation code or
|
|
|
- behaviour, but I think it would be good to use the ESS functions
|
|
|
- from a political point of view. Plus of course it has the normal
|
|
|
- benefits of an API (insulates us from any underlying changes etc). [DED]
|
|
|
+#+resname: python-get-from-session
|
|
|
+: 98
|
|
|
|
|
|
- I'll look into this. I believe that I looked at and rejected these
|
|
|
- functions initially but now I can't remember why. I agree with
|
|
|
- your overall point about using API's where available. I will take
|
|
|
- a look back at these and either switch to using the ess commands,
|
|
|
- or at least articulate under this TODO the reasons for using our
|
|
|
- custom R-interaction commands. [Eric]
|
|
|
+**** DONE shell
|
|
|
|
|
|
- ess-execute
|
|
|
+#+srcname: task-shell-sessions
|
|
|
+#+begin_src sh :session what
|
|
|
+WHAT='patton'
|
|
|
+#+end_src
|
|
|
|
|
|
- Lets just replace =org-babel-R-input-command= with =ess-execute=.
|
|
|
+#+srcname: task-shell-sessions-what
|
|
|
+#+begin_src sh :session what :results replace
|
|
|
+echo $WHAT
|
|
|
+#+end_src
|
|
|
|
|
|
- I tried this, and although it works in some situations, I find that
|
|
|
- =ess-command= will often just hang indefinitely without returning
|
|
|
- results. Also =ess-execute= will occasionally hang, and pops up
|
|
|
- the buffer containing the results of the command's execution, which
|
|
|
- is undesirable. For now these functions can not be used. Maybe
|
|
|
- someone more familiar with the ESS code can recommend proper usage
|
|
|
- of =ess-command= or some other lower-level function which could be
|
|
|
- used in place of [[file:lisp/org-babel-R.el::defun%20org-babel%20R%20input%20command%20command][org-babel-R-input-command]].
|
|
|
+#+resname: task-shell-sessions-what
|
|
|
+: patton
|
|
|
|
|
|
-*** ess functions
|
|
|
-
|
|
|
-#+begin_quote ess-command
|
|
|
-(ess-command COM &optional BUF SLEEP NO-PROMPT-CHECK)
|
|
|
+**** DONE R
|
|
|
|
|
|
-Send the ESS process command COM and delete the output
|
|
|
-from the ESS process buffer. If an optional second argument BUF exists
|
|
|
-save the output in that buffer. BUF is erased before use.
|
|
|
-COM should have a terminating newline.
|
|
|
-Guarantees that the value of .Last.value will be preserved.
|
|
|
-When optional third arg SLEEP is non-nil, `(sleep-for (* a SLEEP))'
|
|
|
-will be used in a few places where `a' is proportional to `ess-cmd-delay'.
|
|
|
-#+end_quote
|
|
|
+#+srcname: task-R-session
|
|
|
+#+begin_src R :session what :results replace
|
|
|
+a <- 9
|
|
|
+b <- 8
|
|
|
+a + b
|
|
|
+#+end_src
|
|
|
|
|
|
-#+begin_quote ess-execute
|
|
|
-(ess-execute COMMAND &optional INVERT BUFF MESSAGE)
|
|
|
+#+resname: task-R-session
|
|
|
+: 17
|
|
|
|
|
|
-Send a command to the ESS process.
|
|
|
-A newline is automatically added to COMMAND. Prefix arg (or second arg
|
|
|
-INVERT) means invert the meaning of
|
|
|
-`ess-execute-in-process-buffer'. If INVERT is 'buffer, output is
|
|
|
-forced to go to the process buffer. If the output is going to a
|
|
|
-buffer, name it *BUFF*. This buffer is erased before use. Optional
|
|
|
-fourth arg MESSAGE is text to print at the top of the buffer (defaults
|
|
|
-to the command if BUFF is not given.)
|
|
|
-#+end_quote
|
|
|
+#+srcname: another-task-R-session
|
|
|
+#+begin_src R :session what :results replace
|
|
|
+a + b
|
|
|
+#+end_src
|
|
|
|
|
|
-*** out current setup
|
|
|
+*** DONE function to bring up inferior-process buffer [4/4]
|
|
|
|
|
|
- 1) The body of the R source code block is wrapped in a function
|
|
|
- 2) The function is called inside of a =write.table= function call
|
|
|
- writing the results to a table
|
|
|
- 3) The table is read using =org-table-import=
|
|
|
+This should be callable from inside of a source-code block in an
|
|
|
+org-mode buffer. It should evaluate the header arguments, then bring
|
|
|
+up the inf-proc buffer using =pop-to-buffer=.
|
|
|
|
|
|
-** DEFERRED Rework Interaction with Running Processes [2/5]
|
|
|
-*** DONE robust to errors interrupting execution
|
|
|
+For lack of a better place, lets add this to the
|
|
|
+`org-shiftmetadown-hook' hook.
|
|
|
|
|
|
-#+srcname: long-runner-ruby
|
|
|
-#+begin_src ruby :results silent
|
|
|
- sleep(10)
|
|
|
- :patton_is_an_grumpy
|
|
|
+**** DONE ruby
|
|
|
+
|
|
|
+#+srcname: task-ruby-pop-to-session
|
|
|
+#+begin_src ruby :var num=9 :var another="something else"
|
|
|
+num.times{|n| puts another}
|
|
|
#+end_src
|
|
|
|
|
|
-*** DEFERRED use =C-g= keyboard-quit to push processing into the background
|
|
|
-This may be possible using the `run-with-timer' command.
|
|
|
+**** DONE python
|
|
|
|
|
|
-I have no idea how this could work...
|
|
|
+#+srcname: task-python-pop-to-session
|
|
|
+#+begin_src python :var num=9 :var another="something else"
|
|
|
+another * num
|
|
|
+#+end_src
|
|
|
+**** DONE R
|
|
|
|
|
|
-#+srcname: long-runner-ruby
|
|
|
-#+begin_src ruby :results silent
|
|
|
- sleep(10)
|
|
|
- :patton_is_an_grumpy
|
|
|
+#+srcname: task-R-pop-to-session
|
|
|
+#+begin_src R :var a=9 :var b=8
|
|
|
+a * b
|
|
|
#+end_src
|
|
|
|
|
|
-*** TODO ability to select which of multiple sessions is being used
|
|
|
- Increasingly it is looking like we're going to want to run all
|
|
|
- source code blocks in comint buffer (sessions). Which will have
|
|
|
- the benefits of
|
|
|
- 1) allowing background execution
|
|
|
- 2) maintaining state between source-blocks
|
|
|
- - allowing inline blocks w/o header arguments
|
|
|
+**** DONE shell
|
|
|
|
|
|
-**** R sessions
|
|
|
- (like ess-switch-process in .R buffers)
|
|
|
-
|
|
|
- Maybe this could be packaged into a header argument, something
|
|
|
- like =:R_session= which could accept either the name of the
|
|
|
- session to use, or the string =prompt=, in which case we could use
|
|
|
- the =ess-switch-process= command to select a new process.
|
|
|
-
|
|
|
-*** TODO evaluation of shell code as background process?
|
|
|
- After C-c C-c on an R code block, the process may appear to
|
|
|
- block, but C-g can be used to reclaim control of the .org buffer,
|
|
|
- without interrupting the R evalution. However I believe this is not
|
|
|
- true of bash/sh evaluation. [Haven't tried other languages] Perhaps
|
|
|
- a solution is just to background the individual shell commands.
|
|
|
+#+srcname: task-shell-pop-sessions
|
|
|
+#+begin_src sh :var NAME="eric"
|
|
|
+echo $NAME
|
|
|
+#+end_src
|
|
|
|
|
|
- The other languages (aside from emacs lisp) are run through the
|
|
|
- shell, so if we find a shell solution it should work for them as
|
|
|
- well.
|
|
|
-
|
|
|
- Adding an ampersand seems to be a supported way to run commands in
|
|
|
- the background (see [[http://www.emacswiki.org/emacs/ExecuteExternalCommand#toc4][external-commands]]). Although a more extensible
|
|
|
- solution may involve the use of the [[elisp:(progn (describe-function 'call-process-region) nil)][call-process-region]] function.
|
|
|
-
|
|
|
- Going to try this out in a new file [[file:lisp/org-babel-proc.el][org-babel-proc.el]]. This should
|
|
|
- contain functions for asynchronously running generic shell commands
|
|
|
- in the background, and then returning their input.
|
|
|
+*** DEFERRED function to dump last N lines from inf-proc buffer into the current source block
|
|
|
|
|
|
-**** partial update of org-mode buffer
|
|
|
- The sleekest solution to this may be using a comint buffer, and
|
|
|
- then defining a filter function which would incrementally interpret
|
|
|
- the results as they are returned, including insertion into the
|
|
|
- org-mode buffer. This may actually cause more problems than it is
|
|
|
- worth, what with the complexities of identifying the types of
|
|
|
- incrementally returned results, and the need for maintenance of a
|
|
|
- process marker in the org buffer.
|
|
|
+Callable with a prefix argument to specify how many lines should be
|
|
|
+dumped into the source-code buffer.
|
|
|
|
|
|
-**** 'working' spinner
|
|
|
- It may be nice and not too difficult to place a spinner on/near the
|
|
|
- evaluating source code block
|
|
|
+*** REJECTED comint notes
|
|
|
|
|
|
-*** TODO conversion of output from interactive shell, R (and python) sessions to org-babel buffers
|
|
|
- [DED] This would be a nice feature I think. Although an org-babel
|
|
|
- purist would say that it's working the wrong way round... After
|
|
|
- some interactive work in a *R* buffer, you save the buffer, maybe
|
|
|
- edit out some lines, and then convert it to org-babel format for
|
|
|
- posterity. Same for a shell session either in a *shell* buffer, or
|
|
|
- pasted from another terminal emulator. And python of course.
|
|
|
+Implementing comint integration in [[file:lisp/org-babel-comint.el][org-babel-comint.el]].
|
|
|
+
|
|
|
+Need to have...
|
|
|
+- handling of outputs
|
|
|
+ - split raw output from process by prompts
|
|
|
+ - a ring of the outputs, buffer-local, `org-babel-comint-output-ring'
|
|
|
+ - a switch for dumping all outputs to a buffer
|
|
|
+- inputting commands
|
|
|
+
|
|
|
+Lets drop all this language specific stuff, and just use
|
|
|
+org-babel-comint to split up our outputs, and return either the last
|
|
|
+value of an execution or the combination of values from the
|
|
|
+executions.
|
|
|
+
|
|
|
+**** comint filter functions
|
|
|
+: ;; comint-input-filter-functions hook process-in-a-buffer
|
|
|
+: ;; comint-output-filter-functions hook function modes.
|
|
|
+: ;; comint-preoutput-filter-functions hook
|
|
|
+: ;; comint-input-filter function ...
|
|
|
+
|
|
|
+#+srcname: obc-filter-ruby
|
|
|
+#+begin_src ruby :results last
|
|
|
+1
|
|
|
+2
|
|
|
+3
|
|
|
+4
|
|
|
+5
|
|
|
+#+end_src
|
|
|
|
|
|
** DONE Remove protective commas from # comments before evaluating
|
|
|
org inserts protective commas in front of ## comments in language
|
Eric Schulte