getting started on automatic loading of elisp from org-mode files

lisp/org-babel-tangle.el

@@ -39,7 +39,7 @@ and shebang(#!) line to use when writing out the language to
 file.")
 
 (defun org-babel-tangle ()
-  "Extract the bodies of all source code blocks form the current
+  "Extract the bodies of all source code blocks from the current
 file into their own source-specific files."
   (interactive)
   (save-excursion
@@ -57,13 +57,15 @@ file into their own source-specific files."
                            (with-temp-file filename
                              (funcall lang-f)
                              (when she-bang (insert (concat she-bang "\n")))
-                             (comment-region (point) (progn (insert "generated by org-babel-tangle") (point)))
+                             (comment-region
+			      (point) (progn (insert "generated by org-babel-tangle") (point)))
                              (mapc #'org-babel-spec-to-string (reverse specs)))))
              ;; if there are multiple sessions then break out by session
              (if (> (length by-session) 1)
                  (mapc (lambda (session-pair)
                          (setq block-counter (+ block-counter (length (cdr session-pair))))
-                         (to-file (format "%s-%s.%s" base-name (car session-pair) ext) (cdr session-pair)))
+                         (to-file (format
+				   "%s-%s.%s" base-name (car session-pair) ext) (cdr session-pair)))
                        by-session)
                (setq block-counter (+ block-counter (length (cdr (car by-session)))))
                (to-file (format "%s.%s" base-name ext) (cdr (car by-session)))))))

org-babel.org

@@ -1,5 +1,5 @@
 #+TITLE: org-babel --- facilitating communication between programming languages and people
-#+SEQ_TODO: TODO PROPOSED | DONE DEFERRED REJECTED
+#+SEQ_TODO: PROPOSED TODO STARTED | DONE DEFERRED REJECTED
 #+OPTIONS: H:3 num:nil toc:t
 #+STARTUP: oddeven hideblocks
 
@@ -25,11 +25,10 @@ In this document:
 - The [[* Sandbox][Sandbox]] :: demonstrates much of the early/basic functionality
      through commented source-code blocks.
 
-Also see the [[file:library-of-babel.org][Library of Babel]] an extensible collection of ready-made
+Also see the [[file:library-of-babel.org][Library of Babel]], an extensible collection of ready-made
 and easily-shortcut-callable source-code blocks for handling common
 tasks.
 
-
 * Introduction
 
 Org-Babel enables *communication* between programming languages and
@@ -139,7 +138,13 @@ table, allowing the test suite to be run be evaluation of the table
 and the results to be collected in the same table.
 
 *** Emacs initialization files stored in Org-Mode buffers
-Once org-babel-tangle is completed this could be a very compelling use case.
+Using `org-babel-tangle' it is possible to embed your Emacs
+initialization into org-mode files.  This allows for folding,
+note-taking, todo's etc... embedded with the source-code of your Emacs
+initialization, and through org-mode's publishing features aids in
+sharing your customizations with others.
+
+**** my customization
 
 
 ** features
@@ -191,8 +196,35 @@ would then be [[#sandbox][the sandbox]].
 #+end_src
 
 
-* Tasks [27/42]
-** TODO support for working with =*Org Edit Src Example*= buffers [1/4]
+* Tasks [27/43]
+** TODO support for working with =*Org Edit Src Example*= buffers [2/4]
+*** TODO optionally evaluate header references when we switch to =*Org Edit Src*= buffer
+That seems to imply that the header references need to be evaluated
+and transformed into the target language object when we hit C-c ' to
+enter the *Org Edit Src* buffer [DED]
+
+Good point, I heartily agree that this should be supported [Eric]
+
+(or at least before the first time we attempt to evaluate code in that
+buffer -- I suppose there might be an argument for lazy evaluation, in
+case someone hits C-c ' but is "just looking" and not actually
+evaluating anything.) Of course if evaluating the reference is
+computationally intensive then the user might have to wait before they
+get the *Org Edit Src* buffer. [DED]
+
+I fear that it may be hard to anticipate when the references will be
+needed, some major-modes do on-the-fly evaluation while the buffer is
+being edited.  I think that we should either do this before the buffer
+is opened or not at all, specifically I think we should resolve
+references if the user calls C-c ' with a prefix argument.  Does that
+sound reasonable? [Eric]
+
+Yes [Dan]
+
+[Dan] So now that we have org-src-mode and org-src-mode-hook, I guess
+org-babel should do this by using the hook to make sure that, when C-c
+C-' is issued on a source block, any references are resolved and
+assignments are made in the appropriate session.
 *** TODO set buffer-local-process variables appropriately [DED]
     I think something like this would be great. You've probably
 already thought of this, but just to note it down: it would be really
@@ -207,8 +239,19 @@ I had not thought of that, but I agree whole heartedly. [Eric]
 
 Once this is done every variable should be able to dump regions into
 their inferior-process buffer using major-mode functions.
+*** DEFERRED send code to inferior process
+Another thought on this topic: I think we will want users to send
+chunks of code to the interpreter from within the *Org Edit Src*
+buffer, and I think that's what you have in mind already. In ESS that
+is done using the ess-eval-* functions. [DED]
+
+I think we can leave this up to the major-mode in the source code
+buffer, as almost every source-code major mode will have functions for
+doing things like sending regions to the inferior process.  If
+anything we might need to set the value of the buffer local inferior
+process variable. [Eric]
 
-*** TODO some possible requests/proposed changes for Carsten [2/3]
+*** DONE some possible requests/proposed changes for Carsten [4/4]
     While I remember, some possible requests/proposed changes for Carsten
     come to mind in that regard:
 
@@ -232,7 +275,8 @@ buffer."
 
 I think this is great, but I think it should be implemented in the
 org-mode core
-**** TODO Rename buffer and minor mode?
+
+**** DEFERRED Rename buffer and minor mode?
      Something shorter than *Org Edit Src Example* for the buffer
      name. org-babel is bringing org's source code interaction to a
      level of maturity where the 'example' is no longer
@@ -253,45 +297,10 @@ org-mode core
      perhaps we will also have more functionality to add to that minor
      mode, making it even more of a misnomer. Perhaps something like
      org-src-mode would be better.
-
-**** DEFERRED a hook called when the src edit buffer is created
-This should be implemented in the org-mode core
+**** DONE Changed minor mode name and added hooks
      
-
-*** DEFERRED send code to inferior process
-Another thought on this topic: I think we will want users to send
-chunks of code to the interpreter from within the *Org Edit Src*
-buffer, and I think that's what you have in mind already. In ESS that
-is done using the ess-eval-* functions. [DED]
-
-I think we can leave this up to the major-mode in the source code
-buffer, as almost every source-code major mode will have functions for
-doing things like sending regions to the inferior process.  If
-anything we might need to set the value of the buffer local inferior
-process variable. [Eric]
-
-*** TODO optionally evaluate header references when we switch to =*Org Edit Src*= buffer
-That seems to imply that the header references need to be evaluated
-and transformed into the target language object when we hit C-c ' to
-enter the *Org Edit Src* buffer [DED]
-
-Good point, I heartily agree that this should be supported [Eric]
-
-(or at least before the first time we attempt to evaluate code in that
-buffer -- I suppose there might be an argument for lazy evaluation, in
-case someone hits C-c ' but is "just looking" and not actually
-evaluating anything.) Of course if evaluating the reference is
-computationally intensive then the user might have to wait before they
-get the *Org Edit Src* buffer. [DED]
-
-I fear that it may be hard to anticipate when the references will be
-needed, some major-modes do on-the-fly evaluation while the buffer is
-being edited.  I think that we should either do this before the buffer
-is opened or not at all, specifically I think we should resolve
-references if the user calls C-c ' with a prefix argument.  Does that
-sound reasonable? [Eric]
-
-Yes [Dan]
+**** DEFERRED a hook called when the src edit buffer is created
+     This should be implemented in the org-mode core
 
 ** TODO resolve references to other org buffers/files
    This would allow source blocks to call upon tables, source-blocks,
@@ -319,6 +328,17 @@ languages which almost always produce graphical output should set
 results in the buffer.  Then if there is a combination of =silent= and
 =file= =:results= headers we could drop the results to a temp buffer
 and pop open that buffer...
+** TODO =\C-c \C-o= to open results of source block
+by adding a =defadvice= to =org-open-at-point= we can use the common
+=\C-c \C-o= keybinding to open the results of a source-code block.
+This would be especially useful for source-code blocks which generate
+graphical results and insert a file link as the results in the
+org-mode buffer.  (see [[* figure out how to handle graphic output][TODO figure out how to handle graphic output]]).
+This could also act reasonably with other results types...
+
+- file :: use org-open-at-point to open the file
+- scalar :: open results unquoted in a new buffer
+- tabular :: export the table to a new buffer and open that buffer
 
 ** TODO Finalise behaviour regarding vector/scalar output
 *** DONE Stop spaces causing vector output
@@ -343,12 +363,13 @@ msg + " y python"
 #+begin_src emacs-lisp :var msg="org-babel speaks"
 (concat msg " elisp")
 #+end_src
-** TODO share org-babel
+** STARTED share org-babel [1/4]
 how should we share org-babel?
 
-- post to org-mode and ess mailing lists
-- create a org-babel page on worg
-- create a short screencast demonstrating org-babel in action
+*** DONE post to org-mode
+*** TODO post to ess mailing list
+*** TODO create a org-babel page on worg
+*** TODO create a short screencast demonstrating org-babel in action
 
 *** examples
 we need to think up some good examples
@@ -1908,13 +1929,66 @@ This could probably be added to [[file:lisp/org-babel-script.el][org-babel-scrip
 (see [[* file result types][file result types]])
 
 
-* Bugs [16/21]
+* Bugs [18/23]
+** TODO Allow source blocks to be recognised when #+ are not first characters on the line
+   I think Carsten has recently altered the core so that #+ can have
+   preceding whitespace, at least for literal/code examples. org-babel
+   should support this.
+
+** TODO non-orgtbl formatted lists
+for example
+
+#+srcname: this-doesn't-match-orgtbl
+#+begin_src emacs-lisp :results replace
+'((:results . "replace"))
+#+end_src
+
+#+resname: this-doesn't-match-orgtbl
+
+
+** TODO collapsing consecutive newlines in string output
+
+#+srcname: multi-line-string-output
+#+begin_src ruby :results output
+"the first line ends here
+
+
+     and this is the second one
+
+even a third"
+#+end_src
+
+#+resname: multi-line-string-output
+
+
+** TODO cursor movement when evaluating source blocks
+   E.g. the pie chart example. Despite the save-window-excursion in
+   org-babel-execute:R. (I never learned how to do this properly: org-R
+   jumps all over the place...)
+   
 ** PROPOSED external shell execution can't isolate return values
 I have no idea how to do this as of yet.  The result is that when
 shell functions are run w/o a session there is no difference between
 the =output= and =value= result arguments.
 
-** TODO weird escaped characters in shell prompt break shell evaluation
+Yea, I don't know how to do this either.  I searched extensively on
+how to isolate the *last* output of a series of shell commands (see
+[[* last command for
+ shells][last command for shells]]).  The results of the search were basically
+that it was not possible (or at least not accomplish-able with a
+reasonable amount of effort).
+
+That fact combined with the tenancy to all ways use standard out in
+shell scripts led me to treat these two options (=output= and =value=)
+as identical in shell evaluation.  Not ideal but maybe good enough for
+the moment.
+
+In the `results' branch I've changed this so that they're not quite
+identical: output results in raw stdout contents, whereas value
+converts it to elisp, perhaps to a table if it looks tabular. This is
+the same for the other languages. [Dan]
+
+** DEFERRED weird escaped characters in shell prompt break shell evaluation
    E.g. this doesn't work. Should the shell sessions set a sane prompt
    when they start up? Or is it a question of altering
    comint-prompt-regexp? Or altering org-babel regexps?
@@ -1942,38 +2016,13 @@ the =output= and =value= result arguments.
    the user's regular emacs init.  I can't think of a way for us to
    set this automatically, and we are SOL without a regexp to match
    the prompt.
-
-** TODO non-orgtbl formatted lists
-for example
-
-#+srcname: this-doesn't-match-orgtbl
-#+begin_src emacs-lisp :results replace
-'((:results . "replace"))
-#+end_src
-
-#+resname: this-doesn't-match-orgtbl
-
-** TODO collapsing consecutive newlines in string output
-
-#+srcname: multi-line-string-output
-#+begin_src ruby :results replace
-"the first line ends here
-
-
-     and this is the second one
-
-even a third"
-#+end_src
-
-#+resname:
-: the first line ends here
-: 	     and this is the second one
-: 	return even a third
-
-** TODO cursor movement when evaluating source blocks
-   E.g. the pie chart example. Despite the save-window-excursion in
-   org-babel-execute:R. (I never learned how to do this properly: org-R
-   jumps all over the place...)
+** DONE make :results replace the default?
+   I'm tending to think that appending results to pre-existing results
+   creates mess, and that the cleaner `replace' option should be the
+   default. E.g. when a source block creates an image, we would want
+   that to be updated, rather than have a new one be added.
+   
+   I agree.
 ** DONE ruby evaluation not working under ubuntu emacs 23
    With emacs 23.0.91.1 on ubuntu, for C-h f run-ruby I have the
    following, which seems to conflict with [[file:lisp/langs/org-babel-ruby.el::let%20session%20buffer%20save%20window%20excursion%20run%20ruby%20nil%20session%20current%20buffer][this line]] in org-babel-ruby.el.
@@ -2157,7 +2206,7 @@ mean(mean(table))
 mean(table)
 #+end_src
 
-** DEFERRED org bug/request: prevent certain org behaviour within code blocks
+** DONE org bug/request: prevent certain org behaviour within code blocks
    E.g. [[]] gets recognised as a link (when there's text inside the
    brackets). This is bad for R code at least, and more generally
    could be argued to be inappropriate. Is it difficult to get org to
@@ -2166,6 +2215,9 @@ mean(table)
    I believe Carsten addressed this recently on the mailing list with
    the comment that it was indeed a difficult issue.  I believe this
    may be one area where we could wait for an upstream (org-mode) fix.
+
+   [Dan] Carsten has fixed this now in the core.
+
 ** DONE with :results replace, non-table output doesn't replace table output
    And vice versa. E.g. Try this first with table and then with len(table) [DED]
 #+begin_src python :var table=sandbox :results replace
@@ -2629,12 +2681,14 @@ out...
 table.first.join(" - ")
 #+end_src
 
-: "1 - 2 - 3"
+#+resname:
+: 1 - 2 - 3
 
-#+begin_src python :var table=sandbox :results replace
+#+begin_src python :var table=sandbox
 table[0]
 #+end_src
 
+#+resname:
 | 1 | 2 | 3 |
 
 #+begin_src ruby :var table=sandbox :results replace