In short this is an example document of a project-less literate Emacs/orgmode page with embedded SVGs that can be directly evaluated and exported to HTML. Everything is contained in one .org file. There are no additional files needed and no external image files.
I take a lot of notes using Emacs’ Orgmode. Often I will have little code snippets to add diagrams or plots. In some languages this is easier than in others. For instance the octave integration in Emacs is excellent and plots can be added pretty easily. However sometimes I want the power of a full programming language, so here I’ve tried to develop a simplified workflow that will work with Clojure.
- Projectless
- Clojure is dynamically typed and uses a REPL which can make it feel very interactive. However if you’ve used something like MATLAB.. well you don’t have the same level of seemless integration. B/c the language is general purpose, the language core includes very few features. Unlike Octave/R or similar languages, you typically won’t just drop into a REPL and go. You need to set up a project folder, source code directory structure, defined in a
deps.ednconfiguration file, specify what libraries you want, make some namespaces.. etc etc. I want to short circuit as much of that as possible. - Embedded SVG
- I also wanted to go a step further and have plots/diagrams directly included in the
.orgfile itself. It won’t work for every situation. You won’t want to inline a jpeg .. but for most situations this is sufficient in my use cases. And I want this embedded SVG to be in the exported HTML. SVG and HTML play nice together. I also don’t want to limit myself to some hacked together plotting library to imitate MATLAB. With the generic SVG primitives provided bything/geomyou aren’t just limited to plotting. You can do much much more (scroll to the bottom of the page for a TLDR)
Places of potential improvement are in green Note boxes. Feedback/Suggestions are very welcome :)
You naturally need Clojure installed. Then in Emacs you will need to have ob-clojure and cider enabled. Details here: https://www.orgmode.org/worg/org-contrib/babel/languages/ob-doc-clojure.html
Note: In principle this could be pared down further. This could work without
cider- just with aclojureorcljinferior process. However my ELisp is no-bueno and I wasn’t able to get this working.
The trick to getting this to work project-less (ie. without using a dep.edn) is to leverage an experimental tool in Clojure called add-libs that is in the tools.deps.alpha library/namespace. This is described in detail by Alex Miller: https://insideclojure.org/2018/05/04/add-lib/ (note: since publication the tool was renamed from add-lib to add-libs)
At the moment this is experimental and living on a branch. Maybe some day it’ll be part of core and this step can be skipped. We want this library to be automatically loaded when your projectless cider session gets launched. I’ve included the following in ~~/.clojure/deps.edn~ so that it is enable globally.
{
:deps { org.clojure/tools.deps.alpha {:git/url "https://github.com/clojure/tools.deps.alpha.git"
:sha "241cd24c35ba770aea4773ea161d45276e5d3a73"}}
}Note: Ideally this would only be included in cider sessions that are projectless (are missing a
deps.edn). If you know how to make such a conditional switch, please let me know - or post an answer here: clojure-emacs/cider#3025
We then cider-jack-in a cider session (preferrably in a directory where there with no Clojure project)
Note: ideally orgmode would automatically launch a session somehow in a temp folder.. somewhere in the background..
And we can start using add-libs directly to load libraries dynamically:
(use 'clojure.tools.deps.alpha.repl)
(add-libs {'thi.ng/geom {:mvn/version "1.0.0-RC4"}})
(use 'thi.ng.geom.viz.core)
(use 'thi.ng.geom.svg.core)This simply: 1) loads the add-libs namespace, 2) adds an SVG plotting library (fetching it from Maven) 3) loads some namespaces from that library
Unfortunately I need to rerun this block twice. The first run always seems to timeout:
executing Clojure code block...
nrepl-send-sync-request: Sync nREPL request timed out (ns user op eval code (use 'clojure.tools.deps.alpha.repl)
Note: I have no idea what’s going on here.. if anyone knows what’s up, please let me know
Once we have a library loaded and included in the default namespace, we can start using it directly. To demonstrate including SVG I’m using the fantastic ~thi-ng/geom~ library from Karsten Schmidt. The project is a bit intimidating in scope, bit it’s actually very modular bare-Clojure with no magic/macros. Each piece is usually easy to read and understand in isolation. The library has some very minimal SVG/Math/Vec cores and then other small practical namespaces built on top of this (it’s best to think of it as a collection of interconnecting mini-libraries). Here I’m using the plotting utility from geom-viz to make a very simple stacked interval plot which is then output in SVG hiccup and serialized to SVG XML. This was pulled directly from the examples: https://github.com/thi-ng/geom/blob/master/geom-viz/src/core.org#stacked-intervals
(->> {:x-axis (linear-axis
{:domain [-10 310]
:range [50 550]
:major 100
:minor 50
:pos 150})
:y-axis (linear-axis
{:domain [0 4]
:range [50 150]
:visible false})
:data [{:values [[0 100] [10 90] [80 200] [250 300] [150 170] [110 120]
[210 280] [180 280] [160 240] [160 170]]
:attribs {:stroke-width "10px" :stroke-linecap "round" :stroke "#0af"}
:layout svg-stacked-interval-plot}]}
(svg-plot2d-cartesian)
(svg {:width 600 :height 200})
(serialize)
symbol)What’s great here is that this plot was immediately inserted (as shown) when I evaluated the source code block. I exported it to HTML and the SVG was automatically inlined. There are no .svg files. (Though we could ~spit~ one if we want). Now I can just grab this .html file and email it to some friends :)
Some things to note:
The source code block here required a header argument to specify that result should come out as an HTML block:
geom’s serialize (the second to last line) transforms its own SVG hiccup format to a valid SVG/XML string. If we stopped here then the cider session would show an output sourrounded by double quotes (as it does for all strings). We don’t want these quotes in the resulting HTML, so the result needs to be piped through symbol to remove those.
Note: I seem to be abusing
symbol, b/c this isn’t its intended use - but I don’t know of a better alternative. Please let me know if there is something better (that doesn’t involve tweakingob-clojure)
If you have multiple evaluations in one source code block (like function definitions) then all their outputs will show up in the #+RESULTS block (not just the last form). I’ve separate non-drawing stuff into their own source code block that has the results disabled (note how the last code block is one big threading macro)
By default #+RESULTS: blocks of evaluated org-mode source code blocks are not exported to the HTML so you will need to add a orgmode configuration at the top of your org file to enable this:
#+PROPERTY: header-args :results org :eval never-export :exports both
And that’s it! You can now evaluate code blocks (C-c C-c block-by-block or C-c C-v C-b for the whole buffer) and export to HTML and everything necessary will be in the .html file - with no images to drag around.
I think the end result is .. satisfactory. Admittedly it’s not yet as concise as writing code block in something purpose-built like Octave, but ultimately the flexibility of Clojure (even if a tad verbose) may be worth it in some situations. Probably with a good suite/library of convenience functions built on top of thing/geom one could get to much smaller focused domain-specific pieces of code.
This is mostly just to show what’s possible with thing/geom. The Clojure code is near instantaneous, export is immediate. However, running on a Core M-5Y10c, the final 437kb SVG string took a few minute to insert into the #+RESULTS block (I suspect an issue in orgmode itself).
So this might be pushing the limits of waht you wanna do :)
Code from: https://github.com/thi-ng/geom/blob/master/geom-svg/src/examples.org
(require
'[thi.ng.geom.core :as g]
'[thi.ng.geom.vector :refer [vec3]]
'[thi.ng.geom.matrix :as mat]
'[thi.ng.geom.circle :as c]
'[thi.ng.geom.polygon :as p]
'[thi.ng.geom.gmesh :as gm]
'[thi.ng.geom.mesh.subdivision :as sd]
'[thi.ng.geom.svg.core :as svg]
'[thi.ng.geom.svg.shaders :as shader]
'[thi.ng.geom.svg.renderer :as render]
'[thi.ng.math.core :as m])
(def width 640)
(def height 480)
(def model (g/rotate-y (mat/matrix44) m/SIXTH_PI))
(def view (apply mat/look-at (mat/look-at-vectors 0 1.75 0.75 0 0 0)))
(def proj (mat/perspective 60 (/ width height) 0.1 10))
(def mvp (->> model (m/* view) (m/* proj)))
(def diffuse (shader/normal-rgb (g/rotate-y (mat/matrix44) m/PI)))
(def uniforms {:stroke "white" :stroke-width 0.25})
(def shader-diffuse
(shader/shader
{:fill diffuse
:uniforms uniforms
:flags {:solid true}}))
(defn ring
[res radius depth wall]
(-> (c/circle radius)
(g/as-polygon res)
(g/extrude-shell {:depth depth :wall wall :inset -0.1 :mesh (gm/gmesh)})
(g/center)))
(def mesh
(->> [[1 0.25 0.15] [0.75 0.35 0.1] [0.5 0.5 0.05] [0.25 0.75 0.05]]
(map (partial apply ring 40))
(reduce g/into)
(sd/catmull-clark)
(sd/catmull-clark)))
;; 2d text label w/ projected anchor point
(defn label-3d
[p mvp screen [l1 l2]]
(let [p' (mat/project-point p mvp screen)
p2' (mat/project-point (m/+ p 0 0 0.2) mvp screen)
p3' (m/+ p2' 100 0)]
(svg/group
{:fill "black"
:font-family "Arial"
:font-size 12
:text-anchor "end"}
(svg/circle p' 2 nil)
(svg/line-strip [p' p2' p3'] {:stroke "black"})
(svg/text (m/+ p3' 0 -5) l1 {})
(svg/text (m/+ p3' 0 12) l2 {:font-weight "bold"}))))(let [screen (mat/viewport-matrix width height)
max-z (/ 0.75 2)]
(->> (svg/svg
{:width width :height height}
(render/mesh mesh mvp screen shader-diffuse)
(label-3d (vec3 0 0 max-z) mvp screen ["Shader" "Normal/RGB"]))
(svg/serialize)
symbol))