Clojure 101

Embedding a Clojure REPL in your production application

2009-05-21T08:07:00.001-07:00

You can create an interactive production application by embedding a Clojure REPL in your application. This can help you inspect and interrogate the running application for runtime status, allow you to make bug fixes and monitor running code.

When working on the open source project enclojure-clojure-lib, we created a org.enclojure.repl library which have all the necessary server and client functions for creating a socket based clojure.main/repl in your application.

You can add the following code in your application which starts a socket REPL at the speicified port and wait for a connection.

;Use specified socket
(def server-socket (org.enclojure.repl.main/run-repl-server 11345))

;Use next available socket port
(def server-socket (org.enclojure.repl.main/run-repl-server 0))
(def server-port (.getLocalPort server-socket))

You can have your monitoring application - connect to the remote REPL as follows:
(def+ {:keys [repl-fn result-fn close-fn]} (create-repl-client host port))

create-repl-client returns a map:
{:repl-fn repl-fn
:result-fn result-fn
:close-fn close-fn}

You can listen to the incoming socket data using the result-fn in another thread:
(.start (Thread. #(write-to-window (result-fn))))

You can send the command to the remote REPL as follows:
(repl-fn "(give-me-status)")

To close this client connection you call
(close-fn)

You can use the following command to shutdown the remote REPL server
(repl-fn "(org.enclojure.repl.main/close-server)")

If you use Enclojure Plugin in Netbeans - You can create a REPL window connected to your REPL enabled application. Check out the Remote unmanaged REPLs.

Embedding a Clojure REPL in your application for Local use (Non-socket)

Sometimes you also want to create a REPL for use with in your application. For example: Enclojure has a Netbeans IDE REPL which is running in Netbeans process and hence can manipulate the Netbeans IDE via the REPL. This helped us with the Enclojure plugin development. You can create a REPL for local use in Application as follows:

(def+ {:keys [repl-fn result-fn]} (create-clojure-repl))

You can use the repl-fn and result-fn as explained before. Following is the code snippet copied from the org.enclojure.repl.main. create-clojure-repl is wrapping the clojure.main/repl to start the REPL in a thread. This is the function used in both local and remote Socket based REPL.


(def *printStackTrace-on-error* false)

(defn is-eof-ex? [throwable]
(and (instance? clojure.lang.LispReader$ReaderException throwable)
(or
(.startsWith (.getMessage throwable) "java.lang.Exception: EOF while reading")
(.startsWith (.getMessage throwable) "java.io.IOException: Write end dead"))))

(defn create-clojure-repl []
"This function creates an instance of clojure repl using piped in and out.
It returns a map of two functions repl-fn and result-fn - first function
can be called with a valid clojure expression and the results are read using
the result-fn."
(let [cmd-wtr (PipedWriter.)
        result-rdr (PipedReader.)
        piped-in (clojure.lang.LineNumberingPushbackReader. (PipedReader. cmd-wtr))
        piped-out (PrintWriter. (PipedWriter. result-rdr))
        repl-thread-fn #(binding [*printStackTrace-on-error* *printStackTrace-on-error*
                                                       *in* piped-in
                                                       *out* piped-out
                                                       *err* *out*]
               (try
                 (clojure.main/repl
                   :init (fn [] (in-ns 'user))
                   :read (fn [prompt exit]
                           (read))
                   :caught (fn [e]
                             (when (is-eof-ex? e)
                               (throw e))
                             (if *printStackTrace-on-error*
                               (.printStackTrace e *out*)
                               (prn (clojure.main/repl-exception e)))
                             (flush))
                   :need-prompt (constantly true))
                 (catch clojure.lang.LispReader$ReaderException ex
                   (prn "REPL closing"))
                 (catch java.lang.InterruptedException ex)
                 (catch java.nio.channels.ClosedByInterruptException ex)))]
(.start (Thread. repl-thread-fn))
{:repl-fn (fn [cmd]
     (if (= cmd ":CLOSE-REPL")
       (do
         (.close cmd-wtr)
         (.close result-rdr))
       (do
         (.write cmd-wtr cmd)
         (.flush cmd-wtr))))
;//??Using CharArrayWriter to build the string from each read of one byte
;Once there is nothing to read than this function returns the string read.
;Using partial so that CharArrayWriter is only created and once and reused.
;There could be better way.
:result-fn (partial
       (fn [wtr]
         (.write wtr (.read result-rdr))
         (if (.ready result-rdr)
           (recur wtr)
           (let [result (.toString wtr)]
             (.reset wtr)
             result)))
       (CharArrayWriter.))}))

Frequency table from sequences

2009-04-17T13:53:00.000-07:00

Following function creates the frequency table from the given sequences.

(defn frequency-table  [& seqs]
 (let [count-fn (fn [s item]
                  (update-in s [item] #(if % (inc %) 1)))]
   (reduce #(reduce count-fn %1 %2) {} seqs)))

(frequency-table [1 2 3])
=> {3 1, 2 1, 1 1}

(frequency-table ["fred" "ethel" "lucy"] ["lucy" "rick"] ["lucy" "ethel" "john"])
=> {"john" 1, "rick" 1, "lucy" 3, "ethel" 2, "fred" 1}

(frequency-table '(1 2 3) #{4 2 1})
=> {4 1, 3 1, 2 2, 1 2}

(frequency-table {:a 1 :b 3} {:a 1 :b 4})
=> {[:b 4] 1, [:b 3] 1, [:a 1] 2}

Sorting the list of sequences using max-key

2009-04-15T17:48:00.001-07:00

(max-key k x y)

max-key applies the k function (which should return integer) to the rest of the arguments and returns the argument which yields the greatest number. This can be used to bubble the largest sequence to the front.

(max-key count #{1 2 3} #{4 5})
=> #{1 2 3}

(max-key count #{1 2 3} #{4 5 6 7})
=> #{4 5 6 7}

(apply max-key count '(#{3 4 5} #{1 2} #{3}))
=> #{3 4 5}

Following function takes a list of sequences and return them in the order of their size. So the sequence with most number of elements are returned first.

(defn sort-seqs [seqs]
 (lazy-seq
   (when-not (empty? seqs)
     (let [largest-seq (apply max-key count seqs)]
       (cons largest-seq (sort-seqs (remove #(identical? largest-seq %) seqs)))))))

(sort-seqs '(#{3} #{3 4 5} #{1 2}))
=> (#{3 4 5} #{1 2} #{3})

(sort-seqs '([1 2] [3] [4 5 6 7]))
=> ([4 5 6 7] [1 2] [3])

(sort-seqs '({:a 1} {a 2 :b 4} {:c 5}))
=> ({a 2, :b 4} {:c 5} {:a 1})

Manipulating an inner item - (in-place editing)

2009-04-09T14:39:00.001-07:00

If you have nested associative structure than using assoc-in and update-in you can manipulate the inner structured item and get back a newly nested structure.

update-in takes the sequence of keys to find the location and a function to apply to the old value at that location:

(def m {:a [1 2 3]})
(update-in m [:a] conj 4)
=> {:a [1 2 3 4]}

Nested map update
(def m {:a {:b [1 2 3]}})
(update-in m [:a :b] conj 4)
=> {:a {:b [1 2 3 4]}}

Apply merge function to an inner map
(def m {:a 1 :config {:b 1}})
(update-in m [:config] merge {:b 3})
=> {:a 1, :config {:b 3}}

We can also apply the same functions to the vectors also. Vectors are associative maps and index are the keys.

(def v [:a {:c 2, :b 1} :d])
(v 2)
=> :d ;Item at the 3rd index

We can use both sequential and associative de-structuring binds on vectors:
(let [{fred 2} v]
fred)
=> :d

assoc-in also takes the sequence of keys and a new value. The following code is locating the 2nd item in the vector. assoc-in is adding the [:rows 42] key value pair to the found map and update-in is applying assoc to the found map with the given key value pairs.

(assoc-in v [1 :rows] 42)
=> [:a {:rows 42, :c 2, :b 1} :d]

(update-in v [1] assoc :rows 42 :cols 21)
=> [:a {:cols 21, :rows 42, :c 2, :b 1} :d]

Long term relationship function - Var object

2009-04-09T12:31:00.000-07:00

When you pass a function to another function as parameter than the value of the function var is passed. So if you redefine the passed function - the other function still sees the old defined function and doesn't use the new function. So if you want to establish a long term relationship between the two functions - you need to pass the Var object itself and not its value.
e.g.


(defn worker []
 (Thread/sleep 2000)
 (prn "not working"))

Try two commands on the repl:
user> worker
#<user$worker__319 user$worker__319@cbb3fa>

user> (var worker)
#'user/worker

user> #'worker
#'user/worker

The reader macro #'x expands to (var x). If you have the following function which calls the worker function in a thread:

(defn worker-thread [worker-fn]
(.start (Thread. #(dotimes [i 10] (worker-fn)))))

Launch the worker thread

user> (worker-thread worker)

Fix the worker function:

(defn worker []
(Thread/sleep 2000)
(prn "working"))

You will see that the thread still printing "not working" every 5 seconds.

If you had launched the worker thread as follows:

user> (worker-thread #'worker)

and now you fix the function:

(defn worker []
(Thread/sleep 2000)
(prn "really working"))

Thread will now print the new "really printing" message.

Destructuring binding in def

2009-04-01T13:17:00.000-07:00

A lot of times when I am debugging a clojure function in REPL. I end up doing defs for all my lets and if my let is using the destructuring bind than it become a little tedious process. The following macro helps me substitute the let with def+ to define the symbols in my REPL:

(defmacro def+

"def with binding (def+ [{:keys [a b d]} {:a 1 :b 2 :d 3}])"

[bindings]

(let [let-expr (macroexpand `(let ~bindings))

vars (filter #(not (.contains (str %) "__"))

(map first (partition 2 (second let-expr))))

def-vars (map (fn [v] `(def ~v ~v)) vars)]

(concat let-expr def-vars)))

;Example usage:

(def+ [a 1 b 2])

(def+ [{:keys [a b]} {:a 1 :b 2}])

(def+ [z 1

{:keys [a b]} {:a 1 :b 2}])

This should not be used in the final code - though it will work - but probably using in REPL is a better use for this macro.