Doooks Tech Stuff

How to fix Tomcat5 XPathFactory#newInstance() failed to create an XPathFactory for the default object model: http://java.sun.com/jaxp/xpath/dom with the XPathFactoryConfigurationException: javax.xml.xpath.XPathFactoryConfigurationException: No XPathFctory implementation found for the object model: http://java.sun.com/jaxp/xpath/dom

2010-09-23T12:19:00.000+01:00

Saw this exception on a RHEL5 box - our application stopped working after a yum update.

To fix it I deleted the symlink to /usr/share/java/xml-commons-apis.jar in /var/lib/tomcat5/common/endorsed and restarted tomcat.

The stack trace looked something like:

- XPathFactory#newInstance() failed to create an XPathFactory for the default object model: http://java.sun.com/jaxp/xpath/dom with the XPathFactoryConfigurationException: javax.xml.xpath.XPathFactoryConfigurationException: No XPathFctory implementation found for the object model: http://java.sun.com/jaxp/xpath/dom
java.lang.RuntimeException: XPathFactory#newInstance() failed to create an XPathFactory for the default object model: http://java.sun.com/jaxp/xpath/dom with the XPathFactoryConfigurationException: javax.xml.xpath.XPathFactoryConfigurationException: No XPathFctory implementation found for the object model: http://java.sun.com/jaxp/xpath/dom
at javax.xml.xpath.XPathFactory.newInstance(Unknown Source)
at com.sun.tools.xjc.reader.internalizer.Internalizer.(Internalizer.java:81)
at com.sun.tools.xjc.reader.internalizer.DOMForest.transform(DOMForest.java:448)
at com.sun.tools.xjc.api.impl.s2j.SchemaCompilerImpl.bind(SchemaCompilerImpl.java:221)
at org.apache.cxf.common.util.ReflectionInvokationHandler.invoke(ReflectionInvokationHandler.java:52)

How to Stop Debian Ignoring /etc/security/limits.conf

2010-08-18T13:59:00.002+01:00

I was trying to increase the number of open files allowed for a user on a Debian 5 box. Changing /etc/security/limits.conf was being ignored until I added the following line to /etc/pam.d/common-session.

session required pam_limits.so

Logging out and in again after adding that line gave the new session the ulimit I was trying to configure.

Took quite a lot of googling to find this: http://www.zimbra.com/forums/administrators/16066-solved-too-many-open-files-error.html

Faker for Clojure 1.1

2010-07-12T20:39:00.001+01:00

I wanted to use Sebastián Galkin's Faker library to create some sample data. It requires some bleeding edge stuff from Clojure 1.2. but my current project is built against Clojure 1.1, so I back-ported it to work with 1.1.

You can find the code at http://github.com/doooks/faker

How to Fix Tomcat5 on RHEL5 org.apache.xalan.processor.TransformerFactoryImpl not found

2010-04-27T10:26:00.001+01:00

I got this exception trying to start a Grails app on a RHEL5 box using OpenJDK and Tomcat5, all installed from Yum.

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'annotationHandlerAdapter': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter]: Constructor threw exception; nested exception is javax.xml.transform.TransformerFactoryConfigurationError: Provider org.apache.xalan.processor.TransformerFactoryImpl not found at java.lang.Thread.run(Thread.java:636)

Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class
[org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter]: Constructor threw exception; nested exception is javax.xml.transform.TransformerFactoryConfigurationError: Provider org.apache.xalan.processor.TransformerFactoryImpl not found

To fix I did a yum install of xerces and xalan, created a symlink from $catalina_home/common/endorsed/xalan.jar to /usr/share/java/xalan-j2.jar and restarted Tomcat.

Project Euler Problem 14 in Clojure

2010-03-15T20:06:00.000Z

This one was fun. I also learned a good lesson the hard way about lazy sequences. When I was playing with the code in the REPL everything worked fine, but I got confused when I hooked everything up and started getting empty lists out of functions. It looked like functions were returning the values pointed to by the atoms before I modified them. I'd read about how lazy sequences don't get evaluated until they are required, but it didn't occur to me that "for" would return immediately, without evaluating anything. In the REPL this doesn't happen of course.

Was also my first look at maps, and the loop macro.

Solves in about 40sec

(defn- next-int [n]
  (if (even? n)
    (/ n 2)
    (+ (* n 3) 1)))

(defn- grow [start-num]
  (let [sequence (atom (list start-num))]
    (loop [head (first @sequence)]
      (let [next (next-int head)]
        (swap! sequence conj next)
        (if (= 1 next)
          @sequence
          (recur next))))))

(defn- measure-sequences []
  (let [counts (atom {})]
    (doall
      (for [start-num (range 1 1000000)]
        (let [sequence (grow start-num)]
          (swap! counts assoc start-num (count sequence)))))
    @counts))

(defn max-map-entry [map]
  (first (sort (comparator #(> (val %1) (val %2))) (seq map))))

(defn solve []
  (first (max-map-entry (measure-sequences))))

Project Euler Problem 13 in Clojure

2010-03-15T19:53:00.000Z

Really not very interesting

;; defining big list of numbers elided

(defn solve[]
(.substring (str (apply + numbers)) 0 10))

Project Euler Problem 9 in Clojure

2010-03-15T19:48:00.001Z

Solves in about 16sec. I quite liked the solution though of course the forum posts on the Project Euler site show some much cleverer solutions.

(defn pythagorean-triplet? [a b c]
  (and
    (< a b)
    (< b c)
    (= (+ (* a a) (* b b)) (* c c))))

(defn solve []
  (first
    (for [a (range 1 1000) b (range a 1000) c (range b 1000)
      :when (and
        (pythagorean-triplet? a b c)
        (= 1000 (+ a b c)))]
    (* a b c))))

Project Euler Problem 8 in Clojure

2010-03-15T19:44:00.001Z

Not very interesting solution, since I use int/string conversions. As a clojure newcomer it feels a bit weird using the Java String API, just because you know its there. Writing functions to wrap the Java String API is probably not idiomatic Clojure, but I think it might be nice to wrap them in macros one day, to make the substring calls look a bit more Clojure-ish.

Solves in about 20msec.

(def bignumber "73167176531330624919......") ;; elided

(defn- calc-offsets []
  (range 0 (- 1000 5)))

(defn- five-digit-list [bigstring offset]
  (let [substr (.substring bigstring offset (+ 5 offset))]
    (map #(Integer/parseInt (str %)) substr)))

(defn- five-digit-lists [bigstring]
  (map #(five-digit-list bigstring %) (calc-offsets)))

(defn solve []
  (apply max (map #(apply * %) (five-digit-lists bignumber))))

Project Euler Problem 7 in Clojure

2010-03-11T21:56:00.002Z

Another idea stolen from Ed. This defines a lazy sequence of primes, and holds on to the head, so whichever primes have been realised are memoised, and so are cached for future access.

Takes about 45 seconds to run.

(def primes
  (let [known-primes (atom (list))]
    (for [n (iterate inc 2)
      :when (or
        (= 2 n)
        (not-any? #(zero? (mod n %)) @known-primes))]
    (do
      (swap! known-primes conj n)
      n))))

(defn solve []
  (nth primes 10000))

Project Euler Problem 6 in Clojure

2010-03-09T20:55:00.001Z

Wrote a square function so that the result wasn't cast to Double, so I didn't have to mess about with the formatting of the answer.

(defn square [num]
  (* num num))

(defn sum-squares [max]
  (reduce + (map #(square %) (range 1 max))))

(defn square-sum [max]
  (square (apply + (range 1 max))))

(defn solve []
  (- (square-sum 101) (sum-squares 101)))

Project Euler Problem 5 in Clojure

2010-03-08T22:21:00.000Z

Not very impressive performance, this brute-force method takes about 30 seconds to run. Could probably be a lot quicker if factors were chosen more intelligently.

(def factors (into () (range 20 3 -1)))

(defn- all-factors [number]
  (if (every? #(zero? (mod number %)) factors)
    number
    (recur (inc number))))

(defn solve[]
  (all-factors 1))

Project Euler Problem 4 in Clojure

2010-03-08T21:09:00.001Z

Most of my time was spent playing with for, trying to work out if it was best to do it all in the list comprehension. In the end I thought it was more readable to pull the creation of the products out into its own function.

(defn palindrome? [str]
  (if (= (first str) (last str))
    (if (<= 2 (count str))
      (recur (rest (butlast str)))
      true)
    false))

(defn three-digit-numbers []
  (range 100 1000))

(defn- products []
  (distinct
    (for [n1 (three-digit-numbers) n2 (three-digit-numbers)]
      (* n1 n2))))

(defn solve []
  (apply max (filter #(palindrome? (str %)) (products))))

Project Euler Problem 3 in Clojure - part 2

2010-03-07T11:46:00.001Z

So the solution in part 1 was no good - it took half an hour to run. I spent a while trying and failing to make a lazy, purely functional version but after talking to Ed about his solution I came up with this, which runs in about 500msec :

(defn factor? [big small]
  (zero? (mod big small)))

(defn prime? [n]
  (not-any? #(factor? n %) (range 2 n)))

(defn primes []
  (filter #(prime? %) (iterate inc 2)))

(defn prime-factors [n]
  (let [factors (atom ())]
    (last
     (for [candidate (primes) :while (< (apply * @factors) n)]
       (if (factor? n candidate)
      (swap! factors conj candidate))))))

(defn solve []
  (first (prime-factors 600851475143)))

Project Euler Problem 3 in Clojure - part 1

2010-03-04T22:46:00.006Z

This one is proving a bit tougher. First you need a list of prime numbers. I'm using the Euler's Sieve to produce primes.

First I wrote this function:

(defn sieve-numbers [position numbers]
  (if (>= position (count numbers))
  numbers
    (let [prime (nth numbers position)]
      (let [products (map #(* prime %) numbers)]
        (let [filtered (remove #(some #{%} products) numbers)]
          (recur (inc position) filtered))))))

which worked but was dog-slow:

;; (dotimes [_ 5] (time (sieve-euler 1000)))
;; "Elapsed time: 484.981564 msecs"
;; "Elapsed time: 459.624835 msecs"
;; "Elapsed time: 462.771112 msecs"
;; "Elapsed time: 460.312279 msecs"
;; "Elapsed time: 462.203786 msecs"
;; nil

To solve the problem we need to primes up to ceil(sqrt(N)) which is 775147 so that first method would take days.

To speed up the filtering I rewrote to use a SortedSet:

(defn sieve-numbers [primes candidates]
  (if (empty? candidates)
  primes
  (let [prime (first candidates)]
    (let [products (map #(* prime %) candidates)]
      (let [filtered (difference candidates products #{prime})]
        (recur (conj primes prime) filtered))))))

which is about 32 times faster:

;; (dotimes [_ 5] (time (sieve-euler 1000)))

;; "Elapsed time: 13.061787 msecs"
;; "Elapsed time: 12.994529 msecs"
;; "Elapsed time: 13.716897 msecs"
;; "Elapsed time: 13.237088 msecs"
;; "Elapsed time: 14.33136 msecs"

Still not good enough though. I need a faster sieve before I can even attempt to factor the large(ish) number and solve problem 3 inside a minute. I think my next step will be to pre-filter some of the non-primes with a Wheel Factorisation. This method wont remove all the non-primes, but it might be fun to implement.

I'm leaving the current sieve chugging away tonight to see how long it takes anyway, for comparison.

Project Euler Problem 2 in Clojure

2010-03-03T22:40:00.003Z

;; lazy fibo stolen from programming clojure p137
(defn fibo []
(map first (iterate (fn [[a b]] [b (+ a b)]) [1 2])))

(defn solve []
(reduce + (filter even? (take-while #(> 4000000 %) (fibo)))))

Project Euler Problem 1 in Clojure

2010-03-03T22:30:00.004Z

I'm trying to solve Project Euler using Clojure as an exercise to learn the language. My aim is to favour readability over performance, but still execute each problem inside a minute.

(defn is-multiple [number factor]
(zero? (mod number factor)))

(defn is-multiple-3-or-5 [number]
(or (is-multiple number 3)
(is-multiple number 5)))

(defn solve []
(reduce + (filter is-multiple-3-or-5 (range 1 1000))))

Easy way to get information about a running JVM

2010-01-05T13:46:00.002Z

kill -3 $PID

outputs a load of information about your running JVM to stdout. Very handy.

Happy Hacking Keyboard Lite 2 Xmodmap settings for Ubuntu Linux

2009-11-11T17:09:00.006Z

During the upgrade to Ubuntu 9.10 I managed to clobber my hard-won Xmodmap settings to make my Happy Hacking Keyboard Lite 2 (the USB version, with the arrow keys) work properly.

Here is the commented xmodmap file I use to modify the default key mappings, which is slightly British oriented. To use the file, run xmodmap path_to_file.

Some features:

£ and € symbols are mapped to right-diamond-3 and right-diamond-e, respectively.
Arrow keys work.
PageUp/PageDown on Fn-Arrows work.
Left Diamond is another Alt.
Tilde, pipe, backslash and backtick behave themselves.

I just hope I don't ever have to work all this out again.

------------------- SNIP FILE -------------------

! This is an xmodmap input file for the Happy Hacking Keyboard, model HHKB Lite 2. It assumes that the
! keymap is already set up for a gb-105 keyboard, then applies fixes for arrow keys and British conventions
!
! This file makes the following changes:
!
! Fix the arrow keys
!
keycode 111 = Up
keycode 116 = Down
keycode 113 = Left
keycode 114 = Right
!
! Fix the Fn-Arrows to give PgUp, PgDown, Home and End
!
keycode 112 = Prior
keycode 117 = Next
keycode 110 = Home
keycode 115 = End
!
! Fix the Alt and Diamond keys
!
keycode 133 = Super_L
keycode 134 = Mode_switch
keycode 64 = Alt_L
keycode 108 = Alt_R
!
! Put the @ on shift-' as British keyboards do
!
keycode 48 = apostrophe at
!
! Put the " on shift-2 as British keyboards do
!
keycode 11 = 2 quotedbl
!
! Put the Pound symbol on right-diamond-3
!
keycode 12 = 3 numbersign sterling sterling
!
! Put the Euro symbol on right-diamond-e
!
keycode 26 = e E EuroSign
!
! Fix tilde
!
keycode 49 = grave asciitilde
!
! Fix backslash
!
keycode 51 = backslash bar
!
!
! Remove all the existing modifier key mappings
!
clear Shift
clear Lock
clear Control
clear Mod1
clear Mod2
clear Mod3
clear Mod4
clear Mod5
!
! Put the modifier key mappings back
!
add Shift = Shift_L Shift_R
add Lock = Caps_Lock
add Control = Control_L Control_R
!
! including this line which makes both Alt keys and the left diamond key act as Alt keys
!
add Mod1 = Alt_L Alt_R Super_L
!
! makes the right diamond acts as the Mode_switch modifier (mapped to right diamond above)
!
add Mod3 = Mode_switch
add Mod5 = Scroll_Lock

------------------------- END OF FILE -----------------------

Share a git repository with unix groups

2009-08-20T15:52:00.004+01:00

If you have a shared Linux/Unix box that all your developers use to share code with git, there are various ways of setting up the shared repository. If you just use git over ssh, you don't need to maintain any more servers.

The simplest possible solutions assume that there is a single user account and different developers are authenticated on the server with their ssh public keys. I needed to restrict which developers get access to different repositories, and to be able to switch these permissions on and off. Unix has already solved this problem of course, with users and groups. Here is how to use real users and groups on your unix server to control git access.

Set up unix accounts for your users. "adduser jim"
Set up groups for each project, "addgroup wonderproject-developers"
Add the appropriate developers to the group "adduser jim wonderproject-developers"
In a private directory on your server, logged in as yourself, make a directory for the git repository "mkdir wonderproject.git"
Make the new group the group owner of the git repository "chgrp wonderproject-developers wonderproject.git/"
Fix the perms so only the group and the owner can read and write the repository "chmod 770 wonderproject.git/"
Make the repository setgid, so that any new files created inside it have group ownership the same as the directory "chmod g+s wonderproject.git"
cd wonderproject.git
create the git repository "git --bare init --shared=group" the --shared option ensures that git keeps the correct permissions when the various developers push commits into the repository.
Now, if your project is empty, clone this new repository onto your desktop and start committing "git clone ssh://jim@git.example.com:port/path/to/repos/wonderproject.git
If you have an existing repository on your desktop and you want to push it into the empty shared repository, edit your .git/config remote url to point at the new server url and then do a "git push --all"

Switching off Expect: 100-Continue header with Apache HttpClient

2009-06-30T11:01:00.003+01:00

Apache HTTP client version 4.0 (at least, the version bundled with the Android SDK) adds the header "Expect: 100-Continue" by default, which confuses many web servers. The following snippet switches this header off:

HttpClient client = new DefaultHttpClient();
client.getParams().setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, false);

I was seeing an error response like the following before I switched this header off:

HTTP/1.1 417 Expectation Failed.
Date: Tue, 30 Jun 2009 10:03:59 GMT.
Server: hi.
Vary: Accept-Encoding.
Content-Length: 364.
Connection: close.
Content-Type: text/html; charset=iso-8859-1.

417 Expectation Failed
Expectation Failed
The expectation given in the Expect request-header field could not be met by this server.
The client sent Expect: 100-Continue but we only allow the 100-continue expectation.

Copying package selections with debian/ubuntu

2009-06-30T10:17:00.005+01:00

This is documented in lots of places but it seems to always take me a while to find it. Here is how to install the set of packages from one machine to another.

On the source machine: sudo dpkg --get-selections > output_file
Transfer the output file to the target machine
On the target machine: sudo dpkg --set-selections <>
On the target machine: sudo apt-get -u dselect-upgrade

Screen Tweaks on Ubuntu Jaunty

2009-05-03T11:08:00.009+01:00

Screen is a must-have tool for anyone that spends time at the command line. I move my .screenrc file around with me whenever I move machine, and hadn't thought about it much for a while, but when I started screen on a Ubuntu Jaunty box with no .screenrc, I noticed that the system-wide screen config in /etc/screenrc is now quite useful.

The screenshot below shows the status line from the default config.

Using two whole lines for status messages seems a bit much but I'm a little old school when it comes to preserving display space. So you get load, CPU, free memory, a clock and named screens without any effort now.

I copied the /etc/screenrc to ~/.screenrc and added a few options

bind x
bind ^x
ignorecase on

The bind options stop C-ax, which normally detaches the terminal from screen, from working. I tend to type that keystroke by accident fairly frequently. The last option ignores case during history searches.

If you use lots of gnome-terminal windows or even tabs to manage your terminal sessions you should try screen, it will speed you up a lot once the keystrokes get wired into your muscle memory.

MyFaces NoSuchMethodError

2009-02-16T15:26:00.008Z

java.lang.NoSuchMethodError - org.apache.myfaces.component.html.ext.HtmlDataTable.refresh(Ljavax/faces/context/FacesContext;)V

This exception can be thrown by MyFaces when the page backing bean doesn't return a String that is mapped in the navigation-rules in faces-config.xml (or wherever you have your mapping file).

Check for typos or a missing entry in the navigation rules.

Hibernate gotcha: in clause with positional and named query parameters

2009-02-16T13:52:00.011Z

Here's a gotcha that can break JPA QL queries with an IN clause when using Hibernate as the persistence provider. The problem happens when you use positional parameters rather than named parameters.

Code such as the following will break with positional parameters:

List states = widgetService.findSomeWidgetStates();
Query q = entityManager.createQuery("select count(w) from Widget w where w.state in (?)");
q.setParameter(1, states);

with the error message:

Caused by: java.lang.IllegalArgumentException: Can not set java.lang.Integer field example.Widget.id to....

But if you change to named parameters, as in:

Query q = entityManager.createQuery("select count(w) from Widget w where w.state in (:widgetStates)");
q.setParameter("widgetStates", states);

Everything just works. So it seems named parameters are worth a little extra typing.