vikasing

Introduction to node.js

noreply@blogger.com (Vikash Singh) — Thu, 12 Apr 2012 12:02:00 PDT

I recently had to give a presentation on a technology, I sorted out two: a graph db (neo4j) and node.js. I had heard a lot of about node.js and was not very sure what it really was, so I choose it for the presentation, because that made learn about it in more details. Here is the presentation:

Introduction to Node.js

View more presentations from Vikash Singh

Extracting meaningful text from webpages

noreply@blogger.com (Vikash Singh) — Mon, 26 Mar 2012 13:30:00 PDT

I was trying to extract the meaningful text from a webpage for a given URL for crowl. For example if I visit any news site for a particular article, I will find a lot of crap (clutter) with the news text, this includes: ads, related news stories, top news stories, comments on the article, other web site links and much more.
Lets take an example of this The Times of India article:

http://timesofindia.indiatimes.com/tech/news/hardware/83-year-old-woman-sues-Apple-for-1m/articleshow/12415012.cms

The useful text in the The Times of India article has around 30% share of total content, the remaining 70% is the clutter. You may argue that you need those links related to most popular stories, related stories etc. But sill a lot of extra stuff is there which we really don't care about. (Meaningful) Information extraction from such a page is a big nightmare. We can start with getting the HTML source and stripping the HTML tags from the text. Using regular expressions, lets remove all the links too. The resultant content will look like:

83-year-old woman sues Apple for $1m - The Times of India | The Times of India | | More More ADVERTISEMENT Hardware The Times of India The Times of India Indiatimes Web (by Google) Video Photos You are here:  »   »   » Hardware Breaking News: 83-year-old woman sues Apple for $1m The writer has posted comments on this articleANI | Mar 26, 2012, 04.42PM IST My Saved articles Read more:||||||| SHARE AND DISCUSS NEW YORK: An 83-year-old American woman has sued for 1 million dollars after she failed to see the glass door at the tech giant's office and smashed her face. Evelyn Paswall, a former Manhattan fur-company vice president, went to to return an on December 13. While approaching the store, Paswall didn't realize she was heading straight for a wall of glass. She smashed her face against it, breaking her nose, Paswall claims in her suit filed in the US Eastern District federal court. Now the Forest Hills, Queens, resident, Paswall claimed in her lawsuit that the company was negligent not elderly-proofing the store's see-through fa ade, The New York Post reports. She argues that Apple should have put marks on the glass that older people could spot before they come face-to-face with disaster. "The defendant was negligent . . . in allowing a clear, see-through glass wall and/or door to exist without proper warning," Paswall suit said. Hi ! Do you like this story? My saved articles RELATED COVERAGE Articles Blogs LATEST NEWS » ......

As you can observe the above text has a lot of extra text which we don't want. Attempts have been made to get extract the main content, here is one such article: How to Extract a Webpage’s Main Article Content
The Java program to get the above text: (Jsoup can be downloaded from here)

 public static void main(String[] args) throws Exception {
     String href="(.*?<\\/a>)";
     Document doc = Jsoup.connect("http://timesofindia.indiatimes.com/tech/news/hardware/83-year-old-woman-sues-Apple-for-1m/articleshow/12415012.cms").get();
            String source = doc.html();
            source = source.replaceAll(href, "");
     System.out.println(Jsoup.parse(source).text());
 }

The best Java lib I could find to get the main text from a web page was boilerpipe, and the same can be tested here. It does a pretty good job of removing the clutter around the meaningful text. Running the The Times of India news article link through boilerpipe gives the following text:

Tweet
NEW YORK: An 83-year-old American woman has sued Apple for 1 million dollars after she failed to see the glass door at the tech giant's office and smashed her face.
Evelyn Paswall, a former Manhattan fur-company vice president, went to Apple's Manhasset store to return an iPhone on December 13.
While approaching the store, Paswall didn't realize she was heading straight for a wall of glass.
She smashed her face against it, breaking her nose, Paswall claims in her suit filed in the US Eastern District federal court.
Now the Forest Hills, Queens, resident, Paswall claimed in her lawsuit that the company was negligent not elderly-proofing the store's see-through fa ade, The New York Post reports.
She argues that Apple should have put marks on the glass that older people could spot before they come face-to-face with disaster.
"The defendant was negligent . . . in allowing a clear, see-through glass wall and/or door to exist without proper warning," Paswall suit said.
Hi !

The above text is very close to what we want. Boilerpipe library is based on this paper. By combining Jsoup (to get the page title) with boilerpipe (to get the page content) we can get the meaningful content from a webpage.

A Note on YCSB

noreply@blogger.com (Vikash Singh) — Sat, 09 Jul 2011 13:02:00 PDT

Recently we had to benchmark a number of In-Memory databases available, mainly open source ones. I didn't know about YCSB until my architect told me about it.

YCSB = Yahoo! Cloud Serving Benchmark

It didn't impress me at first because it was from Yahoo! no offense but Yahoo! still expects us to pay for it's email POP3 access (Yahoo! Plus), they haven't learned anything from GMail, immaturity at its best. Nevertheless we started our benchmarking with Oracle and MongoDB. I know neither of them is an in-memory database but we liked the concept of memory mapped data of MongoDB.

I wrote the Oracle client for YCSB and MongoDB client was included with the benchmark code (thanks to Yen Pai). Writing a client for YCSB is fairly simple and that's what impressed me. But my impressions were washed away by horrible glitches I found in the included drivers as well as in YCSB code itself. There are a number of forks (including mine, which is a dead one by the way) which provide a lot of patches to the original YCSB code and include many new clients as well but the owner of the project Brian Frank Cooper has a very small interest in reviewing them.

I ran the first benchmark on 1,00,000 data sets for all the work loads provided with YCSB. Default workloads are not sufficient to test all the operation properly, which forced me to create my own workload configuration. It turned out that MongoDB was just 2-4 times faster than Oracle and that didn't impressed us much. So we considered Gemfire and Hazelcast as well, both "real" in-memory databases, one open source and other commercial (a 60 day trial in this case).

Again I had to write the clients for both the new DBs and it turned out to be a piece of cake. I have to admit YCSB has a great pluggability, plugging a client for any db just requires the driver libs + some 20 lines of code and you are done . YCSB can also run on multiple machines. YCSB offers a great platform for benchmarking any kind of database out there and same should be realized by Yahoo! or Brian Cooper who can put some more effort in its development.

Here are the results of MongoDB, Gemfire and Hazelcast benchmarks on 100000 data sets:

Operation (100,000)	DBs Throughput (operations/sec)
	Gemfire	MongoDB	Hazelcast
Write (ops/sec)	3032.324	5123.475	3709.336
Read (ops/sec)	7634.170	7825.338	4315.367

MongoDB turns out to be the winner, the reason which I can think of is that both Gemfire and Hazelcast use JVM but MongoDB leaves everything to OS by mapping the data into memory.

More about YCSB can be found here and on the wiki

Find Me Lazy

noreply@blogger.com (Vikash Singh) — Sun, 26 Jun 2011 02:42:00 PDT

I was supposed to write a New Year post six months back, I didn't. Someone last week asked me what are you best at, I didn't (or couldn't) answer, now I guess laziness is what I am best at. Last year's new year post can be found here which I posted just after 3 days of new year. This year I am late by just 6 months. So I'll summarize whats happened in last 18 months span:

Series I finished:
The Wire
OZ
Generation Kill
Six Feet Under
Twin Peaks
The Life and Times of Tim
24
The Lost Room
The Pacific
Daria
Long Way Round
An Idiot Abroad
Spartcus: Blood and Sand

Series which I started following
Breaking Bad
Game of Thrones
In Treatment
Its Always Sunny In Philadelphia
The Ricky Gervais Show
The Venture Bros.
The IT Crowd
Boardwalk Empire
Fringe

Apart from series, movies and games; few more insignificant things happened in my life:
Started a project Crowl and released first revision (0.1)
Shifted to Noida from Bangalore.
Started gizmoage.com.
Finished a couple of novels.
Finished following games:

Crysis 2
Blur
Battlefield: Bad Company 2
Need for Speed: Hot Pursuit
Call of Duty Black Ops
Call Of Duty Modern Warfare 2
Just Cause 2

To add to the list I bought a car and still learning how to drive with L sign on front as well as back. In Jack Sparrow way: the feeling which someone should have after getting a car, I don't have it.

Caution: A Blurry Pic Ahead!

A Simple URL Shortening Algorithm in JAVA

noreply@blogger.com (Vikash Singh) — Sat, 20 Nov 2010 15:25:00 PST

We have so many url shortening services available today, I am not sure what kind of algorithm they use to shorten a particular url. Given the limitations over the characters which can be used in a url it becomes pretty much obvious that we are limited to 62 alpha numeric chars i.e. [a-z 0-9 A-Z]. Though - (hyphen) and _ (underscore) are allowed in a url still we want to avoid them for many good reasons. Very obvious would be a bad looking url like http://xyz.com/c0--rw_ or http://xyz.com/______-.
Following is the simple implementation of base10 to base62 converter, that's all we need to shorten a url. With 62 chars and a unique string 7 char long we can shorten:

62⁷ = 3,521,614,606,208 urls

that's a lots of urls.

How shortening works in the present case:

Suppose you have a table with following columns:

1. unique auto increment id (long),
2. url (string),
3. base62 string (string)

Now the trick is that we convert unique id to base62 string not the url, and then the url is mapped to the unique id. For example if we want to shorten the following url:

http://news.xinhuanet.com/english2010/world/2010-11/18/c_13612801.htm

First we need to look for the last unique id in the table then add 1 to it and convert the resulting number to base62. Suppose last unique id was 678544325 now the next id 678544326 will be mapped to the above url and base62 of a 678544326 will be:

45*62⁴+57*62³+6*62²+23*62¹+20*62⁰

means a five char url, having following array indexes {45}{57}{6}{23}{20} in

String[] elements = {
                "a","b","c","d","e","f","g","h","i","j","k","l","m","n","o",
                "p","q","r","s","t","u","v","w","x","y","z","1","2","3","4",
                "5","6","7","8","9","0","A","B","C","D","E","F","G","H","I",
                "J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X",
                "Y","Z"
                };

which will give a base62 string: JVgxu and a shortened url can be http://xyz.com/JVgxu

Following is the java code to convert a number to base62 string

/**
 * @author vikasing
 *
 */
public class Base62Converter {
    private final int LENGTH_OF_URL_CODE=6;
    public String convertTo62Base(long toBeConverted)
    {
        String[] elements = {
                "a","b","c","d","e","f","g","h","i","j","k","l","m","n","o",
                "p","q","r","s","t","u","v","w","x","y","z","1","2","3","4",
                "5","6","7","8","9","0","A","B","C","D","E","F","G","H","I",
                "J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X",
                "Y","Z"
                };
        String convertedString="";
        int numOfDiffChars= elements.length;
        if(toBeConverted<numOfDiffChars+1 && toBeConverted>0)
        {
            convertedString=elements[(int) (toBeConverted-1)];
        }
        else if(toBeConverted>numOfDiffChars)
        {
            long mod = 0;
            long multiplier = 0;
            boolean determinedTheLength=false;
            for(int j=LENGTH_OF_URL_CODE;j>=0;j--)
            {
                multiplier=(long) (toBeConverted/Math.pow(numOfDiffChars,j));
                if(multiplier>0 && toBeConverted>=numOfDiffChars)
                {
                    convertedString+=elements[(int) multiplier];
                    determinedTheLength=true;
                }
                else if(determinedTheLength && multiplier==0)
                {
                    convertedString+=elements[0];
                }
                else if(toBeConverted<numOfDiffChars)
                {
                    convertedString+=elements[(int) mod];
                }
                
                mod=(long) (toBeConverted%Math.pow(numOfDiffChars,j));
                toBeConverted=mod;                
            }
            
        }
        return convertedString;
    }

}

Above code is part of the project Crowl on which I have been working for a while. File can be browsed under org.crow.utils package.

Update: found this precise code for base62 conversion on the web:

public String converter ( int base, long decimalNumber)
 {
  
   String tempVal = decimalNumber == 0 ? "0" : "";
         long mod = 0;

         while( decimalNumber != 0 ) {
             mod = decimalNumber % base;
             tempVal = baseDigits.substring( (int)mod, (int)mod + 1 ) + tempVal;
             decimalNumber = decimalNumber / base;
         }
         System.out.print(tempVal);
         return tempVal;
 }

I didn't check the performance of the above code but it is smaller than the first one but both give the same output.

10 TV series You Don't Want to Miss

noreply@blogger.com (Vikash Singh) — Sun, 24 Oct 2010 15:17:00 PDT

I watched (or started following) 15 TV series in 2009 itself and in 2010 I was able to finish another 6. Here is a list of some of the great series you wouldn't wanna miss:

1. The Sopranos: You'll have to get through a couple of episodes first to get to know whats really going on. Its one of the best series HBO ever produced, depicting a New Jersey mafia family. Watch Tony Sopranos running a mafia family and struggling with his own.

2. OZ: This is my favorite from HBO, it shows the daily life of prisoners in a maximum security penitentiary. It has got every possible face of crime. One of the many things I like about HBO is that it gives its characters a complete freedom and doesn't hesitate in showing anything, things which can't be seen on any other channel. The narration of character Augustus Hill is one of the best part of the series.

3. Dexter: When it comes to the narration Dexter takes the cake, Michael C. Hall is a great actor and he fits perfectly in the role of a serial killer. Although many times a factor of luck plays an important role in the life of Dexter, the character of Dexter is very strong. A serial killer who can't feel any emotions: that's new and fresh.

4. Curb Your Enthusiasm: Co-creator of Seinfeld Larry David unites with HBO to produce this masterpiece. Believe me this series is better than Seinfeld, I have to admit I could not finish watching Seinfeld beyond 5th season because of its repetitive expressions/actions/dialogues. I liked Kramer but others became dull and boring after a certain number of episodes. Curb Your Enthusiasm shows David's daily life and his unusual way of handling the everyday matters.

5. The Band of Brothers and The Pacific: If you liked Saving Private Ryan, you don't wanna miss these two mini-series produced by HBO with the collaboration of Steven Spielberg and Tom Hanks. Aren't the names of these two masters enough? Both are based on World War-II.

6. Rome: BBC has produced a few documentary series's on Rome before. For this 2 season series BBC unites with HBO to produce a great series, which covers the most important time in history of Roman Empire, the lives of Julius Caesar and Augustus when the Rome expanded the most.

7. Carnivàle: This is again from HBO, a kind of serious and disturbing drama series set in 1934. Its a story of a Carnivàle which travels around and a fugitive Ben who joins the Carnivàle. It is great to watch the superb acting from these nobodies. Its a (sadly) 2 season series with some amazing background music.

8. Arrested Development: Its neither like FRIENDS nor like Seinfeld, its a different kind of comedy series from Fox. A story of a broken (or stupid) family. Jason Bateman tries to fix the problems of the family only to find himself in the funniest situations.

9. Twin Peaks: Twin Peaks is a name of the town in which a murder takes place and an FBI agent visits the town to investigate the murder, this investigation covers the whole two seasons. The series is known for its bizarre characters e.g the main character Dale Cooper likes black coffee and cherry pie and it can be seen very often. Twin Peaks became a huge hit and almost gave birth to a cult.

10. Californication: I was surprised to see this kind of stuff coming from a non-HBO channel. This is an amazing drama/comedy series from Showtime. The main character Hank Moody is played by David Duchovny who played Fox Mulder in The X-Files. Moody is an interesting character who tries to live his life in his own way and does many 'nasty' things. Another interesting character is Charlie Runkle played by Evan Handler.

available memory less than 128mb!!! -1 half life 2

noreply@blogger.com (Vikash Singh) — Fri, 08 Jan 2010 13:08:00 PST

Half Life 2 has got some problem on Vista 64 bit, it keeps crashing with a pop up message:

available memory less than 128mb!!! -1

Solution for this is to run Half Life in Windows XP Compatibility Mode, right click on the short-cut of Half Life, click on the Compatibility tab, select Run this program in compatibility mode for:

Enable GZip Compression on Glassfish v3

noreply@blogger.com (Vikash Singh) — Tue, 05 Jan 2010 12:39:00 PST

Login to https://localhost:4848 (admin panel). Go to the Network Config > Network Listener and select the listener for which you want to enable gzip. Click on the HTTP tab, see below:

Scroll down until you get the following entries:

Select Compression= on, Put the mime types and click on Save button.
Its done!!

mysql and jtpl template engine tutorial

noreply@blogger.com (Vikash Singh) — Sun, 03 Jan 2010 08:33:00 PST

jtpl is a light-weight java template engine, which is good for small size application but becomes sludgy for data intensive apps. Following example shows how to use jtpl with mysql.

jtpl replaces everything which is put inside {} and it uses html comments as entry and exit points

< ! -- BEGIN: main -- >
{This will be replaced by jtpl}
{ThisToo}
< ! -- END: main -- >

this template file should be saved with the extension .jtpl
In your servlet you need to create a Template object which will take template file as input parameter.

Template tpl = new Template(new File("FULL_PATH\\home.jtpl"));

next you need to assign the value to the template parameters like:

tpl.assign("ThisToo", "Assigned Value Here !");

in the end template is parsed using

tpl.parse("main");

If you have nested template regions in a template like this:

< ! -- BEGIN: main -- >
< ! -- BEGIN: header-- >
{Links}
< ! -- BEGIN: header-- >
< ! -- END: main -- >

Everything remains same except when you parse, you'll have to parse the inner region first, like:

tpl.parse("main.header");

then the outer (or main) region:

tpl.parse("main");

You can put as many regions you want inside a main region.
Using jtpl with mysql(or any other db) is simple as explained above. Here is a sample template file home.jtpl:

<!-- BEGIN: main1 -->
<html>
    <head>
        <title>{PTITLE}</title>       
    </head>
    <body>
            <div>               
                    <div>
                       <a class="a" href="/anylink1">{LINK1}</a>
                       <a class="a" href="/anylink2">{LINK2}</a>
                       <a class="a" href="/anylink3">{LINK3}</a>
                    </div>
            </div>
            <div>
                <div>
                    <div>
                        <!-- BEGIN: div -->
                            <div>
                                <a target="_blank" href ="{LINK}">{TITLE}</a>
                                <br><span>{CONTENT}</span>
                            </div>
                        <!-- END: div -->
                    </div>
                </div>
        </div>
    </body>
</html>
<!-- END: main1 -->

and here is the Servlet:

import net.sf.jtpl.Template;

import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.sql.*;

/**
 *
 * @author viksin
 */
public class sample extends HttpServlet {

    @Override
    protected void doGet(HttpServletRequest request,
            HttpServletResponse response) throws ServletException, IOException {
        PrintWriter out = response.getWriter();
        try {
            out.print(this.generatePage());
        } catch (Exception e) {
            e.printStackTrace(out);
        } finally {
            out.close();
        }
    }

    protected String generatePage() throws Exception {
        Template tpl = null;
        Connection conn = null;
        Statement st = null;
        ResultSet rs = null;
        String Content = "";
        TimeCalc tc = new TimeCalc();
        tpl = new Template(new File("FULL_PATH\\home.jtpl"));
        tpl.assign("PTITLE", "MySite");
        tpl.assign("LINK1", "Home");
        tpl.assign("LINK2", "News");
        tpl.assign("LINK3", "About");
        try {
            Class.forName("org.gjt.mm.mysql.Driver").newInstance();
            conn = DriverManager.getConnection("mysql_URL", "USERNAME", "PASSWORD");
            st = conn.createStatement();
            rs = st.executeQuery("select title,link,content from table");
            while (rs.next()) {
                tpl.assign("TITLE", rs.getString("title"));
                tpl.assign("CONTENT", rs.getString("content"));
                tpl.assign("LINK", rs.getString("link"));
                tpl.parse("main1.div");
            }           
            tpl.parse("main1");
        } catch (Exception ex) {
            return ex.toString();
        } finally {
            if (rs != null) {
                rs.close();
            }
            if (st != null) {
                st.close();
            }
            if (conn != null) {
                conn.close();
            }
        }
        return (tpl.out());
    }
}

jtpl is meant for small and simple applications, it does not have many features which other template engines like velocity, stringtemplate etc. have.
jtpl also uses the SingleThreadModel which is not recommended also it gets slower with large data.
Presently I am using StringTemplate which is faster and better than jtpl.

Bad Old Year! Happy New Year!

noreply@blogger.com (Vikash Singh) — Sat, 02 Jan 2010 16:40:00 PST

2009: One of the worst years of my life!

Here are a few facts which support above statement:

1. Got cheated for USB modem in March.

2. Got chicken-pox in April.

3. Lost money in shares in June

4. Father got into a major accident in Oct.

5. Got back pain in Nov.

6. Troubles at workplace (Whole Year!)

There were a few good moments too:

1. Got a beautiful niece (Asmi) on 7th March.

2. Had fun with college friends in Ahemdabad and Mumbai.

3. Watched following TV Series:

1) The Sopranos
2) Carnivale
3) Scrubs
4) Dead Like Me
5) 24
6) The Life and Times of Tim
7) Rome
8) John Adams
9) Avatar the last Airbender
10) Psych
11) That 70's Show
12) Curb Your Enthusiasm
13) Dexter
14) Heroes
15) Bones

2010: Expectations!

1. Complete the development of 9am.in, algowiki.com, showthe.info/about, paltan.org, index3r.com

2. Complete remaining TV series: The Wire, OZ, Deadwood, Futurama, Meerkat Manor, Generation Kill, Mash, Six Feet Under,Twin Peaks, From the Earth to the Moon
3. Self Employment
4. Get a powerful server for natmac.org

Glassfish v3 and NetBeans 6.8

noreply@blogger.com (Vikash Singh) — Sun, 13 Dec 2009 13:57:00 PST

Finally they are out but after using the combo, I have realized that the best integration of both was glassfish v2.1 and netbeans 7.1.

The new combo has got some troubles in integration I think (or I may be doing something wrong!). Whenever I try to restart the glassfish from netbeans, it stops but does not start properly, when browsing an application I get blank pages. Again I stop the server and start it from the console it works fine.

Creating virtual server is still a nightmare as it was in Beta release of GF.

Keep getting this exception in server log all the time:

SEVERE: PWC3989: An exception or error occurred in the container during the request processing
java.lang.ClassCastException: com.sun.grizzly.config.ContextRootInfo cannot be cast to org.apache.catalina.Context
at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:511)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:273)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:233)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at com.sun.grizzly.comet.CometEngine.executeServlet(CometEngine.java:473)
at com.sun.grizzly.comet.CometEngine.handle(CometEngine.java:341)
at com.sun.grizzly.comet.CometAsyncFilter.doFilter(CometAsyncFilter.java:84)
at com.sun.grizzly.arp.DefaultAsyncExecutor.invokeFilters(DefaultAsyncExecutor.java:161)
at com.sun.grizzly.arp.DefaultAsyncExecutor.interrupt(DefaultAsyncExecutor.java:137)
at com.sun.grizzly.arp.AsyncProcessorTask.doTask(AsyncProcessorTask.java:88)
at com.sun.grizzly.http.TaskBase.run(TaskBase.java:189)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:619)

One reason may be enabling the Comet support in GF, but no exact idea.

Don't put anything unknown inside C:\Program Files\sges-v3\glassfish\domains\domain1\docroot, you'll start receiving this warning and any JSP application will stop working.

WARNING: StandardWrapperValve[jsp]: PWC1406: Servlet.service() for servlet jsp threw exception
org.apache.jasper.JasperException: PWC6345: There is an error in invoking javac. A full JDK (not just JRE) is required
        at org.apache.jasper.compiler.DefaultErrorHandler.jspError(DefaultErrorHandler.java:89)
        at org.apache.jasper.compiler.ErrorDispatcher.dispatch(ErrorDispatcher.java:375)
        at org.apache.jasper.compiler.ErrorDispatcher.jspError(ErrorDispatcher.java:116)
        at org.apache.jasper.compiler.Jsr199JavaCompiler.compile(Jsr199JavaCompiler.java:200)
        at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:380)
        at org.apache.jasper.compiler.Compiler.compile(Compiler.java:449)
        at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:620)
        at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377)
        at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:483)
        at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:373)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
        at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:279)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
        at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
        at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)
        at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:332)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:233)
        at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)
        at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
        at com.sun.grizzly.comet.CometEngine.executeServlet(CometEngine.java:473)
        at com.sun.grizzly.comet.CometEngine.handle(CometEngine.java:341)
        at com.sun.grizzly.comet.CometAsyncFilter.doFilter(CometAsyncFilter.java:84)
        at com.sun.grizzly.arp.DefaultAsyncExecutor.invokeFilters(DefaultAsyncExecutor.java:161)
        at com.sun.grizzly.arp.DefaultAsyncExecutor.interrupt(DefaultAsyncExecutor.java:137)
        at com.sun.grizzly.arp.AsyncProcessorTask.doTask(AsyncProcessorTask.java:88)
        at com.sun.grizzly.http.TaskBase.run(TaskBase.java:189)
        at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
        at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
        at java.lang.Thread.run(Unknown Source)

Waiting for the fixes!!

tormac.info++

noreply@blogger.com (Vikash Singh) — Thu, 10 Dec 2009 14:04:00 PST

tormac.info is a torrent search engine which uses Google custom search service to search 250+ torrent sites.
Created it after getting frustrated jumping from one torrent site to another site for a video.

It also displays latest feeds updated on a few torrent sites.

News:

Netbeans 6.8 and Glassfish v3 released today.
Got a new 1.4kva UPS
updated natmac.org home page

General Problems encountered when parsing RSS or ATOM feeds

noreply@blogger.com (Vikash Singh) — Wed, 18 Nov 2009 08:24:00 PST

RSS and ATOM are XML specifications for web syndication. 9am completely depends on RSS/ATOM feeds discovered on the web, it uses Argotic Syndication Framework .Net library for parsing the feed, it is able to do the job most of the time, but whenever it encounters an unwanted element in the feed it throws an exception, which is expected. But my question is why these unwanted elements are there at first place? Have a look at these two RSS 2.0 feeds:


<?xml version="1.0"?>
<rss version="2.0">
<channel>
<title>Liftoff News</title>
<link>http://liftoff.msfc.nasa.gov/ </link>
<description>Liftoff to Space Exploration.</description>
<language>en-us</language>
<pubDate>Tue, 10 Jun 2003 04:00:00 GMT</pubDate>
<lastBuildDate>Tue, 10 Jun 2003 09:41:01 GMT</lastBuildDate>
<docs>http://blogs.law.harvard.edu/tech/rss </docs>
<generator>Weblog Editor 2.0</generator>
<managingEditor>editor@example.com</managingEditor>
<webMaster>webmaster@example.com</webMaster>
<item>
<title>Star City</title>
<link>http://liftoff.msfc.nasa.gov/news/2003/news-starcity.asp </link>
<description>How do Americans get ready to work with Russians aboard the International Space Station? They take a crash course in culture, language and protocol at Russia's Star City </description>
<pubDate>Tue, 03 Jun 2003 09:39:21 GMT</pubDate>
<guid>http://liftoff.msfc.nasa.gov/2003/06/03.html#item573 </guid>
</item>
</channel>
</rss>

Listing 1


<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
<channel>
<item>DDI NEWS</item>
<link>http://www.ddinews.com </link>
<description>The latest news from Doordashan news - India's largest broadcaster</description>
<copyright>Copyright: (C) Doordarshan News</copyright>
<item>
<title>Climate talks make progress, pressure on US  </title>
<description>Environment ministers made progress on Tuesday towards a scaled-down climate deal in Copenhagen next month, with Washington facing pressure to promise deep cuts by 2020 in greenhouse gas emissions. </description>
<link>http://www.ddinews.gov.in/Homepage/Homepage+-+Headlines/Climate+talks+make+progress.htm</link>
<pubDate>11/18/2009 1:53:18 PM</pubDate>
</item>
</channel>
</rss>

Listing 2

Listing 1 shows the correct format of an RSS 2.0 feed. Listing 2 is incorrect since tag <item> is there after tag <channel> which is not the right place for <item> tag. Argotic Syndication Framework throws an exception for the Listing 2 and doesn't parse it further.

This is one of the problems which I have observed along with the following:

1. Tag <language> missing or most of the time it has value "en-us" no matter what is the language used in the feed.
2. Tag <pubDate> is missing or having some wrong value.
3. Mismatch between the actual format of the feed and the format declared in <rss version="??"> tag.
4. Description is missing.
5. Feed Title is missing.
6. Item Title is missing ...most stupid and irritating.
7. Incomplete/Relative url in <link> tag

These are a few problems which I have seen with the web feeds, although I am talking about only RSS 2.0 here, similar problems can be imagined with ATOM/BLOGML and other syndication formats.

PITA 5: NetBeans 6.8 Beta & Glassfish v3 Problems!

noreply@blogger.com (Vikash Singh) — Sun, 15 Nov 2009 09:25:00 PST

Installed NetBeans 6.8 Beta last week, firstly I had some trouble creating virtual server:

Cause: Unable to create a new instance of 'com.sun.faces.application.resource.ResourceHandlerImpl': java.lang.NullPointerException

had to use default virtual server.

and now after a glassfish update last night, I am not able to build, deploy and undeploy any web app on glassfish. Keep getting this in server log:

org.apache.catalina.LifecycleException: PWC2769: Manager has not yet been started
at org.apache.catalina.session.StandardManager.stop(StandardManager.java:892)
at org.apache.catalina.core.StandardContext.stop(StandardContext.java:5383)
at com.sun.enterprise.web.WebModule.stop(WebModule.java:531)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:5211)
at com.sun.enterprise.web.WebModule.start(WebModule.java:500)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:928)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:912)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:694)
at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1932)
at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1604)
at com.sun.enterprise.web.WebApplication.start(WebApplication.java:90)
at org.glassfish.internal.data.EngineRef.start(EngineRef.java:126)
at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:241)
at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:236)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:337)
at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:340)
at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:163)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:87)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:84)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:128)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:457)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:401)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:125)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:915)
at org.jvnet.hk2.osgimain.Main.start(Main.java:140)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1077)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
at java.lang.Thread.run(Thread.java:619)
|#]
org.apache.catalina.LifecycleException: java.lang.NullPointerException
at org.apache.catalina.core.StandardContext.start(StandardContext.java:5216)
at com.sun.enterprise.web.WebModule.start(WebModule.java:500)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:928)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:912)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:694)
at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1932)
at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1604)
at com.sun.enterprise.web.WebApplication.start(WebApplication.java:90)
at org.glassfish.internal.data.EngineRef.start(EngineRef.java:126)
at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:241)
at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:236)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:337)
at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:340)
at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:163)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:87)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:84)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:128)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:457)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:401)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:125)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:915)
at org.jvnet.hk2.osgimain.Main.start(Main.java:140)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1077)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at org.glassfish.deployment.common.ClassDependencyBuilder.getAnnotatedClasses(ClassDependencyBuilder.java:57)
at org.glassfish.deployment.common.ClassDependencyBuilder.computeResult(ClassDependencyBuilder.java:41)
at org.glassfish.web.loader.ServletContainerInitializerUtil.checkAgainstInterestList(ServletContainerInitializerUtil.java:405)
at org.glassfish.web.loader.ServletContainerInitializerUtil.getInitializerList(ServletContainerInitializerUtil.java:313)
at org.apache.catalina.core.StandardContext.callServletContainerInitializers(StandardContext.java:5258)
at com.sun.enterprise.web.WebModule.callServletContainerInitializers(WebModule.java:551)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:5190)
... 37 more
|#]
java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: java.lang.NullPointerException
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:932)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:912)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:694)
at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1932)
at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1604)
at com.sun.enterprise.web.WebApplication.start(WebApplication.java:90)
at org.glassfish.internal.data.EngineRef.start(EngineRef.java:126)
at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:241)
at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:236)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:337)
at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:340)
at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:163)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:87)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:84)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:128)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:457)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:401)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:125)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:915)
at org.jvnet.hk2.osgimain.Main.start(Main.java:140)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1077)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
at java.lang.Thread.run(Thread.java:619)
|#]
java.lang.Exception: java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: java.lang.NullPointerException
at com.sun.enterprise.web.WebApplication.start(WebApplication.java:118)
at org.glassfish.internal.data.EngineRef.start(EngineRef.java:126)
at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:241)
at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:236)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:337)
at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:340)
at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:163)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:87)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:84)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:128)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:457)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:401)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:125)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:915)
at org.jvnet.hk2.osgimain.Main.start(Main.java:140)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1077)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
at java.lang.Thread.run(Thread.java:619)
|#]
java.lang.Exception: java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: java.lang.NullPointerException
at com.sun.enterprise.web.WebApplication.start(WebApplication.java:118)
at org.glassfish.internal.data.EngineRef.start(EngineRef.java:126)
at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:241)
at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:236)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:337)
at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:340)
at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:163)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:87)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:84)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:128)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:457)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:401)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:125)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:915)
at org.jvnet.hk2.osgimain.Main.start(Main.java:140)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1622)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1077)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
at java.lang.Thread.run(Thread.java:619)
|#]
|#]
|#]

For now I am using Tomcat and waiting for final release of glassfish v3.
Also I keep getting following error all the time whenever I try to edit a jsp file in NB:

java.lang.NullPointerException
at org.netbeans.modules.web.core.jsploader.BaseJspEditorSupport$4.propertyChange(BaseJspEditorSupport.java:176)
at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:339)
at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:276)
at org.openide.text.CloneableEditorSupport.firePropertyChange(CloneableEditorSupport.java:527)
at org.openide.text.CloneableEditorSupport.fireDocumentChange(CloneableEditorSupport.java:2320)
at org.openide.text.CloneableEditorSupport.closeDocument(CloneableEditorSupport.java:2201)
at org.openide.text.CloneableEditorSupport.access$2500(CloneableEditorSupport.java:123)
at org.openide.text.CloneableEditorSupport$StrongRef.run(CloneableEditorSupport.java:2681)
at org.netbeans.modules.openide.util.ActiveQueue.run(ActiveQueue.java:64)
[catch] at java.lang.Thread.run(Thread.java:619)

Hope this too gets resolved in final 6.8 release.

PITA 4: MySQL Error Messages

noreply@blogger.com (Vikash Singh) — Sun, 25 Oct 2009 14:32:00 PDT

Today while connecting to my DB Server, I got a strange error message:

Got error 28 from storage engine

Tried to check the .err file located in var/lib/mysql (Xubuntu), couldn't open the file, when checked, the file size was more than 50GB.

This was the reason of the above error (No free space on the disk to write!). Stopped MySQL server and deleted the .err file. After starting the server I checked the file for errors and found this error repeated for millions times:

091026 ***** InnoDB: Error: page ***** log sequence number * *********
InnoDB: is in the future! Current system log sequence number 0 330424282.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/mysql/en/backing-up.html for more information.

Since I copied the complete data folder from one computer to another, the log files for InnoDB got corrupted. I could not restore them.

The only option I had to avoid any data loss or corruption was to change the Engine type to MyISAM, which doesn't look for log files and can be repaired with less effort.

Lucene and MySQL (Correction!)

noreply@blogger.com (Vikash Singh) — Sat, 03 Oct 2009 18:10:00 PDT

In my 25 July 2009 post I mentioned about creating a lucene index of MySQL table, however there is glitch in the getSqlData() method, which can be corrected as following:


public void GetSqlData() {
        String IndexStoreDir = "F:/NewsIndex";      
        StandardAnalyzer analyzer = new StandardAnalyzer();
        Connection con = null;
        Statement stmt = null;
        ResultSet rs=null;
        try {
            String connUrl = "jdbc:mysql://localhost:3306/SQLDb"; 
           // put your mysql database name in place of SQLDb
            Connection con = DriverManager.getConnection(connUrl, "root", "SQLPassword");
           // put your mysql password in place of SQLPassword
            stmt = con.createStatement();
            Class.forName("com.mysql.jdbc.Driver");
            stmt.executeQuery("select * from table");
            IndWri = new IndexWriter(IndexStoreDir, analyzer, false, IndexWriter.MaxFieldLength.UNLIMITED);
            int numberOfUpdates=0;
            rs = stmt.getResultSet();

            while (rs.next()) {
                Document docu = new Document();
                Field title= new Field("title", rs.getString("title"), Field.Store.YES, Field.Index.ANALYZED);
                Field content= new Field("content", rs.getString("content"), Field.Store.YES, Field.Index.ANALYZED);
                docu.add(title);
                docu.add(content);
                IndWri.addDocument(docu);
            }
            IndWri.close();
           
        } catch (Exception ex) {
        }
        finally {
            if(rs!=null){try{rs.close();} catch(Exception ex2){}}
            if(stmt!=null){try{stmt.close();} catch(Exception ex1){}}
            if(con != null){try{con.close(); }catch(Exception e){}}

        }
    }

There is no need of addDoc() method. In previous case the index was getting created for (number of fields)*(number of records) number of docs because every time addDoc() method was called which created a new document for a field.

PITA 3: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

noreply@blogger.com (Vikash Singh) — Sat, 03 Oct 2009 17:28:00 PDT

I was trying to update around 70k records in a MySQL table through a loop, and kept getting the following exception after 16k+ records:

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
Last packet sent to the server was 0 ms ago.

Tried everything which I could get on the web, but nothing happened. Few suggestions which I found on the web are:

1. replace localhost with 127.0.0.1 in MySQL connection string... In some cases this works because Windows Defender removes localhost entry from C:\Windows\system32\drivers\etc\hosts file therefore MySQL won't be able to resolve the host and throw that exception.

2. modify registry to connect from TCP ports greater than 5000... this also didn't work for me, you can see the opened connection by typing the command netstat -b.
Details about this are given here http://support.microsoft.com/kb/196271

3. add wait_timeout = X in my.ini (or my.cnf on *nix env.)... this will help if you have problems with MySQL itself, for that check MySQL error log located in mysql Data directory. If you have any errors about connections setting wait_timeout may help, but in my case there are no errors from MySQL site.

How did I resolve it??... Instead of processing all the records in one execution I processed only 1000 records at a time.

However the exact reason is still unknown.

PITA 2: Edit my.cnf file in Ubuntu

noreply@blogger.com (Vikash Singh) — Sun, 13 Sep 2009 15:48:00 PDT

I tried to bind the local machine's global address to allow the remote connections for mysql (5.1) in my.cnf file, it kept giving me:

Error: no write permission for file "my.cnf"

tried edit etc/mysql/my.cnf and gedit etc/mysql/my.cnf also tried to change the permission of the file and the mysql folder by chmod with no success.

Until I found out putting sudo is necessary before every command if the user account is not root.
http://ubuntuforums.org/showthread.php?p=7077569

Used sudo gedit etc/mysql/my.cnf and it worked.

Feed Crawler Revisit Policy

noreply@blogger.com (Vikash Singh) — Sat, 29 Aug 2009 10:23:00 PDT

Every crawler has some revisit policy for crawling a web document. If a web site changes very frequently, the crawler will try to visit the site more than a web site which does not change so frequently, for example a crawler will be more interested in a news site than a static About Us page. This saves time for the crawler and avoids unwanted traffic for the web servers.

A feed crawler is crawler which tries to get the data from web site feeds, which may include: rss, atom, blogml etc. A feed crawler should expect more rapid changes than a normal web crawler and also it should penalize the slow changes. I have used an exponential function to implement the revisit policy for my feed crawler. Following graphs show three different functions which can be used as a revisit policy:

Definitions:

nd : Number of days to wait till next visit
x : Number of visits without a feed update (Blank Visits)
lc : Last Change (or Update)
nv : Next Visit
lv : Last Visit
s: staleness

Function 1: nd = 2^(x+1)/x

Function 2: nd = x²-2*x+k

Function 3: nd = e^x/2*x

The revisit algorithm works in following way (for Function 3):

s=lv-lc;
IF (s==0)
THEN nv=TOMORROW;
ELSE nv=TODAY + e^x/2*x --(any function can be used!)

If a feed is not updated after a certain number of visits the crawler should assume the feed as a dead feed and stop visiting that page. Also if a feed gets updated after certain time after some blank visits (say y), next visit can be calculated in following way (for Function 3):

nv= TODAY + e^(y-1)/2*(y-1)

Example: Take Function 2: x²-2*x+k (say k =3)

If a feed is updated after 5 blank visits, then: y=5;

F(y-1)= (y-1)²-2*(y-1)+3
= (5-1)²-2*(5-1)+3
=16-8+3=11

so next visit will be after 11 days.

If a feed is not updated after 5 blank visits, then: k=3 & y=5;

F(y) = (y)²-2*(y)+3
= (5)²-2*(5)+3
=25-10+3=18

now next visit will be after 18 days.

Lucene and MySQL

noreply@blogger.com (Vikash Singh) — Sat, 25 Jul 2009 10:07:00 PDT

I read about the full text search provided by MySQL over the web and had to look for a better option for indexing my data which grows everyday. Sphinx looked a better choice but the Java api is not fully mature yet so I had to choose Lucene which takes little more time than Sphinx in displaying the search results but is far better than MySQL full text search, comparison is given here. In some cases Lucene outperforms other two. Other advantage of Lucene is its sub-projects like: Solr, Compass, Nutch etc.

Here is the sample code for indexing the MySQL data: (Please find the corrected code here for the method getSqlData())

 
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.search.TopDocCollector;
import java.sql.*;
import java.io.IOException;

/**
*
* @author viksin
*/
public class LuceneImp {

public void getSqlData() {
   String IndexStoreDir = "F:/LuceneIndex";
   ResultSet rs;
   StandardAnalyzer analyzer = new StandardAnalyzer();
   IndexWriter writer = null;

   try {
       String connUrl = "jdbc:mysql://localhost:3306/SQLDb"; 
        // put your mysql database name in place of SQLDb
       Connection con = DriverManager.getConnection(connUrl, "root", "SQLPassword");
       // put your mysql password in place of SQLPassword
       Statement stmt = con.createStatement();
       Class.forName("com.mysql.jdbc.Driver");
       stmt.executeQuery("select * from table");
       writer = new IndexWriter(IndexStoreDir, analyzer, true, IndexWriter.MaxFieldLength.UNLIMITED);
       rs = stmt.getResultSet();
       while (rs.next()) {
      
           addDoc(writer, "title", rs.getString("title"));
           addDoc(writer, "content", rs.getString("content"));

       }
       writer.close();
       rs.close();
       stmt.close();
   } catch (Exception ex) {
   }
}

private static void addDoc(IndexWriter w, String field, String value) throws IOException {
   Document doc = new Document();
   doc.add(new Field(field, value, Field.Store.YES, Field.Index.ANALYZED));
   w.addDocument(doc);
}
}

If you want to add a new sql row to an existing index replace true with false in the following line and run the above program.

 writer = new IndexWriter(IndexStoreDir, analyzer, true, IndexWriter.MaxFieldLength.UNLIMITED);

You can put a column in the sql table (a flag) for checking and updating the indexing status of the row. In that case the you'll have to add following lines in the above code:

int indexStatus = 0;            
while (rs.next()) {
                indexStatus = rs.getInt("isindexed");
                if (indexStatus != 1) {
                    addDoc(writer, "title", rs.getString("title"));
                    addDoc(writer, "content", rs.getString("content"));
                    addDoc(writer, "link", rs.getString("link"));
                    addDoc(writer, "source", rs.getString("source"));
                    updateIndexStatus(rs.getString("id"));
                }

updateIndexStatus method will look like this:

            
    private static void updateIndexStatus(String Id) {
        try {
           String connUrl = "jdbc:mysql://localhost:3306/SQLDb"; 
           // put your mysql database name in place of SQLDb
           Connection con = DriverManager.getConnection(connUrl, "root", "SQLPassword");
           // put your mysql password in place of SQLPassword
            Statement stmt = con.createStatement();
            Class.forName("com.mysql.jdbc.Driver");
            stmt.executeQuery("update table set isindexed='1' where id="+Id+"");
            stmt.close();
        } catch (Exception ex) {
        }
    }

For searching your index this method can help:

 private static void searchIndex() {
      StandardAnalyzer analyzer = new StandardAnalyzer();
      String qu = "Java and Linux"; // put your keyword here
      String IndexStoreDir = "F:/LuceneIndex";
      try {
          Query q = new QueryParser("title", analyzer).parse(qu);
          int hitspp = 100; //hits per page
          IndexSearcher searcher = new IndexSearcher(IndexStoreDir);
          TopDocCollector collector = new TopDocCollector(hitspp);
          searcher.search(q, collector);
          ScoreDoc[] hits = collector.topDocs().scoreDocs;
          System.out.println("Found " + hits.length + " hits.");
          for (int i = 0; i < hits.length; ++i) {
                int docId = hits[i].doc;
                Document d = searcher.doc(docId);
                System.out.println((i + 1) + ". " + d.get("title"));
            }
            searcher.close();
        } catch (Exception ex1) {
        }
    }

Or you can use Luke for browsing index created by Lucene.

Phenom II X4 940 BE Temperature Analysis

noreply@blogger.com (Vikash Singh) — Fri, 12 Jun 2009 14:22:00 PDT

Little Overclocking and you'll hit the Max Temperature allowed for Phenom II X4 940 BE which is 62°C. I hit that temperature when I overclocked to 3.6 GHz with 1.4 V. Cool'n'Quiet helps in decreasing the CPU temperature by 5-6°C. My current idle temperature is 39°C-45°C with Cool'n'Quiet enabled and full load temperature goes upto 55°C (with playing FarCry2). Here is the analysis (using Biostar TA790GX 128MB):

Figure 1 (Y-axis: Temperature in °C, X-axis: Time in Mins)

Figure 1 shows the temperature for 3 different cases:

No Overclocking + Cool'N'Quiet enabled (Blue Line)
OCed to 3.4GHz + Cool'N'Quiet disabled + stock voltage (Red Line)
No Overclocking + Cool'N'Quiet disabled (Green Line)

Ran the test for one hour for each setting. The maximum temperature I hit was 55°C under full load and the lowest temperature was 39°C. My room temperature stays between 25-35°C so idle CPU temperature is not bad but its not good either. With stock CPU cooler I'll suggest to enable the cool n quiet feature. And if you want to overclock, don't exceed more than 3.4GHz, because that's the max you can go with the stock voltage. The Figure 2 shows the % of CPU usage in three conditions I mentioned above.

Figure 2

When OCed the CPU usage doesn't go more than 60%, but in case of Cool N Quiet enabled the CPU usage is around 90% most of the time.

Opening the case (CoolerMaster CM 690) from both the sides decreases the CPU temperature by 5°C and idle temperature hovers around 34°C-40°C (Figure 3). Will put more fans to improve the air flow, also will apply a new thermal paste some time soon. I am expecting an idle temperature around 30°C.

Figure 3

Here are a few screen shots of CPUID HWMonitor, CoreTemp64, CPU-Z:

Overclocking Phenom II 940BE with Biostar TA790GX 128MB

noreply@blogger.com (Vikash Singh) — Sun, 07 Jun 2009 09:28:00 PDT

I tried the Automate OverClock given in the Bios but it didn’t work, kept getting the blue screens on Vista x64. Tried to get the info about manual overclock option on the Google with no luck. Therefore I explored all the options which were available in the Bios. Here is how to over clock Phenom II 940 by increasing the Multiplier: (Will show how to overclock upto 3.6Ghz)

Step0: Disable Cool N Quiet in the Bios

Step1: Go to your bios T-Series tab, select Manual OverClock option

Step2: Click on the CPU FID/UID Control and Enable the Custom P-States

Step3: Select a Core FID

Note: Select a multiplier which your system can handle, higher multiplier means higher frequency hence it requires higher voltage which means higher CPU temperature. You need a very good cooling system (Cabinet!) for keeping your CPU temp low at higher frequencies.

Step4: Select a suitable voltage according the frequency selected in Step3 by clicking on the Core VID on the same screen. For 3.6Ghz select a voltage 1.4000V or little high but not very high.

Note: Over Clocking reduces the CPU life because of high voltage and temp. So if you want your CPU to last long (5+ years) don't go for over clocking.

Crysis Ends

noreply@blogger.com (Vikash Singh) — Sat, 23 May 2009 16:12:00 PDT

Finished playing Crysis, it was a great fun ride with a lot of action and also with a last moment crysis. Last level of Crysis has got a huge memory leak, it starts using 100% RAM and hangs. Patch 1.2 helps and running the game in DX 9 mode improves performance. This is how to set the Crysis in DirectX 9 mode: Open the Game Explorer (Vista) and right click on the Crysis icon.

The game is smaller than I expected, I was thinking in the line of FarCry but still it was fun playing Crysis. Downloading Crysis Warhead. One screenshot from last level of Crysis:

Also finished playing NFS Undercover, it is similar to Carbon, but again its just another NFS (Let me forget the Prostreet Horror!).

State of Broadband in India

noreply@blogger.com (Vikash Singh) — Mon, 18 May 2009 13:59:00 PDT

Today I received a surprise call from my Broadband provider, Tata Indicom. The person on the other end talked about their new Fair Usage Policy for those who have unlimited plans. The details of this policy are given here. The Indicom guy said that it was forced by the government (a lie!!). I have an unlimited 384Kbps plan, now according to this policy I can not upload+download more than 70GB (limitation!). I wonder if both speed and data transfer are restricted, why they call it an unlimited plan. On asking the same question from the Indicom guy, he said that this policy enables other users to use the Internet fairly (I use it unfairly!).

I have around 200GB in the downloading queue and with these restrictions in the place it will take around 4 months to download (not to forget the downtime and power cuts!). I can not get rid of Tata Indicom because Airtel does not cover my area (heard they also have similar policy!), Reliance broadband is worse than Indicom, BSNL guys are deaf and they don't really care about a thing in the world (govt employees!). Other option, USB Modems is another nightmare, never heard a good thing about them. Only BSNLs EVDO is good (based on rumours!) but again its availability and support is NIL in the Indian IT Hub.

To be frank, India (a third world country!!) is still far behing in this field, we pay huge amounts to get nothing but headache in the name of broadband. Lets pray that one day after hundred years we'll get an unlimited 32 Mbps connection.

Here are my speedtest results:

And I pay 1100+ Rs. for this, which works 15 days a month and 15 hrs in a day.

Gaming Begins

noreply@blogger.com (Vikash Singh) — Sun, 17 May 2009 13:48:00 PDT

I bought my first PC in 2004 having configuration: 2.5 Gz P4, 128MB of RAM, 40 GB HDD, CD-R, 15'' Monitor. With 16 MB shared graphics I was left with 112MB of RAM but the worst part was its SiS chipset which is known for its bad graphics and also a 4x AGP slot (not 8x !) and with this combination gaming was just a dream. However in 2007 I did a full upgrade with 1GB RAM (Max supported!), an Nvidia FX 5 series 8x 256 MB card and a 160 GB hard drive, this enabled me to play the games which were released in 2005 or before. This was a branded PC from Compaq IL-5030 Series.

This time I decided to assemble a PC with my configuration, which can play latest games coming out in the market as well as next two years games with a little part upgrades. So I went for this configuration:

AMD Phenom II X4 940 BE
ATI 4870 1 GB DDR5
4GB DDR2 800 MHz (will upgrade to 8GB soon!)
Biostar TA 790GX 128MB Mobo with Crossfire (I can put one more Gfx Card :D)
500GB HDD + 320GB (Ext) + 500GB (Ext) - will add another 1.5 TB internal HDD

Here are a few screenshots of NFS Undercover (with 1440x900 @ 75 Hz):

Click on the image to see the full resolution image. FYI currently I am playing NFS and Crysis.