tag:blogger.com,1999:blog-87220489930941130712024-02-19T05:02:18.725-08:00The world is amazingUnknownnoreply@blogger.comBlogger170125tag:blogger.com,1999:blog-8722048993094113071.post-35578334313426484732022-01-20T14:01:00.002-08:002022-01-20T14:07:12.153-08:00InvalidTestClassError - No runnable methods<pre><span id="docs-internal-guid-1d04c775-7fff-2e1f-797b-7b184447a253"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Recently I encountered an interesting error while running a unit test. This test was passing from my Intellij IDE, but was failing from the maven. Below is the error I got from maven</span></span></pre><pre>[ERROR] com.test.SampleTest.initializationError Time elapsed: 0 s <<< ERROR!
org.junit.runners.model.InvalidTestClassError:
Invalid test class 'com.test.SampleTest':
1. No runnable methods
</pre>
The error was coming because my test class has following<div><div><ul style="text-align: left;"><li>extending <i>BaseTest</i> that extends the junit's <i>TestCase</i>.</li><li>running my test using <i>@Test</i> from <i>org.testng.annotations</i>.</li><li>using <i>@RunWith(JMockit.class)</i></li></ul></div>
<pre>@RunWith(JMockit.class)
class SampleTest extends BaseTest {
@BeforeClass // testng
public void setUp(){
super.setUp();
}
... then test cases
}
</pre><span id="docs-internal-guid-36cbb8aa-7fff-7624-3129-e108b0f8a52a"><p dir="ltr" style="background-color: white; line-height: 1.38; margin-bottom: 0pt; margin-top: 12pt; padding: 0pt 0pt 12pt;"><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">If I run the test without <i>JMockit</i> removes the </span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">No runnable methods</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> error. But it causes few other errors as my test was relying on mocked data. So I removed all the testng </span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">@Test</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> and replaced it with junit </span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">@Test</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">. </span></p><p dir="ltr" style="background-color: white; line-height: 1.38; margin-bottom: 0pt; margin-top: 12pt; padding: 0pt 0pt 12pt;"><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Then it raised another issue, </span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">BaseTest</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> class already has a </span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">setUp()</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> method which is not static, and I can't use it from a static method (</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">@BeforeClass</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> for </span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">setUp()</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">). So I pushed this </span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">super.setUp()</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> inside the </span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">@Before</span><span style="background-color: transparent; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> which runs every time the test runs. But I added a small trick to load it only once.</span></p></span><pre>@RunWith(JMockit.class)
class SampleTest extends BaseTest {
private boolean initialised = false;
@Before // junit
public void setUp(){
if(!initialised){
super.setUp();
initialised = true;
}
}
... then test cases
} </pre></div><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Another thing I tried was adding </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">@Ignore</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> for the </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">BaseTest</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> so that junit avoids it. But that didn't work for me.</span><div><br /></div><div>I was using the following command to run it from maven
<pre>mvn verify -Dtest=SampleTest -DfailIfNoTests=false
</pre></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-10106337492382524992022-01-06T15:25:00.002-08:002022-01-06T15:32:50.913-08:00Map with expiry time in Java<a href="https://docs.oracle.com/javase/8/docs/api/java/util/Map.html" target="_blank">Map</a> in Java holds the value as key value pair. And it keeps the data there until the end of the application lifecycle. But sometimes a case could arise where you need to use the map with an expiry time, after which your particular entry will be removed. For that purpose here are a few examples from the various libraries of Java.
<br><br>
<h1>1. <a href="https://commons.apache.org/proper/commons-collections/apidocs/org/apache/commons/collections4/map/PassiveExpiringMap.html" target="_blank">PassiveExpiringMap</a></h1>
<br><br>
Apache collection has a simple implementation for expiring hashmap which is PassiveExpiringMap. This uses the implementation of Map underneath. We just need to construct the object with a given expiry time after which an entry will be removed from the map.
<br><br>
<b>Maven dependency:</b>
<br><br>
<pre><dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.0</version>
</dependency>
</pre>
<br><br>
Code snippet to show the usage. In this example we are setting the expiry time 1000 milliseconds, and adding entries to the map every 500 milliseconds.
<br><br>
<pre>
int expiryInMills = 1000;
PassiveExpiringMap<String, Integer> map = new PassiveExpiringMap<>(expiryInMills);
for (int i = 0; i < 10; i++) {
map.put(String.valueOf(i), i);
System.out.println(map.keySet());
Thread.sleep(500);
}
</pre>
<br><br>
<b>Pros:</b>
<ul>
<li>Usage is very simple.</li>
<li>Comes with Apache collection which is sometimes available in the project.</li>
</ul>
<br><br>
<b>Cons:</b>
<ul>
<li>Not thread safe.</li>
<li>Removes expired values only when the <b>get()</b>,<b> keySet()</b>, or <b>size()</b> methods are called. More precisely, if we only do <b>put()</b> and do nothing, nothing will be removed from the map.</li>
</ul>
<br><br>
<h1>2. <a href="https://github.com/google/guava/wiki/CachesExplained" target="_blank">Guava cache</a></h1>
<br><br>
This is a Cache implementation that comes with the Guava. The main purpose of this is actually to serve as an in-memory cache. It has many features related to caching but among all two features are interesting (expire after write, and expire after access). For our needs we can use the expireAfterWrite() option which we can use to set the expiry time for an entry. Once that is set, any particular entry having that time will be removed.
<br><br>
<b>Maven dependency:</b>
<br><br>
<pre><dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>10.0.1</version>
</dependency>
</pre>
<br><br>
Code snippet to show the usage. In this example we are using expiry time 1000 milliseconds and adding an entry to the cache after 500 milliseconds. We are setting the expiry time through expireAfterWrite().
<br><br>
<pre>
int expiryInMills = 1000;
LoadingCache<String, Integer> cache = CacheBuilder.newBuilder()
.expireAfterWrite(expiryInMills, TimeUnit.MILLISECONDS)
.build(
new CacheLoader<String, Integer>() {
public Integer load(String key) {
return 1;
}
});
for (int i = 0; i < 10; i++) {
cache.put(String.valueOf(i), i);
System.out.println(cache.asMap().keySet());
Thread.sleep(500);
}
</pre>
<br><br>
<b>Pros:</b>
<ul>
<li>Implementation uses the concurrent hashmap underneath which is thread safe.</li>
<li>Comes with features for in-memory cache.</li>
</ul>
<br><br>
<b>Cons:</b>
<ul>
<li>Designed for Cache, and could be heavy if those features are not needed.</li>
<li>I think the latest version is still in beta. Otherwise none.</li>
</ul>
<br><br>
<h1>3. <a href="https://github.com/ben-manes/caffeine" target="_blank">Caffeine Cache</a></h1>
<br><br>
Caffeine cache was introduced after being inspired by the Guava cache (but adding few features and improvements). For our purpose we don't need it. We only need the expiring entries from the map after the given time.
<br><br>
<b>Maven dependency:</b>
<br><br>
<pre><dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
<version>3.0.5</version>
</dependency>
</pre>
<br><br>
Code snippet to show the usage. The usage is similar to the Guava cache.
<br><br>
<pre>
int expiryInMills = 1000;
Cache<String, Integer> cache = Caffeine.newBuilder()
.expireAfterWrite(expiryInMills, TimeUnit.MILLISECONDS)
.build();
for (int i = 0; i < 10; i++) {
cache.put(String.valueOf(i), i);
System.out.println(cache.asMap().keySet());
Thread.sleep(500);
}
</pre>
<br><br>
<b>Pros:</b>
<ul>
<li>Similar to Guava cache. And comes with more features which Guava is missing.</li>
</ul>
<br><br>
<b>Cons:</b>
<ul>
<li>Designed for Cache, and could be heavy if those features are not needed.</li>
</ul>
<br><br>
<h1>Output</h1>
<br><br>
For all the above code snippet, the output is same, and it is
<br><br>
<pre>[0]
[1, 0]
[1, 2]
[3, 2]
[3, 4]
[5, 4]
[5, 6]
[7, 6]
[7, 8]
[8, 9]
</pre>
<br><br>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-90072220330553401722022-01-04T11:36:00.011-08:002022-01-11T02:36:21.683-08:00Analyse large heap dump using Eclipse MATIt is easy to analyse a heapdump when the dump file size is reasonable. But to analyse a large heap dump (specially a dump from production) is not straightforward. I personally prefer the <a href="https://www.eclipse.org/mat/downloads.php" target="_blank">Eclipse Memory Analyser</a>. When I try to load the large dump using the UI it just get crashed. Here are the steps I am showing how to analyse the heap from commandline and then view it using the UI.
<br><br>
<h1>Generate index from command-line</h1>
My heap dump size is approximately 30gb. To generate index and I provide 22gb memory from commandline. My Java version is 17 (Eclipse requires version 11 or later). And I executed the command in Mac which has 32gb memory. <b>Below is the command</b>.
<br><br>
<pre>
/Applications/mat.app/Contents/MacOS/MemoryAnalyzer -consolelog -application org.eclipse.mat.api.parse "/Heapdump/analyse.dump" -vm /jdk-17.0.1.jdk/Contents/Home/bin/java -vmargs -Xmx22g -XX:-UseGCOverheadLimit
</pre>
Above command takes several minutes to finish and below is the output for my heap dump.
<pre>
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.eclipse.osgi.internal.framework.SystemBundleActivator (file:/Applications/mat.app/Contents/Eclipse/plugins/org.eclipse.osgi_3.16.300.v20210525-1715.jar)
WARNING: Please consider reporting this to the maintainers of org.eclipse.osgi.internal.framework.SystemBundleActivator
WARNING: System::setSecurityManager will be removed in a future release
Task: Parsing analyse.dump
[Task: Parsing /Heapdump/analyse.dump
[
Subtask: Scanning /Heapdump/analyse.dump
[....................
[INFO] Wrote threads call stacks to /Heapdump/analyse.dump.threads
[....................
[INFO] Heap /Heapdump/analyse.dump.dump contains 263,511,984 objects
[....................
Subtask: Extracting objects from /Heapdump/analyse.dump.dump
[................................................................................Task: Removing unreachable objects
[
Subtask: Searching for unreachable objects
[
Subtask: Marking reachable objects
[.
Subtask: Re-indexing objects
[.
[INFO] Removed 174,619 unreachable objects using 14,634,072 bytes
[..
Subtask: Re-indexing classes
[...
Subtask: Writing /Heapdump/analyse.dump.idx.index
[....
Subtask: Writing /Heapdump/analyse.dump.o2c.index
[.....
Subtask: Writing /Heapdump/analyse.dump.a2s.index
[......
Subtask: Re-indexing outbound index
[.......
Subtask: Writing /Heapdump/analyse.dump.inbound.index
[........
Subtask: Writing /Heapdump/analyse.dump.outbound.index
[.........
Subtask: Writing /Heapdump/analyse.dump.o2hprof.index
[.........Task: Calculating Dominator Tree
[
Subtask: Dominator Tree calculation
[
Subtask: Depth-first search
[....................
Subtask: Computing dominators
[..............................................
Subtask: Calculate retained sizes
[............................................................
Subtask: Create dominators index file
[................................................................................]
</pre>
<h1>Load generated index into UI</h1>
This command basically generates several index files ends with <i>analyse.outbound.index</i> and a single file <i>analyse.threads</i>. As you have the index generated, it would be much easier to load the dump into the UI. But before doing that you need to point Eclipse MAT to a java that is version 11 or greater, and also provide some memory to it. I provide 22gb memory to it. Here is you you can configure.
1. Open the <i>/Applications/mat.app/Contents/Info.plist</i> and update the <b>-vm</b> path.
<pre>
<!-- To use a specific Java version (instead of the default) uncomment the following option and
edit it to add a VM. Installed VMs can be found via $/usr/libexec/java_home -V
<string>-vm</string><string>/Library/Java/JavaVirtualMachines/jdk-11.jdk/Contents/Home/bin/java</string>
-->
<string>-vm</string><string>/jdk-17.0.1.jdk/Contents/Home/bin/java</string>
</pre>
2. Update <b>-vm</b> in <i>/Applications/mat.app/Contents/Eclipse/MemoryAnalyzer.ini</i>. Make sure you put it before the <b>-vmargs</b>. And also update how much memory you want to put. Here is mine:
<pre>
-startup
../Eclipse/plugins/org.eclipse.equinox.launcher_1.6.200.v20210416-2027.jar
--launcher.library
../Eclipse/plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.2.200.v20210527-0259
-vm /jdk-17.0.1.jdk/Contents/Home/bin/
-vmargs
-Xmx22g
-Dorg.eclipse.swt.internal.carbon.smallFonts
-XstartOnFirstThread
</pre>
All set! now open your Eclipse MAT, click <b>Open a Heap Dump</b>, and then select the analyse.dump to open. This will take a few minutes to generate a report which later you can analyse. You could generate the report from the command line too. But as my UI could do it for me I didn't try the commandline.
<br><br><br>
<h1>Notes:</h1>
If you do not provide the <b>-vm</b> from the commandline you'd end up with error like below:
<pre>
2022-01-04 19:08:16.855 MemoryAnalyzer[23910:1364729] Cannot find executable for CFBundle 0x7f88b3508420 </Library/Internet Plug-Ins/JavaAppletPlugin.plugin> (not loaded)
</pre>
<div align="center"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjF1AH4o_hmD5VWI9bNpaDgRzTRA-0-k4AyTbOu2KotucdDLIwmoOGYoEO6DHIX9r0aCT0dpHCIyLMR8qOjUoml7wo4-5XJdHyoI4sfAOFqGBuQCwriICLpreWVcqpiuZ6CjLAt5r5dlvqtt_39aRB6cDBq3Euk0jR_N1JCGYwFf_7bezawIYFc3Ncf2Q" ><img alt="" border="0" height=250 width=218 src="https://blogger.googleusercontent.com/img/a/AVvXsEjF1AH4o_hmD5VWI9bNpaDgRzTRA-0-k4AyTbOu2KotucdDLIwmoOGYoEO6DHIX9r0aCT0dpHCIyLMR8qOjUoml7wo4-5XJdHyoI4sfAOFqGBuQCwriICLpreWVcqpiuZ6CjLAt5r5dlvqtt_39aRB6cDBq3Euk0jR_N1JCGYwFf_7bezawIYFc3Ncf2Q"/></a></div>
<div align="center">Image showing error: <b>The JVM shared library "/Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/../lib/server/libjvm.dylib"
does not contain the JNI_CreateJavaVM symbol.</b></div>
<br><br><br>
If you do not provide the -vm in Info.plist, You'd see the same error while opening the Eclipse MAT ui.
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-2010149969131419472017-11-26T13:27:00.000-08:002017-11-26T13:27:50.945-08:00Get function name programaticaly - Python<div align="justify">
This little piece of code will help you to get the function name programatically. This is very helpful when you are implementing the debugging prints from your program.
<br><br><br>
<div align="left">
<code><pre>
import sys
def new_function():
function_name = sys._getframe().f_code.co_name
print function_name
new_function()
</pre></code></div>
<br><br><br>
You of course can write the function name manually(<i>function_name="new_function"</i>). But that case if you change the function's name, you have to change the string as well from the variable function_name.
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-29697138915020590392013-11-27T13:06:00.002-08:002013-11-27T13:06:21.373-08:00SyntaxError: Non-ASCII character '\xe9' in file on line 22<div align="justify">
The Error:
<br><br>
SyntaxError: Non-ASCII character '\xe9' in file test.py on line 22, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
<br><br><br>
This is mainly occurred when you are trying to print the unicode text into the console. Basically the error itself has the solution. Browsing the link will show you what exactly have to be done.
<br><br><br>
Just writing the below code at the very first or the second line at the script. This will set the encoding of the console and will evade the error you are encountering.
<br><br><br>
<code># -*- coding: utf-8 -*-</code>
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-85797876798889021542013-11-27T12:47:00.001-08:002017-11-26T13:26:14.827-08:00How to display contents from a file into textarea - PHP<div align="justify">
How to display contents from a file into the <b>textarea</b> of HTML?
<br><br>
This is very easy and slightly tricky. Here is the PHP code:
<br><br>
<code>
<pre>
print '<textarea>';
include($_some_file_name);
print '</textarea>';
</pre>
</code>
<br><br>
Basically you are including the whole contents from the file into the PHP source which later rendered as HTML. Its just a tricky but effective solution.
<br><br><br>
</div>
Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8722048993094113071.post-84529096128555360672013-11-09T07:32:00.003-08:002022-01-07T18:39:09.473-08:00Upgrading Python in Linux without YUM<div align="justify">
Upgrading the Python at Linux box is slightly risky as the <b>Yum</b> is heavily dependent upon the Python. You have to be very careful while doing this on linux.
<br><br><br>
I was having the Python 2.4.3 as default with my linux installation. To upgrade the Python to 2.7.3 I have followed the below instructions. It was simple. Just downloading, compiling and installing.
<br><br><br>
<b>Notes:</b> Make sure you have the linux development tools, gcc, zlib and similar things are already installed. Otherwise you can not perform the following tasks.
<br><br><br>
1) Download the Python.
<div align="left"><code>wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2</code></div>
<br><br><br>
2) Extract the zip file.<br>
<div align="left"><code>tar xf Python-2.7.3.tar.bz2</code></div>
<br><br><br>
3) Enter inside the extracted directory.<br>
<div align="left"><code>cd Python-2.7.3</code></div>
<br><br><br>
4) Configuring the installation.<br>
<div align="left"><code>./configure --prefix=/usr/local</code></div>
<br><br><br>
5) Make and install at the same time.<br>
<div align="left"><code>make && make altinstall</code></div>
<br><br><br>
6) Add symbolic link in /usr/local/bin.<br>
<div align="left"><code>ln -s /usr/local/bin/python2.7 /usr/local/bin/python</code></div>
<br><br><br>
That's it. Enjoy your upgraded Python!
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-44323726546181514242013-11-09T07:20:00.003-08:002017-11-26T13:28:28.096-08:00Extracting an attribute value with beautifulsoup - Python<div align="justify">
Previously I have demonstrate <a href="http://icfun.blogspot.com/2013/11/python-get-div-content-using.html" target="_blank">to parse the div content</a> from the html source. Now I am showing you how to parse the attribute value from a desired tag from a given html.
<br><br><br>
For this example, lets consider the the attribute name is <b>__VIEWSTATE</b>. This is very important when you are scraping the sites of Microsoft. In this example I am going to show you how to parse this <b>__VIEWSTATE</b> using beautifulsoup.
<br><br>
<div align="left">
<code><pre>
soup = BeautifulSoup(content)
viewstate = soup.findAll("input", {"type": "hidden", "name": "__VIEWSTATE"})
print viewstate[0]['value']
</pre></code>
<br><br>
<b>__VIEWSTATE</b> mainly resides as hidden parameter under <i>input</i> of the <i>forms</i>. So I have looked for the tag named <i>input</i> that is having a <i>type=hidden</i> and <i>name=__VIEWSTATE</i>. Example code returns all the matches from the html.
<br><br>
Here, I have selected the first match(means index 0). But in your case it can be in different indexes if multiple matches are found. Also, its very important to check the count before accessing the array(using function <b>len()</b>). Otherwise Python will popup with an Array index out of bound exception in case no matches are found.
<br><br><br>
</div>
</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-84070817600593379502013-11-09T07:09:00.001-08:002013-11-09T07:09:32.742-08:00Python: Get div content using BeautifulSoup<div align="justify">
I am going to show you an example that will parse a div content from the given html. Lets consider the python variable <b>content</b> holds the html source in it.
<br><br><br>
<div align="left">
<code><pre>
soup = BeautifulSoup(content)
div_content = soup.find("div", {"class": "product"})
print div_content
print str(div_content)
</pre></code>
</div>
<br><br>
In this example, the code is basically parsing the div tags which are only having the <b>class=product</b>. You can add more attributes like id=... and so on. For example,
<br><br>
<div align="left">
<code><pre>
div_content = soup.find("div", {"class": "product", "id": "myDiv"})
</pre></code>
</div>
<br><br><br>
</div>
Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-8722048993094113071.post-4559245823355164732013-11-09T07:01:00.002-08:002013-11-09T07:01:26.817-08:00Python: How to install BeautifulSoup<div align="justify">
Python library <b>BeautifulSoup</b> is very handy while handling the html parsing and other tasks. This makes life easier for the scrappers to parse the data, specially those who are not that much expert on <b>REGEX</b>. This is much simpler than <b>REGEX</b> and any a newbie developer can even write a scraper using this <b>BeautifulSoup</b>.
<br><br><br>
Here is how you can install this <b>BeautifulSoup</b> on your development PC.
<br><br>
1) Directly using the pip. The command is like below:
<br><br>
<code>pip install BeautifulSoup</code>
<br><br>
2) If you do not have the pip installed with your Python(it doesn't come by default with Python installation). That case you can install the <b>BeautifulSoup</b> soup by following ways.
<br><br>
a) Download the latest <b>BeautifulSoup</b> from the link: <a href="https://pypi.python.org/pypi/BeautifulSoup" target="_blank">BeautifulSoup</a>
<br><br>
b) Unzip this into a folder.
<br><br>
c) Enter into the folder and run the following command to install <b>BeautifulSoup</b>.
<br><br>
<code>python setup.py install</code>
<br><br>
3) Or simply you can place the BeautifulSoup.py file with your python source code. It will automatically be loaded from your Python script. But its not recommended as you have to move this file from folder to folder and it looks ugly.
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-21164576554031546822013-11-09T06:51:00.000-08:002013-11-09T06:51:20.560-08:00ImportError: libcurl.so.4: cannot open shared object file<div align="justify">
I have encountered this error at my CentOS server while using the <b>pycurl</b> from the Python.
<br><br><br>
<div align="left"><code>ImportError: libcurl.so.4: cannot open shared object file: No such file or directory</code></div>
<br><br><br>
Python basically searches the paths for dynamic libraries which are only defined at the environment variable <b>LD_LIBRARY_PATH</b>. For example in my case, the <i>libcurl.so</i> library is located at the path <b>/usr/local/lib</b> and its not defined at the env varaible <b>LD_LIBRARY_PATH</b>. So I have added the path on it, and worked.
<br><br><br>
You can add the path to the variable by any of the following ways.
<br><br><br>
<div align="left"><code>export LD_LIBRARY_PATH=/usr/local/lib</code></div>
<br><br><br>
or, if you want to keep the previously set variables are intact, then
<br><br><br>
<div align="left"><code>export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH</code></div>
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-55107886526873429042013-11-05T09:14:00.001-08:002013-11-05T09:14:35.110-08:00Using TRUNC function in JPQL<div align="justify">
If you are finding the <b>TRUNC</b> function is not working with your <b>JPQL</b>, then here is the solution. You can alternately use the <a href="http://icfun.blogspot.com/2013/11/using-native-database-functions-in-jpql.html" target="_blank">FUNC</a> function of <b>JPQL</b> to translate the <b>TRUNC</b> into <b>JPQL</b>. Here is the example.
<br><br><br>
<code><pre>Select o from Example o where FUNC('TRUNC', o.date) = ...</pre></code>
<br><br><br>
Hope this little trick will work with you.
<br><br><br>
</div>
Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-8722048993094113071.post-15340997794424036132013-11-05T09:01:00.000-08:002013-11-05T09:07:03.849-08:00Using native database functions in JPQL<div align="justify">
Java Persistence Query language(<b>JPQL</b>) is defined by the Java Persistence API(<b>JPA</b>) and it is very easy to use for querying database with entities. But sometime it becomes pretty hard to find similar function in <b>JPQL</b> which is available only in a particular database. To overcome this issue, it has an option and it is the <b>FUNC</b> function of <b>JPQL</b>.
<br><br><br>
This function basically interpret the database native functions with the JPQL. The procedure to use the function is like below.
<br><br><br>
<div align="left">
<code></pre>
FUNC('SOME_NATIVE_FUNCTION', 'PARAMETER');
FUNC('SOME_NATIVE_FUNCTION', 'PARAMETER_1', PARAMETER_2, 'PARAMETER_3');
</pre></code>
</div>
<br><br><br>
I am using the <b>TO_NUMBER()</b> as an example here, but it can be any function.
<br><br>
<code></pre>SELECT o FROM result o WHERE FUNC('TO_NUMBER', e.total) < 90</pre></code>
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-44177592907328991142013-11-05T08:38:00.000-08:002013-11-05T08:38:03.913-08:00Ruby: Adding logger with Mechanize<div align="justify">
Ruby <b>mechanize </b>is having a built-in logging option that you can use very easily. It will help you debugging the HTTP calls along with the headers that were handled by the <b>mechanize</b>. Here is the example:
<br><br><br>
<code><pre>
require 'rubygems'
require 'mechanize'
require 'logger'
agent = Mechanize.new { |a|
a.user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0'
a.follow_meta_refresh = true
}
agent.log = Logger.new "mechanize.log"
</pre></code>
<br><br><br>
you'll find all your info and debugging output inside the <b>mechanize.log</b> file.
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-77045038401048773562013-11-04T13:31:00.000-08:002013-11-04T13:31:03.945-08:00Ruby: Mechanize get html body as string<div align="justify">
Below ruby mechanize code will help you to retrieve the HTML content from the provide link.
<br><br><br>
<code>
agent = Mechanize.new { |a|<br>
a.user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0'<br>
a.follow_meta_refresh = true<br>
}<br>
<br>
page = $agent.get("http://icfun.blogspot.com/")<br>
content = page.body<br>
</code>
<br><br><br>
Ruby mechanize is a very sophisticated library to perform lot of powerful task including mimicking the browser itself. You'll find lot of online tutorial regarding this issue.
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-48508132390966891812013-11-04T13:25:00.001-08:002013-11-04T13:27:35.294-08:00Ruby: Get the redirected url using mechanize<div align="justify">
This example is ideal for the case when actually you want to get the url that was redirected from another url. For example you have requested for url-1, and its redirected into url-2 and then into url-3. Now you want to get the url-3 on a variable. I have provided a small mechanize code for you.
<br><br><br>
<code>
agent = Mechanize.new { |a|<br>
a.user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0'<br>
a.follow_meta_refresh = true<br>
}<br>
<br>
page = $agent.get("http://icfun.blogspot.com/")<br>
page.uri.to_s <br>
</code>
<br><br><br>
In this case it will print the blog's link itself as it doesn't have any redirection.
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-43083798119287599312013-10-31T11:41:00.001-07:002013-10-31T11:41:40.623-07:00got exception 'CONNECTION_REFUSED [os error=0, line 1715 of ../nsapi/URL.cpp]<div align="justify">
<b>Error:</b>
<br><br><br>
<code>
[error] [client xx.xx.xx.xx] ap_proxy: trying POST /test/TestService at backend host 'xx.xx.xx.xx/xxxx; got exception 'CONNECTION_REFUSED [os error=0, line 1715 of ../nsapi/URL.cpp]: Error connecting to host xx.xx.xx.xx:xxxx'
</code>
<br><br><br>
<b>Solution:</b>
<br><br><br>
You can not eradicate this problem completely. But you can reduce error occurrence. You have to increase the <b>AcceptBacklog</b> for your servers, from the tuning tab. It can be found from:
<br><br>
<code>Environment - Servers - Select server - Configuration - Tuning tab</code>
<br><br><br>
If you are confused, then <a href="http://docs.oracle.com/cd/E21764_01/apirefs.1111/e13952/taskhelp/tuning/TuneConnectionBacklogBuffering.html" target="_blank">click the link</a> for a detailed description.
<br><br><br>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-74736903542522711342013-10-30T14:42:00.003-07:002013-10-30T14:43:58.319-07:00EJB: Class has two properties of the same name<div align="justify">
<b>Error:</b>
<br><br><br>
<code>
Exception in thread "main" com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException:<br>
1 counts of IllegalAnnotationExceptions<br>
Class has two properties of the same name "xyz"<br>
this problem is related to the following location:<br>
</code>
<br><br><br>
<b>Cause:</b>
<br><br><br>
This is because you are having public field(s) inside your class which is exposed through the webservice. And the JAX-WS uses default binding option <code>XmlAccessType.PUBLIC_MEMBER</code> for the webservices. Which means every public members and the public getter/setters will be bound. Lets see the following example:
<br><br><br>
<code>
public class Test{<br>
public String xyz;<br>
<br>
public String getXyz(){<br>
}<br>
<br>
public void setXyz(){<br>
}<br>
}
</code>
<br><br><br>
Since you have used public field, the JAXB will bind both the public field and the public getter/setter for your webservice. And that's why its always saying <i>Class has two properties of the same name "xyz"</i>(one from field, and one from getter/setter).
<br><br><br>
<b>Solution:</b>
<br><br><br>
Best solution is to use the private for the field, instead of using public. I mean your <code><b>public</b> String xyz</code> should be <code><b>private</b> String xyz</code>.
<br><br><br>
If you have no option rather than using public for the field, then another solution is to use the following annotation above the java class to force the JAXB to use only the members and discarding the getter/setters for binding.
<br><br><br>
<code>
@XmlRootElement(name = "test")<br>
@XmlAccessorType(XmlAccessType.FIELD)<br>
public class Test{<br>
...<br>
</code>
<br><br><br>
</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-5703899477078851332013-10-30T14:23:00.005-07:002013-10-30T14:24:22.146-07:00IllegalAnnotationsException: 2 counts of IllegalAnnotationExceptions<div align="justify">
<b>Error:</b>
<br><br><br>
<code>
Caused by: com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 2 counts of IllegalAnnotationExceptions
<br>
Two classes have the same XML type name "{http://.../}YourClass". Use @XmlType.name and @XmlType.namespace
<br>
to assign different names to them.
<br>
this problem is related to the following location:
<br>
</code>
<br><br><br>
<b>Cause:</b>
<br><br>
A probable reason is that you might be having two Java classes which you are trying to expose through the webservice. That's why the JAX-WS is confused with this and popping out the exception.
<br><br>
<b>Solution:</b>
<br><br>
Using the <b>XmlType</b> annotation can rescue you from this odd situation. Just use the annotation at the top of your java class and give a different name through the name attribute. For example:
<br><br>
<b><code>@XmlType(name="newClassName")</code></b>
<br><br><br>
</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-67330637476549022292013-10-28T14:02:00.002-07:002013-10-28T14:02:23.333-07:00org.jgroups.TimeoutException: timeout sending message to master:ha-server-1/ejb<div align="justify">
<b>Error:</b>
<br><br>
[org.infinispan.transaction.tm.DummyTransaction] (pool-10-thread-1) ISPN000111: afterCompletion() failed for SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=[master:ha-server-1/ejb, master:ha-server-2/ejb], isMarkedForRollback=false, transaction=DummyTransaction{xid=DummyXid{id=1}, status=3}, lockedKeys=null, backupKeyLocks=null, viewId=1} org.infinispan.transaction.synchronization.SyncLocalTransaction@1} org.infinispan.transaction.synchronization.SynchronizationAdapter@20: org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to master:ha-server-1/ejb
<br><br><br>
<b>Cause:</b>
<br><br>
In my case it was for having multiple clusters configured into different server. For example, I have a Cluster-A is configured into the box 10.10.10.10, and having Cluster-B is configured inside the box 10.10.10.20. Both the clusters are independent. But as they are in same network, and having the same cluster configuration(for example host.xml, domain.xml), each of the clusters discovered each other and tried to combine them as one Cluster. They basically discover each other using the multicast Ip address(JGroups issue).
<br><br><br>
<b>Solution:</b>
<br><br>
You have to change the configuration file and use a different one for the new cluster. It should work for you, if you change the grouping name, multicast ip, and multicast port from the configuration file.
<br><br><br>
</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-5561289879610333572013-10-28T13:54:00.001-07:002013-10-28T13:54:24.422-07:00IJ000453: Unable to get managed connection for java:jboss/datasources/<div align="justify">
<b>Error:</b>
<br><br>
Internal Exception: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/TestDS
<br><br><br>
<b>Solution:</b>
<br><br>
1) Check for the database, whether its up or not.
<br><br><br>
2) Check for the datasource configuration from JBoss console(usually http://localhost:9990/console). Make sure the connection parameters are correct.
<br><br><br>
3) Check for the network connection between the database server and the Jboss server is ok.
<br><br><br>
4) If the databse is PostgreSql, then make sure that the database is not limited for opening number of concurrent connection.
<br><br><br>
</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-76424370124527644632013-10-20T06:32:00.000-07:002013-10-20T06:34:49.726-07:00no physical address for 766de5c9-8ac2-6d30-89ef-78d39aa5f7eb, dropping message<div align="justify">
<b>Error:</b>
<br><br><br>
<code>[Server:ha-server-3] 13:59:13,122 WARNING [org.jgroups.protocols.UDP] (OOB-15,null) null: no physical address for 766de5c9-8ac2-6d30-89ef-78d39aa5f7eb, dropping message</code>
<br><br><br>
In My case, it was due to having multiple jboss clusters at a same network and each of the clusters were having same name. For example ha-server-1 and ha-server-2 existed at two different clusters in different machines.
<br><br>
<code>
Cluster-1(10.10.10.10):<br>
|<br>
+- ha-server-1<br>
+- ha-server-2<br>
<br>
Cluster-2(10.10.10.20):<br>
|<br>
+- ha-server-1<br>
+- ha-server-2<br>
</code><br>
I have resolved this problem by changing the ha-server names.
Note: Both were independent cluster. I assume it happened due to the multicast issue of JGroups. Any further explanation from an expert like you will be nice.
</div>
Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8722048993094113071.post-51517972031155327982013-10-10T12:44:00.000-07:002013-10-10T12:44:19.273-07:00JBAS011843: Failed instantiate InitialContextFactory <div align="justify">
This message usually pops out from the application that is deployed to the JBoss.
<br><br><br>
<b>Error:</b>
<br><br>
<code>Exception at Initial Conext: JBAS011843: Failed instantiate InitialContextFactory org.jboss.naming.remoe.client.InitialContextFactory from classloader ModuleClassLoader for Module "dployment ... ... :main" from Service Module Loader</code>
<br><br><br>
<b>reason:</b>
<br><br>
The library jboss-client.jar is couldn't be loaded from your deployed application at JBoss. You must add this jar to the classpath to eradicate this error.
<br><br><br>
</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8722048993094113071.post-79967381213070679942013-05-13T01:43:00.003-07:002013-05-13T02:04:24.886-07:00How to take a jvm Heap Dump<div align="justify">
You can take a heap dump of your <i>jvm</i> for the memory leak debugging purpose. <i>jmap</i> comes as a default with the <i>jdk</i>. You can use the <i>locate</i> command to find the <i>jmap</i> from your <i>linux</i>.
<br><br>
<code>locate jmap</code>
<br>
This command will show you your <i>jmap</i> path. Using the path, you can use a <i>jvm</i> heap dump command like the following.
<br><br>
<code>/home/oracle/Middleware/jdk160_24/bin/jmap -dump:file=/home/icfun/dump.bin 23968</code>
<br>
here <b>23968</b> is the pid. To find a pid for your running java process, use the following command.
<br><br>
<code>ps ax | grep java</code>
<br>
This command will display a list of running java processes from your <i>linux</i>.
<br><br>
<b>Error:</b>
<br><br>
You may encounter errors while taking heap dump using <i>jmap</i>. One error can be:
<br>
<i>23968: well-known file is not secure</i>
<br><br>
This can cause if your running process and the heap dump output file belongs to different users. For example if /home/icfun/dump.bin belongs to the root user and the pid 23968 belongs to the oracle user. Better change the output dump file directory in such a way that it belongs to the oracle user(or the user that is same to your pid). Also try to run the <i>jmap</i> dump command using the oracle user.
</div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8722048993094113071.post-77780750001224883772013-03-18T04:38:00.000-07:002013-03-18T04:39:15.350-07:00Get row count of all tables from oracle schema<div align="justify">
It is possible to get the total row count of all tables from a given schema using a single sql. Below one will work.
<br><br><br>
<code>
select table_name, to_number( extractvalue( xmltype( dbms_xmlgen.getxml('select count(*) c from '||table_name ) ) ,'/ROWSET/ROW/C' ) ) count from user_tables;
</code>
<br><br><br>
Hope this one helped you... :-)
<br><br><br>
</div>Unknownnoreply@blogger.com0