<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" media="screen" href="/~d/styles/rss2full.xsl"?><?xml-stylesheet type="text/css" media="screen" href="http://feeds.feedburner.com/~d/styles/itemcontent.css"?><rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0" version="2.0">

<channel>
	<title>Joshua Zhu's Blog</title>
	
	<link>http://blog.zhuzhaoyuan.com</link>
	<description />
	<pubDate>Tue, 17 Jan 2012 10:11:56 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.7.1</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="self" type="application/rss+xml" href="http://feeds.feedburner.com/JoshuaZhusBlog" /><feedburner:info uri="joshuazhusblog" /><atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="hub" href="http://pubsubhubbub.appspot.com/" /><item>
		<title>A Mechanism to Help Write Web Application Firewalls for Nginx</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/YAyPeXvwUJo/</link>
		<comments>http://blog.zhuzhaoyuan.com/2012/01/a-mechanism-to-help-write-web-application-firewalls-for-nginx/#comments</comments>
		<pubDate>Mon, 16 Jan 2012 14:38:09 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[C10K]]></category>

		<category><![CDATA[Life]]></category>

		<category><![CDATA[nginx]]></category>

		<category><![CDATA[tengine]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=210</guid>
		<description><![CDATA[Developing a Web Application Firewall module for Nginx is not an easy task. The lack of input body filters makes it harder. Nginx is an outstanding web server, but it is not perfect. Actually, nothing is perfect.
So we added the input body filter mechanism to our own Nginx distribution, which is named Tengine. By taking [...]]]></description>
			<content:encoded><![CDATA[<p>Developing a Web Application Firewall module for Nginx is not an easy task. The lack of input body filters makes it harder. Nginx is an outstanding web server, but it is not perfect. Actually, nothing is perfect.</p>
<p>So we added the input body filter mechanism to our own Nginx distribution, which is named <a href="http://tengine.taobao.org/" onclick="javascript:pageTracker._trackPageview('/outbound/article/tengine.taobao.org');">Tengine</a>. By taking advantage of this mechanism, processing the request body is not that complicated anymore (In standard Nginx, request body may be buffered to disk file and you have to deal with up to two buffers)</p>
<p>Here I have an example to demonstrate how to write an input body filter. It is a simple module to fight hash collision DoS attacks.</p>

<div class="wp_syntax"><div class="code"><pre class="c" style="font-family:monospace;"><span style="color: #808080; font-style: italic;">/*
 * Copyright (C) Joshua Zhu, http://www.zhuzhaoyuan.com
 */</span>
&nbsp;
&nbsp;
<span style="color: #339933;">#include &lt;ngx_config.h&gt;</span>
<span style="color: #339933;">#include &lt;ngx_core.h&gt;</span>
<span style="color: #339933;">#include &lt;ngx_http.h&gt;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">typedef</span> <span style="color: #993333;">struct</span> <span style="color: #009900;">&#123;</span>
    ngx_flag_t                            enable<span style="color: #339933;">;</span>
    ngx_uint_t                            max_post_params<span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span> ngx_http_anti_hashdos_loc_conf_t<span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">typedef</span> <span style="color: #993333;">struct</span> <span style="color: #009900;">&#123;</span>
    ngx_uint_t                            post_params<span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span> ngx_http_anti_hashdos_ctx_t<span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> ngx_int_t ngx_http_anti_hashdos_input_body_filter<span style="color: #009900;">&#40;</span>ngx_http_request_t <span style="color: #339933;">*</span>r<span style="color: #339933;">,</span>
    ngx_buf_t <span style="color: #339933;">*</span>buf<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #993333;">static</span> <span style="color: #993333;">void</span> <span style="color: #339933;">*</span>ngx_http_anti_hashdos_create_loc_conf<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #993333;">static</span> <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>ngx_http_anti_hashdos_merge_loc_conf<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #339933;">,</span> <span style="color: #993333;">void</span> <span style="color: #339933;">*</span>parent<span style="color: #339933;">,</span>
    <span style="color: #993333;">void</span> <span style="color: #339933;">*</span>child<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #993333;">static</span> ngx_int_t ngx_http_anti_hashdos_init<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> ngx_command_t ngx_http_anti_hashdos_filter_commands<span style="color: #009900;">&#91;</span><span style="color: #009900;">&#93;</span> <span style="color: #339933;">=</span> <span style="color: #009900;">&#123;</span>
&nbsp;
    <span style="color: #009900;">&#123;</span> ngx_string<span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;anti_hashdos&quot;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span>
      NGX_HTTP_MAIN_CONF<span style="color: #339933;">|</span>NGX_HTTP_SRV_CONF<span style="color: #339933;">|</span>NGX_HTTP_LOC_CONF<span style="color: #339933;">|</span>NGX_CONF_FLAG<span style="color: #339933;">,</span>
      ngx_conf_set_flag_slot<span style="color: #339933;">,</span>
      NGX_HTTP_LOC_CONF_OFFSET<span style="color: #339933;">,</span>
      offsetof<span style="color: #009900;">&#40;</span>ngx_http_anti_hashdos_loc_conf_t<span style="color: #339933;">,</span> enable<span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span>
      <span style="color: #000000; font-weight: bold;">NULL</span> <span style="color: #009900;">&#125;</span><span style="color: #339933;">,</span>
&nbsp;
    <span style="color: #009900;">&#123;</span> ngx_string<span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;anti_hashdos_max_post_params&quot;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span>
      NGX_HTTP_MAIN_CONF<span style="color: #339933;">|</span>NGX_HTTP_SRV_CONF<span style="color: #339933;">|</span>NGX_HTTP_LOC_CONF<span style="color: #339933;">|</span>NGX_CONF_TAKE1<span style="color: #339933;">,</span>
      ngx_conf_set_num_slot<span style="color: #339933;">,</span>
      NGX_HTTP_LOC_CONF_OFFSET<span style="color: #339933;">,</span>
      offsetof<span style="color: #009900;">&#40;</span>ngx_http_anti_hashdos_loc_conf_t<span style="color: #339933;">,</span> max_post_params<span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span>
      <span style="color: #000000; font-weight: bold;">NULL</span> <span style="color: #009900;">&#125;</span><span style="color: #339933;">,</span>
&nbsp;
      ngx_null_command
<span style="color: #009900;">&#125;</span><span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> ngx_http_module_t ngx_http_anti_hashdos_filter_module_ctx <span style="color: #339933;">=</span> <span style="color: #009900;">&#123;</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* preconfiguration */</span>
    ngx_http_anti_hashdos_init<span style="color: #339933;">,</span>           <span style="color: #808080; font-style: italic;">/* postconfiguration */</span>
&nbsp;
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* create main configuration */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* init main configuration */</span>
&nbsp;
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* create server configuration */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* merge server configuration */</span>
&nbsp;
    ngx_http_anti_hashdos_create_loc_conf<span style="color: #339933;">,</span><span style="color: #808080; font-style: italic;">/* create location configuration */</span>
    ngx_http_anti_hashdos_merge_loc_conf  <span style="color: #808080; font-style: italic;">/* merge location configuration */</span>
<span style="color: #009900;">&#125;</span><span style="color: #339933;">;</span>
&nbsp;
&nbsp;
ngx_module_t ngx_http_anti_hashdos_filter_module <span style="color: #339933;">=</span> <span style="color: #009900;">&#123;</span>
    NGX_MODULE_V1<span style="color: #339933;">,</span>
    <span style="color: #339933;">&amp;</span>ngx_http_anti_hashdos_filter_module_ctx<span style="color: #339933;">,</span> <span style="color: #808080; font-style: italic;">/* module context */</span>
    ngx_http_anti_hashdos_filter_commands<span style="color: #339933;">,</span><span style="color: #808080; font-style: italic;">/* module directives */</span>
    NGX_HTTP_MODULE<span style="color: #339933;">,</span>                      <span style="color: #808080; font-style: italic;">/* module type */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* init master */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* init module */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* init process */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* init thread */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* exit thread */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* exit process */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                                 <span style="color: #808080; font-style: italic;">/* exit master */</span>
    NGX_MODULE_V1_PADDING
<span style="color: #009900;">&#125;</span><span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> ngx_http_input_body_filter_pt  ngx_http_next_input_body_filter<span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> ngx_int_t
ngx_http_anti_hashdos_input_body_filter<span style="color: #009900;">&#40;</span>ngx_http_request_t <span style="color: #339933;">*</span>r<span style="color: #339933;">,</span>
    ngx_buf_t <span style="color: #339933;">*</span>buf<span style="color: #009900;">&#41;</span>
<span style="color: #009900;">&#123;</span>
    u_char                           <span style="color: #339933;">*</span>p<span style="color: #339933;">;</span>
    ngx_http_anti_hashdos_ctx_t      <span style="color: #339933;">*</span>ctx<span style="color: #339933;">;</span>
    ngx_http_anti_hashdos_loc_conf_t <span style="color: #339933;">*</span>ahlf<span style="color: #339933;">;</span>
&nbsp;
    ahlf <span style="color: #339933;">=</span> ngx_http_get_module_loc_conf<span style="color: #009900;">&#40;</span>r<span style="color: #339933;">,</span> ngx_http_anti_hashdos_filter_module<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>ahlf<span style="color: #339933;">-&gt;</span>enable<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #b1b100;">return</span> ngx_http_next_input_body_filter<span style="color: #009900;">&#40;</span>r<span style="color: #339933;">,</span> buf<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    ctx <span style="color: #339933;">=</span> ngx_http_get_module_ctx<span style="color: #009900;">&#40;</span>r<span style="color: #339933;">,</span> ngx_http_anti_hashdos_filter_module<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span>ctx <span style="color: #339933;">==</span> <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        ctx <span style="color: #339933;">=</span> ngx_pcalloc<span style="color: #009900;">&#40;</span>r<span style="color: #339933;">-&gt;</span>pool<span style="color: #339933;">,</span> <span style="color: #993333;">sizeof</span><span style="color: #009900;">&#40;</span>ngx_http_anti_hashdos_ctx_t<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span>ctx <span style="color: #339933;">==</span> <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
            <span style="color: #b1b100;">return</span> NGX_HTTP_INTERNAL_SERVER_ERROR<span style="color: #339933;">;</span>
        <span style="color: #009900;">&#125;</span>
&nbsp;
        ctx<span style="color: #339933;">-&gt;</span>post_params <span style="color: #339933;">=</span> <span style="color: #0000dd;">1</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #b1b100;">for</span> <span style="color: #009900;">&#40;</span>p <span style="color: #339933;">=</span> buf<span style="color: #339933;">-&gt;</span>pos<span style="color: #339933;">;</span> p <span style="color: #339933;">&lt;</span> buf<span style="color: #339933;">-&gt;</span>last<span style="color: #339933;">;</span> p<span style="color: #339933;">++</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
&nbsp;
        <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span><span style="color: #339933;">*</span>p <span style="color: #339933;">==</span> <span style="color: #ff0000;">'&amp;'</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
            ctx<span style="color: #339933;">-&gt;</span>post_params<span style="color: #339933;">++;</span>
        <span style="color: #009900;">&#125;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span>ctx<span style="color: #339933;">-&gt;</span>post_params <span style="color: #339933;">&gt;</span> ahlf<span style="color: #339933;">-&gt;</span>max_post_params<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        ngx_log_error<span style="color: #009900;">&#40;</span>NGX_LOG_ERR<span style="color: #339933;">,</span> r<span style="color: #339933;">-&gt;</span>connection<span style="color: #339933;">-&gt;</span>log<span style="color: #339933;">,</span> <span style="color: #0000dd;">0</span><span style="color: #339933;">,</span>
                      <span style="color: #ff0000;">&quot;anti hashdos: <span style="color: #000099; font-weight: bold;">\&quot;</span>%V<span style="color: #000099; font-weight: bold;">\&quot;</span> blocked, too many post params: %d&quot;</span><span style="color: #339933;">,</span>
                      <span style="color: #339933;">&amp;</span>r<span style="color: #339933;">-&gt;</span>connection<span style="color: #339933;">-&gt;</span>addr_text<span style="color: #339933;">,</span>
                      ctx<span style="color: #339933;">-&gt;</span>post_params<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #b1b100;">return</span> NGX_HTTP_BAD_REQUEST<span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #b1b100;">return</span> ngx_http_next_input_body_filter<span style="color: #009900;">&#40;</span>r<span style="color: #339933;">,</span> buf<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> <span style="color: #993333;">void</span> <span style="color: #339933;">*</span>
ngx_http_anti_hashdos_create_loc_conf<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #009900;">&#41;</span>
<span style="color: #009900;">&#123;</span>
    ngx_http_anti_hashdos_loc_conf_t <span style="color: #339933;">*</span>conf<span style="color: #339933;">;</span>
&nbsp;
    conf <span style="color: #339933;">=</span> ngx_pcalloc<span style="color: #009900;">&#40;</span>cf<span style="color: #339933;">-&gt;</span>pool<span style="color: #339933;">,</span> <span style="color: #993333;">sizeof</span><span style="color: #009900;">&#40;</span>ngx_http_anti_hashdos_loc_conf_t<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span>conf <span style="color: #339933;">==</span> <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #b1b100;">return</span> <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    conf<span style="color: #339933;">-&gt;</span>enable <span style="color: #339933;">=</span> NGX_CONF_UNSET<span style="color: #339933;">;</span>
    conf<span style="color: #339933;">-&gt;</span>max_post_params <span style="color: #339933;">=</span> NGX_CONF_UNSET_UINT<span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">return</span> conf<span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>
ngx_http_anti_hashdos_merge_loc_conf<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #339933;">,</span> <span style="color: #993333;">void</span> <span style="color: #339933;">*</span>parent<span style="color: #339933;">,</span> <span style="color: #993333;">void</span> <span style="color: #339933;">*</span>child<span style="color: #009900;">&#41;</span>
<span style="color: #009900;">&#123;</span>
    ngx_http_anti_hashdos_loc_conf_t <span style="color: #339933;">*</span>prev <span style="color: #339933;">=</span> parent<span style="color: #339933;">;</span>
    ngx_http_anti_hashdos_loc_conf_t <span style="color: #339933;">*</span>conf <span style="color: #339933;">=</span> child<span style="color: #339933;">;</span>
&nbsp;
    ngx_conf_merge_value<span style="color: #009900;">&#40;</span>conf<span style="color: #339933;">-&gt;</span>enable<span style="color: #339933;">,</span> prev<span style="color: #339933;">-&gt;</span>enable<span style="color: #339933;">,</span> <span style="color: #0000dd;">0</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    ngx_conf_merge_uint_value<span style="color: #009900;">&#40;</span>conf<span style="color: #339933;">-&gt;</span>max_post_params<span style="color: #339933;">,</span> prev<span style="color: #339933;">-&gt;</span>max_post_params<span style="color: #339933;">,</span>
                              <span style="color: #0000dd;">120</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">return</span> NGX_CONF_OK<span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> ngx_int_t
ngx_http_anti_hashdos_init<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #009900;">&#41;</span>
<span style="color: #009900;">&#123;</span>
    ngx_http_next_input_body_filter <span style="color: #339933;">=</span> ngx_http_top_input_body_filter<span style="color: #339933;">;</span>
    ngx_http_top_input_body_filter <span style="color: #339933;">=</span> ngx_http_anti_hashdos_input_body_filter<span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">return</span> NGX_OK<span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span></pre></div></div>

<p>The code looks quite straight forward, right? And it is similar to an output body filter, just a few steps:</p>
<p>1) Implement your own input body filter function. e.g.</p>

<div class="wp_syntax"><div class="code"><pre class="c" style="font-family:monospace;"><span style="color: #993333;">static</span> ngx_int_t
ngx_http_anti_hashdos_input_body_filter<span style="color: #009900;">&#40;</span>ngx_http_request_t <span style="color: #339933;">*</span>r<span style="color: #339933;">,</span> ngx_buf_t <span style="color: #339933;">*</span>buf<span style="color: #009900;">&#41;</span>
<span style="color: #009900;">&#123;</span>
    <span style="color: #808080; font-style: italic;">/* Do the input body filtering here */</span>
<span style="color: #009900;">&#125;</span></pre></div></div>

<p>2) In your input body filter function, return an HTTP error code if something is wrong. Otherwise, call ngx_http_next_input_body_filter(r, buf) directly to pass the buf to the next input body filters.</p>
<p>3) Install your input body filter in the post_configuration hook function. Push your input body filter to the head of the input body filter chain. e.g.</p>

<div class="wp_syntax"><div class="code"><pre class="c" style="font-family:monospace;"><span style="color: #993333;">static</span> ngx_int_t
ngx_http_anti_hashdos_init<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #009900;">&#41;</span>
<span style="color: #009900;">&#123;</span>
    ngx_http_next_input_body_filter <span style="color: #339933;">=</span> ngx_http_top_input_body_filter<span style="color: #339933;">;</span>
    ngx_http_top_input_body_filter <span style="color: #339933;">=</span> ngx_http_anti_hashdos_input_body_filter<span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">return</span> NGX_OK<span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span></pre></div></div>

<p>NOTE: This is just a demonstration to show how to write input body filters.<strong> If you want to fight hash collision DoS attacks completely, you have to write more code and various POST content types should be processed.</strong></p>
<p>Download the code here:<br />
<a href="http://www.zhuzhaoyuan.com/download/tengine/anti_hashdos.tar.gz" >http://www.zhuzhaoyuan.com/download/tengine/anti_hashdos.tar.gz</a></p>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/YAyPeXvwUJo" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2012/01/a-mechanism-to-help-write-web-application-firewalls-for-nginx/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2012/01/a-mechanism-to-help-write-web-application-firewalls-for-nginx/</feedburner:origLink></item>
		<item>
		<title>Tengine, a customized Nginx, goes to open source</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/ErWTKyU6c8Y/</link>
		<comments>http://blog.zhuzhaoyuan.com/2011/12/tengine-a-customized-nginx-goes-to-open-source/#comments</comments>
		<pubDate>Fri, 02 Dec 2011 11:31:15 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[C10K]]></category>

		<category><![CDATA[nginx]]></category>

		<category><![CDATA[tengine]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=205</guid>
		<description><![CDATA[We&#8217;re glad to announce that Tengine, our home-baked Nginx at Taobao now becomes an open source project.
Taobao is the largest e-commerce website in Asia and ranked #12 on Alexa’s top global sites list. Our website serves billions of pageviews per day. For busy website as us, Nginx is obviously the best choice. Thanks to Nginx&#8217;s [...]]]></description>
			<content:encoded><![CDATA[<p>We&#8217;re glad to announce that <a href="http://tengine.taobao.org" onclick="javascript:pageTracker._trackPageview('/outbound/article/tengine.taobao.org');">Tengine</a>, our home-baked Nginx at Taobao now becomes an open source project.</p>
<p><a href="http://www.taobao.com" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.taobao.com');">Taobao</a> is the largest e-commerce website in Asia and ranked #12 on Alexa’s top global sites list. Our website serves billions of pageviews per day. For busy website as us, Nginx is obviously the best choice. Thanks to Nginx&#8217;s high performance, small footprint and flexibility, we have done more with less.</p>
<p>We first learned the Nginx internals by using it as a traditional web server and developing dozens of modules. Then from June of this year we started hacking the Nginx core to expand its capabilities. As some of the features we have developed may also benefit other Nginx users and websites, so why not open source them? We do not want to be just open source software users, but also open source contributors. That&#8217;s why the Tengine open source project came out.</p>
<p>Tengine is based on the latest stable version of Nginx (Nginx-1.0.10). There are a few features and bug fixes you may be interested in Tengine:</p>
<ul>
<li>Logging enhancement. It supports syslog (local and remote) and pipe logging. You can also do log sampling, i.e. not all requests have to be written.</li>
<li>Protects the server when the system load and memory use goes high.</li>
<li>Combines multiple CSS or JavasScript requests into one request to reduce the downloading time.</li>
<li>Sets the worker process number and CPU affinities automatically. Setting Nginx&#8217;s worker_cpu_affinity is not a pain any more.</li>
<li> Enhanced limit_req module with whitelist support and more limit_req directives in one location.</li>
<li> More operations engineer friendly server information, so host can be located easily when error happens.</li>
<li> More command lines support. You can list all modules compiled in and the directives supported, even the content of configuration file itself.</li>
<li> Set expiration for files according to specific content type.</li>
<li> Error pages can be set back to &#8216;default&#8217;.</li>
</ul>
<p>Basically, Tengine can be considered as a better or superset of Nginx. You can download the tar ball here:<br />
<a href="http://tengine.taobao.org/download/tengine-1.2.0.tar.gz" onclick="javascript:pageTracker._trackPageview('/outbound/article/tengine.taobao.org');">http://tengine.taobao.org/download/tengine-1.2.0.tar.gz</a></p>
<p>We want to say thank you to the Nginx team, especially to Igor. Thank you very much for your great work! We would love to donate the patches against the Nginx-1.1 branch later if you think the patches are okay.</p>
<p>Frankly, I&#8217;m not sure whether the features in Tengine right now can impress you guys or not. It&#8217;s the first step we moving towards open source after all. We have built a team working on Tengine and have quite a long to-do list. I promise you more enhancements are coming out.</p>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/ErWTKyU6c8Y" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2011/12/tengine-a-customized-nginx-goes-to-open-source/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2011/12/tengine-a-customized-nginx-goes-to-open-source/</feedburner:origLink></item>
		<item>
		<title>I’m Back</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/vfTgCDS3Y8M/</link>
		<comments>http://blog.zhuzhaoyuan.com/2011/11/im-back/#comments</comments>
		<pubDate>Wed, 16 Nov 2011 09:57:27 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[Life]]></category>

		<category><![CDATA[nginx]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=197</guid>
		<description><![CDATA[I&#8217;m so terrible about keeping my blog up-to-date. This blog has not been updated for 2 years. Sorry guys.
Yet today I have good news for you, if you are an Nginx fan. We are going to open source our home-baked Nginx! This Nginx fork is named Tengine. It serves on thousands of production servers at [...]]]></description>
			<content:encoded><![CDATA[<p>I&#8217;m so terrible about keeping my blog up-to-date. This blog has not been updated for 2 years. Sorry guys.</p>
<p>Yet today I have good news for you, if you are an Nginx fan. We are going to open source our home-baked Nginx! This Nginx fork is named Tengine. It serves on thousands of production servers at our website taobao.com. (For those know nothing about Taobao: Taobao is the largest e-commerce website in Asia, and is ranked #15 on Alexa&#8217;s top global sites list)</p>
<p>There are quite a few features that you may be interested. I&#8217;ll release the more detailed announcement soon.</p>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/vfTgCDS3Y8M" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2011/11/im-back/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2011/11/im-back/</feedburner:origLink></item>
		<item>
		<title>Nginx Internals (Slides &amp; Video)</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/BZGJF6BjIEs/</link>
		<comments>http://blog.zhuzhaoyuan.com/2009/09/nginx-internals-slides-video/#comments</comments>
		<pubDate>Tue, 22 Sep 2009 08:14:57 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[C10K]]></category>

		<category><![CDATA[Software]]></category>

		<category><![CDATA[nginx]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=186</guid>
		<description><![CDATA[Last Saturday I gave the talk &#8220;Nginx Internals&#8221; in Guangzhou. Here are the presentation slides and the video of the talk.
Nginx Internals
View more documents from Joshua Zhu.

Nginx Internals Video part 1 (in Chinese):

Nginx Internals Video part 2 (in Chinese):
UPDATE: This part is lost, saddly, and I have no backup  

Nginx Internals Video part 3 [...]]]></description>
			<content:encoded><![CDATA[<p>Last Saturday I gave the talk &#8220;Nginx Internals&#8221; in Guangzhou. Here are the presentation slides and the video of the talk.</p>
<div style="width:425px;text-align:left" id="__ss_2028238"><a style="font:14px Helvetica,Arial,Sans-serif;display:block;margin:12px 0 3px 0;text-decoration:underline;" href="http://www.slideshare.net/joshzhu/nginx-internals" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.slideshare.net');" title="Nginx Internals">Nginx Internals</a><object style="margin:0px" width="425" height="355"><param name="movie" value="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=nginxinternals-090920203544-phpapp01&#038;rel=0&#038;stripped_title=nginx-internals" /><param name="allowFullScreen" value="true"/><param name="allowScriptAccess" value="always"/><embed src="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=nginxinternals-090920203544-phpapp01&#038;rel=0&#038;stripped_title=nginx-internals" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="355"></embed></object>
<div style="font-size:11px;font-family:tahoma,arial;height:26px;padding-top:2px;">View more <a style="text-decoration:underline;" href="http://www.slideshare.net/" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.slideshare.net');">documents</a> from <a style="text-decoration:underline;" href="http://www.slideshare.net/joshzhu" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.slideshare.net');">Joshua Zhu</a>.</div>
</div>
<p>Nginx Internals Video part 1 (in Chinese):</p>
<div><object id="ssss" width="480" height="370" ><param name="allowScriptAccess" value="always" /><embed pluginspage="http://www.macromedia.com/go/getflashplayer" src="http://p.you.video.sina.com.cn/player/outer_player.swf?auto=1&#038;vid=24664517&#038;uid=1648944337" type="application/x-shockwave-flash" name="ssss" allowFullScreen="true" allowScriptAccess="always" width="480" height="370"></embed></object></div>
<p>Nginx Internals Video part 2 (in Chinese):<br />
UPDATE: This part is lost, saddly, and I have no backup <img src='http://blog.zhuzhaoyuan.com/wp-includes/images/smilies/icon_sad.gif' alt=':(' class='wp-smiley' /> </p>
<div><object id="ssss" width="480" height="370" ><param name="allowScriptAccess" value="always" /><embed pluginspage="http://www.macromedia.com/go/getflashplayer" src="http://p.you.video.sina.com.cn/player/outer_player.swf?auto=1&#038;vid=24664759&#038;uid=1648944337" type="application/x-shockwave-flash" name="ssss" allowFullScreen="true" allowScriptAccess="always" width="480" height="370"></embed></object></div>
<p>Nginx Internals Video part 3 (in Chinese):</p>
<div><object id="ssss" width="480" height="370" ><param name="allowScriptAccess" value="always" /><embed pluginspage="http://www.macromedia.com/go/getflashplayer" src="http://p.you.video.sina.com.cn/player/outer_player.swf?auto=1&#038;vid=24672660&#038;uid=1648944337" type="application/x-shockwave-flash" name="ssss" allowFullScreen="true" allowScriptAccess="always" width="480" height="370"></embed></object></div>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/BZGJF6BjIEs" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2009/09/nginx-internals-slides-video/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2009/09/nginx-internals-slides-video/</feedburner:origLink></item>
		<item>
		<title>Nginx Internals Talk in Guangzhou, China</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/8cQa9RCFrYw/</link>
		<comments>http://blog.zhuzhaoyuan.com/2009/08/nginx-internals-talk-guangzhou/#comments</comments>
		<pubDate>Mon, 31 Aug 2009 03:37:52 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[C10K]]></category>

		<category><![CDATA[Software]]></category>

		<category><![CDATA[nginx]]></category>

		<category><![CDATA[server]]></category>

		<category><![CDATA[source]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=170</guid>
		<description><![CDATA[

I&#8217;m going to give a free talk on nginx&#8217;s internals next month (September 19), in Guangzhou, China.
I&#8217;ve been reading the source code of nginx for a few days. Digging into this charming code is really a pleasant experience, though at first glance it appeared a little bit difficult to understand. Nginx becomes more and more [...]]]></description>
			<content:encoded><![CDATA[<p style="text-align: left;">
<p><div id="attachment_173" class="wp-caption aligncenter" style="width: 179px"><a href="http://blog.zhuzhaoyuan.com/wp-content/uploads/2009/08/nginxmap.jpg" ><img class="size-medium wp-image-173 " title="nginxmap" src="http://blog.zhuzhaoyuan.com/wp-content/uploads/2009/08/nginxmap-169x300.jpg" alt="nginx map" width="169" height="300" /></a><p class="wp-caption-text">nginx map (click to view large image)</p></div></p>
<p style="text-align: left;">
<p style="text-align: left;">I&#8217;m going to give a free talk on nginx&#8217;s internals next month (September 19), in Guangzhou, China.</p>
<p>I&#8217;ve been reading the source code of nginx for a few days. Digging into this charming code is really a pleasant experience, though at first glance it appeared a little bit difficult to understand. Nginx becomes more and more popular, but unfortunately there is not enough documentation on its architecture and implementation. Now that I have spent a considerable amount of time reading the source code and have gained some knowledge, why not share it with those who want to know things under the hood?</p>
<p>So, if you are interested in this talk and you can be in Guangzhou that day, feel free to join in. Please comment on this post or drop me an email to let me know which parts you are interested in (see the mind map above, draft version though).</p>
<p>There might be a thousand Hamlets in a thousand people&#8217;s eyes. Note that I&#8217;m not Igor, and the only way I try to understand the nuts and bolts is by reverse engineering it, hence I can&#8217;t guarantee you no mistakes or misunderstandings in my talk. And frankly, it is not a trivial topic after all, not only because of the size of nginx&#8217;s code base, but also its elaborate design.</p>
<p>The speech will be in Chinese while slides will be in English. Specifics of time and location are coming soon. Stay tuned.</p>
<p>Update:<br />
Time: 14:30-17:30, September 19, 2009<br />
Location: Netease Building Tower E, Guangzhou Information Port #16 Keyun RD. Tianhe District, Guangzhou<br />
Registration: <a href="http://blog.laiyonghao.com/2009/09/programming-tech-party/370" onclick="javascript:pageTracker._trackPageview('/outbound/article/blog.laiyonghao.com');">http://blog.laiyonghao.com/2009/09/programming-tech-party/370</a></p>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/8cQa9RCFrYw" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2009/08/nginx-internals-talk-guangzhou/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2009/08/nginx-internals-talk-guangzhou/</feedburner:origLink></item>
		<item>
		<title>A Handy Strace Option</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/1Mr2Kza7LTg/</link>
		<comments>http://blog.zhuzhaoyuan.com/2009/08/a-handy-strace-option/#comments</comments>
		<pubDate>Thu, 20 Aug 2009 07:39:32 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[Programming]]></category>

		<category><![CDATA[tool]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=167</guid>
		<description><![CDATA[I didn&#8217;t notice the &#8216;-ff&#8217; option of strace until I came across it today. By turning it on, not only fork(2)s can be followed, but also each process&#8217;s trace will be written to tracefile.pid, where pid is the process id of each process. Typical usage might look like this:

# strace -o tracelog.txt -ff -T command

This [...]]]></description>
			<content:encoded><![CDATA[<p>I didn&#8217;t notice the &#8216;-ff&#8217; option of strace until I came across it today. By turning it on, not only fork(2)s can be followed, but also each process&#8217;s trace will be written to tracefile.pid, where pid is the process id of each process. Typical usage might look like this:</p>

<div class="wp_syntax"><div class="code"><pre class="sh" style="font-family:monospace;"># strace -o tracelog.txt -ff -T command</pre></div></div>

<p>This option can be quite handy, when debugging programs that spawn child processes.</p>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/1Mr2Kza7LTg" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2009/08/a-handy-strace-option/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2009/08/a-handy-strace-option/</feedburner:origLink></item>
		<item>
		<title>Creating a Hello World! Nginx Module</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/hxYL1UTTLH8/</link>
		<comments>http://blog.zhuzhaoyuan.com/2009/08/creating-a-hello-world-nginx-module/#comments</comments>
		<pubDate>Thu, 13 Aug 2009 07:53:09 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[Programming]]></category>

		<category><![CDATA[nginx]]></category>

		<category><![CDATA[howto]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=156</guid>
		<description><![CDATA[Some of my friends think nginx modules are very difficult to write. Sure it&#8217;s not so easy but it&#8217;s not that hard either. The only problem is that the documentation is not enough. But don&#8217;t let this scare yourself away, the situation is improving.
So I write a hello world nginx module here. It&#8217;s pretty short [...]]]></description>
			<content:encoded><![CDATA[<p>Some of my friends think nginx modules are very difficult to write. Sure it&#8217;s not so easy but it&#8217;s not that hard either. The only problem is that the documentation is not enough. But don&#8217;t let this scare yourself away, the situation is improving.</p>
<p>So I write <a href="http://www.zhuzhaoyuan.com/download/nginx/hello.tar.gz" >a hello world nginx module</a> here. It&#8217;s pretty short and with enough comments. If you&#8217;re a new nginx module developer, feel free to take it as an example and replace <strong><em>hello</em></strong> with whatever your module name is, then start your happy nginx module hacking journey. And I&#8217;ll write more topics on nginx soon. Look for it!  <img src='http://blog.zhuzhaoyuan.com/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<p>ngx_http_hello_module.c:</p>

<div class="wp_syntax"><div class="code"><pre class="c" style="font-family:monospace;">&nbsp;
<span style="color: #808080; font-style: italic;">/*
 * Copyright (C) Joshua Zhu, http://www.zhuzhaoyuan.com
 */</span>
&nbsp;
&nbsp;
<span style="color: #339933;">#include &lt;ngx_config.h&gt;</span>
<span style="color: #339933;">#include &lt;ngx_core.h&gt;</span>
<span style="color: #339933;">#include &lt;ngx_http.h&gt;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>ngx_http_hello<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #339933;">,</span> ngx_command_t <span style="color: #339933;">*</span>cmd<span style="color: #339933;">,</span>
    <span style="color: #993333;">void</span> <span style="color: #339933;">*</span>conf<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #993333;">static</span> ngx_command_t ngx_http_hello_commands<span style="color: #009900;">&#91;</span><span style="color: #009900;">&#93;</span> <span style="color: #339933;">=</span> <span style="color: #009900;">&#123;</span>
    <span style="color: #009900;">&#123;</span> ngx_string<span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;hello&quot;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span>
      NGX_HTTP_LOC_CONF<span style="color: #339933;">|</span>NGX_CONF_NOARGS<span style="color: #339933;">,</span>
      ngx_http_hello<span style="color: #339933;">,</span>
      <span style="color: #0000dd;">0</span><span style="color: #339933;">,</span>
      <span style="color: #0000dd;">0</span><span style="color: #339933;">,</span>
      <span style="color: #000000; font-weight: bold;">NULL</span> <span style="color: #009900;">&#125;</span><span style="color: #339933;">,</span>
&nbsp;
    ngx_null_command
<span style="color: #009900;">&#125;</span><span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> u_char ngx_hello_string<span style="color: #009900;">&#91;</span><span style="color: #009900;">&#93;</span> <span style="color: #339933;">=</span> <span style="color: #ff0000;">&quot;Hello, world!&quot;</span><span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> ngx_http_module_t ngx_http_hello_module_ctx <span style="color: #339933;">=</span> <span style="color: #009900;">&#123;</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* preconfiguration */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* postconfiguration */</span>
&nbsp;
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* create main configuration */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* init main configuration */</span>
&nbsp;
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* create server configuration */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* merge server configuration */</span>
&nbsp;
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* create location configuration */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span>                           <span style="color: #808080; font-style: italic;">/* merge location configuration */</span>
<span style="color: #009900;">&#125;</span><span style="color: #339933;">;</span>
&nbsp;
&nbsp;
ngx_module_t ngx_http_hello_module <span style="color: #339933;">=</span> <span style="color: #009900;">&#123;</span>
    NGX_MODULE_V1<span style="color: #339933;">,</span>
    <span style="color: #339933;">&amp;</span>ngx_http_hello_module_ctx<span style="color: #339933;">,</span>    <span style="color: #808080; font-style: italic;">/* module context */</span>
    ngx_http_hello_commands<span style="color: #339933;">,</span>       <span style="color: #808080; font-style: italic;">/* module directives */</span>
    NGX_HTTP_MODULE<span style="color: #339933;">,</span>               <span style="color: #808080; font-style: italic;">/* module type */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* init master */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* init module */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* init process */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* init thread */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* exit thread */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* exit process */</span>
    <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">,</span>                          <span style="color: #808080; font-style: italic;">/* exit master */</span>
    NGX_MODULE_V1_PADDING
<span style="color: #009900;">&#125;</span><span style="color: #339933;">;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> ngx_int_t
ngx_http_hello_handler<span style="color: #009900;">&#40;</span>ngx_http_request_t <span style="color: #339933;">*</span>r<span style="color: #009900;">&#41;</span>
<span style="color: #009900;">&#123;</span>
    ngx_int_t    rc<span style="color: #339933;">;</span>
    ngx_buf_t   <span style="color: #339933;">*</span>b<span style="color: #339933;">;</span>
    ngx_chain_t  out<span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* we response to 'GET' and 'HEAD' requests only */</span>
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span><span style="color: #009900;">&#40;</span>r<span style="color: #339933;">-&gt;</span>method <span style="color: #339933;">&amp;</span> <span style="color: #009900;">&#40;</span>NGX_HTTP_GET<span style="color: #339933;">|</span>NGX_HTTP_HEAD<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #b1b100;">return</span> NGX_HTTP_NOT_ALLOWED<span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* discard request body, since we don't need it here */</span>
    rc <span style="color: #339933;">=</span> ngx_http_discard_request_body<span style="color: #009900;">&#40;</span>r<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span>rc <span style="color: #339933;">!=</span> NGX_OK<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #b1b100;">return</span> rc<span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* set the 'Content-type' header */</span>
    r<span style="color: #339933;">-&gt;</span>headers_out.<span style="color: #202020;">content_type_len</span> <span style="color: #339933;">=</span> <span style="color: #993333;">sizeof</span><span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;text/html&quot;</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">-</span> <span style="color: #0000dd;">1</span><span style="color: #339933;">;</span>
    r<span style="color: #339933;">-&gt;</span>headers_out.<span style="color: #202020;">content_type</span>.<span style="color: #202020;">len</span> <span style="color: #339933;">=</span> <span style="color: #993333;">sizeof</span><span style="color: #009900;">&#40;</span><span style="color: #ff0000;">&quot;text/html&quot;</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">-</span> <span style="color: #0000dd;">1</span><span style="color: #339933;">;</span>
    r<span style="color: #339933;">-&gt;</span>headers_out.<span style="color: #202020;">content_type</span>.<span style="color: #202020;">data</span> <span style="color: #339933;">=</span> <span style="color: #009900;">&#40;</span>u_char <span style="color: #339933;">*</span><span style="color: #009900;">&#41;</span> <span style="color: #ff0000;">&quot;text/html&quot;</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* send the header only, if the request type is http 'HEAD' */</span>
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span>r<span style="color: #339933;">-&gt;</span>method <span style="color: #339933;">==</span> NGX_HTTP_HEAD<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        r<span style="color: #339933;">-&gt;</span>headers_out.<span style="color: #202020;">status</span> <span style="color: #339933;">=</span> NGX_HTTP_OK<span style="color: #339933;">;</span>
        r<span style="color: #339933;">-&gt;</span>headers_out.<span style="color: #202020;">content_length_n</span> <span style="color: #339933;">=</span> <span style="color: #993333;">sizeof</span><span style="color: #009900;">&#40;</span>ngx_hello_string<span style="color: #009900;">&#41;</span> <span style="color: #339933;">-</span> <span style="color: #0000dd;">1</span><span style="color: #339933;">;</span>
&nbsp;
        <span style="color: #b1b100;">return</span> ngx_http_send_header<span style="color: #009900;">&#40;</span>r<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* allocate a buffer for your response body */</span>
    b <span style="color: #339933;">=</span> ngx_pcalloc<span style="color: #009900;">&#40;</span>r<span style="color: #339933;">-&gt;</span>pool<span style="color: #339933;">,</span> <span style="color: #993333;">sizeof</span><span style="color: #009900;">&#40;</span>ngx_buf_t<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span>b <span style="color: #339933;">==</span> <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #b1b100;">return</span> NGX_HTTP_INTERNAL_SERVER_ERROR<span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* attach this buffer to the buffer chain */</span>
    out.<span style="color: #202020;">buf</span> <span style="color: #339933;">=</span> b<span style="color: #339933;">;</span>
    out.<span style="color: #202020;">next</span> <span style="color: #339933;">=</span> <span style="color: #000000; font-weight: bold;">NULL</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* adjust the pointers of the buffer */</span>
    b<span style="color: #339933;">-&gt;</span>pos <span style="color: #339933;">=</span> ngx_hello_string<span style="color: #339933;">;</span>
    b<span style="color: #339933;">-&gt;</span>last <span style="color: #339933;">=</span> ngx_hello_string <span style="color: #339933;">+</span> <span style="color: #993333;">sizeof</span><span style="color: #009900;">&#40;</span>ngx_hello_string<span style="color: #009900;">&#41;</span> <span style="color: #339933;">-</span> <span style="color: #0000dd;">1</span><span style="color: #339933;">;</span>
    b<span style="color: #339933;">-&gt;</span>memory <span style="color: #339933;">=</span> <span style="color: #0000dd;">1</span><span style="color: #339933;">;</span>    <span style="color: #808080; font-style: italic;">/* this buffer is in memory */</span>
    b<span style="color: #339933;">-&gt;</span>last_buf <span style="color: #339933;">=</span> <span style="color: #0000dd;">1</span><span style="color: #339933;">;</span>  <span style="color: #808080; font-style: italic;">/* this is the last buffer in the buffer chain */</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* set the status line */</span>
    r<span style="color: #339933;">-&gt;</span>headers_out.<span style="color: #202020;">status</span> <span style="color: #339933;">=</span> NGX_HTTP_OK<span style="color: #339933;">;</span>
    r<span style="color: #339933;">-&gt;</span>headers_out.<span style="color: #202020;">content_length_n</span> <span style="color: #339933;">=</span> <span style="color: #993333;">sizeof</span><span style="color: #009900;">&#40;</span>ngx_hello_string<span style="color: #009900;">&#41;</span> <span style="color: #339933;">-</span> <span style="color: #0000dd;">1</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* send the headers of your response */</span>
    rc <span style="color: #339933;">=</span> ngx_http_send_header<span style="color: #009900;">&#40;</span>r<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
    <span style="color: #b1b100;">if</span> <span style="color: #009900;">&#40;</span>rc <span style="color: #339933;">==</span> NGX_ERROR <span style="color: #339933;">||</span> rc <span style="color: #339933;">&gt;</span> NGX_OK <span style="color: #339933;">||</span> r<span style="color: #339933;">-&gt;</span>header_only<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #b1b100;">return</span> rc<span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">/* send the buffer chain of your response */</span>
    <span style="color: #b1b100;">return</span> ngx_http_output_filter<span style="color: #009900;">&#40;</span>r<span style="color: #339933;">,</span> <span style="color: #339933;">&amp;</span>out<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span>
&nbsp;
&nbsp;
<span style="color: #993333;">static</span> <span style="color: #993333;">char</span> <span style="color: #339933;">*</span>
ngx_http_hello<span style="color: #009900;">&#40;</span>ngx_conf_t <span style="color: #339933;">*</span>cf<span style="color: #339933;">,</span> ngx_command_t <span style="color: #339933;">*</span>cmd<span style="color: #339933;">,</span> <span style="color: #993333;">void</span> <span style="color: #339933;">*</span>conf<span style="color: #009900;">&#41;</span>
<span style="color: #009900;">&#123;</span>
    ngx_http_core_loc_conf_t <span style="color: #339933;">*</span>clcf<span style="color: #339933;">;</span>
&nbsp;
    clcf <span style="color: #339933;">=</span> ngx_http_conf_get_module_loc_conf<span style="color: #009900;">&#40;</span>cf<span style="color: #339933;">,</span> ngx_http_core_module<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    clcf<span style="color: #339933;">-&gt;</span>handler <span style="color: #339933;">=</span> ngx_http_hello_handler<span style="color: #339933;">;</span> <span style="color: #808080; font-style: italic;">/* handler to process the 'hello' directive */</span>
&nbsp;
    <span style="color: #b1b100;">return</span> NGX_CONF_OK<span style="color: #339933;">;</span>
<span style="color: #009900;">&#125;</span></pre></div></div>

<p>config:</p>

<div class="wp_syntax"><div class="code"><pre class="sh" style="font-family:monospace;">ngx_addon_name=ngx_http_hello_module
HTTP_MODULES=&quot;$HTTP_MODULES ngx_http_hello_module&quot;
NGX_ADDON_SRCS=&quot;$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_hello_module.c&quot;</pre></div></div>

<p>hello.conf:</p>

<div class="wp_syntax"><div class="code"><pre class="conf" style="font-family:monospace;"># hello
&nbsp;
worker_processes 1;
&nbsp;
events {
       worker_connections 1024;
}
&nbsp;
http {
     server {
     	    listen 8080;
	    server_name localhost;
&nbsp;
	    location / {
	    	     hello;
	    }
     }     
}</pre></div></div>

<p>If you want more detailed information, please consult <a href="http://www.evanmiller.org/nginx-modules-guide.html" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.evanmiller.org');">Evan Miller&#8217;s Guide To Nginx Module Development.</a></p>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/hxYL1UTTLH8" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2009/08/creating-a-hello-world-nginx-module/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2009/08/creating-a-hello-world-nginx-module/</feedburner:origLink></item>
		<item>
		<title>Learn from Memcached’s Success</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/3vtNJv2oEqI/</link>
		<comments>http://blog.zhuzhaoyuan.com/2009/07/learn-from-memcacheds-success/#comments</comments>
		<pubDate>Mon, 13 Jul 2009 06:51:41 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[C10K]]></category>

		<category><![CDATA[Programming]]></category>

		<category><![CDATA[memcached]]></category>

		<category><![CDATA[server]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=147</guid>
		<description><![CDATA[Memcached becomes more and more popular nowadays. It is widely used by many heavy loaded sites. Why does it succeed?
Well, of course the first and the most important reason is that it meets the need for speed of the web 2.0 sites, by caching data and objects in memory. However, from the point of view [...]]]></description>
			<content:encoded><![CDATA[<p>Memcached becomes more and more popular nowadays. It is widely used by many heavy loaded sites. Why does it succeed?</p>
<p>Well, of course the first and the most important reason is that it meets the need for speed of the web 2.0 sites, by caching data and objects in memory. However, from the point of view of a server developer, what I want to emphasize is that it is the simplicity of memcached&#8217;s protocol design makes it more successful. Take a look at memcached&#8217;s protocol:</p>
<li>storage: (&quot;set&quot;, &quot;add&quot;, &quot;replace&quot;, &quot;append&quot;, &quot;prepend&quot;, &quot;cas&quot;)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;command name&gt; &lt;key&gt; &lt;flags&gt; &lt;exptime&gt; &lt;bytes&gt; [noreply]rn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cas &lt;key&gt; &lt;flags&gt; &lt;exptime&gt; &lt;bytes&gt; &lt;cas unique&gt; [noreply]rn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reply: (&quot;ERRORrn&quot;, &quot;CLIENT_ERROR &lt;error&gt;rn&quot;, &quot;SERVER_ERROR &lt;error&gt;rn&quot;,<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;STOREDrn&quot;, &quot;NOT_FOUNDrn&quot;, &quot;EXISTSrn&quot;, &quot;NOT_FOUNDrn&quot;)
</li>
<li>retrieval: (&quot;get&quot;, &quot;gets&quot;)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;get &lt;key&gt;rn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;gets &lt;key&gt;rn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reply: (&quot;ENDrn&quot;,<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;VALUE &lt;key&gt; &lt;flags&gt; &lt;bytes&gt; [&lt;cas unique&gt;]rn&lt;data block&gt;rn&quot;)
</li>
<li>
deletion:<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;delete &lt;key&gt; [&lt;time&gt;] [noreply]rn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reply: (&quot;DELETEDrn&quot;, &quot;NOT_FOUNDrn&quot;)
</li>
<li>
increment/decrement: (&quot;incr&quot;, &quot;decr&quot;)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;incr &lt;key&gt; &lt;value&gt; [noreply]rn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;decr &lt;key&gt; &lt;value&gt; [noreply]rn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reply: (&quot;NOT_FOUNDrn&quot;,<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;&lt;value&gt;rn&quot;)
</li>
<li>
statistics: (&quot;stat&quot;)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;statsrn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;stats &lt;args&gt;rn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reply: (&quot;STAT &lt;name&gt; &lt;value&gt;rn&quot;,<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;STAT items:&lt;slabclass&gt;:&lt;stat&gt; &lt;value&gt;rn&quot;<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;ENDrn&quot;)
</li>
<li>
other:<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;flush_all<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reply: (&quot;OKrn&quot;)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;versionrn<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reply: (&quot;VERSION &lt;version&gt;rn&quot;)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;verbosity<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reply: (&quot;OKrn&quot;)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quit
</li>
<p>With the textual protocol as shown above, memcache can be easily supported and implemented in various programming languages. No wonder dozens of different memcache clients appear. And then it consequently boosts memcached&#8217;s use. Simple thing usually will withstand the test of time. The old simple textual protocols, e.g., HTTP, FTP, SMTP and POP3 are still in use on the modern Internet. Not only because textual protocols can be easily parsed and extended, but also they are convenient for human being to read and debug. This is where the UNIX philosophy shines.</p>
<p>In conclusion, always prefer textual protocol when designing your own application. It would turn out to be really a wise decision. </p>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/3vtNJv2oEqI" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2009/07/learn-from-memcacheds-success/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2009/07/learn-from-memcacheds-success/</feedburner:origLink></item>
		<item>
		<title>Tips on High Performance Server Programming</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/NOtOldvdjYE/</link>
		<comments>http://blog.zhuzhaoyuan.com/2009/06/tips-on-high-performance-server-programming/#comments</comments>
		<pubDate>Thu, 11 Jun 2009 02:58:10 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[C10K]]></category>

		<category><![CDATA[Programming]]></category>

		<category><![CDATA[howto]]></category>

		<category><![CDATA[server]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=126</guid>
		<description><![CDATA[Yesterday, I gave a talk entitled &#8220;Tips on High Performance Server Programming&#8221; to some computer science graduate students at Jinan University. Below are the slides I used for the talk.
Tips on High Performance Server Programming
View more Microsoft Word documents from Joshua Zhu.

]]></description>
			<content:encoded><![CDATA[<p>Yesterday, I gave a talk entitled &#8220;<a href="http://zhuzhaoyuan.com/docs/server-tips.pdf" onclick="javascript:pageTracker._trackPageview('/downloads/docs/server-tips.pdf');">Tips on High Performance Server Programming</a>&#8221; to some computer science graduate students at <a href="http://www.jnu.edu.cn/" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.jnu.edu.cn');">Jinan University</a>. Below are the slides I used for the talk.</p>
<div style="width:425px;text-align:left" id="__ss_1565494"><a style="font:14px Helvetica,Arial,Sans-serif;display:block;margin:12px 0 3px 0;text-decoration:underline;" href="http://www.slideshare.net/joshzhu/tips-on-high-performance-server-programming?type=powerpoint" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.slideshare.net');" title="Tips on High Performance Server Programming">Tips on High Performance Server Programming</a><object style="margin:0px" width="425" height="355"><param name="movie" value="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=server-tips-090610212259-phpapp01&#038;rel=0&#038;stripped_title=tips-on-high-performance-server-programming" /><param name="allowFullScreen" value="true"/><param name="allowScriptAccess" value="always"/><embed src="http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=server-tips-090610212259-phpapp01&#038;rel=0&#038;stripped_title=tips-on-high-performance-server-programming" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="355"></embed></object>
<div style="font-size:11px;font-family:tahoma,arial;height:26px;padding-top:2px;">View more <a style="text-decoration:underline;" href="http://www.slideshare.net/" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.slideshare.net');">Microsoft Word documents</a> from <a style="text-decoration:underline;" href="http://www.slideshare.net/joshzhu" onclick="javascript:pageTracker._trackPageview('/outbound/article/www.slideshare.net');">Joshua Zhu</a>.</div>
</div>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/NOtOldvdjYE" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2009/06/tips-on-high-performance-server-programming/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2009/06/tips-on-high-performance-server-programming/</feedburner:origLink></item>
		<item>
		<title>Dstat, an Excellent Replacement for Vmstat</title>
		<link>http://feedproxy.google.com/~r/JoshuaZhusBlog/~3/y07JGernhFs/</link>
		<comments>http://blog.zhuzhaoyuan.com/2009/03/dstat-an-excellent-replacement-for-vmstat/#comments</comments>
		<pubDate>Tue, 17 Mar 2009 10:59:47 +0000</pubDate>
		<dc:creator>Joshua</dc:creator>
		
		<category><![CDATA[Software]]></category>

		<category><![CDATA[benchmark]]></category>

		<category><![CDATA[graphing]]></category>

		<guid isPermaLink="false">http://blog.zhuzhaoyuan.com/?p=109</guid>
		<description><![CDATA[For a long time, I was not satisfied with vmstat, because it does not generate timestamps. Then I came across a cool program named Dstat yesterday, which can be an excellent replacement for vmstat.
By using Dstat, now not only all my needs can be met but also it is pretty easy to do resource usage [...]]]></description>
			<content:encoded><![CDATA[<p>For a long time, I was not satisfied with vmstat, because it does not generate timestamps. Then I came across a cool program named <a href="http://dag.wieers.com/home-made/dstat/" onclick="javascript:pageTracker._trackPageview('/outbound/article/dag.wieers.com');">Dstat</a> yesterday, which can be an excellent replacement for vmstat.</p>
<p><div id="attachment_110" class="wp-caption alignnone" style="width: 310px"><a href="http://blog.zhuzhaoyuan.com/wp-content/uploads/2009/03/dstat.png" ><img class="size-medium wp-image-110" title="dstat" src="http://blog.zhuzhaoyuan.com/wp-content/uploads/2009/03/dstat-300x198.png" alt="A screenshot of Dstat" width="300" height="198" /></a><p class="wp-caption-text">A screenshot of Dstat</p></div></p>
<p>By using Dstat, now not only all my needs can be met but also it is pretty easy to do resource usage analysis and graphing.</p>
<p>As the author described, &#8220;Dstat is a <strong>versatile</strong> replacement for vmstat, iostat, netstat, nfsstat and ifstat. Dstat overcomes some of their limitations and adds some extra features, more counters and flexibility. Dstat is handy for monitoring systems during performance tuning tests, benchmarks or troubleshooting.&#8221; So why use three or more tools when one tool can give you everything you need?</p>
<p>Here is an example showing how to use it.</p>
<p>First, capture the resource usage information (CPU and memory) to a file, e.g. stat.dat:</p>

<div class="wp_syntax"><div class="code"><pre class="sh" style="font-family:monospace;">$ dstat -tcmn &gt; stat.dat</pre></div></div>

<p>Then use the scripts below to create CPU and memory usage graphs:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/usr/bin/gnuplot</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">set</span> terminal png
<span style="color: #000000; font-weight: bold;">set</span> output <span style="color: #ff0000;">&quot;cpu.png&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> title <span style="color: #ff0000;">&quot;CPU usage&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> xlabel <span style="color: #ff0000;">&quot;time&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> ylabel <span style="color: #ff0000;">&quot;percent&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> xdata <span style="color: #000000; font-weight: bold;">time</span>
<span style="color: #000000; font-weight: bold;">set</span> timefmt <span style="color: #ff0000;">&quot;%d-%m %H:%M:%S&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> format x <span style="color: #ff0000;">&quot;%H:%M&quot;</span>
plot <span style="color: #ff0000;">&quot;stat.dat&quot;</span> using <span style="color: #000000;">1</span>:<span style="color: #000000;">4</span> title <span style="color: #ff0000;">&quot;system&quot;</span> with lines, 
<span style="color: #ff0000;">&quot;stat.dat&quot;</span> using <span style="color: #000000;">1</span>:<span style="color: #000000;">3</span> title <span style="color: #ff0000;">&quot;user&quot;</span> with lines, 
<span style="color: #ff0000;">&quot;stat.dat&quot;</span> using <span style="color: #000000;">1</span>:<span style="color: #000000;">5</span> title <span style="color: #ff0000;">&quot;idle&quot;</span> with lines</pre></div></div>


<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/usr/bin/gnuplot</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">set</span> terminal png
<span style="color: #000000; font-weight: bold;">set</span> output <span style="color: #ff0000;">&quot;memory.png&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> title <span style="color: #ff0000;">&quot;memory usage&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> xlabel <span style="color: #ff0000;">&quot;time&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> ylabel <span style="color: #ff0000;">&quot;size(M Bytes)&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> xdata <span style="color: #000000; font-weight: bold;">time</span>
<span style="color: #000000; font-weight: bold;">set</span> timefmt <span style="color: #ff0000;">&quot;%d-%m %H:%M:%S&quot;</span>
<span style="color: #000000; font-weight: bold;">set</span> format x <span style="color: #ff0000;">&quot;%H:%M&quot;</span>
plot <span style="color: #ff0000;">&quot;stat.dat&quot;</span> using <span style="color: #000000;">1</span>:<span style="color: #000000;">9</span> title <span style="color: #ff0000;">&quot;used&quot;</span> with lines, 
<span style="color: #ff0000;">&quot;stat.dat&quot;</span> using <span style="color: #000000;">1</span>:<span style="color: #000000;">10</span> title <span style="color: #ff0000;">&quot;buff&quot;</span> with lines, 
<span style="color: #ff0000;">&quot;stat.dat&quot;</span> using <span style="color: #000000;">1</span>:<span style="color: #000000;">11</span> title <span style="color: #ff0000;">&quot;cach&quot;</span> with lines, 
<span style="color: #ff0000;">&quot;stat.dat&quot;</span> using <span style="color: #000000;">1</span>:<span style="color: #000000;">12</span> title <span style="color: #ff0000;">&quot;free&quot;</span> with lines</pre></div></div>

<p>Resource usage graph examples:<br />
<a href="http://blog.zhuzhaoyuan.com/wp-content/uploads/2009/03/cpu.png" ><img class="alignnone size-medium wp-image-116" title="cpu" src="http://blog.zhuzhaoyuan.com/wp-content/uploads/2009/03/cpu-300x225.png" alt="cpu" width="300" height="225" /></a></p>
<p><a href="http://blog.zhuzhaoyuan.com/wp-content/uploads/2009/03/memory.png" ><img class="alignnone size-medium wp-image-117" title="memory" src="http://blog.zhuzhaoyuan.com/wp-content/uploads/2009/03/memory-300x225.png" alt="memory" width="300" height="225" /></a></p>
<p>Download the scripts.<br />
<a href="http://www.zhuzhaoyuan.com/download/dstat/cpu.sh" >http://www.zhuzhaoyuan.com/download/dstat/cpu.sh</a><br />
<a href="http://www.zhuzhaoyuan.com/download/dstat/memory.sh" >http://www.zhuzhaoyuan.com/download/dstat/memory.sh</a></p>
<img src="http://feeds.feedburner.com/~r/JoshuaZhusBlog/~4/y07JGernhFs" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://blog.zhuzhaoyuan.com/2009/03/dstat-an-excellent-replacement-for-vmstat/feed/</wfw:commentRss>
		<feedburner:origLink>http://blog.zhuzhaoyuan.com/2009/03/dstat-an-excellent-replacement-for-vmstat/</feedburner:origLink></item>
	</channel>
</rss>

