Musings of a Software Developer

Moved to Github

noreply@blogger.com (Krishna Bhargava Vangapandu) — Wed, 11 Jun 2014 19:50:00 +0000

Moved my blog from here to krishnabhargav.github.io

Few handy things to know in Go Library (Command line arguments, Reflection, Setting Bits, Hash functions)

noreply@blogger.com (Krishna Bhargava Vangapandu) — Fri, 03 Jan 2014 14:45:00 +0000

Command line arguments in Go

In order to access the command line arguments passed to a Go executable, we work with the "flag" package from the standard Go library. Shown below is an example of passing command line arguments.

go run cla.go Krishna

Or if you have an executable obtained from go build, you can run it as

./cla Krishna

Shown below is an example go snippet that accesses the first argument passed & prints it.

package main

import (
    "fmt"
    "flag"
)

func main() {
 //Without the call to Parse(), arguments may not be accessed
 flag.Parse()
 fmt.Printf("%s\n",flag.Arg(0))
}

Notice that you will have to invoke the call to flag.Parse() without which you will not be able to access the arguments.

Reflection support in Go

We take the previous example, add some code for reflection which will help us understand how reflection library may be used. Just some simple example without having to go deep. For that, read this post from the official Go blog.

So what is the type returned by flag.Arg(0) ? We will start from there and go use some functions.

package main

import (
 "fmt" //regular format package
 "flag" //package from command line arguments
 "reflect" //package for reflection
)

func main() {
 //Without the call to Parse(), arguments may not be accessed
 flag.Parse()
 fmt.Printf("%s\n",flag.Arg(0))

 passed := flag.Arg(0)
 //TypeOf actually returns a Type type
 fmt.Println("typeof : ",reflect.TypeOf(passed))

 //ValueOf actually returns a Value
 fmt.Println("valueof : ",reflect.ValueOf(passed))

 //Example functions from the Value type
 fmt.Println("type: ",reflect.ValueOf(passed).Type())
 fmt.Println("kind: ", reflect.ValueOf(passed).Kind())
 fmt.Println("interface:", reflect.ValueOf(passed).Interface())
}

Big Integer & Set Bits

In the future post, I would like to implement a Bloom filter in Go but for that I will need to know how to do the following:

- Create a big number and set some bit "x" to 1 or 0.

- Accept a string, convert it to byte array, then hash the byte array.

To support the above, I wanted to see how simple big numbers would work. I am only looking enough to support my purpose.

Big Numbers (multi-precision numbers) are supported using "math/big" package in Go library. Shown below is an example code that does that.

package main

import ( 
   "fmt"
   "math/big"

   "unsafe"
)

func main() {
 
 //create a new big integer
 filter := big.NewInt(0)

 fmt.Println("filter value: ",filter)
 
 //accessing the sizeof from unsafe package
 fmt.Println("size of : ",unsafe.Sizeof(filter))
 
 //example of how to cast uintptr to int
 filter.SetBit(filter,int(unsafe.Sizeof(filter)*8)-1,1)
 
 fmt.Println("filter after setbit last to 1 : ",filter)
}

The above example shows a few things:
- Creating an instance of big.Int using the NewInt() call.
- Accessing the size of the big.Int using unsafe package exported SizeOf() function
- SizeOf() returns uintptr and we convert that to integer - that is a good example of casting
- How to set bit on the big.Int using SetBit

Working with Hash functions

Example below demonstrates the following

how to convert a string to a byte array
how to use FNV Hash from the standard Go library

package main

import (
 "fmt"
 "hash/fnv"
)

func main() {
        //Create a hash fnv function
 hash := fnv.New32()
 
 name := "Krishna"

 //converting string to byte array
 namebytes := []byte(name)

 fmt.Println("hash reset : ",hash.Sum32())
 
 hash.Write(namebytes)

 fmt.Println("hash of Krishna : ",hash.Sum32())

 hash.Reset() //resets the hash

 fmt.Println("hash reset : ",hash.Sum32())

 //write again
 hash.Write(namebytes)
 
 fmt.Println("hash of Krishna : ", hash.Sum32())
}

Hash libraries in the Go "hash" package implements the Hash interface. The implementations in the standard Go library as of this time are adler32, crc32, crc64, fnv-1/fnv-1a. The above example creates an instance of FNV hash, writes a byte array to it and then gets the value through Sum32() call. You can "forget" what is written to it by Reset() function which will allow you to reuse the hash instance.

I am newly learning Go and along the way sharing some things that I learnt. If you know of a better way to accomplish what I have shared here, please feel free to comment.

Working with "Packages" in Go language

noreply@blogger.com (Krishna Bhargava Vangapandu) — Thu, 02 Jan 2014 04:30:00 +0000

Packages in Golang

The specification states that all source files for a package be stored in the same directory.
You can import a package as shown below.

import "fmt"

In this case, the Printf method may be accessed as fmt.Printf(). All other identifiers exported by the package is also accessible as fmt.XXXX().

You can also do the following.

import f "fmt"

In this case, the qualifier is "f" instead of the default "fmt". So you access as "f.Printf()".

One other way is the following.

 import . "fmt"

In this case, you dont need a qualifier for the exported identifiers from the fmt package. You may also use the blank identifier (underscore : _) as the qualifier name for cases where you wish the initialize the package but not necessarily import its exported identifiers.
Source files of a package have an init() function. See below for more details.
The previous example of a simple Web Server is now split into two files

//GoWebServer.go
package main

import f "fmt"

//init() is a special function that gets initialized.
func init(){
 f.Printf ("Package Initialized\n")
}

func main() {
 f.Printf("Web Server will be started at port 8080\n")
 runWebServer()
}

//WebServerHelper.go

package main

import (
 . "fmt"
 "io"
 web "net/http"
)

func init(){
 Printf("Initializing WebServerHelper.go \n")
}

func runWebServer(){
 rootHandler := web.HandlerFunc(func(w web.ResponseWriter, r *web.Request) {
  io.WriteString(w, "Hi there, I love "+r.URL.Path)
 })

 srv := web.Server{
  Handler: rootHandler,
  Addr:    ":8080",
 }
 srv.ListenAndServe()
}

Few things that I learnt when trying to make this work.
- You need to specify all the files required for your main program to build properly. So you may

go build

Or you may run

go build *.go

- You can also specify output to be generated as

go build -o output/gowebserver

Or be more explicit

go build -o output/gowebserver *.go

- Go complier does not like if a package is imported but not used. This is good. I cannot tell how annoying i find C# using statements that are just there but no one really uses them.

Notice that both the files have an init() function defined and when I build everything and run the program, the output is as follows:

Package Initialized
Initializing WebServerHelper.go 
Web Server will be started at port 8080

Such init() functions may be defined multiple times even in the same source file & the execution order is unspecified/undefined. So we should not write our initialization logic based or the execution order.
And init() cannot be called by anyone else. Packages are initialized only once. So if you import packages P & Q but Q also imports P, then P is initialized only once.

Package initialization is comprised of variable initialization (package level variables are all assigned initial values) & invocation of init() functions - performed one package at a time in a single go routine.

See below for a basic program that indicates the above said sequence - variable initialization followed by init() method.

package main

import "fmt"

//x := 2000
var x int = 2000

func init(){
 var y int = x
 fmt.Printf("X value is %d\n",y)
}

func main(){ }

x is a global variable (which by the way cannot use the shortcut notation that is commented) that is initialized to 2000 and that is what gets printed when the program is executed.

Local Packages in Go

When you say

import "fmt"

the package "fmt" is looked for in the standard Go tree. Then it is looked for in "$GOPATH/pkg" directory where $GOPATH is the environment variable set to some directory on your machine. Commands such as "go get" also looks for this directory. You can read more about it here (look for Go path). So what about local packages - the one you do not want to put in GOPATH but leave it in the current directory for your application? To make this work, I renamed the package declaration (which originally was main) for the WebServerHelper.go as

package ws

Then I created a folder with the same name as the package & placed the .go file in the folder. Then my import statements on the GoWebServer.go (the main file) has changed to

import (
 f "fmt"
 ws "./ws"
)

Notice the "./ws" declaration. So the import "XXX" statement says look for "XXX" file path. So without "./" it was looking in standard GO tree, then it looks at GOPATH. If there was a "./", it was just looking at the local directory. You may also give absolute paths. Also you don't need to build this directory specially. Just have your main file compile and you are good.

One other things that I had to change in the code to make the application work as before.
- In packages, if your function is to be exported, then it should start with upper case. In general, a field or a function or other identifiers within a package that starts with upper case letter are exported.

So if you are trying to make the previous code run (after the directory changes as described above), you will get the following

./GoWebServer.go:15: cannot refer to unexported name ws.runWebServer
./GoWebServer.go:15: undefined: ws.runWebServer

So just change the name of the function to "RunWebServer" & it becomes exported and ready to use.

That's it for now!

Writing some Go! code

noreply@blogger.com (Krishna Bhargava Vangapandu) — Tue, 31 Dec 2013 05:36:00 +0000

This post is my notes when writing some code in Go lang. To start off, I will write a Hello World program in Go language. See below.

package main

import "fmt"

func main(){
 messageToPrint := "Welcome to Go!"
 fmt.Printf(messageToPrint + "\n")
}

Now to run this, you can do one of the following :

go run HelloWorld.go

go build HelloWorld.go

In the first case, "go run" compiles the program and runs it. It does not create an executable as such. (It could be creating a temporary executable somewhere.) The second command actually builds an executable which you can run. Go also has a tool called "gofmt" which you can run directly as gofmt or can run as

go fmt HelloWorld.go

. The formatting tool formats your source code so that it matches the Go-defined formatting. This is a good thing as each one of us need not invent our own coding conventions. Gofmt can also be used to perform some code refactoring. In order to perform a rename variable, the following command does the trick.

gofmt -l -w -r 'messageToPrint -> mesg' ./HelloWorld.go

The above command would rename the messageToPrint variable as mesg. In general, you pass 'pattern -> replaceWith' style of commands when using the -r option. More information on what gofmt -r can be used for is available in this presentation

Writing a Web Server in Go!

The following code snippet shows a Web Server that is written in Go! and also shows how to write anonymous functions in Go!

package main

import (
 "fmt"
 "net/http"
)

func main() {
 fmt.Printf("Web Server will be started at port 8080\n")
        //See below for anonymous function
 http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path)
 })
 http.ListenAndServe(":8080", nil)
}

Now a better way (in my opinion) to write the same code as above is shown below.

package main

import (
 "fmt"
 "net/http"
)

func main() {
 fmt.Printf("Web Server will be started at port 8080\n")
 rootHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path)
 })

 srv := http.Server{
  Handler: rootHandler,
  Addr:    ":8080",
 }

 srv.ListenAndServe()
}

In this example, we are actually creating an instance of a Http Server as opposed to the previous example where the http global helper methods were used to start the Http Web Service.

I was just curious to see how well this naive web service performs. Using apache bench, when I run 10000 requests at concurrency level of 100-700, the sample (stupid) program can serve around 5000 requests/second on my age old laptop. Just for kicks, the server in Go is atleast 5 times faster for each request as compared to the nodejs http server.

Back to Go! ..

Package has to be "main" for your main program. Otherwise, it does not work.
Passing -x flag to your helper commands such as "go run", "go fmt" or "go build" will display the complete output and the commands internally executed.
The "go build" has all compiler optimizations enabled.

Hello World! with Scala Web Application using Play 2.0 and Scalate

noreply@blogger.com (Krishna Bhargava Vangapandu) — Wed, 14 Nov 2012 04:22:00 +0000

In this post, we will look at how to make a hello-world web application in scala using the Play 2.0 Web Framework and change the template engine to work with Scalate. We assume your machine is all set up with Java and Scala configured.

Step 1 : DOWNLOAD AND CONFIGURE Play 2.0

Download the Play Framework 2.0 from http://download.playframework.org/releases/play-2.0.4.zip. Then extract the zip file to some location. I put mine at C:\Development\tools\play-2.0.4\. It is usually a good idea to leave the version number intact so as to easily know what version of any tool you are currently having on your machine.

The configuration portion is pretty simple – Update your PATH environment variable to include the root directory of the play-2.0.4 folder which has the “play.bat” (I am running windows).

To verify, launch your command line or powershell and type the command “play”. You should see something like shown below.

Step 2 : CREATE A NEW PLAY 2.0 WEB APPLICATION

Using command line (change directory – cd), go to a folder where you wish to place your application. Run the following command

play new helloworld-scalate

Here helloworld-scalate will be my application name.

Then from the command line go into the “helloworld-scalate” folder. And then run the following command.

play run

The first time you run, it takes a few seconds to initialize the project and upon success, you will see on-screen instructions that the application is hosted at 9000 port on localhost.

Verify by opening your browser and loading http://localhost:9000

You should see a welcome page.

Step 3 : INTEGRATING SCALATE INT YOUR APPLICATION

I found the proper instructions to do this here. But I will repeat them again.

Go to helloworld-scalate\project\Build.scala

3.a Add the following lines to appDependencies.

"org.fusesource.scalate" % "scalate-core" % "1.5.3"

What did we do? What we are doing is to add a dependency on scalate-core in an SBT friendly style. Play 2.0 takes care of fetching this dependency through its repositories which is listed at helloworld-scalate\project\plugins.sbt

To verify if things are fine, go back to command line and run the command

play run

You should see that SBT updates the dependencies. Just verify if your application can still be browsed from localhost:9000.

3.b Add ScalaIntegration.scala to helloworld-scalate\app\lib

You will need to create a folder “lib” inside helloworld-scalate\app folder. And then from the URL given below copy the content into a new file – ScalaIntegration.scala

https://raw.github.com/janhelwich/Play-2-with-Scala-and-Scalate/master/app/controllers/ScalateIntegration.scala

This does the necessary bootstrapping such that you can now use scalate in the current project.

Run the “play run” command again and reload the page at localhost:9000 to make sure you haven’t made a mistake.

Step 4 : UPDATE VIEWS & ACTION WITH TEMPLATE OF YOUR CHOICE

Scalate supports multiple template engines like Jade, Mustache, Scaml. The documentation indicates that Scaml gives the best performance given the strongly typed nature. But I might have misread the docs so read and check yourself.

To do this, rename your files inside “helloworld-scalate\app\views” folder such that extension is changed from .scala.html to .scaml. Note that you will see two files – the layout file (main) and the view file (index).

In the main.scaml, add the following code.

-@ var body: String
-@ var title: String = "Page"

%html
  %head
    %title= title
  %body
    != body

In the index.scaml, add the following code.

-@ var message: String = "Page"
/Layout template file given here
-attributes("layout") = "main.scaml"

%h1= message

Notice that in the index.scaml, we specified that we want to apply the main.scaml layout using the attributes(“layout”) declaration.

You also need to change the application controller which is inside the helloworld-scala\app\controllers\application.scala

The Application controller has an action called “index” which is the one that gets executed when you were loading the browser. We need to change this action such that it renders using Scalate instead of the default one.

package controllers

import play.api._
import play.api.mvc._

object Application extends Controller {

  def index = Action {
    //Ok(views.html.index("Your new application is ready."))
    Ok(Scalate("index.scaml").render(‘message -> "Hello World"))
  }
}

As you can see in the index action, I commented the existing line added this one line.

Ok(Scalate("index.scaml").render(‘message-> "Hello World"))

Please pay attention to the ‘ in front of the title. Here the ‘message is a symbol literal. You can read more about them here.

Now run the “play run” command again and you should see plain html page that prints “Hello World”.

I hope this step-by-step helped you get started with a new play 2.0 web application in Scala and also made you familiar with integrating the scalate engine. Note that the original url which I mentioned earlier has the same steps but step 4 where default Scalate template type is specified in the <application folder>\conf\application.conf is not performed. I skipped this step and it doesn’t appear that it will cause any problems. But again, after reading this tutorial, if you are a beginner, then you and I are at the same level so please feel free to explore more and learn.

Thanks!