In Neo4j 3.2 and earlier if you added a String to a Double it would coerce the Double to a String and concatenate the values. The following would therefore be valid Cypher:

RETURN toFloat("1.0") + " Mark"

╒══════════╕
│"result"  │
╞══════════╡
│"1.0 Mark"│
└──────────┘

This behaviour has changed in the 3.3 series and will instead throw an exception:

RETURN toFloat("1.0") + " Mark"

Neo.ClientError.Statement.TypeError: Don't know how to add `Double(1.000000e+00)` and `String(" Mark")`

We can workaround that by forcing our query to run in 3.2 mode:

CYPHER 3.2
RETURN toFloat("1.0") + " Mark" AS result

or we can convert the Double to a String in our Cypher statement:

RETURN toString(toFloat("1.0")) + " Mark" AS result

The post Neo4j: Cypher – Neo.ClientError.Statement.TypeError: Don’t know how to add Double and String appeared first on Mark Needham.

This is an example of the JSON representation of one business:

$ cat dataset/business.json | head -n1 | jq
{
  "business_id": "FYWN1wneV18bWNgQjJ2GNg",
  "name": "Dental by Design",
  "neighborhood": "",
  "address": "4855 E Warner Rd, Ste B9",
  "city": "Ahwatukee",
  "state": "AZ",
  "postal_code": "85044",
  "latitude": 33.3306902,
  "longitude": -111.9785992,
  "stars": 4,
  "review_count": 22,
  "is_open": 1,
  "attributes": {
    "AcceptsInsurance": true,
    "ByAppointmentOnly": true,
    "BusinessAcceptsCreditCards": true
  },
  "categories": [
    "Dentists",
    "General Dentistry",
    "Health & Medical",
    "Oral Surgeons",
    "Cosmetic Dentists",
    "Orthodontists"
  ],
  "hours": {
    "Friday": "7:30-17:00",
    "Tuesday": "7:30-17:00",
    "Thursday": "7:30-17:00",
    "Wednesday": "7:30-17:00",
    "Monday": "7:30-17:00"
  }
}

The businesses reside in different countries so I wanted to extract the area/county/state and the country for each of them. I found the reverse-geocoder library which is perfect for this problem.

You give the library a lat/long or list of lat/longs and it returns you back a list containing the nearest lat/long to your points along with the name of the place, Admin regions, and country code. It’s way quicker to pass in a list of lat/longs than to call the function individually for each lat/long so we’ll do that.

We can write the following code to extract location information for a list of lat/longs:

import reverse_geocoder as rg

lat_longs = {
    "FYWN1wneV18bWNgQjJ2GNg": (33.3306902, -111.9785992),
    "He-G7vWjzVUysIKrfNbPUQ": (40.2916853, -80.1048999),
    "KQPW8lFf1y5BT2MxiSZ3QA": (33.5249025, -112.1153098)
}

business_ids = list(lat_longs.keys())
locations = rg.search(list(lat_longs.values()))

for business_id, location in zip(business_ids, locations):
    print(business_id, lat_longs[business_id], location)

This is the output we get from running the script:

$ python blog.py 
Loading formatted geocoded file...
FYWN1wneV18bWNgQjJ2GNg (33.3306902, -111.9785992) OrderedDict([('lat', '33.37088'), ('lon', '-111.96292'), ('name', 'Guadalupe'), ('admin1', 'Arizona'), ('admin2', 'Maricopa County'), ('cc', 'US')])
He-G7vWjzVUysIKrfNbPUQ (40.2916853, -80.1048999) OrderedDict([('lat', '40.2909'), ('lon', '-80.10811'), ('name', 'Thompsonville'), ('admin1', 'Pennsylvania'), ('admin2', 'Washington County'), ('cc', 'US')])
KQPW8lFf1y5BT2MxiSZ3QA (33.5249025, -112.1153098) OrderedDict([('lat', '33.53865'), ('lon', '-112.18599'), ('name', 'Glendale'), ('admin1', 'Arizona'), ('admin2', 'Maricopa County'), ('cc', 'US')])

It seems to work fairly well! Now we just need to tweak our script to read in the values from the Yelp JSON file and generate a new JSON file containing the locations:

import json

import reverse_geocoder as rg

lat_longs = {}

with open("dataset/business.json") as business_json:
    for line in business_json.readlines():
        item = json.loads(line)
        if item["latitude"] and item["longitude"]:
            lat_longs[item["business_id"]] = {
                "lat_long": (item["latitude"], item["longitude"]),
                "city": item["city"]
            }

result = {}

business_ids = list(lat_longs.keys())
locations = rg.search([value["lat_long"] for value in lat_longs.values()])

for business_id, location in zip(business_ids, locations):
    result[business_id] = {
        "country": location["cc"],
        "name": location["name"],
        "admin1": location["admin1"],
        "admin2": location["admin2"],
        "city": lat_longs[business_id]["city"]
    }

with open("dataset/businessLocations.json", "w") as business_locations_json:
    json.dump(result, business_locations_json, indent=4, sort_keys=True)

And that’s it!

The post Yelp: Reverse geocoding businesses to extract detailed location information appeared first on Mark Needham.

It was a bit trickier than I expected, mostly because I’m not that familiar with deploying Ruby applications, but I thought I’d capture what I’ve done for future me.

I have the following Gemfile that installs asciidoctor-pdf and its dependencies:

Gemfile

source 'https://rubygems.org'

gem 'prawn'
gem 'addressable'
gem 'prawn-svg'
gem 'prawn-templates'
gem 'asciidoctor-pdf'

I don’t have permissions to install gems globally on the build agents so I’m bundling those up into the vendor directory. It’s been a long time since I worked on a Ruby application so perhaps that’s par for the course.

bundle install --path vendor/
bundle package

On the build agent I’m running the following script:

export PATH="$PATH:/home/teamcity/.gem/ruby/2.3.0/bin"
mkdir $PWD/gems
export GEM_HOME="$PWD/gems"
gem install bundler --user-install --no-rdoc --no-ri && bundle install
./vendor/ruby/2.3.0/bin/asciidoctor-pdf -a allow-uri-read blog.adoc

I override where gems should be installed and then execute the asciidoctor-pdf executable from the vendor directory.

It all seems to work quite nicely but if there’s a better approach that I should be taking so let me know in the comments.

The post Running asciidoctor-pdf on TeamCity appeared first on Mark Needham.

I had some CSV files containing nodes that I wanted to import into Neo4j. Their contents look like this:

$ cat people_header.csv 
name:ID(Person)

$ cat people.csv 
"Mark"
"Michael"
"Ryan"
"Will"
"Jennifer"
"Karin"

$ cat companies_header.csv 
name:ID(Company)

$ cat companies.csv 
"Neo4j"

I find it easier to use separate header files because I often make typos with my column names and it’s easier to update a single line file than to open a multi-million line file and change the first line.

I ran the following command to create a new Neo4j database from these files:

$ ./bin/neo4j-admin import \
	--database=blog.db \
	--mode=csv \
	--nodes:Person people_header.csv,people.csv \
	--nodes:Company companies_heade.csv,companies.csv

which resulted in this error message:

Neo4j version: 3.3.3
Importing the contents of these files into /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/data/databases/blog.db:
Nodes:
  :Person
  /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/people_header.csv
  /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/people.csv

  :Company
  /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/companies.csv

...

Import error: Mixing specified and unspecified group belongings in a single import isn't supported
Caused by:Mixing specified and unspecified group belongings in a single import isn't supported
java.lang.IllegalStateException: Mixing specified and unspecified group belongings in a single import isn't supported
	at org.neo4j.unsafe.impl.batchimport.input.Groups.getOrCreate(Groups.java:52)
	at org.neo4j.unsafe.impl.batchimport.input.csv.InputNodeDeserialization.initialize(InputNodeDeserialization.java:60)
	at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.initialize(InputEntityDeserializer.java:68)
	at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda$new$0(ParallelInputEntityDeserializer.java:104)
	at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing.lambda$submit$1(TicketedProcessing.java:103)
	at org.neo4j.unsafe.impl.batchimport.executor.DynamicTaskExecutor$Processor.run(DynamicTaskExecutor.java:237)

The output actually helpfully indicates which files it’s importing from and we can see under the ‘Company’ section that the header file is missing.

As a result of the typo I made when trying to type companies_header.csv, the tool now treats the first line of companies.csv as the header and since we haven’t specified a group (e.g. Company, Person) on that line we receive this error.

Let’s fix the typo and try again:

$ ./bin/neo4j-admin import \
	--database=blog.db \
	--mode=csv \
	--nodes:Person people_header.csv,people.csv \
	--nodes:Company companies_header.csv,companies.csv

Neo4j version: 3.3.3
Importing the contents of these files into /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/data/databases/blog.db:
Nodes:
  :Person
  /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/people_header.csv
  /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/people.csv

  :Company
  /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/companies_header.csv
  /Users/markneedham/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-b59e33d5-2060-4a5d-bdb8-0b9f6dc919fa/installation-3.3.3/companies.csv

...

IMPORT DONE in 1s 5ms. 
Imported:
  7 nodes
  0 relationships
  7 properties
Peak memory usage: 480.00 MB

Success!

The post Neo4j Import: java.lang.IllegalStateException: Mixing specified and unspecified group belongings in a single import isn’t supported appeared first on Mark Needham.

The most recent one I wrote does the formatting around the Featured Community Member of the Week. I call it like this from the asciidoc, passing in the name of the person and a link to an image:

featured::https://s3.amazonaws.com/dev.assets.neo4j.com/wp-content/uploads/20180202004247/this-week-in-neo4j-3-february-2018.jpg[name="Suellen Stringer-Hye"]

The code for the macro has two parts. The first is some wiring code that registers the macro with Asciidoctor:

lib/featured-macro.rb

RUBY_ENGINE == 'opal' ? (require 'featured-macro/extension') : (require_relative 'featured-macro/extension')

Asciidoctor::Extensions.register do
  if (@document.basebackend? 'html') && (@document.safe < SafeMode::SECURE)
    block_macro FeaturedBlockMacro
  end
end

And this is the code for the macro itself:

lib/featured-macro/extension.rb

require 'asciidoctor/extensions' unless RUBY_ENGINE == 'opal'

include ::Asciidoctor

class FeaturedBlockMacro < Extensions::BlockMacroProcessor
  use_dsl

  named :featured

  def process parent, target, attrs
    name = attrs["name"]

    html = %(

                

                    
                
            
            

              #{name} - This Week's Featured Community Member
            
)

    create_pass_block parent, html, attrs, subs: nil
  end
end

When we convert the asciidoc into HTML we need to tell asciidoctor about the macro, which we can do like this:

asciidoctor template.adoc \
  -r ./lib/featured-macro.rb \
  -o -

And that's it!

The post Asciidoctor: Creating a macro appeared first on Mark Needham.

My model is based on one from the text classification tutorial. The tutorial shows how to create custom Estimators which we can learn more about in a post on the Google Developers blog.

Imports

Let’s get started. First, our imports:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import numpy as np
import pandas as pd
import tensorflow as tf
from sklearn import preprocessing
from sklearn.model_selection import train_test_split

We’ve obviously got Tensorflow, but also scikit-learn which we’ll use to split our data into a training and test sets as well as convert the author names into numeric values.

Model building functions

Next we’ll create a function to create a bag of words model. This function calls another one that creates different EstimatorSpecs depending on the context it’s called from.

EMBEDDING_SIZE = 50
MAX_LABEL = 3
WORDS_FEATURE = 'words'  # Name of the input words feature.


def bag_of_words_model(features, labels, mode):
    bow_column = tf.feature_column.categorical_column_with_identity(WORDS_FEATURE, num_buckets=n_words)
    bow_embedding_column = tf.feature_column.embedding_column(bow_column, dimension=EMBEDDING_SIZE)
    bow = tf.feature_column.input_layer(features, feature_columns=[bow_embedding_column])
    logits = tf.layers.dense(bow, MAX_LABEL, activation=None)
    return create_estimator_spec(logits=logits, labels=labels, mode=mode)


def create_estimator_spec(logits, labels, mode):
    predicted_classes = tf.argmax(logits, 1)
    if mode == tf.estimator.ModeKeys.PREDICT:
        return tf.estimator.EstimatorSpec(
            mode=mode,
            predictions={
                'class': predicted_classes,
                'prob': tf.nn.softmax(logits),
                'log_loss': tf.nn.softmax(logits),
            })

    loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)
    if mode == tf.estimator.ModeKeys.TRAIN:
        optimizer = tf.train.AdamOptimizer(learning_rate=0.01)
        train_op = optimizer.minimize(loss, global_step=tf.train.get_global_step())
        return tf.estimator.EstimatorSpec(mode, loss=loss, train_op=train_op)

    eval_metric_ops = {
        'accuracy': tf.metrics.accuracy(labels=labels, predictions=predicted_classes)
    }
    return tf.estimator.EstimatorSpec(mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)

Loading data

Now we’re ready to load our data.

Y_COLUMN = "author"
TEXT_COLUMN = "text"
le = preprocessing.LabelEncoder()

train_df = pd.read_csv("train.csv")
X = pd.Series(train_df[TEXT_COLUMN])
y = le.fit_transform(train_df[Y_COLUMN].copy())
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

The only interesting thing here is the LabelEncoder. We’ll keep that around as we’ll use it later as well.

Transform documents

At the moment our training and test dataframes contain text, but Tensorflow works with vectors so we need to convert our data into that format. We can use the VocabularyProcessor to do this:

MAX_DOCUMENT_LENGTH = 100
vocab_processor = tf.contrib.learn.preprocessing.VocabularyProcessor(MAX_DOCUMENT_LENGTH)

X_transform_train = vocab_processor.fit_transform(X_train)
X_transform_test = vocab_processor.transform(X_test)

X_train = np.array(list(X_transform_train))
X_test = np.array(list(X_transform_test))

n_words = len(vocab_processor.vocabulary_)
print('Total words: %d' % n_words)

Training our model

Finally we’re ready to train our model! We’ll call the Bag of Words model we created at the beginning and build a train input function where we pass in the training arrays that we just created:

model_fn = bag_of_words_model
classifier = tf.estimator.Estimator(model_fn=model_fn)

train_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={WORDS_FEATURE: X_train},
    y=y_train,
    batch_size=len(X_train),
    num_epochs=None,
    shuffle=True)
classifier.train(input_fn=train_input_fn, steps=100)

Evaluating our model

Let’s see how our model fares. We’ll call the evaluate function with our test data:

test_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={WORDS_FEATURE: X_test},
    y=y_test,
    num_epochs=1,
    shuffle=False)

scores = classifier.evaluate(input_fn=test_input_fn)
print('Accuracy: {0:f}, Loss {1:f}'.format(scores['accuracy'], scores["loss"]))

INFO:tensorflow:Saving checkpoints for 1 into /var/folders/k5/ssmkw9vd2yb3h5wnqlxnqbkw0000gn/T/tmpb6v4rrrn/model.ckpt.
INFO:tensorflow:loss = 1.0888131, step = 1
INFO:tensorflow:Saving checkpoints for 100 into /var/folders/k5/ssmkw9vd2yb3h5wnqlxnqbkw0000gn/T/tmpb6v4rrrn/model.ckpt.
INFO:tensorflow:Loss for final step: 0.18394235.
INFO:tensorflow:Starting evaluation at 2018-01-28-22:41:34
INFO:tensorflow:Restoring parameters from /var/folders/k5/ssmkw9vd2yb3h5wnqlxnqbkw0000gn/T/tmpb6v4rrrn/model.ckpt-100
INFO:tensorflow:Finished evaluation at 2018-01-28-22:41:34
INFO:tensorflow:Saving dict for global step 100: accuracy = 0.8246673, global_step = 100, loss = 0.44942895
Accuracy: 0.824667, Loss 0.449429

Not too bad! I managed to get a log loss score of ~ 0.36 with a scikit-learn ensemble model but it is better than some of my first attempts.

Generating predictions

I wanted to see how it’d do against Kaggle’s test dataset so I generated a CSV file with predictions:

test_df = pd.read_csv("test.csv")

X_test = pd.Series(test_df[TEXT_COLUMN])
X_test = np.array(list(vocab_processor.transform(X_test)))

test_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={WORDS_FEATURE: X_test},
    num_epochs=1,
    shuffle=False)

predictions = classifier.predict(test_input_fn)
y_predicted_classes = np.array(list(p['prob'] for p in predictions))

output = pd.DataFrame(y_predicted_classes, columns=le.classes_)
output["id"] = test_df["id"]
output.to_csv("output.csv", index=False, float_format='%.6f')

Here we go:

The score is roughly the same as we saw with the test split of the training set. If you want to see all the code in one place I’ve put it on my Spooky Authors GitHub repository.

The post Tensorflow: Kaggle Spooky Authors Bag of Words Model appeared first on Mark Needham.

$ asciidoctor --help
Usage: asciidoctor [OPTION]... FILE...
Translate the AsciiDoc source FILE or FILE(s) into the backend output format (e.g., HTML 5, DocBook 4.5, etc.)
By default, the output is written to a file with the basename of the source file and the appropriate extension.
Example: asciidoctor -b html5 source.asciidoc

    -b, --backend BACKEND            set output format backend: [html5, xhtml5, docbook5, docbook45, manpage] (default: html5)
                                     additional backends are supported via extensions (e.g., pdf, latex)

I don’t want to have to convert my code to one of these formats each time – I want to convert asciidoc to asciidoc!

For example, given the following files:

mydoc.adoc

= My Blog example

== Heading 1

Some awesome text

== Heading 2

include::blog_include.adoc[]

blog_include.adoc

Some included text

I want to generate another asciidoc file where the contents of the include file are exploded and displayed inline.

After a lot of searching I came across an excellent script written by Dan Allen and put it in a file called adoc.rb. We can then call it like this:

$ ruby adoc.rb mydoc.adoc
= My Blog example

== Heading 1

Some awesome text

== Heading 2

Some included text

Problem solved!

In my case I actually wanted to explode HTTP includes so I needed to pass the -a allow-uri-read flag to the script:

$ ruby adoc.rb mydoc.adoc -a allow-uri-read 

And now I can generate asciidoc files until my heart’s content.

The post Asciidoc to Asciidoc: Exploding includes appeared first on Mark Needham.

I record my runs with the Strava app and it has an API that returns lat/long coordinates for each run in the Google encoded polyline algorithm format.

We can use the polyline library to decode these values into a list of lat/long tuples. For example:

import polyline
polyline.decode('u{~vFvyys@fS]')
[(40.63179, -8.65708), (40.62855, -8.65693)]

Once we’ve got the route defined as a set of coordinates we need to compare them. My Googling led me to an algorithm called Dynamic Time Warping

DTW is a method that calculates an optimal match between two given sequences (e.g. time series) with certain restrictions.

The sequences are “warped” non-linearly in the time dimension to determine a measure of their similarity independent of certain non-linear variations in the time dimension.

The fastdtw library implements an approximation of this library and returns a value indicating the distance between sets of points.

We can see how to apply fastdtw and polyline against Strava data in the following example:

import os
import polyline
import requests
from fastdtw import fastdtw

token = os.environ["TOKEN"]
headers = {'Authorization': "Bearer {0}".format(token)}

def find_points(activity_id):
    r = requests.get("https://www.strava.com/api/v3/activities/{0}".format(activity_id), headers=headers)
    response = r.json()
    line = response["map"]["polyline"]
    return polyline.decode(line)

Now let’s try it out on two runs, 1361109741 and 1346460542:

from scipy.spatial.distance import euclidean

activity1_id = 1361109741
activity2_id = 1346460542

distance, path = fastdtw(find_points(activity1_id), find_points(activity2_id), dist=euclidean)

>>> print(distance)
2.91985018100644

These two runs are both near my house so the value is small. Let’s change the second route to be from my trip to New York:

activity1_id = 1361109741
activity2_id = 1246017379

distance, path = fastdtw(find_points(activity1_id), find_points(activity2_id), dist=euclidean)

>>> print(distance)
29383.492965394034

Much bigger!

I’m not really interested in the actual value returned but I am interested in the relative values. I’m building a little application to generate routes that I should run and I want it to come up with a routes that are different to recent ones that I’ve run. This score can now form part of the criteria.

The post Strava: Calculating the similarity of two runs appeared first on Mark Needham.

Pre requisites

We have the following HTML to define the div that will contain the map.

We also need to import the following Javascript and CSS files:

Polyline representing part of a route

The following code creates a polyline for a Strava segment that I often run.

var map = L.map('map');
L.tileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {maxZoom: 18,}).addTo(map);

var rawPoints = [
  { "latitude": 51.357874010145395, "longitude": -0.198045110923591 },
  { "latitude": 51.3573858289394, "longitude": -0.19787754933584795 },
  { "latitude": 51.35632791810057, "longitude": -0.19750254941422557 },
  { "latitude": 51.35553240304241, "longitude": -0.197232163894512 },
  { "latitude": 51.35496267279901, "longitude": -0.1970247338143316 },
  { "latitude": 51.35388700570004, "longitude": -0.19666483094752069 },
  { "latitude": 51.3533898352570, "longitude": -0.1964976504847828 },
  { "latitude": 51.35358452733139, "longitude": -0.19512563906602554 },
  { "latitude": 51.354762877995036, "longitude": -0.1945622934585907 },
  { "latitude": 51.355610110109986, "longitude": -0.19468697186046677 },
  { "latitude": 51.35680377680643, "longitude": -0.19395063336295112 },
  { "latitude": 51.356861596801075, "longitude": -0.1936180154828497 },
  { "latitude": 51.358487396611125, "longitude": -0.19349660642888197 }
];

var coordinates = rawPoints.map(rawPoint => new L.LatLng(rawPoint["latitude"], rawPoint["longitude"]))

let polyline = L.polyline(
    coordinates,
    {
        color: 'blue',
        weight: 3,
        opacity: .7,
        lineJoin: 'round'
    }
);

polyline.addTo(map);

I wanted to centre the map around the polyline and initially wrote the following code to do this:

let lats = rawPoints.map(c => c.latitude).reduce((previous, current) => current += previous, 0.0);
let longs = rawPoints.map(c => c.longitude).reduce((previous, current) => current += previous, 0.0);

const position = [lats / rawPoints.length, longs / rawPoints.length];
map.setView(position, 17);

This works fine but the zoom factor was wrong when I drew longer polylines so I needed a better solution.

I should have RTFM because there’s a much simpler way to do this. I actually found the explanation in a GitHub issue from 2011! We can replace the previous snippet with this single line of code:

map.fitBounds(polyline.getBounds());

And this is how it looks on the screen:

The post Leaflet: Fit polyline in view appeared first on Mark Needham.

In the latest version of web3 the API has changed to be based on promises so I decided to translate Thomas’ example.

Let’s get started.

Install npm libraries

We need to install these libraries before we start:

npm install web3
npm install abi-decoder
npm install ethereumjs-testrpc

What do these libraries do?

• web3 is a client library for interacting with an Ethereum blockchain
• abi-decoder is used to decode the hash of a smart contract so that we can work out what was in it.
• ethereum-testrpc lets us spin up a local test version of Ethereum

Smart contract

We’ll still use the same smart contract as Thomas did. Token.sol is a smart contract written in the Solidity language and describes money being transferred between addresses:

contracts/Token.sol

pragma solidity ^0.4.0;

contract Token {
    mapping (address => uint) public balances;
  
    function Token() {
        balances[msg.sender] = 1000000;
    }

    function transfer(address _to, uint _amount) {
        if (balances[msg.sender] < _amount) {
            throw;
        }

        balances[msg.sender] -= _amount;
        balances[_to] += _amount;
    }
}

Whenever somebody tries to transfer some money we'll put 1,000,000 in their account and then transfer the appropriate amount, assuming there's enough money in the account.

Start local Ethereum node

Let's start a local Ethereum node. We'll reduce the gas price - the amount you 'pay' to execute a transaction - so we don't run out.

$ ./node_modules/.bin/testrpc --gasPrice 20000
EthereumJS TestRPC v6.0.3 (ganache-core: 2.0.2)

Listening on localhost:8545

Pre requisites

We need to load a few Node.js modules:

const fs = require("fs"),
      abiDecoder = require('abi-decoder'),
      Web3 = require('web3'),
      solc = require('solc');

Compile smart contract

Next we'll compile our smart contract:

const input = fs.readFileSync('contracts/Token.sol');
const output = solc.compile(input.toString(), 1);
const bytecode = output.contracts[':Token'].bytecode;
const abi = JSON.parse(output.contracts[':Token'].interface);

Connect to Ethereum and create contract object

Now that we've got the ABI (Application Binary Interface) we'll connect to our local Ethereum node and create a contract object based on the ABI:

let provider = new Web3.providers.HttpProvider("http://localhost:8545");
const web3 = new Web3(provider);
let Voting = new web3.eth.Contract(abi);

Add ABI to decoder

Before we interact with the blockchain we'll first add the ABI to our ABI decoder to use later:

abiDecoder.addABI(abi);

Find (dummy) Ethereum accounts

Now we're ready to create some transactions! We'll need some Ethereum accounts to play with and if we call web3.eth.getAccounts we can get a collection of accounts that the node controls. Since our node is a test one these are all dummy accounts.

web3.eth.getAccounts().then(accounts => {
  accounts.forEach(account => {
    console.log(account)
  })
});

0xefeaE7B180c7Af4Dfd23207422071599c7dfd2f7
0x3a54BaAFDe6747531a28491FDD2F36Cb61c83663
0x367e1ac67b9a85E438C7fab7648964E5ed12061e
0xB34ECD20Be6eC99e8e9fAF641A343BAc826FFFf1
0xE65587a2951873efE3325793D5729Ef91b15d5b5
0xdA232aEe954a31179E2F5b40E6efbEa27bB89c87
0x7119fEbab069d440747589b0f1fCDDBAdBDd105d
0xCacB2b61dE0Ca12Fd6FECe230d2f956c8Cdfed34
0x4F33BF93612D1B89C8C8872D4Af30Fa2A9CbfaAf
0xA1Ebc0D19dB41A96B5278720F47C2B6Ab2506ccF

Transfer money between accounts

Now that we have some accounts let's transfer some money between them.

var allAccounts;
web3.eth.getAccounts().then(accounts => {
  allAccounts = accounts;
  Voting.deploy({data: bytecode}).send({
    from: accounts[0],
    gas: 1500000,
    gasPrice: '30000000000000'
  }).on('receipt', receipt => {
    Voting.options.address = receipt.contractAddress;
    Voting.methods.transfer(accounts[1], 10).send({from: accounts[0]}).then(transaction => {
      console.log("Transfer lodged. Transaction ID: " + transaction.transactionHash);
      let blockHash = transaction.blockHash
      return web3.eth.getBlock(blockHash, true);
    }).then(block => {
      block.transactions.forEach(transaction => {
        console.log(abiDecoder.decodeMethod(transaction.input));
      });

      allAccounts.forEach(account => {
          Voting.methods.balances(account).call({from: allAccounts[0]}).then(amount => {
            console.log(account + ": " + amount);
          });
      });
    });
  });
});

Let's run in:

Transfer lodged. Transaction ID: 0x699cbe40121d6c2da7b36a107cd5f28b35a71aff2a0d584f8e734b10f4c49de4

{ name: 'transfer',
  params: 
   [ { name: '_to',
       value: '0xeb25dbd0931386eeab267981626ae3908d598404',
       type: 'address' },
     { name: '_amount', value: '10', type: 'uint256' } ] }

0x084181d6fDe8bA802Ee85396aB1d25Ddf1d7D061: 999990
0xEb25dbD0931386eEaB267981626AE3908D598404: 10
0x7deB2487E6Ac40f85fB8f5A3bC6896391bf2570F: 0
0xA15ad4371B62afECE5a7A70457F82A30530630a3: 0
0x64644f3B6B95e81A385c8114DF81663C39084C6a: 0
0xBB68FF2935080c807D5A534b1fc481Aa3fafF1C0: 0
0x38d4A3d635B451Cb006d63ce542950C067D47F58: 0
0x7878bA9138361A08522418BD1c8376Af7220a506: 0
0xf400c0e749Fe02E7073E08d713E0A207dc91FBeb: 0
0x7070d1712a25eb7FCf78A549F17705AA66B0aD47: 0

This code:

The full example is available in my ethereum-nursery GitHub repository. Thomas also has a follow up post that shows how to deploy a contract on a remote node where client side signatures become necessary.

The post Ethereum Hello World Example using solc and web3 appeared first on Mark Needham.