Creating Anti-Squish Slides for observing insects using Foldscope

Introduction

Every since I got my hands on a Foldscope, I have been pretty much obsessed with it. I have used the Foldscope to see and discover many interesting things. One fun activity, I am sure every Foldscope user must have tried, is observing insects using a Foldscope.

If you have ever tried observing insects you might have realized that, there is a tendency of insects getting ‘squished’ due the magnets and foccusing mechanism of the Foldscope. I wanted to find a way to capture live and moving insects using Foldscope. The below write-up discusses in-detail a method of creating a ‘anti-squish’ slide for foldscope and observing insects using it.

Items needed

  • Transparent nail polish (or some sort of transparent super glue).
  • Glass slide
  • Cover slip
  • Plastic thread (something similar to one used for attaching price tags to apparels in malls and super markets).
  • Bits of white paper
  • Some patience !!

items required to create the anti-squish slide

Creating the ‘anti-squish’ slide

Air-gap created after gluing the thread and slide
One side is kept open so that specimen can be inserted

Actually the idea is very simple. We want to glue the thread between the cover slip and the slide, along the borders of cover slip, such that a cavity or air-gap is created between the slide and cover slip.

For my experiment, I used a rectangular coverslip because it is easiler to bend the thread along the edges of coverslip and gule it.

Only glue three sides of the coverslip to the slide. We will leave one side open as that will allow us to push small insects in the small ‘glass-box’ we created.

I used transparent nail-polish as a glue, but using a commercial transparent super-glue should work much better in my opinion.

It must be noted that the air-gap needs to be very small, which means the plastic thread used to create the gap should be kind of thin. This results in a limitation that only very small insects (smaller than ants) can be viewed. You may be tempted to use a much thicker thread to create more head-room, but I should caution you that you may struggle with foccusing. However, if you are able to design something that allows larger insects like ants to be observed please let me know !!

Observing insects using this slide

Using paper bits for scattering light
Ok, this is the fun part. This is what I typically do:

  • Use a pointed object like needle or toothpick to transfer the insect or larvae on the glass slide first (near the open end).

  • Now use a long strip of paper and gently push the spicemen inside the cavity we created. Be gentle and try not to harm the insect in process.

  • Push the specimen all the way inside towards the sealed walls. Since we have left one side of our glass-box open, there is a tendency of the cover slip bending near the open slide and ‘squishing’ the specimen, when we try to focus using our focussing mechanism of Foldscope.

  • I have also tried to insert very tiny bits of papers inside the slide surrounding the specimen. It serves two purposes:

    • It puts limitations on movement of the insect. Also provides required support to the insect to re-orient itself and possibly expose different organs as it tries to moves through the bits of papers.

    • It allows for scattering of light. This is very important for us to be able to see all sides of the insects

  • Use a strong LED light source for illuminating the specimen.

Doing more

Though the primary intention of this technique was to observe insects, there are many cool things you can try to observe using this technique, like :

  • You can culture things like fungi and moss inside the slide and monitor their growth.

  • You can try to observe various stages of an insect’s lifecycle (eggs –> larvae –> insect).

  • Observe how insects feed. How they interact with each other.

  • Germinating seeds can be observed. It would be fun to see cotelydons,roots and shoots !!

Finally

Hope you found my experiment intresting. If you decide to make this type of slide, please do share your findings with me and the Foldscope community.

Diwali decoration using Arduino and WS2812B LED strip

Since a long time I wanted to create something cool using LEDs and Arduino for Diwali. For those who don’t know, ‘Diwali’ is known as ‘festival of lights’ in India. During Diwali you will see houses and buildings decorated with earthen lamps (called ‘diyas’), lanterns and LED lights. Shown below is the cool programmable color changing lantern decoration which created using Arduino.

The lanterns display in daylight(left) and at night (right)

What I used

  • Arduino Uno (or clone)
  • Led strip (WS2812b 5v) x 1 meter
  • Small Diwali lantern x 4 [the translucent ones, which can be fitted with a small bulb]
  • Plastic string or wire to tie the lanterns one below another.

Construction

Construction is simple but is very important to select the right lanterns. In India, small golden or silver colored lanterns are easily available and are cheap. But these are not translucent (they are intended for daytime decoration). If you search in the shops, you will find translucent, similar sized lanterns (basically these are the smallest sized replicas of common lanterns). Given below is the photo of a correct type of lantern.

I used small sized translucent lanterns.

Closeup of the LED strip.

I tied all the four lanterns one below the other. There can be many ways to do this. I used thin plastic rope to tie-up the lanterns together. These lanterns have inner plastic ring with spokes at the top, which can be used to tie the string/rope. I ensured that the rope is not easily visible and mostly hidden when suspended. Also the height of my lantern chain when suspended had to be be roughly equal to one meter. In my case, I decided to only use 28 out of 30 LEDs on the strip. Once the lanterns were tied and a chain is created, next step was to fix the LED strip inside the lantern. I simply tied one end of the strip to the inner ring of the topmost lantern and suspend the remaining portion of strip into the ‘hollow’ of the lantern chain. Interfacing the LED strip was very simple. The strip has three wires which I connected as follows :

LED stripArduino
DataPD3
VCC5V
GNDGND

Important

Lot of articles on the Internet advocate using a separate power supply for the LED strip. I managed to get it work by directly connecting it to my board (I was using Arduino clone). Maybe because, I only had 1 meter long strip. For anything longer please consider adding a separate power supply.

Once I was done with my hardware, it was time to connect the strip to my Arduino board and write some code.

The Software

To work with the LED strip you need to FastLED library for Arduino. The library comes with cool examples and descent documentation. Initially I wrote a few programs for the LED strip using basic RGB color manipulation (tried to play with random values with RGB). I soon realized that this does not produce a nice visual effect. The best way to produce visual effects with distinguishable colors was to use FastLED HSV colors

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#include "FastLED.h"

FASTLED_USING_NAMESPACE

#if FASTLED_VERSION < 3001000
#error "Requires FastLED 3.1 or later; check github for latest code."
#endif

#define DATA_PIN 3
#define LED_TYPE WS2811
#define COLOR_ORDER GRB
#define NUM_LEDS 28
#define BRIGHTNESS 96
#define MIN_VAL 50
#define MAX_VAL 255

CRGB leds[NUM_LEDS];

void setup() {
delay(3000);
FastLED.addLeds<LED_TYPE, DATA_PIN, COLOR_ORDER>(leds, NUM_LEDS).setCorrection(TypicalLEDStrip);
FastLED.setBrightness(BRIGHTNESS);
}

void loop()
{
byte old_h = 0;
byte h = 0;
while (true) {

while (old_h == h)
{
h = random(0, 8);
}

for (byte i = 0; i < 4; i++) {
set_latern_color(i, h * 32);
FastLED.show();
delay(2000);
}

old_h = h;
}
}

void set_latern_color(byte lantern_no, byte hue)
{
for (byte i = 0; i < 7; i = i + 1)
{
leds[lantern_no * 7 + i].setHue(hue);
}
}

Finally !

Hope you enjoyed reading this article. If you creating something similar, please feel free to share with me !

Creating a temperature & humidity tweet bot using IFTTT and ESP8266

Tweets from our tweet bot

In my previous articles, I had written in detail about creating wifi enabled temperature and humidity logger using ESP8266 + NodeMCU and configuring a NoSQL(CouchDB) database to store data transmitted by it. In this article, we are going make our IoT device tweet temperature and humidity readings using IFTTT. Once we are able to connect our device to IFTTT, it opens a plethora of interesting things we can do with our device. Being able to tweet temperature and humidity is just one of them.

What is IFTTT?

Block diagram illustrating role of IFTTT

IFTTT is a service which allows us to define actions for specific events. An example could be, ‘When I receive an email , send me an SMS’. IFTTT is made of ‘Channels’ and ‘Recipies’. Each ‘Channel’, supports certain actions. If you ask me, ‘Channels’ can be classified into two main categories. First type of ‘Channel’ basically poll or listen to a feed/stream and provide triggers. The other type of ‘Channel’, allows you to do something like, sending SMS, writing email or updating a document on Google Drive.

You create a Recipe using two Channels, one typically to listen to something (example: incoming sensor data) and the other to do something (example: post sensor data to Twitter). All this without writing a single line of code. A lot of people are of the opinion that IFTTT is the next big thing that is happening to the Internet.

Getting started with IFTTT

First, you should create an account on IFTTT. I strongly recommend that you familiarize yourself with IFTTT by creating few simple Recipes. Once you do that, head straight to the Maker Channel , which has been specially created to support DIY electronics. Once you ‘connect’ to this Channel, you should be able to see your ‘key’ on the Channel’s main page. This is a sceret key that should be used by your IoT device to send data over HTTPS to IFTTT. Everytime you send data using the key, the ‘Maker Channel’ is triggered. The idea is simple, we have to create a Recipe involving the ‘Maker’ and ‘Twitter’ Channel (If data received by Maker, then post it to Twitter).

Sending data to the ‘Maker’ Channel is simple. You need to send a HTTP POST request to below URL :

https://maker.ifttt.com/trigger/{event}/with/key/your_secret_key

with data inside the body of the request in JSON format shown below:

1
2
		
{ "value1" : "sensor_reading1", "value2" : "sensor_reading2", "value3" : "" }

One important thing to remmember is to replace {event} with a friendly name that corresponds to sensor or process sending data. For example, if we had two of temperature sensors, we could create two events called ‘sensor1_data_rcvd’ and ‘sensor2_data_rcvd’. When you create a Recipe involving the ‘Maker’ Channel, you need to input the name of event as parameter.

Creating a IFTTT Recipe receive for tweeting Temperature & Humidity

Now that we are familiar with IFTTT and ‘Maker’ Channel, let us create a Recipe which will post temperature and humidity to Twitter. For this you need to ‘connect’ to ‘Maker’ and ‘Twitter’ Channels (you need to have a Twitter account). Next, follow below steps to create a new Recipe.

  • Step1 Choose Channel Trigger Select the ‘Maker’ Channel.

  • Step2 Choose a Trigger Simply click on ‘Receive a Web Request’ link.

  • Step3 Complete Trigger Fields Enter ‘tempupdt’ as the event name. Our device will publish under this event name

  • Step4 Choose an Action Channel Select ‘Twitter’.

  • Step5 Choose an Action Click on ‘Post a tweet’ link.

  • Step6 In the ‘Tweet Text’ textbox enter the text as shown below

    Adding tweet text in Step 6

  • Step7: Click on ‘Create Action’ and we are done.

Now we have successfully created a Recipe which will post temperature and humidity readings to Twitter.

Programing the IoT device (temperature and humidity logger)

The IoT device is a temperature and humidity logger (ESP8266 running NodeMCU and DHT11 sensor) which I had described in my previous article. So I would not be discussing details about it’s construction and setup required to program it. In this section we will discuss in detail about the code part. One important thing to note is, we need to make HTTPS request to send data to IFTTT. For this we need to get latest version of NodeMCU firmware. We need NodeMCU v1.5.1 for this. Older versions of the firmware could be directly downloaded from their site and burned into ESP8266. However starting v1.5.1 , pre-built binaries are not provided.


Source : https://github.com/nodemcu/nodemcu-firmware

This means either you have to build it from source (the last thing I want to do) or use their ‘custom-build’ service.

Luckily their custom-build service is super cool and allows us to create a custom build with only the components which we want. When creating a custom build, please ensure that HTTP, Crypto and DHT modules are checked,in addition to default selections.

Make sure HTTP, Crypto and DHT are selected in your custom build.

Once you download a custom build of NodeMCU v1.5.1, please follow instructions mentioned here to burn it to ESP8266. After that is done you will need to connect the DHT11 sensor and upload the following script to ESP8266

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
		
wifi_nwid = "wifi_network_id"
wifi_pwd ="wifi_password"
interval = 30000
---------------------------------
iscon = false
temp_old = 0.0
humi_old = 0.0

function checkConnection()
if wifi.sta.getip() ~= nil then
iscon = true
else
iscon = false
print('> Wifi error. Retrying')
wifi.sta.connect()
end
end

function get_payload()
status,temp,humi = dht.read11(4)

if( status == dht.OK ) then
if(temp_old == temp) and (math.abs(humi_old - humi) < 5) then
return nil
end
temp_old = temp
humi_old = humi
return string.format("{\"value1\": \"%d\",\"value2\":\"%d\"}",
math.floor(temp),math.floor(humi))
end

return nil
end

function post_values()
pl = get_payload()
if pl then
http.post('https://maker.ifttt.com/trigger/tempupd/with/key/{your_secret_key}',
'Content-Type: application/json\r\n',
pl,
function(code, data)
if (code < 0) then
print("HTTP ERROR")
else
print("SENT.")
end
end)
end
end

function run()
checkConnection()
if iscon == true then
post_values()
end
tmr.alarm(0, interval, 0, function()
run()
end )

end

print('> Booting..')
wifi.setmode(wifi.STATION)
wifi.sta.config(wifi_nwid, wifi_pwd)
wifi.sta.connect()

In the above code please remember to change values of wifi_nw_id and wifi_pwd variables before uploading. Also please substitute {your_secret_key} with actual secret key of your ‘Maker’ Channel as described in the earlier section

Things to consider when posting to Twitter

When posting to twitter we need to be aware of daily limits on tweets. Our IoT device an in practice can generate lot of tweets. When I first programmed the device, it would give a reading every 30 seconds. Then I thought of changing the code so that it posted the data from sensor every five minutes. However,a much better way was to make our device tweet only when there is 1 degree Celcius or more change in temperature or 10% or more fluctuation in humidity readings. This way we would be able to tweet more meaningful data. Also with this approach, I believe our tweets should stay under daily limit.

Doing more with IFTTT

Tweeting temperature and humidity is just one of the many cool things we can do with IFTTT. We can also create a Recipe that will write temperature and humidity readings to a spreadsheet on Google Drive. Shown below is output from one such Recipe.

Top: Row format expression used in IFTTT Recipe to write temperature & humidity values to spreadsheet.Bottom: Output of the IFTTT Recipe; spreadsheet containing data.

Another interesting use would be to control air conditioner based on temperature reading.

One important thing to note here is how easy it was to make our device post data to IFTTT’s , ‘Maker’ Channel. All we had to do was to write some code that made a HTTP POST request. Once we did that, our device is ready to talk to scores of different services without having to configure anything at our end. This is the real power of IFTTT.

A word on IFTTT and IoT

I want to end this article with my thoughts on IFTTT. I think it is a great idea, certainly one of the next big things. In todays world building a platform from scratch is not an option. IFTTT is awesome, because it serves as a glue between many established software services and platforms. There are many consumer grade IoT products/appliances (wifi enabled plugs, smart lights, surveillance cameras..) which come with out of the box integration with IFTTT. In future (and even now), I believe such compatibility will be a key selling point for such items. If you are thinking of buying a smart appliances, make sure that it has a IFTTT Channel , if not then you should certainly be reading this article.

IoT Project - Using CouchDB as a data store for IoT systems (Part 2)

CouchDB enabled an IoT platform

In the Part 1 article , I had written in detail about creating wifi enabled temperature and humidity logger using ESP8266 and NodeMCU. In this article (which is the Part 2), I will discuss about the storing and accessing data generated by the device which we created in Part 1. If you recollect, the hardware which we created only used two components ESP8266 and DHT11, and was incredibly cheap. It had its limitations, mainly lack of memory and lack of RTC (real time clock). With some more effort we could have tried interfacing memory and a RTC, but that would have made our design complicated and expensive. Instead what we can do is try and log infromation to a server in near realtime. Though this has its own limitations, but it is the most simple and least expensive solution we have.

Why CouchDB

Relational databases require a fixed schema. In a system like ours, it poses a challenge when we want to log additional data attributes or omit any. Second issue with relation databases is, you anyways have to write an application layer to access them. This can be very time consuming. Most systems that use relational databases , have a thin service layer (WCF, REST) which sits on top of it. Suitability of technology used in application layer for interacting with IoT device is also important.For example, it will be very hard to access a WCF service from a ESP8266 running NodeMCU. A RESTful API on the other hand is better suited.

Recently I have been reading quite a bit about CouchDB. I have written about CouchDB in the past. One of the most impressive features CouchDB has is native support for RESTful API. In fact, it is the only way to interact with the database. CouchDB acts like a data store and application server at the same time. In a single instance of CouchDB you can store your data as well as your client side code (html,js,css,images). Your javascript code can query your data using RESTful API exposed by CouchDB. All this is not achived by some hack, but it is a very well thought out and supported feature of CouchDB. Besides, CouchDB also has many features which are commonly found in NoSQL databases.

CouchDB is open source and new features are been added very actively to it. Installation of CouchDB for any operating system is super easy and has been detailed in my earlier article.

Configuring CouchDB for our IoT device

Before we begin, I assume have been successfully able to install it and have gone through my article on CouchDB. The first thing we need to do is to enable remote access to our CouchDB instance. This can be done by changing following lines from file /etc/couchdb/default.ini

1
2
3
[httpd]
port = 5984
bind_address = 127.0.0.1

to

1
2
3
[httpd]
port = 5984
bind_address = 0.0.0.0

Once we do that we have to create a new database for our IoT device. I will call it ‘temperature’ (not the best of the name !). Once that is done the most important thing is to create the design document.

Inside the design document (_design/app) we create a ‘document update handler’, called ‘inplace’. This handler appends a timestamp and unique id field (_id) to our document(data) before saving it to CouchDB. This is required, because we do not have a real time clock in our hardware or any means to generate unique ids reliably. Also note that this handler adds a ‘creator’ field which will allow us to know ‘who created the document’. Rest all fields are simply copied over from the incomeing document.

1
2
3
4
5
6
7
8
9
function(doc, req) {
if (!doc) {
var data = JSON.parse(req.body);
data['_id'] = req.uuid;
data.creator = req.userCtx.name;
data.ts = new Date().getTime()
}
return [data, req.body]
}

The above handler takes care of updating data. Now we create a view called ‘s1data’ which when queried returns all the data stored for ‘s1’, the unique id of our device. Given below are JSON samples which are sent to CouchDB by our device in different scenarios:

  • When we have temperature and humidity reading is:

    1
    2
    {"t":22,"h":80,did:'s1'} 

  • In case of checksum error :

    1
    2
    3
        	
    {"error" :"CHECKSUM","did":"s1"}

  • For timeout:

    1
    2
    3
        	
    {"error" :"TIMEOUT","did":"s1"}

Given below is the code for our view

1
2
3
4
5
6
7
8
9
10
11
12
function(t) {
if ('s1' == t.did) {
t.t && t.h && t.ts && 's1' == t.did ? emit(t.ts, {
ts: t.ts,
t: t.t,
h: t.h
}) : emit(t.ts, {
ts: t.ts,
e: t.error
})
}
}

Given below is the full design document for CouchDB instance which I used for my testing. You should be able to replicate it simply by copy pasting code following it via Futon. Please note that I removed the attachment portion from below code. As mentioned eariler it , use Futon to create it and upload files provided by me.

1
2
3
4
5
6
7
8
9
10
11
12
{
"_id": "_design/app",
"_rev": "76-fd558e157e14e369601c2849c4d296cd",
"updates": {
"in-place": "function(doc,req){if(!doc){var data=JSON.parse(req.body);data['_id']=req.uuid;data.creator=req.userCtx.name;data.ts=new Date().getTime()}return[data,req.body]}"
},
"views": {
"s1data": {
"map": "function(t){if('s1'==t.did){t.t&&t.h&&t.ts?emit(t.ts,{ts:t.ts,t:t.t,h:t.h}):emit(t.ts,{ts:t.ts,e:t.error})}}"
}
}
}

Screenshot showing design document of the 'temperature' database.

In a real world scenario, there could be hundreds of such devices feeding data to our CouchDB instance. Also note that the view returns a very light-weight JSON compared to what is actually stored inside CouchDB. We will create a web page to consume this JSON data and display a auto-refreshing chart.

Creating web page for viewing temperature and humidity

Web page displaying sensor data in realtime.

I have used jQuery and d3.js to create a interactive web page, that displays temperature and humidity readings in realtime or on a specific date. Temperature is shown using a bar, humidity using a circle and error using a square. Hovering over any element will show a small tooltip with details. Explaining the entire code is out of the scope of this article, but you can download the html page and required javascript files from here. Since there are only three files, you can easily manually upload them to CouchDB using Futon.

Tooltip showing temperature.

Tooltip showing humidity.

Tooltip showing error.

Finally

With this we conclude the second part of my IoT tutorial. Hope you have enjoyed reading this. Please feel free to comment and ask questions. Thanks for reading !

IoT Project Temperature And Humidity Logger Using ESP8266 And DHT11 (Part 1)

The wifi enabled temperature and humidity logger.

In this project I have used ESP8266 and DHT11 sensor to create a temperature and humidity logger, that transmits data over wifi. Currently, data is transmitted to a CouchDB instance, but with minor modifications to the code it should be possible to send data somewhere else. I have divided this tutorial into two parts. In this part, I will discuss basics of interfacing DHT11 with ESP8266 and transmitting data over wifi. In the next part (Part 2) I will discuss in detail, how to setup CouchDB to receive and store data from our device and create chart using d3.js to monitor temperature and humidity in realtime.

Hardware

This project uses the same layout which I have used in my previous IoT projects. DHT11 is an integrated device which works as a temperature and humidity sensor. You can find more about it here https://www.adafruit.com/product/386. The only difference is, instead of an OLED display, we are using the DHT11 sensor. DHT11 uses propritery 1-wire bus, which also happens to require a pull-up register(R2), which was already soldered on my board (to support I2C communication with OLED display).

Schematics for connecting DHT11 to ESP8266.

From what I read on the internet the DHT11 is not very accurate, it’s counterpart DHT22 has much better accuracy. But the real difference is cost, I could get DHT11 for around USD 1.00 in India from an online retailer. The pins on device are thin, and don’t fit snuggily when plugged into female jumper cables. If you are using jumper cables (like me) twist the pins so that jumper cable fits properly.

I have also seen DHT11 been sold in form of ‘sensor module’, which is nothing but a DHT11 attached to a tiny board with header. The header comes in handy when you want to use jumper cables to connect it to a circuit board (arduino/rasberry pi). Most DHT11 modules are unreasonably expensive compared to the standalone device.

Software

When I researched about interfacing DHT11 , I came to know that it requires precision timing in order to read the sensor’s output. ESP8266 running NodeMCU seemed like a poor choice for the purpose. But I still decided to give it a try mainly because NodeMCU has built in library for reading DHT11 and DHT22. I would term my experiment mostly successful, because with the below code, I am able to read the sensor twice within a minute and log the data over HTTP. However, there are those rare occassions when I get a timeout or checksum error from the sensor.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
couch_ip="couchdb_ip"
couch_port="port"
wifi_nwid = "wifi_ssid"
wifi_pwd ="pwd"
interval = 30000
---------------------------------
iscon = false
did = "s1"

function checkConnection()
if wifi.sta.getip() ~= nil then
iscon = true
else
iscon = false
print('> Wifi error. Retrying')
wifi.sta.connect()
end
end

function get_payload()
local rval = "{}";
status,temp,humi,temp_decimial,humi_decimial = dht.read11(4)
if( status == dht.OK ) then

rval = string.format(
"{\"t\":%d.%03d,\"h\":%d.%03d,\"did\":\""..did.."\"}",
math.floor(temp),
temp\_decimial,
math.floor(humi),
humi_decimial)

elseif( status == dht.ERROR_CHECKSUM ) then
rval = "{\"error\" :\"CHECKSUM\",\"did\":\""..did.."\"}"
elseif( status == dht.ERROR\_TIMEOUT ) then
rval = "{\"error\":\"TIMEOUT\",\"did\":\""..did.."\"}"
end
return rval
end

function post_values()
pl = get_payload()
req = "POST /temperature/_design/app/_update/in-place HTTP/1.0 \r\n"
.."Host:"..couch_ip..":"..couch\_port.."\r\n"
.."Content-Type: application/json\r\n"
..string.format("Content-length:%d\r\n\r\n",string.len(pl))
..pl

print(pl)
local conn=net.createConnection(net.TCP, 0)
conn:on("receive", function(conn, payload) print(payload) end )
conn:on("connection", function(c)
conn:send(req)
conn:close()

end)
conn:connect(couch_port,couch_ip)
end

function run()
checkConnection()
if iscon == true then
post_values()
end
tmr.alarm(0, interval, 0, function()
run()
end )

end

print('> Booting..')
wifi.setmode(wifi.STATION)
wifi.sta.config(wifi_nwid, wifi_pwd)
wifi.sta.connect()
run()

Here are some important points about the lua code which I have shared above. To send data to CouchDB, our code simply sends a http POST request to a CouchDB url with ‘Content-Type: application/json’. It is important to get ‘Content-Length’ correct, else CouchDB will respond with ‘Bad request’. The payload when I have temperature and humidity reading is:

1
{"t":22,"h":80,did:'s01'}	 

In case of checksum error I send:

1
2
	
{"error" :"CHECKSUM","did":"s01"}

For timeout:

1
2
	
{"error" :"TIMEOUT","did":"s01"}

The “did” field in the JSON corresponds to the device id of the each sensor (which can be anything). The code inside the function ‘checkConnection’ keeps on checking if it is connected to the internet. If the device gets disconnected, it will attempt to connect again.

Why CouchDB

For this project I have used CouchDB as the datastore. I have talked about CouchDB in one of my articles. CouchDB is a NoSQL datastore, which works over HTTP. This means that CRUD operations on data can be done via sending JSON inside standard HTTP request. In my next article I will discuss in detail, why I chose CouchDB and how to configure it to work as backend for IoT projects. I will also tell you how I created the interactive bar-chart shown below to monitor temperature and humidity readings.

Interactive bar-chart for displaying temperature and humidity readings.

IoT Project - Wifi Enabled Portable Display

The wifi enabled portable display.

After experimenting with and reading a lot about IoT technologies, I decided to create something. So here my first IoT project: The wifi enabled portable display. This tiny little display connects to a internet over Wifi and waits for messages. It is possible to send text messages or images to the display via Internet , from your PC , mobile phone or tablet. Checkout the below video to see it in action :

Project Goals

Before we get into details, I want to go over list of goals which I had set for this project:

  • Create a low cost wifi enabled display using ESP8266 and 128x64 bit OLED display. It had to be portable and battery powered.
  • The display should be able to show text messages and monochrome graphics. It should automatically scroll long text messages (which cannot fit on screen).
  • Create a GUI front-end which can be used with desktop or mobile browser to send images and text to the display.
  • The display should handle interruptions in Wifi/Internet connection. It should automatically try to reconnect in case it gets disconnected.
  • Use mostly open source tools and free stuff for software development.

High Level Design

High level design showing components and how they are interfaced

The above diagram illustrates the high level design of the project. The table given below summarizes important software and hardware design decisions

Hardware DesignSoftware Design
The board consists of a single ESP8266 module which handles CPU tasks and networking. There is no microcontroller. For me the cost of all components was around USD 15 (here in India).The ESP8266 is using the NodeMCU firmware . All the code to control the display is written in Lua.
I have used a 128x64 bit OLED display, which allows for displaying of descent number of characters and monochrome bitmap graphics.To show things on OLED display, I have used the U8gLib which is available out of the box with NodeMCU firmware.
The display is interfaced with ESP8266 using I2C protocol. This is interesting, because we only require 2 PINs.The MQTT protocol is used to send data to the display. The ESP8266 subscribes to a MQTT broker and listens for messages.
The board works on 3.3v and is mainly designed to be powered via battery. When powered by battery (6v-20v) , a 78L33 based regulator provides 3.3v . There is also the provision to bypass the regulator and provide 3.3v regulated input to board.There is a HTML based front end , which uses Websocket to push messages over MQTT to the display. This front end has been written using AngularJs and Paho library .This makes it possible to send messages to the display using any PC or mobile device.
The board has PINs for serial communication with ESP8266. These can be used to modify software inside ESP8266 and debugging.MQTT hosting provider was CloudMQTT. They have a free plan which awesome for DIY enthusiasts, really grateful to them. to try out their desigs.

The Hardware

The hardware used for this project is very similar to what I have described in my previous article . The only difference being , there is an additonal voltage regulator circuit. I have used 78L33 regulator IC which can convert 6-20V to 3.3v(regulated) with max current of 100mA. I use four AA batteries to power the board. In my tests I have found that the circuit takes around 70-80mA, so choice if regulator is well suited for it. On the circuit board the output of voltage regulator is connected via jumper to rest of the components. The ground PIN is exposed on the board, which allows powering the board from an external 3.3v regulated source. The Tx and Rx pins of ESP8266 are also exposed, which allows for upgrading the software very easily.

Schematic diagram of the board and PINout.

Actual circuit showing PINs and jumper setting.

The Software

This is where things get very exciting. As I have already stated, the board uses an ESP8266 with NodeMCU firmware as a CPU. This means all the code for board is written in Lua. I won’t go into details of the code in this article, you can download it from here. The most challenging part about writing the software was getting the timings correct. There are three important things which the software does, which are:

  • Scrolling of text in case of long messages.
  • Displaying of ‘New Message’ splash screen on receiving new message for couple of seconds.
  • Executing the ‘picture loop’ of U8glib (library for interfacing the display) at regular interval.
  • Checking if the board is connected to internet at regular intervals and trying to connect if disconnected.

The most challenging part was use of timers to make above actions possible (forget multithreading or fancy programing constructs !). One problem which I have not been able to overcome is , being able to read complete binary data for a 128x64 bit image. As a result, I changed the code to only accept and display 96x64 bit images. This problem could be due to speed at which my program is able to read data from socket or something to do with internal buffers. Honestly speaking, I have given up on debugging and finding a fix for this issue. With NodeMCU it is simply very time consuming and painful.

Talking to the display from Internet

Okay, so we have a display which connects to internet and waits for messages. But what good is it, if there is nobody to send a message. To address this, I have created a HTML front-end which uses Paho (MQTT over Websocket) javascript library and AngularJs . This front-end, (as shown in the video) can be used to send text and image messages to the display. I have only tested it with Chrome and IE on Windows 8, but it should work with any browser which supports File API and WebSocket.

Downloads and instructions.

Here is list of files downloads:

File NameComments
messenger.htmlThe GUI front-end to send messages to the display. It uses following libraries from CDN:
mqqtws31.jsThe Paho MQTT library. Download it from their website and place in same folder as messenger.html
smart_lcd.luaContains LUA code for the hardware. Needs to be written to ESP8266 as main.lua. Please refer to my old article for details. Also do not forget to change wifi networkId/password and MQTT server details in this file (check out the header.)
smart_lcd_init.luaNeeds to be written to ESP8266 as init.lua. This contains 'dofile('main.lua')' statement, which executes our code when the board powers up. Here again refer to my old article on writing this file along with smart_lcd.lua to ESP8266
Sample 96x64 bit monochrome bitmap images

Right Click over the images and select 'Save ..' in your browser to save these images.

Finally

Hope you liked reading about my first IoT project. I have learned quite a bit about IoT while making this project. This project is an excellent example of how hardware and software work together to make things work. There are many interesting things you can do with this little device, which I will be covering in future articles. As always, suggestions and feedback are always welcomed !

Thanks to:

Interfacing 128x64 pixel OLED display with ESP8266 (with NodeMCU)

Someone rightly said ,

“A picture is worth a thousand words” !

In the electronics world we can say this translates to,

“ A display is worth a thousand LEDs”.

I have always wanted to add some sort of display to my microcontroller projects. The main issues I faced with this were:

  • The most common LCD is 16x2 character display, which is large in size, heavy and needs lots of pins for interfacing. Also it does not support graphics (in short , ‘not so cool’).
  • Your microcontroller needs to have a library to support the display and necessary I/O pins. I know Arduino has both, but something like ATTiny85 will struggle. Even with an ATMega you quickly run out of PINs after interfacing most displays.
  • I don’t like the idea of adding a breakout board or a shield just to support a display.
  • Cost and availability are always a problem (If you live in US/Europe, you are lucky :-) )

Meet the supercool OLED display !

128x64 pixel monochrome OLED display.

Recently, I was doing a project using standalone ESP8266 and wanted to add a display to it. I have discussed about ESP8266 before on my website and I’ll say this again: “At times, I find it more interesting than Arduino”. I have the most common (and cheap) type of ESP8266 module which exposes only 2 GPIO pins. Initially, I was thinking of adding a microcontroller to my project so that I can interface a display. After some research on the internet , I came across this 128x64 pixel monochrome OLED display. It is tiny and supercool, because:

  • Cost is around USD 9 (in India), which is OK.
  • OLED, so no backlight required, self illuminating.
  • Can be controlled with just 2 pins (using I2C bus protocol).
  • Lightweight, tiny and has low power consumption.
  • Operates on 3.3v, so can be directly interfaced with ESP8266.
  • Supports graphics ! NodeMCU has built in support for such displays.

So, I dropped the idea to use a microcontroller and promptly ordered this. I2C is a nice little bus protocol which I was not aware of until now. The best part about it is , it requires only two PINS (GPIOs will do) and multiple devices can be attached to a single I2C bus.

The hardware part

I have covered the process of flashing ESP8266 with NodeMCU firmware in my previous article , so I am skipping that part. I am assuming that you already have a ESP8266 with NodeMCU firmware. Given below is the circuit diagram. One thing to note is, I have added pull-up resistors (R1 and R2) to be on safe side. My research suggests that OLED display has internal pull-ups , but I was not willing to experiment. The recommended value of pull-up resistors is 4.7KOhm, but I had couple of 6.8KOhm ones lying around, so decided to use those. Please check the PINs on your OLED display. Many displays will have more PINs (compared to mine) to support SPI. Mine had only 4 pins, so connections were straightforward.

Connecting components.

Summary of connections:

ESP8266 PINConnected to:
VCCTo 3.3v PIN of UART and OLED display.
GNDGND of UART bridge and of OLED display.
TXRX
RXTX
CH_PD3.3v
GPIO 0SCL Pin of the OLED display. Also pulled-up to VCC using 4.7KOhm resistor
GPIO 1SDA Pin of the OLED display. Also pulled-up to VCC using 4.7KOhm resistor

Important

Most USB to serial adapters support 5v. If you apply 5v to VCC pin of the ESP8266 module it will get damaged permanently. I can tell this from my experience :-) . So please ensure that your serial converter is 3.3v compatible. You can get a serial converter which supports both, 5v and 3.3v pretty cheap.

The software part

Given below is LUA code to display some text on the display. The code is fairly straightforward. It first initializes I2C on GPIO0 and GPIO2, then uses U8glib library bindings available in NodeMCU to draw some text. I will cover U8glib in more detail in my next article. To send the code to ESP8266, use luatool as explained in my previous article .

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
        
a = true
-- setup I2c and connect display
function init_i2c_display()
i2c.setup(0, 4, 3, i2c.SLOW)
disp = u8g.ssd1306_128x64_i2c(0x3c)
end
-- the draw() routine
function draw()
disp:setFont(u8g.font_6x10)
disp:drawStr( 30, 10, "Hello IoT!")
disp:drawLine(0, 25, 128, 25);
disp:setFont(u8g.font_chikita)
disp:drawStr( 5, 40, "More cool stuff coming soon")
disp:drawStr( 30, 50, "on ashishware.com")
end

function display()
disp:firstPage()
repeat
draw()
until disp:nextPage() == false
display();
end
init_i2c_display()
display()

Conclusion

It is fascinating to see how technology is evolving day by day. In just under USD 10-15 , we have a wifi enabled device with cool little display. I’ll write more on this pretty soon. Until then, feel free to make and experiment.

Wifi detector using ESP8266 and NodeMCU

In this article I will show you, how we can create a very basic wifi network detector using a ESP8266 module, a LED and some code. If you are new to ESP8266 I strongly recommend you read my introductory article on it. We will use a ordinary LED connected to one of the GPIO pins of ESP8266 to indicate status of network. If network is available(and is in range) the LED will be on. If network is unavailable or out of range the LED will start blinking. The best part of this project is, it does not need a separate micro-controller and can be battery powered.

The Hardware

You will need following items:

  • USB to UART bridge (should support 3.3v).
  • ESP8266 with NodeMCU firmware .
  • A normal LED and 1KOhm resistor.
  • A breadboard.
  • Jumper wires .

The USB to UART bridge is required to transfer our program to the ESP8266. We will write a program in Lua to search for a wifi network at regular intervals and control the LED. We will use a python based tool called Lua_tool to transfer our programs to ESP8266.

Important

Most USB to serial adapters support 5v. If you apply 5v to VCC pin of the module it will get damaged permanently. I can tell you from my experience :-) . So please ensure that your serial converter is 3.3v compatible. You can get a serial converter which supports both, 5v and 3.3v pretty cheap.

The Software

Given below is a very simple Lua script (wifi_detect.lua), which is used to detect a wifi network and control the blinking of the LED.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
pin = 4
ssid = 'your_wif_nw_ssid'
is_pwm = false
function check(t)
for k,v in pairs(t) do
if k == ssid then
if is_pwm then pwm.stop(4) end
is_pwm = false
gpio.mode(pin,gpio.OUTPUT)
gpio.write(pin,gpio.HIGH)
return end
end
if is_pwm == false then
pwm.setup(4, 1, 512)
pwm.start(4)
is_pwm = true
end
end
tmr.alarm(0, 5000, 1, function() wifi.sta.getap(check) end )

The pin variable is assigned an integer which corresponds to GPIO2 pin of the ESP8266 module. Next, the ssid variable should be set to the SSID (name) of your wifi network. The check function iterates through list of ssids of all available wifi networks and searches for the specified network (your_wifi_nw_ssid in this case). If it finds one, it sets the GPIO2 pin to high, which makes the LED glow steady. If no matching ssid was found, it starts pwm (pulse-width modulation) on GPIO2 which makes the LED blink.

The first parameter in the call to pwm.setup is the index for GPIO2 team. The next parameter is frequency, in our script it is 1 Hz. The last parameter is duty cycle, whose max values is 1023. A value of 512 means 50% duty cycle. This means that for each second the pin will be high 50% time and low for remaining period. This makes the LED blink. The last line, starts a timer which runs our script repeatedly after every 5 seconds.

Writing the Lua script to ESP8266 module

As described earlier, we will use a python based program , called lua_tool to write our Lua script to the wifi module. To do this, first connect the module to a USB to UART bridge as shown below. The value of current limiting resistor should be greater than 470ohm in my opinion, I have used 1KOhm. Also, I am assuming that you have already burned the NodeMCU firmware to your wifi module. Please read my earlier article on ESP8266 for more details.

Connecting components.

So connect USB to UART bridge to USB port of your PC and serial port pins of the bridge to the ESP8266 as shown above. Now you will need to download Lua_tool and Python 2.7 (don’t use the latest version). Once you have installed Python, you can create a upload.bat file with following contents:

1
C:\Python27\python luatool.py --port COM4 --src wifi_detect.lua --dest main.lua --baud 9600

Make sure to specify the correct port, on my machine the bridge always creates COM4 serial port. Run the batch file. Assuming that it ran successfully, you can now execute your script by connecting to COM4 using Putty and simpy typing:

1
dofile('main.lua')    

If you want your program to run automatically on every boot, create a new file called detector_init.lua with following content:

1
2
3
print('init.lua ver 1.2')
wifi.setmode(wifi.STATION)
dofile("main.lua")

The modify the contents of batch file to:

1
2
C:\Python27\python luatool.py --port COM4 --src wifi_detect.lua --dest main.lua --baud 9600
C:\Python27\python luatool.py --port COM4 --src detector_init.lua --dest init.lua --baud 9600

The NodeMCU firmware executes the init.lua after ESP8266 restarts, adding the dofile(‘main.lua’) command executes main.lua file which contains our wifi detector script. Test the program by switching on/off your wifi router and by moving your ESP8266 far from your router. The LED should blink if your wifi network is found.

Final thoughts

This project was simple and fun. It required no special parts and allowed us to program and learn about the ESP8266 module. We actually ended up creating something useful with very effort. Do try it out and share your experience. Thanks for reading !

Quick introduction to the ESP8266 wifi module

IoT (Internet of things) is a hot topic these days. Simply put, IoT refers to a network of small interconnected devices that can communicate with each other.In last few years, we have seen steady flow of inexpensive and powerful programable embedded devices/boards. Arduino is an excellent example. These days every other gadget or device wants to have the ability to connect to internet wirelessly. In this article I will introduce you to the ESP8266 module which you can use in your next IoT inspired project.

What is a ESP8266 module ?

ESP8266 is an IC created by Espressif Systems, which offers wifi networking cabilities. A module, simply integrates this IC with some other components and becomes an easy to use component.In this article I have described the most common form of ESP8266 which comes as a 8 pin module.

Some of its most important features are:

  • Low cost (USD 2 -5 per piece !)
  • Works in normal as well as access point mode.
  • Low voltage operation (3.3v).
  • Serial interface to program/communicate with the module.
  • Has everything required for wifi networking (supply only power).
  • 64KBytes of instruction RAM, 96KBytes of data RAM and 64KBytes boot ROM.
  • Two GPIO pins (support PWM).
  • Ability to be programmed/flashed via serial interface.
  • Can be easily interfaced with a microcontroller or used standalone.
  • Choice of different firmwares.

Though the features of the module look impressive, those of the chip itself are even more amazing. It has lot more GPIO pins and features which are not fully exposed by this particular module.

Programming the ESP8266 module

This is where the module really shines. There are two popular firmwares it can support, Espressif firmware and NodeMCU . the Espressif firmware is AT commands based. Some modules come with this firmware pre-loaded. The NodeMCU firmware basically adds a Lua interpreter to the module. It then becomes possible to use NodeMCU’s rich API to send Lua scripts to the module over serial port. I find it simply amazing to be able to program such a small device using a language like Lua. This also gives me an opportunity to learn Lua. There is also Micro Python firmware for the module which I haven’t expored much.

Toolchains are available for loading different types of firmwares to the module. Loading firmware is done over serial port by connecting GPIO0 pin of the module to GND. In the next section I will be discussing procedure to load the NodeMCU firmware into ESP8266 module and using it.

Burning the NodeMCU firmware

You will need the following things:

  • USB to UART Serial converter (should support 3.3v)
  • A breadboard
  • Jumper cables
  • The latest version of NodeMCU firmware
  • NodeMCU flasher tool for windows (since I am using windows)
  • Putty for windows (or any other program that allows communicating with COM port)

To burn firmware into ESP9266 module, you need to connect it to your serial converter as shown below.

ESP8266 module pin layout

Important

Most USB to serial adapters support 5v. If you apply 5v to VCC pin of the module it will get damaged permanently. I can tell you from my experience :-) . So please ensure that your serial converter is 3.3v compatible. You can get a serial converter which supports both, 5v and 3.3v pretty cheap.

ESP8266 PINSerial converter PIN
VCCTo 3.3v PIN
GNDGND
TXRX
RXTX
CH_PD3.3v
GPIO 0GND

Next connect your serial converter to your PC’s USB port. In ‘Device Manager’ you should be able to see a new COM port as shown below. On my PC, my converter creates a COM4 port.

Virtual COM4 port created by USB to serial converter

Now unzip the ESP8266 flasher tool and run ‘ESP8266Flasher.exe’. Once the program starts click on the ‘Config’ tab and select the downloaded Node MCU firmware file.

Select the firmware file

Click on the ‘Operation’ tab , select the correct COM port and click on ‘Flash’ button. If everything was setup right, you should see a progressbar on the screen showing status of firmware update. The whole process can take around 2-3 minutes.

Clicking the 'Flash' button will burn the firmware to the module.

If nothing happens after clikcing ‘Flash’ , check the connections and re-confirm if selected COM port is correct. You can also change the baud rate and retry.
Once the firmware update is over, you can disconnect the ESP8266 module. Now disconnect GPIO0 from GND, and connect the module back to the serial converter as before. Start Putty.exe and select a ‘serial’ connection as shown below. Then click on ‘Open’. You should be able to see a terminal window with a ‘<’ prompt. Try following command:

1
2
majorVer,minorVer,devVer  = node.info();
print("Firmware: "..majorVer.."."..minorVer.."."..devVer)

Putty console showing Lua commands and output.

You should see the version number of the Node MCU firmware. This indicates that you have successfuly installed NodeMCU firmware on your ESP8266 module.

What next ?

What I really like the most about this module is it allows you to program in a higher-level language without much hassle. Burning firmware and interfacing is also super easy. In some ways I find it more interesting than Arduino (especially if I can get a module with more GPIO pins). Once you have the firmware installed, you should take out some time to visit the API documentation page and play with your module. The interactive Lua enviornment is a joy to work with, in my opinion. You can experiment and learn a lot.

You will quickly realize that when you disconnect the module, your programs are lost. In my next article, I will post information on saving Lua scripts directly to the module itself. Then it is possible to auto-run your Lua script everytime the module is switched on. Keep reading my website for more on this awesome device.

Creating a web application with CouchDB and AngularJS

It has been a long time I posted something on my website. I was busy learning about NoSQL , mainly CouchDB. So, in this article I show you how to create a basic portfolio management web application using CouchDB and AngularJS. Our application will allow the user to add entries to a stock portfolio. Each entry will capture name of the stock, quantity, price, transaction type (bought/sold) and the date when the transaction was carried out. Also a section on the page will show list of entries in the portfolio and allow deleting and selecting a particular entry (for editing).

What is so special about CouchDB ?

Difference between ordinary NoSQL and CouchDB

As you may have already read, many NoSQL databases store data in form of documents, so does CouchDB. What is different with CouchDB is it acts like a webserver for your application. Yes, a single server that acts like a database and application server. Using a framework like AngularJS and flexibility offered by schema-less design of CouchDB, a broad category of web applications can be created painlessly and fast. In this article, I cover the process of creating the Portfolio application with CouchDB.

What do we need.

We need the following things:

  • CouchDB installed on your local machine. Please visit their website to download binaries for your OS. I have used Windows in this tutorial.
  • NodeJS installed on your system.
  • AngularJS and Bootstrap (which we will pickup from CDN).
  • Some knowledge and experience with AngularJS/javascript and JSON (not mandatory).

Creating our database

Luckily, CouchDB comes with online administrative tool by default, know as Futon. You can access Futon by starting CouchDB and visiting following URL http://localhost:5984/_utils/. Using Futon, let us create our first CouchDB database, called ‘portfolioapp’. This should be very easy. Since this is a NoSQL database, we don’t have to worry about the schema. We can concentrate directly on creating our front-end/GUI.

Futon showing the newly created database.

Using CouchApp to scaffold our application

As you might have realized, we didn’t do much. All we did was created a empty database. In this section, we will use AngularJS and create a front-end (GUI). Here, I would like to introduce a tool called CouchApp. This is a very handy tool which can help you create folder structure for your application and add your application files (HTML, images, scripts) to CouchDB very easy. For this tutorial we are using the NodeJS based version of CouchApp. So, to install CoucApp do this:

1
2
3
4
C:\>cd  apps
C:\apps>npm install couchapp
C:\apps>couchapp boiler portfolioApp
C:\apps>cd portfolioApp

The first command simply navigates to a directory where we want to create the new application. Paths may vary on your system, so please remmeber that. Second command installs CouchApp to this directory.The third line creates a new directory ‘portfolioApp’ with boilerplate code our application. For some reasons, third line gives me an error on my machine (Windows), but the directory is created successfully. At this stage if you open C:\apps\portfolioApp\attachments directory, this is what it should look like:

Files added by CouchApp during scaffolding.

There is nothing particulary important about the directory structure. We can change it to our liking. Our application will consist of only two files, a HTML page and javascript file containing angular code. So we delete everything from this folder first. Download the zip file containing the code. copy the html file from the downloaded archive and paste in C:\apps\portfolioApp\attachments folder. Copy the scripts\portfolio.js file from the archive and paste in C:\apps\portfolioApp\attachments\scripts (you may have to create the scripts folder if you deleted it.)

One important thing which we need to do is create a CouchDB view which will show list of stocks in our portfolio. This is a one time activity, which we can be accomplish via Futon. We can also have CouchApp do this for us every time we deploy our application to CouchDB. Simply modify the C:\apps\portfolioApp\app.js file which was generated by CouchApp and change the line

1
ddoc.views = {};

to

1
2
3
4
5
6
7
ddoc.views = {
"show_all": {
"map": "function(doc) { emit(null, { '_rev':doc._rev,'_id':doc._id , 'stockName':doc.stockName," +
"'price': doc.price,'quantity':doc.quantity,'action': doc.action,'date': doc.date});}"
}
};

Interesting thing to note is we have not definied a schema anywhere in our database, yet we are creating a view which assumes and uses some properties from our documents. This is fine because our application makes sure that every document it creates will have these fields. Hence we have not done any checking or error handling in the view (which we should ideally do). Thats it, we are done. We have created a GUI to perform CRUD operations for our application.

Understanding the code

The front-end application (GUI) to consists of a HTML page and a javascript file. The total code is less than 200 lines (including HTML and javascript). AngularJS with its templating and MVVM(two-way binding) capabilities allows us to achieve client side functionality with very little code. In our javascript, we create a service, called ‘portfolioRepository’ to handle iteraction with CouchDB. This service contains methods to perform CRUD operations using the Rest API provided by CouchDB. All it does is make ajax calls to couchDB. One important this to note is we had created a CouchDB view to display list of portfolio items. For displaying the list on GUI we simply a send HTTP GET request to query the view and get results. Given below is code for the service:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
angular.module('portFolio')
.service("portfolioRepository", ['$http', function ($http) {
this.createItem = function (data) {
var req = {
method: 'PUT',
url: '/portfolioapp/' + data._id,
data: data,
};
return $http(req);
}
this.getAllItems = function () {
var req = {
method: 'GET',
url: '/portfolioapp/_design/app/_view/show_all'
};
return $http(req);
}
this.deleteItem = function (data) {
var req = {
method: 'DELETE',
url: '/portfolioapp/' + data._id + '?rev=' + data._rev
};
return $http(req);
}
}]);

Next, we created a controller called ‘portfolioController’ which is bound to our HTML template for display and entry of data. Explaining it in detail is beyhond the scope of this article, but if you have worked with any MVVM framework , you will get an idea. Give below is full code for our controller :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
  angular.module('portFolio')
.controller('portfolioController', ["portfolioRepository","$filter", function (portfolioRepository,$filter) {
var ctr = this;
this.serverError = "";
function refreshData() {
portfolioRepository.getAllItems().success(function (d) {
ctr.portfolioItems = d.rows.map(function (o) { return o.value; })
});
ctr.currentItem = {};
}
refreshData();
this.saveItem = function () {
if (!this.currentItem.stockName) return;
if (!this.currentItem._id) {
this.currentItem._id = "client1" + ((new Date()).getTime()).toString();
portfolioRepository.createItem(this.currentItem)
.success(function () { refreshData(); })
.error(function () { ctr.serverError = "Unable to save." });
this.currentItem = {};
}
else {
portfolioRepository.createItem(this.currentItem)
.success(function () { refreshData(); })
.error(function () { ctr.serverError = "Unable to save." });
}
}
this.deleteItem = function (index) {
portfolioRepository.deleteItem(this.portfolioItems[index])
.success(function () { refreshData(); })
.error(function () { ctr.serverError = "Unable to delete." });
}
this.createNew = function () {
this.currentItem = {};
}
this.setSelectedItem = function (i) {
this.currentItem = angular.copy(this.portfolioItems[i]);
this.currentItem.date = $filter('date')(this.currentItem.date,"yyyy-MM-dd");
}
}]);

The portfolioApp/attachments/index.html is our view which binds to the controller and allows user to view/add/edit/delete items. You can have a look at the file, it contains angular expressions for binding and event handlers on user actions.

Uploading the application to CouchDB and running it

To upload the app to CouchDB using CouchApp simply execute:

1
C:\apps\portfolioApp>push app.js http://localhost:5984/portfolioapp

If everything went successful this is what you should be able to see using Futon. Our files are uploaded as special type of document with id ‘_design/app’ . Inside this documents all individual files are stored as fields.

You can now run the application by accessing http://localhost:5984/portfolioapp/\_design/app/index.html.

Try adding a couple of records like I did. You should be able to see those in our UI as well as inside CouchDB with Futon.

UI showing the two newly added records.

New records in the database.

Final thoughts

I hope this tutorial helps to illustrate how CouchDB can act as database and application server at the same time. This feature can be very useful for certain types of applications (like reporting, data entry, web based adminstrative tools, data driven websites, etc). Tools like Node.js and AngularJS have advocated standardization and consolidation of programing language and tools for creating web applications. CouchDB tries to take this a little further by trying to remove distinction between an application server and database server by binding them with popular and well adopted technologies like javascript, JSON and REST.