Objective
The objective of this tutorial is to create two applications: a Node.js application that functions as a web client, and a Node-RED application that functions as the server. You will create an input form in the client that lets a user send a registration request to a server. The server will save the registration in a database and send a spoken response back to the client using IBM Watson’s Text to Speech service.
Requirements
You must have access to a Bluemix account.
1. Create a Client Application with SDK for Node.js
First, I will create a Node.js web application, which will function as our client and we will create web pages to ask for user input. Technically, this client application is also running on a server, so it is perhaps confusing to call it a client, but in our scenario and architecture, this client application functions as the user interface that initiates the request to a processing server.
- Go to http://bluemix.net and click to ‘LOG IN’ button to log into your Bluemix account,
- If this is the first time you log in, Bluemix will ask to create an organization and a space. You can create multiple spaces, for instance a different space for each
- Click the ‘CATALOG‘ menu on the top right of the page,
- To create a Node.js based application with the ‘SDK for Node.js’, find the ‘SDK for Node.js’ in the ‘Runtimes’ section and click the icon,
- You are taken to the ‘CREATE’ page of the runtime application. In the configuration controls select a ‘Space’, enter a ‘Name’ e.g. ‘neca-nodejs-client1’ (note that the name must be ‘globally unique’, which means that no one else on Bluemix can have the same name, so often I include my username or initials), the ‘Host’ defaults to the ‘Name’ value but you can overwrite it, and for ‘Domain’ leave the value to ‘mybluemix.net’. The ‘Host’ value will become the subdomain value, so that you will be able to access your application via the ‘Host’.mybluemix.net URL. Click the ‘CREATE’ button.
- The ‘SDK for Node.js’ application is being created, a process that is called ‘staging’.
- When the application is finished staging, click on the ‘Overview’ link in the left menu, to go to the ‘application overview’ page.
- In the application overview, you can see the configuration details of the Node-RED starter application. Our application is a Node.js runtime with an ‘SDK for Node.js’, it is 1 instance of a virtual server with 256 MB of memory assigned, and 5.500 GB of memory available.
- In the ‘APP HEALTH’ box make sure that the application finishes staging successfully and says ‘YOUR APP IS RUNNING’
- Click the URL to view your application listed next to ‘Routes:’ under the application name. The URL should be clickable.
- You now have a running client application in Node.js
2. Create a Server Application with Node-RED Starter
Next, I will create a Node-RED Starter application, which will be our server application. The client application will send requests to the server, the server will do work and send the response back to the client application. Node-RED is a so-called visual workflow editor, which lets you create an application workflow by dragging and dropping visual nodes onto an editor.
- To create a Node.js based application with the ‘Node-RED Starter’, find the ‘Node-RED Starter’ in the ‘Boilerplates’ section and click the icon,
- You are taken to the ‘CREATE’ page of the starter application. In the configuration controls select a ‘Space’, enter a ‘Name’ e.g. ‘neca-nodered1’, the ‘Host’ defaults to the ‘Name’ value but you can overwrite it, and for ‘Domain’ leave the value to ‘mybluemix.net’. The ‘Host’ value will become the subdomain value, so that you will be able to access your application via the ‘Host’.mybluemix.net URL. Click the ‘CREATE’ button.
- The ‘Node-RED Starter’ is being created, a process that is called ‘staging’.
- Click on the ‘Overview’ link in the left menu, to go to the ‘application overview’ page.
- In the application overview, you can see the configuration details of the Node-RED starter application. Our application is a Node.js runtime with an ‘SDK for Node.js’, it is 1 instance of a virtual server with 512 MB of memory assigned, and 5.575 GB of memory available. You also will see that the application is configured out-of-the-box with a Cloudant NoSQL database.
- In the ‘APP HEALTH’ box make sure that the application finishes staging successfully and says ‘YOUR APP IS RUNNING’
- Click the URL to view your application listed next to ‘Routes:’ under the application name. The URL should be clickable.
- This should open the homepage of your Node-RED application.
- Click the large red button that says ‘Go to your Node-RED flow editor’, which opens the actual Node-RED flow editor.
Now you have created a Node-RED Starter server application.
3. A Closer Look at the Client Application
We want to add an HTML form for user input, so we must edit the code of the client application. In order to edit the code, you must first do the following:
- On the application overview page of the client application, click the ‘ADD GIT’ button on the top right. This will create a source code repository and also a DevOps environment that includes an online editor.
- In the ‘Create Git Repository’ popup window, make sure the checkbox for ‘Populate the repo with the starter app package and enable the Build & Deploy pipeline’ is checked and click the ‘CONTINUE’ button
- When you get the ‘SUCCESS’ notice click the ‘CLOSE’ button. In the top right of the application overview page, you should see a link to the source code of the Node.js application source code and an ‘EDIT CODE’ button.
- Click the ‘EDIT CODE’ button or the URL to the code editor, which will take you to the DevOps environment that includes the source code repository and the online editor among other.
- If this is the first time, you go to the DevOps environment, you are asked to create a username or display name.
- You are now in the DevOps environment, where you can make code changes, save source code, and run a build and deployment stage, which is similar to the ‘staging’ of the application when we first created the application.
Let’s first look at the most important files we see in the source code for the client.
- ‘manifest.yml’, this is a configuration file for Bluemix, with information about the name, the host name, the services, and the computer configuration like CPU and memory;
- ‘package.json’, this is a configuration file for the Node.js runtime, with information to make sure Node.js can run your application;
- ‘app.js’, this is your actual Node.js application. In the ‘package.json’ file, there is a line that says
"scripts": {
"start": "node app.js"
},
When you start your application, Node.js looks at the ‘package.json’ file to see which command it must run to start your application. This line says that it must run ‘node app.js’, which means that the ‘node’ runtime will run the ‘app.js’ file. - ‘public’ folder, in the ‘app.js’ file, there is this line
app.use(express.static(__dirname + '/public'));
which means that for so-called ‘static’ files like ‘.html’ files are located in the ‘/public’ folder. When your application starts, it will look for the default html file with the name ‘index.html’, which is the web page that says ‘Hello World!’ when you open the web application.
4. Add an HTML Form to the Client
Now you understand the Node.js code of the client application and added DevOps support, you can start changing the code. To allow users to send input to our Node-RED server application, I will change the ‘index.html’ page to have an HTML form instead of the ‘Hello World!’ message.
- Make sure you are in the online editor in the DevOps environment. If you are not, go to the application overview page, click the ‘EDIT CODE’ button to go to the DevOps environment, and in the left menu with the mini icons, click the pencil or the ‘EDIT CODE’ button in the top right, to make sure you are in the online editor page.
- Open the ‘~/public/index.html’ file.
- Delete the ‘Hello World!’ message and the ‘Thanks’ line by removing the
<h1>
and the<p>
tags in the second table data tag.
<h1 id="message">Hello World!</h1>
<p class='description'></p>Thanks for creating a <span class = "blue">NodeJS Starter > - Replace the removed code by the code below, to add a form with two input controls, one for firstname and one for lastname, and a submit button, as follows:
<h3>Register:</h3>
<form action="http://neca-nodered1.mybluemix.net/ping" method="get">
<label for="firstname">First name:</label> <input type="text" name="firstname"></input><br>
<label for="lastname">Last name:</label> <input type="text" name="lastname"></input><br>
<input type="submit" value="Submit"></input>
</form> - Click the ‘Play’ icon for the ‘Deploy the App from the Workspace’ to build and deploy the changed source code to the runtime
- This will change the status of your application in the workspace to ‘deploying’.
5. HTTP Methods, Forms and REST
Communication over the internet is based on the HTTP protocol.
Forms
When you send an HTML form submission, you can use two HTTP methods: GET and POST.
In this HTML form definition, there are two attributes: action, which defines the URL or URI where the form is send to, and method, which is the HTTP method.
<form action="http://neca-nodered1.mybluemix.net/ping" method="get">
When you send a form input over the GET method, input parameters are send as key-value pairs appended as a URL parameter, note that URLs have a maximum length:
http://neca-nodered1.mybluemix.net/ping?firstname=Remko&lastname=de+Knikker
When you need to send sensitive data or large sized data, you should use the POST method. The POST method sends the input data embedded in the body of a request and there is no size limit as there is with the GET method.
REST
These days when your application sends a request to a so called API (Application Programming Interface), you also use different HTTP methods. When you use APIs, the standard protocol to communicate is called REST, which is an extension of HTTP. REST specifies that you must use the following HTTP methods for the corresponding CRUD (Create, Read, Update and Delete) methods on the database:
- Create: POST,
- Read: GET,
- Update: PUT, and
- Delete: DELETE.
6. Add a Request Handler to the Server Application
Now we added an HTML form to the client application, we still need a request handler on the server application that receives the request from the form, stores the data in a database, and then sends a response back the client.
- Go to the Node-RED flow editor: http://<your-host-name>.mybluemix.net/red/#
- Let’s start with a simple exercise. In the left menu, under the ‘input’ section, drag the ‘inject’ node to the ‘Flow 1’ tab.
- In the left menu, scroll down to the ‘output’ section and drag the ‘debug’ node to the ‘Flow 1’ tab.
- Connect the two nodes by drawing a line between the two open dots facing the two nodes.
- After making any changes to a flow in Node-RED, you must deploy the flow by pressing the red ‘Deploy’ button on the top right of the flow editor.
- To test our first flow, make sure in the right column, you click the ‘debug’ tab to see any output from the ‘debug’ node.
- Then click the little square button to the left of the ‘timestamp’ node.
- You should see in the ‘debug’ tab, the timestamp being printed to the debugger.
- Double-click the ‘inject’ node to open its configuration window.
- From the ‘Payload’ dropdown, select ‘string’ to change the input from ‘timestamp’ and set the value to ‘Hello World!’
- Set the ‘Topic’ to ‘saying hi’ and ‘Name’ the ‘inject’ node ‘Hello World Test’.
- Click ‘Done’.
- Deploy the changes to the flow.
- Test the changes.
Now, I will create the server endpoint, to which the form of the client application sends the user input: ‘http://neca-nodered1.mybluemix.net/ping’:
- In the Node-RED flow editor, create a new flow under the test flow, in the same tab.
- To create an ‘http’ endpoint, drag an ‘http’ node under the ‘input’ section to the tab.
- Scroll to the ‘output’ section and drag an ‘http response’ node to the right of the ‘http’ node, and connect the two nodes.
- Double click the ‘http’ node to change the configuration.
- Leave the value for ‘Method’ to ‘GET’, change the value for ‘URL’ to ‘/ping’ and change the ‘Name’ to ‘GET /ping’.
- Click ‘Done’ and deploy the flow.
- Now you can test the client form, go to the client’s URL, complete the form and submit the form. You should see your input as a json response.
Congratulations, you have implemented an HTML input form that sends a request to the server, and you implemented a request handler that sends a response back to the client!
7. Add a Database and Complete Form
I want to store registrations in a database, so that I know who has registered.
First, I will make these changes to the client application:
- Go to the online editor in the DevOps environment of the client application, from the client application overview page, click the ‘EDIT CODE’ button,
- Open the ‘~/public/index.html’ file and change the ‘action’ in the form-tage to ‘http://neca-nodered1.mybluemix.net/register’ and change the method to ‘post’,
<form action="http://neca-nodered1.mybluemix.net/register" method="post">
- Add three new input controls for username, email and password:
<form action="http://neca-nodered1.mybluemix.net/register" method="get">
<label for="firstname">First name:</label> <input type="text" name="firstname"><br>
<label for="lastname">Last name:</label> <input type="text" name="lastname"></input><br><br>
<label for="email">Email:</label> <input type="text" name="email"></input><br>
<label for="username">Username:</label> <input type="text" name="username"></input><br><br>
<label for="password">Password:</label> <input type="password" name="password"></input><br><br>
<input type="submit" value="Submit"></input>
</form>
- Click the ‘Deploy the App from the Workspace’ play button.
- Go to the Node-RED server application overview page. You see there is a ‘Cloudant NoSQL DB’ service bound to the application. Services that are bound to the application, can be accessed by the application in the Node-RED flow editor.
- Node-RED uses this ‘Cloudant NoSQL DB’ service to store all your Node-RED flows in a database called ‘nodered’.
- We can use the same ‘Cloudant NoSQL DB’ service to add our registrations to a new database called ‘registrations’.
- Go back to the flow editor.
- From the ‘input’ section, drag a new ‘http’ node to the tab,
- Double click the ‘http’ node to configure an endpoint with ‘Method’ of value ‘POST’, with ‘URL’ of value ‘/register’ and ‘Name’ of value ‘POST /register’,
- Click ‘Done’,
- From the ‘output’ section, drag a new ‘http response’ to the tab, drop it after the ‘http’ node,
- From the ‘output’ section, drag a ‘debug’ node under the ‘http response’ node,
- From the ‘storage’ section, drag a simple ‘cloudant’ output node, with a single open dot to the left of the node, to store ‘msg’ in a single database.
- Double-click the ‘cloudant’ output node,
- Configure the ‘Service’ to select the cloudant database, do NOT select the option ‘External cloudant or couchdb service’,
- For ‘Database’ enter the name ‘registrations’, for ‘Operation’ select the value ‘insert’, and for ‘Name’ enter the value ‘Store Registrations’,
- Click ‘Done’,
- Connect the ‘http’ node with the three other nodes.
- Click ‘Deploy’ to deploy the changes.
Congratulations! You have now add a database and are storing new registrations in a NoSQL database.
Extra!
Add a Result Page to the Client
When the form request has been handles by the server application, a response was sent.
Extra! Extra!
Let’s Talk
IBM Watson is not just a computer running applications, but it is a smart computer or what is called a ‘cognitive’ computer. One of the Watson services is that it can talk. Let’s say ‘Welcome’ to the new user who just registered.
- First, we must add the Watson service that can translate text to speech. Go to the Node-RED server application overview page,
- Click ‘ADD A SERVICE OR API’,
- The Bluemix catalog will open,
- In the ‘Type to search’ bar enter ‘Text to’ and click the ‘Text to Speech’ service, or browse to the ‘Watson’ section and click the ‘Text to Speech’ service,
- Make sure the ‘Space’ and ‘App’ value are correct and click the ‘CREATE’ button,
- When asked to restage your application, click the ‘RESTAGE’ button,
- Once restaging is finished and the app is running, check the application overview page that the ‘Text to Speech’ service was added successfully,
- Go back to the Node-RED flow editor,
- Delete the connection between the ‘POST /register’ node and the ‘http response’ node,
- In between insert respectively a ‘template’ node from the ‘function’ section, a ‘text to speech’ node from the ‘IBM_Watson’ section, and a ‘function’ node from the ‘function’ section,
- Connect the nodes from left to right, the ‘http’ node to the ‘template’ node, the ‘template’ node to the ‘text to speech’ node, the ‘text to speech’ node to the ‘function’ node, the ‘function’ node to the ‘http response’ node,
- Double-click the ‘template’ node, which will pull out the firstname and lastname from the register request,
- Configure the ‘template’, for ‘Name’ enter the value ‘Welcome’, for ‘Set property’ enter the value ‘msg.payload’, in the template set the value to
Welcome {{payload.firstname}} {{payload.lastname}},
your registration was successfully saved to
our database.
- Click ‘Done’,
- Double-click the ‘text to speech’ node to set the configuration,
- For the ‘Name’ set the value ‘Text to Speech’, for the ‘Language’ set the value to ‘US English’, for ‘Voice’ set the value to ‘Lisa’, and for ‘Format’ select ‘WAV’,
- Click ‘Done’,
- Double-click the ‘function’ node, which will convert the audio file to a proper http response,
- For ‘Name’ set the value to ‘Format Audio Response’, set the ‘Function’ to value
msg.payload = msg.speech;
return msg; - Click ‘Done’,
- Click the ‘Deploy’ button in the top right and click ‘Confirm deploy’
- Try to register a new user in the client application.
Congrats, this is really clever!
I’m looking for a way to do register/login within node-red, using mongodb to store usernames and passwords (bcrypted).
Any idea?
I know settings.js but I’d like users to be able to register, not just static credentials.
Phil,
did you find a solution , I am also looking to do the same thing.