Getting Started: Creating a Java-Liberty App with Watson on Bluemix

Leave a reply

An extended version of this tutorial, adding a Cloudant NoSQL Database and D3js data visualization, is available here.

Requirements:

  • Bluemix account
  • Github account

Steps:

  1. Create the StarterApp
  2. Add the Toolchain or Continuous Integration (CI)
  3. REST API Primer
  4. Authentication in REST API
  5. Getting the Bluemix Configuration
  6. Add the AlchemyData News API Client
  7. Add a Web Form
  8. Implement the AlchemyData News API Request
  9. Create Authorization Header for Basic Auth

1. Create the StarterApp

  1. Sign in to Bluemix.net,
  2. Go to Catalog,
  3. In the left column, under ‘Apps’ > ‘Boilerplates’, click the ‘Java Cloudant Web Starter’ service icon,
  4. For ‘App name’ and ‘Host name’ enter ‘<username>-liberty-watson’,
  5. Click the ‘Create’ button,
  6. When successfully completed, in the ‘Application Details’, in the left column, click the ‘Overview’ link,
    bluemix_application_details
  7. Review the application configuration in the ‘Overview’ link in the left menu of the page,
  8. In the Runtime window, you can see that this boilerplate is built from a so-called ‘BUILDPACK’ called ‘Java Cloudant Web Starter’. Bluemix uses CloudFoundry containers called ‘buildpacks’, roughly comparable to Docker containers, which are also supported by Bluemix,
  9. Scroll down a little bit, in the ‘Connections’ window, you see that this boilerplate by default already created 1 service for you: a Cloudant NoSQL Database,
  10. To add a another service from the Bluemix Catalog to your application, click the ‘Connections’ link in the left menu,
  11. Click the ‘Connect New’ button to add a new AlchemyAPI service,
  12. You are forwarded to the Catalog. From the Catalog, in the left menu of filters, filter by ‘Watson’ and select the AlchemyAPI service > click the ‘Create’ button and > click ‘Restage’ to bind the service to the application,
    bluemix_liberty_services
  13. Note: by default there is a limit to the number of instances you can create for each service under your organization. This limit is different per service. In the public general version of Bluemix, you can create 1 instance of the AlchemyAPI service under your organization. If an AlchemyAPI service instance already exists creating a second instance or connecting an existing service in another space will cause an error. Sometimes, you get an error message after you clicked the Create button. In that case, if an AlchemyAPI instance already exists in your space, click the ‘Connect Existing’ button instead, select the existing AlchemyAPI service and click the ‘Connect’ button, click ‘Restage’ to bind the existing service to the new application,
  14. For each service credentials are created, which you can access via the ‘View Credentials’ button on the service icon under Connections, or via the left menu item ‘Runtime’ > ‘Environment Variables’ > VCAP_SERVICES. You need these credentials in your application code to access the service.

2. Add the Toolchain or Continuous Integration (CI)


Your application is created and you connected a new Bluemix service to the application. The next step is to add the development, build, test and deployment tools to the application, before we can start coding. The service that enables this is called the ‘Toolchain’ that includes among other an online editor ‘Eclipse Orion Web IDE’ based on Eclipse, source code versioning using Github as default repository and issue tracking tool, and a Delivery Pipeline for Continuous Delivery.

  1. Go to the Overview page again, scroll down to ‘Continuous Delivery’ in the bottom right of the application details page,
    bluemix_liberty_enable_toolchain
  2. Under Continuous Delivery click Enable > Create Toolchain > View Toolchain,
    bluemix_toolchain_1
  3. If this is the first time, you enabled the Toolchain then you have not yet authorized Bluemix to access your Github account. You will see a red alert bar with a ‘Failed’ message in the Github icon. Click the ‘Failed’ message and click ‘Configure’ > Authorize App > login to your Github account and agree to authorize Bluemix. Click Save Configuration,
  4. To see the Build and Deployment stages, click Delivery Pipeline,
    bluemix_delivery_pipeline_1
  5. In the Build stage, click the Settings icon > under the Input tab, check to configure build to ‘Run automatically on every Git push’ > and click ‘Save’ or if already checked click ‘Cancel’,
  6. Run the Build stage and review the Deploy stage,
  7. Go back to the Toolchain Overview page,
  8. Click the ‘Code’ icon, which will open the new Github repository,
  9. By default, the newly created Github repository is public. To make the new Github repository private, go to Github and if you have an upgraded account, click Settings > Make Repository private

The Toolchain by default comes with an online client IDE service called ‘Eclipse Orion Web IDE’.

  1. Go to the Toolchain Overview page again and clik the ‘Eclipse Orion Web IDE’ icon,
  2. Click the ‘README.md’ to edit and replace all the markdown with the following:
    # A Java WebSphere Application using Watson AlchemyAPI
    bluemix_orion_ide_1
  3. The IDE will auto-save or click > File > Save, or click command-S,
  4. In the very left margin there is a short list of tiny icons: a back arrow, a pencil, a git icon and a settings cogwheel. Click the Git icon to commit and push the changes,
    bluemix_liberty_orion_leftmenu
  5. You see that 3 files were added for commit: 2 files, .cfignore and .gitignore that Bluemix added after deploying the Buildpack and copying the code from the Buildpack to Github, and the ‘README.md’ file which we changed,
  6. Under the ‘Working Directory Changes’, enter the commit message ‘init’ and press the Commit button,
  7. In the left column, under ‘Outgoing’ press the ‘Push’ button,
  8. Click the ‘back arrow’ icon in the top left margin, that takes you back to the Toolchain overview, and click the ‘Delivery Pipeline’ icon to review the results of the Build stage and the Deploy stage,
  9. Also, verify that the changes were pushed to the Github repository.

3. REST API Primer

Before I start adding code to call the AlchemyAPI, I will go through a quick primer on REST APIs.

REST stands for Representational State Transfer and is Chapter 5 in Roy Thomas Fielding’s dissertation, which talks about a very high level architectural design for distributed systems or a modern Web architecture, based on the HTTP protocol and Uniform Resource Identifiers (URI).

Resources are accessed by URIs, where the HTTP methods indicate if access is a Create, Read, Update or Delete, respectively using HTTP methods for POST, GET, PUT, or DELETE.

For example, a resource like ‘Persons’ might be accessed via URI:
http://www.mydomain.com/api/v1/persons
To create a new person, you submit a POST request with personal data to the endpoint for the Persons resource.
curl 'http://www.mydomain.com/api/v1/persons' -d '{"firstname":"Remko","lastname":"de Knikker"}

To retrieve or read a person from the endpoint for the Persons resource, you submit a GET request to the endpoint for the Persons resource. The following examples use cURL, a command-line tool for transferring data with URLs. Another great tool for testing REST APIs and generating code is Postman.
curl 'http://www.mydomain.com/api/v1/persons/1'
or
curl 'http://www.mydomain.com/api/v1/persons?id=1'

To add Basic Auth to the request
curl -U remkohdev:passw0rd 'http://www.mydomain.com/api/v1/persons/1'

JSON

Although the REST specification itself does not talk about data formats, in practice an industry standard or best practice has developed to write input and output data in JSON format.

JSON stands for ‘JavaScript Object Notation’ and is a subset of the JavaScript Programming Language. JSON uses key-value mapping with curly and square brackets to indicate objects and arrays.

The following is an example of an array of people, each person being an object with a firstname and lastname property:
[
{"firstname":"Remko", "lastname":"de Knikker"},
{"firstname":"John", "lastname":"Doe"}
]

Parsing JSON

GSON is a JSON parser for Java developed and released by Google. The following example create a JSON object from a String, looks for the ‘result’ node, and the ‘docs’ node in the JSON, then it creates a JSON response array with a JSON object with ‘docs’, and returning a String format.
import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
...
// serialize 'result' String from AlchemyData News API to json
Gson gson = new Gson();
JsonObject alchemyJsonObject = gson.fromJson(result, JsonObject.class);
JsonObject resultJsonObject = alchemyJsonObject.get("result").getAsJsonObject();
JsonArray docs = resultJsonObject.get("docs").getAsJsonArray();
// create response object to return 'docs' in json array
JsonObject jsonObject = new JsonObject();
jsonObject.add("docs", docs);
JsonArray jsonArrayResponse = new JsonArray();
jsonArrayResponse.add(jsonObject);
return Response.ok(jsonArrayResponse.toString()).build();

4. Authentication in REST APIs

In general, REST APIs use three ways to authenticate and authorize an application to access endpoints to resources:

  1. API Key
  2. Basic Auth
  3. OAuth2
  4. Database Authentication

API Key

Some services like the Watson AlchemyAPI generate an API Key to authenticate your request. In general you use the API Key as a URL parameter:
curl 'http://www.mydomain.com/api/v1/persons/1?apikey=12abdhy632kabc=='

Basic Auth

Most Bluemix services use Basic Auth to verify authentication. In Basic Auth, you set the HTTP header for ‘Authorization’ to a string that includes a Base64 encoded username and password:
"Basic "+ base64.encode(username+":"+password)
To submit the full request:
curl -U remkohdev:passw0rd 'http://www.mydomain.com/api/v1/persons/1'

OAuth2

OAuth authentication is a 2-step process. When signing up with an email and password for an API service, the service will provide a client ID and client secret. In step 1, your application will send an oauth request to obtain an access token by identifying with the client ID and client secret. In step 2, your application makes the actual service request, by identifying with the access token obtained in step 1.

For example, in Bluemix you can add APIs from Pitney Bowes, which will automatically sign you up and give you an API key (to be used as client ID) and a client secret. You need to Base64 encode the ‘apikey:client secret’ string to make a Basic Auth access token request.
curl -X POST -H "Authorization: Basic a2tLeW1rNz0d6VA==" -H "Content-Type: application/x-www-form-urlencoded" -d 'grant_type=client_credentials' "https://api.pitneybowes.com/oauth/token"
The oauth request returns an access token, the Pitney Bowes oauth request returns the following response:
{
"access_token": "vc8FglTSoK2",
"tokenType": "BearerToken",
"issuedAt": "1479828944765",
"expiresIn": "35999",
"clientID": "kkvxVw1",
"org": "pitneybowes"
}
Then in the next step, you use the ‘access_token’ to set the ‘Authorization’ HTTP Header with a Bearer token.
curl -X GET -H "Authorization: Bearer vc8FglTSoK2" "https://api.pitneybowes.com/location-intelligence/geolife/v1/demographics/byaddress?address=550%20W%2054th%20St%2C%20New%20York%2C%20NY%2010019&country=USA"

Database Authentication

While Database services on Bluemix use regular database authentication via:
protocol://username:password@hostname:port/database

5. Getting the Bluemix Configuration

To authenticate your service request, you need to retrieve the credentials from the Bluemix environment. Services configuration are automatically added to a system environment variable called ‘VCAP_SERVICES’. When you run your application on localhost, the ‘VCAP_SERVICES’ runtime system environment variable cannot be found, and the application throws a ‘RuntimeException: VCAP_SERVICES not found’.

If you plan to run on localhost, add a ‘bluemix.json’ file to the classpath ‘~/WebContent/WEB-INF/classes/’ and in the Singleton class load the file from the classpath.

To read the services configuration on localhost and Bluemix, go to the online ‘Orion Web IDE’ and in the ‘src’ directory, add a new folder ‘com.<username>.dev.liberty.api.’ and to the new folder or package, add a new file or Singleton class ‘BluemixConfig.java’.
bluemix_bluemixconfig_java

Copy-paste the following code to the new BluemixConfig.java class.

In the online Orion IDE from the Toolchain, go to the git IDE via the upper left ‘Git’ icon, and commit-push your changes.

The following step is only necessary, if you are developing on localhost, else skip this step:

  • Create a configuration file ‘~/WebContent/WEB-INF/classes/bluemix.json’,
  • On Bluemix, go to the ‘Application Details’ page > Runtime > ‘Environment Variables’, copy the VCAP_SERVICES json, and paste the json in the bluemix.json file,

Now to get the credentials for a service like Cloudant, you write:
String user = BluemixConfig.getInstance().getCloudantDBUsername();
String password = BluemixConfig.getInstance().getCloudantDBPassword();

To get the apikey for the AlchemyData News API service, you write:
String apikey = BluemixConfig.getInstance().getAlchemyApikey();

  1. Go to the online editor, and open the ‘manifest.yml’ file.
  2. Bluemix uses CloudFoundry to deploy applications, and CloudFoundry uses Application Manifests for deployment. The ‘manifest.yml’ file is used to build and deploy the application to Bluemix, using the application’s path, the configuration settings, and the connections or services.
  3. The path defines the build ‘war’ artefact that is used to deploy to Bluemix,
  4. The ‘host’ and ‘name’ properties are used to deploy the application during deployment in the staging process. The ‘host’ value must correspond to the ‘host’ in the ‘Application details’
  5. The ‘services:’ node is an array of service names that correspond to the ‘name’ properties for each of the services in the ‘VCAP_SERVICES’.

6. Add the AlchemyData News API Client

Now, let’s add a client to call the ‘AlchemyData News API’. First, add a new file or class ‘AlchemyDataNewsAPI.java’ to the ‘com.remkohde.dev.liberty.api’ folder or package. Add the following code to the new class:

Commit-push the changes in the Git client.

7. Add a Web Form

To send user input to the AlchemyData News API Client and display the response back to the user, edit the ‘~/WebContent/index.html’ file and replace the current code by the following code:

Commit-push the changes in the Git client.

Go to the Delivery Pipeline and check that the application was built and deployed successfully. Open the application in your browser,
bluemix-liberty-webform

submit a request and you should get the following response.
bluemix-liberty-webform-response

8. Implement the AlchemyData News API Request

Finally, for the code to call the actual REST API, I need to implement an HTTP client. In this tutorial, choose:

  • Apache HTTP Components, or
  • Watson Developer Cloud Java SDK under the hood uses OK HTTP client. You need to be build the repository using Maven.

Normally you can add dependencies via the ‘Orion Web IDE’ > File > import > HTTP and paste a link to the zip or jar file of your dependencies, but at the time of writing this tutorial there seemed to be a bug in that feature.

As a work-around, clone the repository to your localhost, resolve the dependencies and upload them back to our Github repository. Make sure in the online ‘Orion Web IDE’, to merge the local changes as incoming code from your localhost with the code that is edited via the ‘Orion Web IDE’ client, before you make further changes online, in order to prevent Git conflicts.

  1. Clone to localhost:
    git clone https://github.com/<username>/<username>-liberty-watson.git
  2. Download the dependencies as described below, either using Apache HTTP Components or using the Watson Developer Cloud Java SDK,
  3. Add > commit > push changes to Github,
  4. Then go to the online Orion IDE > git > in the ‘Incoming’ section, click the middle icon to the right to ‘Merge the content from the branch to your active branch’,

Apache HTTP Components

To call the AlchemyData News API using Apache HTTP Components:

  1. Download Apache HTTP Components
  2. Unzip and copy the jar files in ~/lib/*’ to ‘~/WebContent/WEB-INF/lib’,
  3. Edit the ‘build.xml’ ant file which is used by the Delivery Pipeline to build and deploy your application,

  4. Add, commit and push your changes to the Github account,
    cd <username>-liberty-watson
    git status
    git add .
    git status
    git commit -m "added dependencies for http components"
    git push
  5. Now, go back to the online ‘Orion Web IDE’ > Go to Git client > and to the right of ‘Incoming’, click the middle icon to ‘Merge the content from the branch to your active branch’,
    bluemix-liberty-incoming
  6. Go back to the ‘Orion Web IDE’, and open the ‘~/WebContent/WEB-INF/lib’ folder to check that the dependent jars are now there,
  7. Open the ‘AlchemyDataNewsAPI.java’ class,
  8. Add the following import statements to the top of the class:
    import org.apache.http.HttpEntity;
    import org.apache.http.client.methods.CloseableHttpResponse;
    import org.apache.http.client.methods.HttpGet;
    import org.apache.http.impl.client.CloseableHttpClient;
    import org.apache.http.impl.client.HttpClients;
    import org.apache.http.util.EntityUtils;
    import java.io.IOException;
  9. To call the AlchemyData News API, add the implementation of the callDataNews() method as follows:

Commit-push the changes in the Git client.

Go to the Delivery Pipeline and check that the application was built and deployed successfully. Open the application in your browser.

Watson Developer Cloud Java SDK

To install the Watson Developer Cloud Java SDK and its dependencies, you must have Maven installed. Check to see if you have Maven with:
mvn -v

  1. Go to https://github.com/watson-developer-cloud/java-sdk,
  2. Clone the repository to your localhost,
    mkdir dev/lib/java
    cd dev/lib/java
    git clone https://github.com/watson-developer-cloud/java-sdk.git
  3. Build the repository using Maven’s ‘pom.xml’ file,
    cd java-sdk
    mvn clean install
  4. This will produce the following jars:
    • ~/alchemy/target/alchemy-3.5.2-SNAPSHOT.jar
    • ~/all/target/java-sdk-3.5.2-SNAPSHOT-jar-with-dependencies.jar
  5. Copy these jar files to ‘~/WebContent/WEB-INF/lib’,
  6. Download Apache Commons Lang 3.5 from http://mirrors.ibiblio.org/apache//commons/lang/binaries/commons-lang3-3.5-bin.zip,
  7. Unzip and copy the ‘commons-lang3-3.5.jar’ file to ‘~/WebContent/WEB-INF/lib’,
  8. Edit the ‘build.xml’ Ant file which is used by the Delivery Pipeline to build and deploy your application:
    <pathelement location="WebContent/WEB-INF/lib/alchemy-3.5.2-SNAPSHOT.jar"/>
    <pathelement location="WebContent/WEB-INF/lib/java-sdk-3.5.2-SNAPSHOT-jar-with-dependencies.jar"/>
    <pathelement location="WebContent/WEB-INF/lib/commons-lang3-3.5.jar"/>
  9. Add, commit and push your changes to the Github account,
    cd <username>-liberty-watson
    git status
    git add .
    git status
    git commit -m "added dependencies for watson developer cloud java sdk"
    git push
  10. Now, go back to the online ‘Orion Web IDE’ > Go to Git client > and to the right of ‘Incoming’, click the middle icon to ‘Merge the content from the branch to your active branch’,
  11. Go back to the ‘Orion Web IDE’, and open the ‘~/WebContent/WEB-INF/lib’ folder to check that the dependent jars are now there,
  12. Open the ‘AlchemyDataNewsAPI.java’ class,
  13. Add the imports of the required classes to the ‘AlchemyDataNewsAPI.java’ class:
    import java.util.HashMap;
    import java.util.Map;
    import org.apache.commons.lang3.StringUtils;
    import com.ibm.watson.developer_cloud.alchemy.v1.AlchemyDataNews;
    import com.ibm.watson.developer_cloud.alchemy.v1.model.DocumentsResult;
  14. To call the AlchemyData News API, implement the callDataNews() method as follows:

8. Create Authorization Header for Basic Auth

Other services might use Basic Auth instead of using an API Key for Authentication. In those cases you can retrieve the username and password from the System Environment variable VCAP_SERVICES, but you need to Base64 encode them to build the Authorization header.

  1. Download the Apache Commons Codec from http://mirror.reverse.net/pub/apache//commons/codec/binaries/commons-codec-1.10-bin.zip
  2. Unzip and add ‘commons-codec-1.10.jar’ to your classpath,
  3. To create a Basic Auth HTTP header for a REST API requiring Basic Auth, add the following code:
    import org.apache.commons.codec.binary.Base64;
    ...
    String decoded = username + ":" + password;
    String encoded = Base64.encodeBase64String(decoded.getBytes());
    String authHttpHeader = "Basic "+encoded;
  4. Add an HTTP Header ‘Authorization’ and set it to the above base64 encoded string,
    HttpGet httpGet = ...
    httpGet.setHeader("Authorization", authHttpHeader);

Leave a Reply

Your email address will not be published. Required fields are marked *