Category Archives: dev

Open Source driving Security, DevOps and Infrastructure Automation

Leave a reply

If you are an Application Developer, DevOps engineer, Site Reliability Engineer (SRE) or otherwise interested in Cloud Native and application modernization, you will see a lot of action in the fields of security, automation and devops areas on Cloud Native. Venture Capital is investing heavily in devops automation and security, recruiters are aggressively hunting down talent, new job openings are being posted every day, new startups are funded, open source projects and Kubernetes extensions launched with no end.

There are also a lot of really boring companies and technologies, capitalizing on the surge in adoption of cloud native without inspiring vision, like anything with “data-driven”, “single-sign on”, “insights”, “utilizing AI to optimize”, “future of cloud engineering” or “AI driven” makes me fall asleep fast. But there are also some really exciting initiatives, especially in the field of “everything is code”, automation of infrastructure, security and Kubernetes extensions and standardizations.

One thing, that IBM always got right in my opinion, is the decades long strategy to lead innovation by open sourcing technologies and creating community governed foundations to govern the open source standards for it, from the Apache Foundation, the Linux Foundation, Node.js Foundation to the more recent Cloud Native Computing Foundations (CNCF), CD Foundation to Open Cybersecurity Alliance (OCA).

The technologies and startups that I am most excited about, and which I recommend to let your long term career choices and direction be guided by all follow the same model and objectives: to build a service organization, to open source their core technology and to aim to be promoted to foundation projects. Just to name a few startups:

  • env0, two rounds of seed funding in 2020 of 6.8M
  • Snyk, added 200M investments in September 2020 and went from Series A to a $2.6BN valuation in 2.5 years
  • Harness, added 60M in series B funding in 2019, acquiring Drone.io in 2020
  • Pulumi, raised $37.5 million in Series B funding
  • Ambassador, formerly known as datawire.io 4M funding in seed funding in 2019

Or some interesting open source projects,

  • Ansible. Ansible is too widely adopted and established to really mention here, but the reason why I include it here, is because it too keeps innovating to adept to the growing complexity of cloud native architectures and still driving and inspiring a lot of the above mentioned startups probably, for instance with Ansible Molecule and Ansible AWX.
  • OpenSCAP, I am personally very interested in the automation of so-called business controls and compliance with regulatory requirements. OpenSCAP is one of the best tools out there, especially with image, container and NIST scanning. But there are also super exciting developments in the field of Robotic Process Automation (RPA).
  • kustomize, if you are used to Helm to configure your Kubernetes deployments, you will find Kustomize fun to play with. There are other tools like Source-to-Image (S2I) for instance that are similar.
  • skopeo, the longer I work on Cloud Native, the more I find myself drawn back to the basics of managing containers, skopeo is a great tool for this.
  • podman, also a bit out of line here, but if you are interested in security and hate to be stuck with only a single tool in the toolbox, Podman is a great alternative to Docker to build your images, and comes out of the SELinux corner with a much better security design.

Ionic2: if … else … template blocks in Angular2

Leave a reply

Angular2 supported an if-condition via the *ngIf directive.

<div *ngIf="edit == false">
VIEW
</div>
<div *ngIf="edit == true">
EDIT
</div>

Since Angular2 v4, the ngIf directive was extended with an ‘then.. else..’ syntax, in which you can reference the identifier of the HTML tags to include.

<div *ngIf="edit; then editDiv else viewDiv"></div>
<ng-template #editDiv>
EDIT
</ng-template>
<ng-template #viewDiv>
VIEW
</ng-template>

Another way to control HTML input-tags to be editable or disabled, is to use the [attr.disabled] syntax.

<input [attr.disabled]="edit ? null : true">

When using Ionic2, this translates to the following code.

<ion-input [disabled]="!edit"><ion-input>

In the TypeScript component, then in the control the ‘edit’ boolean variable.

I use the [attr.disabled] syntax for instance to display a detail page of an object in view (Read) or edit (Create/Update) form, and adding a Delete option to the view modus to complete a CRUD option. This simplifies my application development, imo, cause now the navigation in my application is mostly from List to Detail pages.

See also: NgIf Else lands in Angular 2.x+/4.0, Ashish Singh (Dec 18, 2016)

Messenger2Watson(1): Connect Facebook to a Watson Chatbot

2 Replies

In Slack2Watson(1) and (2) I created a chatbot and integrated it into Slack Slash Commands. Now, let’s integrate the same chatbot into Facebook Messenger. To accomplish this I need to create a Facebook Application, add the Messenger Platform to my Facebook Application, create a Webhook, have admin access to a Facebook Page to generate a Page Access Token so I can send and receive messages send to the Facebook Page, and create the Node-RED flows to integrate the Facebook Messenger with the IBM Watson Conversation.

Steps:

  1. Create a Flow to Verify the Request for Webhook Edits in Node-RED,
  2. Create an Endpoint for the Redirect URL of the Webhook in Node-RED,
  3. Create a Facebook Application for the Messenger Platform,
  4. Enable Webhooks Integration with Node-RED,

Create a Flow to Verify the Request for Webhook Edits in Node-RED

To prepare the setup and configuration of the Facebook Application, the Messenger platform and Webhooks to enable a chatbot in Facebook Messenger, I will first create the Node-RED flows to implement the required server workflow.

The first flow is to verify the endpoint for the setup of the Facebook Application, using the ‘hub.challenge’ token.

  • Go to your Node-RED application on Bluemix at http://<username>-nodered-slackapp.mybluemix.net/,
  • Click the ‘Go to your Node-RED flow editor’ button,
  • If you’re not logged in yet, log in now,
  • Add a new flow tab and rename the flow ‘Facebook Messenger’,
  • To verify your endpoint during setup of your Webhook, or when you update an existing topic subscription of your Webhook, Facebook sends a GET request. The request will include:
    hub.mode=subscribe
    hub.challenge — a random string
    hub.verify_toke

    • Continue reading →

Slack2Watson(2): Connect Slack to a Watson Chatbot with Node-RED

1 Reply

Using:

  • Slash Commands in Slack,
  • Watson Conversation service to create a ChatBot,
  • Node-RED to configure the Application Flow,

Using a chatbot, I want to automate the following scenario:

  1. user: Hello
  2. chatbot: Hello. Where are you?
  3. user: hi, i am at the Rubin Museum.
  4. chatbot: I love the Rubin Museum. Shall I give you some recomendations of my personal favorites?
  5. user: oh yes, I would love some recommendations.
  6. chatbot: do you like sculpture, paintings or ornaments?
  7. user: i prefer paintings!
  8. chatbot: ok, here are my favorite paintings at the Rubin Museum: a, b, c

Steps:

  1. Create the Watson Conversation for the Rubin Museum Scenario,
  2. Test the Conversation for the Rubin Museum Scenario,
  3. Setup Slash Commands in Slack,
  4. Create the Node-RED Flow to Watson Conversation,
  5. Add Token Validation of the Slack Request to Node-RED Flow,

Create the Watson Conversation for the Rubin Museum Scenario

  • Go to IBM Bluemix and login to your account,
  • To create the Watson Conversation, click the ‘Create Service’ button, which will take you to the Catalog, or
  • Go to the Catalog, Under ‘Services’ filter by ‘Watson’, or in the catalog browse to the Watson section,
  • Click the ‘Conversation’ service,
  • Agree or change the ‘Service name’ and the ‘Credentials name’, and click the ‘Create’ button,
  • Under the ‘Credentials’ tab, you will find the username, password, and workspace ID that you need later to configure access to the conversation service,
  • Click the green ‘Launch tool’ button,
  • First create a workspace, click ‘Create’, name the workspace ‘Watson2Slack-Workspace’, and click ‘Create’,
  • You are now in your Conversation workspace, and you should see 3 tabs: Intents, Entities, and Dialog,

    Continue reading →

Slack2Watson (1): Connect API Connect to Serverless OpenWhisk

1 Reply

‘Slack2Watson (1): Connect API Connect to Serverless OpenWhisk’ is part 1 of an application called Slack2Watson, which integrates a ChatBot into Slack and allows you to retrieve information about events via Slash Commands in Slack. In part 1, I will set up the ‘API Connect’ service and link the APIs in ‘API Connect’ to the OpenWhisk functions.

Integration is managed using API management in ‘API Connect,’ serverless or event-based APIs in OpenWhisk, and IBM Watson to create the Conversation bot. Some data is retrieved via Google Sheets API.

  1. Create API Connect API v1.0.0
  2. Publish the API
  3. Create an OpenWhisk action
  4. Connect API Connect to OpenWhisk

Requirements

This tutorial uses the OpenWhisk service and the API Connect service on http://bluemix.net, the IBM Cloud. Create an account on IBM Bluemix to run this tutorial.

1. Create API Connect API v1.0.0

  • Go to the Bluemix Catalog,
  • In the left menu select the ‘APIs’ category, and click the ‘API Connect’ service,
  • Click the ‘CREATE’ button to create the ‘API Connect’ service,
  • Click the ‘Sandbox’ icon to go to the Sandbox view with a list of products, which currently says ‘There are no products available in the selected catalog.’,
  • Go to the ‘Navigate to’ icon next to the ‘Home’ icon, and click ‘Drafts’, which will take you by default to the ‘Drafts’ view in the ‘Designer’ perspective,
  • In the ‘Drafts’ view, you see a ‘Products’ tab and an ‘APIs’ tab, click the ‘APIs’ tab,
  • Click the ‘ADD’ button and add a ‘New API’ titled ‘<username>-slack2watson’
  • Title: ‘<username>-slack2watson’
  • Name: ‘<username>-slack2watson’
  • Base Path: /slack2watson
  • Version: 1.0.0

Continue reading →

Easy Web Application Development for Beginners with Node-RED (2)

Leave a reply

Objective

In this tutorial I will create a Node-RED server application that will process a request from a client web form. The server will save the request data in a NoSQL database.

Requirements

You must have:

  • Access to a Bluemix account.

1. Create a Node-RED Starter Application

Start to create a Node-RED Starter server application. A client application sends a form request to the Node-RED server, the server processes the form data and sends a response back to the client.

Node-RED is a so-called visual workflow editor, which lets you create an application workflow by dragging and dropping visual nodes onto an editor.
Continue reading →

Creating a Nodejs App with Watson AlchemyAPI and CloudantDB

Leave a reply

WIP

Requirements:

Overview:

  1. Setup
  2. Setup Details
  3. Add Watson AlchemyAPI
  4. Add Cloudant DB

Setup

  1. Login to Bluemix, in a separate tab login to Github,
  2. In Bluemix open Catalog->Boilerplates and create a ‘Node.js Cloudant DB Web Starter’ Boilerplate, named ‘<username>-nodejs-app1’, click CREATE
  3. Go to ‘Overview’, scroll down to ‘Continuous delivery’ and click ‘Enable’
  4. In the ‘Toolchain Settings’, change the name of the Toolchain to ‘<username>-nodejs-app1’, and click Create,
  5. If the Github THINK or CODE icon displays a configuration error, from the block’s dropdown icon in the topright, click the ‘Configure’ link to correct the configuration of your Github account,
  6. Go to your Github account and make sure that the repository for the new Bluemix app is created successfully,
  7. Copy the Git URL, on localhost open a commandline terminal, change to your development directory and clone the new repository,
    cd ~/dev/src/bluemix
    git clone https://github.com/remkohdev/remkohdev-nodejs-app1.git
    cd remkohdev-nodejs-app1
  8. Open the project directory in your favorite editor,
  9. Open the ‘package.json’ file and change the ‘name’ property to ‘<username>-nodejs-app1’,
  10. From the commandline, in your project directory,
    git status
    git add .
    commit -m "change package.json name property"

Continue reading →

Getting Started: Creating a Java-Liberty App with Watson on Bluemix

Leave a reply

An extended version of this tutorial, adding a Cloudant NoSQL Database and D3js data visualization, is available here.

Requirements:

  • Bluemix account
  • Github account

Steps:

  1. Create the StarterApp
  2. Add the Toolchain or Continuous Integration (CI)
  3. REST API Primer
  4. Authentication in REST API
  5. Getting the Bluemix Configuration
  6. Add the AlchemyData News API Client
  7. Add a Web Form
  8. Implement the AlchemyData News API Request
  9. Create Authorization Header for Basic Auth

1. Create the StarterApp

Continue reading →

Add Serverless Programming and Data Integration on Bluemix

Leave a reply

Add Serverless or Event-Based Programming using OpenWhisk:

  1. Go to the OpenWhisk service at https://console.ng.bluemix.net/openwhisk
  2. Click the ‘Develop’ button, which takes you to the Openwhisk editor,
  3. Create a new Action and implement the JavaScript function,
  4. At the bottom right, click the ‘View REST Endpoint’, and in the ‘cURL Example’, click the ‘Show Full Command’ to reveal the Base64 encoded authentication token for the OpenWhisk service,
  5. You can get the username:password credentials by installing the OpenWhisk ‘wsk’ command line and run:
    wsk property get --auth
    Or to Base64 encode it, run:
    wsk property get --auth | awk '{printf("%s", $3)}' | base64
  6. The way to call the OpenWhisk endpoint is using ‘Basic Auth’:
    https://openwhisk.ng.bluemix.net/api/v1/namespaces/<organization_id>_<space>/actions/<action>?blocking=true

    7. Then, for example:

    • Next create a new action and link the two actions together by Extending the first action and link it to the second action.
    • Next create an action to save the result to the Cloudant Database using the ‘Create Document’ action,
    • Next trigger the Slack notification

Using Twitter data via DashDB:

  1. Create a DashDB service
  2. Create a Insights for Twitter service
  3. Open the DashDB dashboard > Load > Load Twitter Data > select the Twitter Service we created > Next
  4. In the Search for Twitter data: “#SEC posted:2016-11-01,2016-11-15”
  5. Click Get Tweet Count > Next
  6. Select ‘Load the data into new tables with this prefix:’ and enter prefix ‘SEC_Twitter’ > Next > Next > Finish

insights_for_twitter_sec_finished

Using public SEC data via DashDB:
https://www.sec.gov/dera/data/financial-statement-data-sets.html
Download the SEC file for 2016Q3: 2016q3.zip

  1. Unzip the ‘2016q3.zip’ archive,
  2. Load the 4 *.txt files: num, pre, sub, tag,
  3. Load > Load from Desktop > drop the file into the upload window,
  4. Change the separator character to ‘tab’,
  5. Don’t check the ‘Does the file have columns that contain dates or times? Yes No’, this may break the dataload sometimes, as the data is not always 100% cleaned correctly,
  6. Rename the tables so they have the prefix ‘SEC_2016Q3_’.

Other datasets:

  • Pitney Bowes Geodata:
    1. Add the Pitney Bowes service to your Bluemix application
    2. Get AccessToken using OAuth2:
      curl -X POST -H "Authorization: Basic a2t2eetceteratVA==" -H "Content-Type: application/x-www-form-urlencoded" -d 'grant_type=client_credentials' "https://api.pitneybowes.com/oauth/token"
    3. Then call the PB service:
      curl -X GET -H "Authorization: Bearer vc84ZaccesstokenoK2" "https://api.pitneybowes.com/location-intelligence/geolife/v1/demographics/byaddress?address=550%20W%2054th%20St%2C%20New%20York%2C%20NY%2010019&country=USA"
  • WeatherCompany Data:
    1. Add the WeatherCompany Data service to your Bluemix application,
    2. Get the username and password from the credentials,
    3. Daily Forecasts:
      curl -X GET "https://<username>:<password>@twcservice.mybluemix.net/api/weather/v1/geocode/33.40/-83.42/forecast/daily/10day.json?units=m&language=en-US"
    4. Current Conditions:
      curl -X GET "https://<username>:<password>@twcservice.mybluemix.net/api/weather/v1/geocode/33.40/-83.42/observations.json?language=en-US&units=m"
    5. Almanac (Historic Data):
      curl -X GET "https://<username>:<password>@twcservice.mybluemix.net/api/weather/v1/geocode/33.40/-83.42/almanac/monthly.json?start=01"