Showing “Get started” button in SUSI Viber bot

When we start a chat with SUSI.AI on Viber i.e. SUSI Viberbot, there should be an option on how to get started with the bot. The response to it are some options like “Visit repository”, “How to contribute” which direct the user to check how SUSI.AI bot is made and prompts him/her to contribute to it. Along with that an option of “start chatting” can be shown to add up some sample queries for the user to try.

To accomplish the task at hand, we will accomplish these sub tasks:

  1. To show the “Get started” button.
  2. To show the reply to “Get started” query.
  3. To respond to the queries, nested in the response of “Get started”

Showing “Get started”:

The Viber developers platform notifies us when a user starts a conversation with our bot. To be exact, a conversation_started event is sent to our webhook and can be handled accordingly. The Viberbot shows a welcome message to the user along with a Get started button to help him/her start.

To send just the welcome message:

if (req.body.event === 'conversation_started') {
       // Welcome Message
       var options = {
           method: 'POST',
           url: 'https://chatapi.viber.com/pa/send_message',
           headers: headerBody,
           body: {
               // some required body properties here
               text: 'Welcome to SUSI.AI!, ' + req.body.user.name + '.',
               // code for showing the get started button here.
        }
           json: true
       };
 
       request(options, function(error, res, body) {
           // handle error
       });
   }

The next step is to show the “Get started” button. To show that we use a keyboard tool, provided by Viber developers platform. So after the “text” key we have the “keyboard” key and a value for it:

keyboard: {
             "Type": "keyboard",
             "DefaultHeight": true,
             "Buttons": [{
                 "ActionType": "reply",
                 "ActionBody": "Get started",
             }]
         }

The action type as shown in the code, can be “reply” or “open-url”. The “reply” action type, triggers an automatic query sent back with “Get started” (i.e. the value of “ActionBody” key), when that button gets clicked.

Hence, this code helps us tackle our first sub task:

Reply to “Get started”:

We target to make each SUSI.AI bot generic. The SUSI FBbot and SUSI Tweetbot shows some options like “Visit repository”, “Start chatting” and “How to contribute?” for the “Get started” query. We render the same answer structure in Viberbot.

The “rich_media” type helps us send buttons in our reply message. As we ought to use three buttons in our message, the button rows are three in the body object:

if(message === "Get started"){
                   var options = {
                       method: 'POST',
                       url: 'https://chatapi.viber.com/pa/send_message',
                       headers: headerBody,
                       body: {
                           // some body object properties here
                           type: 'rich_media',
                           rich_media: {
                               Type: "rich_media",
                               ButtonsGroupColumns: 6,
                               ButtonsGroupRows: 3,
                               BgColor: "#FFFFFF",
                               Buttons: buttons
                           }
                       },
                       json: true
                   };
 
                   request(options, function(error, res, body) {
                       if (error) throw new Error(error);
                       console.log(body);
                   });

As said before, 2 type of Action types are available – “open-url” and “reply”. “Visit repository” button has an “open-url” action type and “How to contribute?” or “start chatting” has a “reply” action type.

Example of “Visit repository” button:

var buttons = [{
                Columns: 6,
                Rows: 1,
                Text: "Visit repository",
                "ActionType": "open-url",
                "ActionBody": "https://www.github.com/fossasia/susi_server",
                // some text styling properties here
             }];

To respond to the “reply” action type queries:

When the “reply” action type button gets clicked, it triggers an automatic query sent back to the bot with the value same as that of the “ActionBody” key. So we just need to apply a check if the message string recieved is “Start chatting” or “How to contribute?”

For the response to “Start chatting”, we plan to show sample queries for the user to try. This can be shown by using buttons with the action type as “reply”.

Code snippet to show a button with the text as “What is FOSSASIA?”:

var buttons = [{
                        Columns: 6,
                        Rows: 1,
                        Text: "What is FOSSASIA? ",
                        "ActionType": "reply",
                        "ActionBody": "What is FOSSASIA?",
                        // text styling here
                    }];

For the response to “How to contribute”, we show some messages to help the user contribute to SUSI.AI. These messages also just need buttons with it, to be able to apply the necessary action.

We respond with 2 messages to the user, both the messages have a button.

For example, a button to visit the SUSI.AI Gitter channel:

var buttons = [{
                    Columns: 6,
                    Rows: 1,
                       Text: "<font color=#323232><b>Chat on Gitter</b></font>",
                      ActionType: "open-url",
                      ActionBody: "https://www.gitter.im/fossasia/susi_server",
                      // text styling here
            }];

This way we have successfully added the “Get started” option to our Viberbot and handled all the subsequent steps.

Resources:

  1. Viber video managing chat extensions by Ingrid Lunden from Tech crunch.
  2. Develop a chat bot with node js by Slobodan Stojanović from smashing magazine.

Showing location response in SUSI.AI bots

SUSI.AI has a capability to tell the basic information about a location, it is asked for. Along with the basic information about that place, it shows a map (i.e. open street map) pointing to that location. The task at hand is to inculcate this “location” feature to the SUSI.AI messenger bots. The SUSI Tweetbot and SUSI Fbbot are used as examples in this blog.

Let’s first check on what response do we get, when a person asks a query like “where is london” to the SUSI API. Along with the basic information about that location, the result as shown below has the type of reply (i.e. map), latitude, longitude and a link to the open street map.

"actions": [
      {
        "type": "answer",
        "language": "en",
        "expression": "Ludhiana is a city and a municipal corporation in Ludhiana district in the Indian state of Punjab, and is the largest city north of Delhi."
      },
      {
        "type": "anchor",
        "link": "https://www.openstreetmap.org/#map=13/30.912040570244187/75.85379021980509",
        "text": "Here is a map",
        "language": "en"
      },
      {
        "type": "map",
        "latitude": "30.912040570244187",
        "longitude": "75.85379021980509",
        "zoom": "13",
        "language": "en"
      }
    ]

The response for a location type query has these 3 main parts:

  1. Clickable static map image.
  2. A basic information of the place asked about.
  3. The link i.e. when the static map image is clicked it should redirect to the corresponding map location.

Let’s try to make up with the 1st part of the response i.e. Static map image.

The map quest API is used to result in a static map image of the location. We need an API key to access the map quest API, which can be requested from their developer site.

Along with that we need the latitude and longitude of the location at hand.

The SUSI API’s response helps us to get the latitude value:

// if body represents the response object 
var lat = body.answers[0].actions[2].latitude;

And the longitude value:

var lon = body.answers[0].actions[2].longitude;

Using the three values that are API key, latitude and longitude, the static image is rendered by this link:

var static_image_url = "https://open.mapquestapi.com/staticmap/v4/getmap?key=API_KEY&size=600,400&zoom=13&center="+lat+","+lon;

The second part is, basic information about the place asked, can be fetched from:

// if body is the JSON response object from SUSI API
var mapMessage = body.answers[0].actions[0].expression;

The link to the map location can be easily fetched from the SUSI API’s response:

var link = body.answers[0].actions[1].link;

As all the three parts are ready, let’s look on how to render them on the SUSI.AI bot’s screen.

Facebook:

Sending a generic template message object:

message = {
        "type":"template",
        "payload":{
                    "template_type":"generic",
                    "elements":[{
                        "title": mapMessage,
                       "image_url": static_image_url,
                       "Item_url": link
                    }]
        }
};

sendTextMessage(sender, message, 1);

Twitter:

The Twitter API does not need a static image of the map to be rendered. It does that work for us. We just need to send an event to the Twitter API with the message data object constituting of the map message, the latitude and longitude values:

"message_data": {
            "text": mapMessage,
            "attachment": {
                "type": "location",
                "location": {
                    "type": "shared_coordinate",
                    "shared_coordinate": {
                        "coordinates": {
                            "type": "Point",
                            "coordinates": [lon, lat]
                        }
                    }
                }
            }
}

Resources:

  1. Speed up customer service with quick replies and welcome messages by Ian Cairns from Twitter blog.
  2. Drive discovery of bots and other customer experiences in direct messages by By Travis Lull from Twitter blog.
  3. By Seth Rosenberg from Facebook developers blogLink Ads to Messenger, Enhanced Mobile Websites, Payments and More.

How to Get Secure Webhook for SUSI Bots in Kubernetes Deployment

Webhook is a user-defined callback which gets triggered by any events in code like receiving a message from a user in SUSI bot is an event. Few bots need webhook URI for callback like in SUSI Viber bot we need to define a webhook URI in the code to receive callbacks and make our Viber bot work. In this blog, we will learn how can we get an SSL activated webhook while deploying our bot to Google container using Kubernetes. We will generate SSL certificate using kube lego service that is included in kubernetes and you will define that in yaml files below. We can also generate SSL certificate using third party services like CloudFlare but by using it we will be dependant on CloudFlare so we will use kube lego.

We will start off by registering a domain first on which we will activate SSL certificate and use that domain as a webhook. Go to freenom and register your account. After logging in, register a free domain of any name and check out that order. Next, you have to set IP for DNS of this domain. To do so we will reserve an IP address in our Google cloud project with this command:

gcloud compute addresses create IPname --region us-central1

You will get a created message. To see your IP go to VPC Network -> External IP addresses. Add this IP to DNS zone of your domain and save it for later use in yaml files that we will use for deployment. Now we will deploy our bot using yaml files but before deployment, we will create a cluster

gcloud container clusters create clusterName

After creating cluster add these yaml files to your bot repository and add your IP address that you have saved above to the yamls/nginx/service.yaml file for “loadBalancerIP” parameter. Replace domain name in yamls/application/ingress-notls.yaml and yamls/application/ingress-tls.yaml with your domain name that you have registered already. Add your email ID to yamls/lego/configmap.yaml for “lego.email” parameter. Replace “image” and “env” parameters in yamls/application/deployment.yaml with your docker image and your environment variables that you are using in your code. After changing yaml files we will use this deploy script to create a deployment. Change paths for yaml files in script according to your yaml files path.

In gcloud shell run the following command to deploy an application using given configurations.

bash ./path-to-deploy-script/deploy.sh create all

This will create the deployment as we have defined in the script. The Kubernetes master creates the load balancer and related Compute Engine forwarding rules, target pools, and firewall rules to make the service fully accessible from outside of Google Cloud Platform. Wait for a few minutes for all the containers to be created and the SSL Certificates to be generated and loaded.

You have successfully created a secure webhook. Test it by opening the domain that you have registered at the start.

Resources

Enabling SSL using CloudFlare: https://jonnyjordan.com/blog/how-to-setup-cloudflare-flexible-ssl-for-wordpress/
https://www.youtube.com/watch?v=qFvwEVkl5gk

Skill Editor in SUSI Skill CMS

SUSI Skill CMS is a web application built on ReactJS framework for creating and editing SUSI skills easily. It follows an API centric approach where the SUSI Server acts as an API server. In this blogpost we will see how to add a component which can be used to create a new skill SUSI Skill CMS.

For creating any skill in SUSI we need four parameters i.e model, group, language, skill name. So we need to ask these 4 parameters from the user. For input purposes we have a common card component which has dropdowns for selecting models, groups and languages, and a text field for skill name input.

<SelectField
    floatingLabelText="Model"
    value={this.state.modelValue}
    onChange={this.handleModelChange}
>
    {models}
</SelectField>
<SelectField
    floatingLabelText="Group"
    value={this.state.groupValue}
    onChange={this.handleGroupChange}
>
    {groups}
</SelectField>
<SelectField
    floatingLabelText="Language"
    value={this.state.languageValue}
    onChange={this.handleLanguageChange}
>
    {languages}
</SelectField>
<TextField
    floatingLabelText="Enter Skill name"
    floatingLabelFixed={true}
    hintText="My SUSI Skill"
    onChange={this.handleExpertChange}
/>
<RaisedButton label="Save" backgroundColor="#4285f4" labelColor="#fff" style={{marginLeft:10}} onTouchTap={this.saveClick} />

This is the card component where we get the user input. We have API endpoints on SUSI Server for getting the list of models, groups and languages. Using those APIs we inflate the dropdowns.
Then the user needs to edit the skill. For editing of skills we have used Ace Editor. Ace is an code
editor written in pure javascript. It matches the features native editors like Sublime and TextMate.

To use Ace we need to install the component.

npm install react-ace --save                        

This command will install the dependency and update the package.json file in our project with this dependency.

To use this editor we need to import AceEditor and place it in the render function of our react class.

<AceEditor
    mode=" markup"
    theme={this.state.editorTheme}
    width="100%"
    fontSize={this.state.fontSizeCode}
    height= "400px"
    value={this.state.code}
    name="skill_code_editor"
    onChange={this.onChange}
    editorProps={{$blockScrolling: true}}
/>

Now we have a page that looks something like this

Now we need to handle the click event when a user clicks on the save button.

First we check if the user is logged in or not. For this we check if we have the required cookies and the access token of the user.

 if(!cookies.get('loggedIn')) {
            notification.open({
                message: 'Not logged In',
                description: 'Please login and then try to create/edit a skill',
                icon: <Icon type="close-circle" style={{ color: '#f44336' }} />,
            });
            return 0;
        }

If the user is not logged in then we show him a error notification and asks him to login.

Then we check if he has filled all the required fields like name of the skill etc. and after that we call an API Endpoint on SUSI Server that will finally store the skill in the skill_data_repo.

let url= “http://api.susi.ai/cms/modifySkill.json”
$.ajax({
    url:url,
    dataType: 'jsonp',
    jsonp: 'callback',
    crossDomain: true,
    success: function (data) {
        console.log(data);
        if(data.accepted===true){
            notification.open({
                message: 'Accepted',
                description: 'Your Skill has been uploaded to the server',
                icon: <Icon type="check-circle" style={{ color: '#00C853' }} />,
            });
           }
    }
});

In the success function of ajax call we check if accepted parameter is true from the server or not. If accepted is true then we show user a notification with a message that “Your Skill has been uploaded to the server”.

To see this component running please visit http://skills.susi.ai/skillEditor.

Resources

Material-UI: http://www.material-ui.com/

Ace Editor: https://github.com/securingsincity/react-ace

Ajax: http://api.jquery.com/jquery.ajax/

Universal Cookies: https://www.npmjs.com/package/universal-cookie

Enhancing SUSI Line Bot UI by Showing Carousels and Location Map

A good UI primarily focuses on attracting large numbers of users and any good UI designer aims to achieve user satisfaction with a user-friendly design. In this blog, we will learn how to show carousels and location map in SUSI line bot to make UI better and easy to use. We can show web search result from SUSI in form of text responses in line bot but it doesn’t follow design rule as it is not a good approach to show more text in constricted space. Users deem such a layout as less attractive. In SUSI webchat, location is returned with a map which looks more promising to users as illustrated below:

Web search is returned as carousels and is viewable as:

We can implement web search in line by sending an array of text responses. We can do this with the code below:

for (var i = 0; i < 5; i++) {
   msg[i] = "";
   msg[i] = {
       type: 'text',
       text: 'text to be sent here'
   }
}
return client.replyMessage(event.replyToken, msg);

If we send web search using text response it looks messy, a user will not like it that much as it is difficult for a user to read and understand a lot of text in one place. You can see it in the screenshot below:

To make it better looking we will implement carousels for web search in SUSI line bot. Carousels are tiles that can be scrolled horizontally to show rich content. We can implement carousels using this code:

for (var i = 1; i < 4; i++) {
   title = 'title of carousel';
   query = title;
   msg = 'description of carousel here';
   link = 'link to be opened here';
   if (title.length >= 40) {
       title = title.substring(0, 36);
       title = title + "...";
   }
   if (msg.length >= 60) {
       msg = msg.substring(0, 56);
       msg = msg + "...";
   }
   carousel[i] = {
       "title": title,
       "text": msg,
       "actions": [{
               "type": "uri",
               "label": "View detail",
               "uri": link
           },
           {
               "type": "message",
               "label": "Ask SUSI again",
               "text": query
           }
       ]
   };
}

In above code, we are checking the length of title and description because line API has a limit for the title up to 40 characters and description up to 60 characters. If limit exceeds we then trim the title or description accordingly and use it. Next, we assign title, description, and link to be opened in action to carousel so that we can use it in below code.

const answer = [{
       type: 'text',
       text: ans
   },
   {
       "type": "template",
       "altText": "this is a carousel template",
       "template": {
           "type": "carousel",
           "columns": [
               carousel[1],
               carousel[2],
               carousel[3]
           ]
       }
   }
]
return client.replyMessage(event.replyToken, answer);

Above code shows an array names answer in which we have added carousels that we have created in last code. After implementing this web search will look like this:

This UI looks neat and easy to read and users will like this. Ask SUSI again will send the title of the carousel to SUSI. To show location map on SUSI line bot just like it is shown on SUSI webchat we will follow this code:

const answer = [{
       type: 'text',
       text: ans
   },
   {
       "type": "location",
       "title": "name of the place here",
       "address": address,
       "latitude": latitude of location here,
       "longitude": longitude of location here
   }
]
return client.replyMessage(event.replyToken, answer);

We will send a location type message which will include latitude and longitude of the location so that Line bot can show location using map. After implementing location type response it will look like this:

On clicking on this location it will open a map inside line app on which we will see a pointer pointing at the location that we have provided.

If you want to learn more about line messaging API refer to this https://devdocs.line.me/en/#messaging-api

Resources
https://devdocs.line.me/en/#messaging-api