Haufe – Innovation Project
Summary A team from Microsoft, Haufe Group and Haufe Akademie set out to build a chatbot with
Microsoft LUIS for the website of Haufe Akademie.
Its main purpose is to interact with the customer and help him searching products on the website,
selecting training subject, date, location. With over 860 training subjects and more than 90.000
participants per year a technical solution is key. We focus on staying connected to our customer
24/7 - around the clock and globe.
Key technologies
• Microsoft Bot Framework
• Cognitive Services LUIS API
• Azure Search
• Azure Storage
• Azure Linux Virtual Machines
• Docker
• Jenkins
All Azure services and the Bot Framework have been programmed in Node.js.
Customer profile Haufe Akademie is one of the leading partners in the development of companies, professionals
and executives in German-speaking countries. We work with around 1,200 trainers and
consultants and provide more than 1,000 diverse topics across 23 fields with around 95,000
participants annually. All DAX 30 companies draw on Haufe Akademie’s expertise for their training
needs, thereby ensuring that they are prepared for the future. Haufe Academy is part of the Haufe
Group. The Group is widely regarded as one of the most innovative media and software
companies in Germany
Problem Statement Haufe Akademie offers their training portfolio through their website. As they offer various topics
of courses it is sometimes overwhelming for the customer to get the right one for the right date
and location. On the product landing page of a specific training they offer human chat support
but customers often don't reach those sites and the human support chat is limited to local
working hours. Here is where ACE - the chat bot is coming into play.
ACE will be available at any time as first contact chat for the customer. The customer can interact
with ACE and ACE will guide the customer to find the right trainings in the location and time
frame they're looking for. If the customer wants, ACE assists in connecting with a human support
buddy if they're logged in into the system.
Solution overview To enhance the user experience on the website we've decided to build a Bot based on the Bot
Framework and LUIS. In the first architecture session we declared the domain and determined the
necessary entities to interact with the Language Understanding Intelligence Service (a.k.a. LUIS).
This diagram shows the process of iterating through the domain
We've identified several entities but decided to go for the first sprint with those 3 entities
• Location
• Date
• Topic
Further we discussed how the bot will interact for the first sprint with the users and have defined
a basic workflow
Our architecture looks something like the diagram below. We have the bot framework app that
provides “channels” to talk to our bot through. One of these channels being the iframe that we
can embed in our website.
The framework app is configurable with the url (the endpoint has to be https!) of the actual bot
server. We used node.js and deployed the application as a docker container on an Azure VM. We
plan on migrating the container to the Azure Container Service.
The SDK allows for an easy setup of a connection to a luis.ai app. To make it work, one needs to
define the intents and entities that should be detected. An intent is basically the verb/action of
the sentence the user typed in, while an entity is a piece of information related to the intent. To
train Luis, we input as many “utterances” as we can think of - the more the better. This has to be
done for each supported intent. In our case, we only had to support “search” and “ask for a
human”. As entities, we have “course type”, “topic”, “datetime” and “location”.
Once the intent was detected as “search”, we simply used the entities as parameters for the Azure
Search Service. Since our data was a spreadsheed file, all we had to do in order to be able query it
was to export it as a csv file, upload it to an Azure Blob Storage and then use an Azure Search
Indexer to add it to our Azure Search Index.
Technical delivery Prerequisites Visual Studio Code (or any Editor of your choice)
We've chosen Visual Studio Code as Editor as it is cross-platform available and integrates great in
different programming languages. It has great Node.js support and debugging capabilities.
The programming langauge has been Node.js which we've used the latest stable version.
The dev machine should also install the Microsoft Bot Framework where you can find the
detailed documentation on how to install it and use the Node.js SDK for the Bot Framework.
Also install the Bot Framework Emulator as you want to test the bot reacting on user input on
your local dev box instead deploying it each time to test.
An active Azure subscription is needed create the Azure Search endpoint and the Azure Storage
endpoint. To create the Azure Search Service follow this instructions. To create the Azure Storage
endpoint follow this instructions How to create a storage account
Defining the LUIS Model The LUIS model we used for this project is pretty simple. It contains exactly two intents.
What was more of a challenge is that we're using LUIS in the German market and not each feature
of LUIS is already available. For instance we have no prebuilt domains we could use therefore we
had to define our one entities. This was especially challenging for the Date.
As this is just a matter of time we expect to use the prebuilt domains in future sprints. With that
set up we've began to enter utterances into the intents to define a good variety of training data
to LUIS. The following show the utterances for the search intent.
Implementing the Bot For the bot implementation we decided to go with LUIS Action Bindings. This makes it pretty
simple to react on certain intents and get easy resolving of the entities.
var SearchAction = { intentName: 'search', friendlyName: 'I want to search for a seminar', confirmOnContextSwitch: false, // allow to abandon this action without confirmation schema: { Location: { type: 'string', message: 'Wo soll das Seminar stattfinden?', optional: true }, StartDate: { type: 'string', message: 'Wann soll das Seminar stattfinden?', optional: true }, Topic: { type: 'string', message: 'Was ist das Thema des Kurses, den Sie suchen?', optional: false }, Type: { type: 'string', message: 'Wofur suchen sie nach?', optional: true }, }, fulfill: function (parameters, callback) { callback("") } }; module.exports = SearchAction;
This binding shows what happens if an entity is missing. For instance if the topic of the search is
missing, the binding automatically reacts and tries to ask the user for more information on this as
the optional parameter was set to false.
There was one caveat in using this bindings as they've been in the current state of the project.
You haven't been able to leverage any session related commands like Adaptive Cards as session
is not delegated into the binding. As of the time of this writing there is an open pull request
https://github.com/Microsoft/BotBuilder-Samples/pull/130 to bridge that gap. We've taken this
basic idea and implemented this into the core sdk in our project. That is the reason you just see
here the callback(""); trigger without parameters. It will be handled by the custom
implementation we did.
The custom implementation assumes that for each binding we'll have an handler. Handlers have
similarity to bindings.
var azureSearchHelper = require("../helpers/azureSearchHelper") var cardBuilderHelper = require("../helpers/cardBuilderHelper") var messageBuilderHelper = require("../helpers/messageBuilderHelper") var SearchHandler = { intentName: "search", handle: function (session, actionModel) { console.log("\nEntities found: " + JSON.stringify(actionModel.parameters) + "\n"); azureSearchHelper.azureSearchLuis(actionModel.parameters, function (result) { // console.log("\nFinal Result: \n" + JSON.stringify(result)); session.send(messageBuilderHelper.getSearchingMessage(actionModel.parameters, session)) if (!result || (result.length === 0)) { console.log("NO RESULT!") session.send('Ich konnte keine passenden Einträge für sie finden. Bitte versuchen sie es erneut.') } else { var reply = (cardBuilderHelper.getCarouselFromResult(result, session)) session.send(reply) session.send("Ich hoffe das Ergebnis ist für Ihnem hilfreich. Bitte fühlen Sie sich frei, eine neue Suchanfrage einzureichen.") }
session.endDialog(); }) } } module.exports = SearchHandler;
There we do the whole implementation of the result like in the above, searching for data and
creating an adaptive card carousel to output to the user in the webchat interface.
The binding and action will be triggered by the fulfillHandler based on the Pull Request of the bot
framework sample mentioned above.
var FulfillReplyHandler = function(session, actionModel) { let foundHandler = false; console.log('\nAction Binding "' + actionModel.intentName + '" completed:\n', actionModel, '\n'); AllHandlers.forEach(function(element) { if(actionModel.intentName === element.intentName) { foundHandler = true; element.handle(session, actionModel); } }, this); if(!foundHandler) { session.endDialog(actionModel.result.toString()); } }
Searching for data and displaying results The bot application is using helpers to retrieve data and display it. The data retrieval is a bit tricky
as the internal training data actually the catalog of trainings is in an internal ERP system which is
producing once a while a CSV list output. This output is stored on the blob storage endpoint
which is then used by the Azure Search service indexer which indexes all relevant CSV files.
This index is used by the helper to retrieve the data
require('dotenv-extended').load({ path: '../env' }); function azureSearch(searchParameters, callback) { // Setup azure-search var AzureSearch = require('azure-search'); var client = AzureSearch({ url: process.env.AZURE_SEARCH_URL, key: process.env.AZURE_SEARCH_KEY }); client.search(process.env.AZURE_SEARCH_INDEX, searchParameters, function (err, results) { if (err) { console.log("\nERROR:\n" + JSON.stringify(err) + "\n") } // console.log("\nAZURE SEARCH RESULTS:\n" + JSON.stringify(results) + "\n") callback(results) }); } function getAzureSearchEntity(parameters) { var Sugar = require("sugar-date"); Sugar.Date.setLocale('de'); var azureSearchEntity = { search: "", top: 200, queryType: "full" }
if (parameters.Topic) { azureSearchEntity.search += "((ThemenbereichUndThemenblock:" + filterKeywords(parameters.Topic) + ")" azureSearchEntity.search += "||(TitelUntertitel:" + filterKeywords(parameters.Topic) + "))" } if (parameters.Location) { azureSearchEntity.search += "&&(Ort:" + filterKeywords(parameters.Location) + ")" } if (parameters.StartDate) { var sugarDate = Sugar.Date.create(parameters.StartDate); var twoDigitDate = sugarDate.getDate(); if (twoDigitDate < 10) twoDigitDate = "0" + twoDigitDate; var twoDigitMonth = sugarDate.getMonth() + 1; if (twoDigitMonth < 10) twoDigitMonth = "0" + twoDigitMonth; azureSearchEntity.filter = "StartDatum ge " + sugarDate.getFullYear() + "-" + twoDigitMonth + "-" + twoDigitDate + "T00:00:00Z"; } console.log("AzureSearchEntity: " + JSON.stringify(azureSearchEntity) + "\n"); return azureSearchEntity } function combineResults(results) { var output = [] for (key in results) { var found = false; for (var key2 in output) { if (output[key2].Titel == results[key].Titel) { output[key2].SpaceTime.push(makeEntryLink(results[key])) found = true } } if (!found) { results[key].SpaceTime = [] results[key].SpaceTime.push(makeEntryLink(results[key])) output.push(results[key]) } } // console.log("\nOutput final: " + JSON.stringify(output)) return output } function makeEntryLink(entry) { var output = { "Location": entry.Ort, "StartDate": entry.Beginn, "AkaId": entry.Veranstaltungskennung, "webLink": "https://www.haufe-akademie.de/" + entry.Veranstaltungskennung } return output } function filterKeywords(topicString) { var temp = "" if (topicString.includes(" ")){ if(topicString.includes("und")){ temp+="/@" + topicString + "/@" temp+="&&/@" + topicString.replace(/ und /g, '/@&&/@') + "/@" } else{ temp += "/@" + topicString.replace(/ /g, '') + "/@" temp += "&&/@" +topicString.replace(/ /g, '/@&&/@') + "/@" } }
else{ temp = "/@" + topicString + "/@" } return temp } function azureSearchLuis(luisEntities, callback) { azureSearch(getAzureSearchEntity(luisEntities), function (azureSearchResults) { callback(combineResults(azureSearchResults)) }) } module.exports.azureSearchLuis = azureSearchLuis
Once the data is retrieved another helper is building the adaptive card outputs to build a nice
card carousel for the customer to iterate through. Here is the code of this helper.
var builder = require("botbuilder"); function getHeroCardFromEntity(entity, session) { var textMessage = "" // textMessage+="**Themenblock**: " + entity.Themenblock + "\n" // textMessage+="<b>Themenbereich\n: </b>" + entity.Themenbereich + "\n" textMessage += "Ort und Startdatum: " for (key in entity.SpaceTime) { if (key > 0) textMessage += ", " textMessage += entity.SpaceTime[key].Location + " " + entity.SpaceTime[key].StartDate } textMessage += ";\n" if (entity.Dauer1 && entity.Dauer2) { textMessage += " Dauer: " + entity.Dauer1 + " " + entity.Dauer2 + "\n" } return new builder.HeroCard(session) .title(entity.Titel) .subtitle(entity.Untertitel) .text(textMessage) .images([ builder.CardImage.create(session, 'https://www.haufe-akademie.de/images/header/logo_195x40.gif') ]) .buttons([ builder.CardAction.openUrl(session, 'https://www.haufe-akademie.de/' + entity.Veranstaltungskennung, 'Learn More') ]) } function getAddaptiveCardFromEntity(entity, session) { var temp = {} temp.contentType = "application/vnd.microsoft.card.adaptive" temp.content = {} temp.content.type = "AdaptiveCard" temp.content.body = [] temp.content.body.push({ "type": "Image", "size": "medium", "url": "https://www.haufe-akademie.de/images/header/logo_195x40.gif" }) temp.content.body.push({ "type": "TextBlock", "text": entity.Titel, "size": "extraLarge", "separation": "strong", "weight": "bolder" }) temp.content.body.push({ "type": "TextBlock", "text": entity.Untertitel, "size": "large",
"separation": "none", "weight": "lighter" }) var locdate = "" for (key in entity.SpaceTime) { if (key > 0) locdate += ", " var loc="" if (entity.SpaceTime[key].Location) loc = entity.SpaceTime[key].Location var date="" if (entity.SpaceTime[key].StartDate) date = entity.SpaceTime[key].StartDate locdate += loc + " " + date } temp.content.body.push({ "type": "TextBlock", "text": "**Ort und Startdatum:** " + locdate, }) if (entity.Dauer1 && entity.Dauer2) { temp.content.body.push({ "type": "TextBlock", "text": "**Dauer:** " + entity.Dauer1 + " " + entity.Dauer2, "separation": "none", }) } // temp.content.body.push({ // "type": "TextBlock", // "text": "**Suchergebnis score:** " + entity["@search.score"], // "separation": "none", // }) temp.content.actions = [] temp.content.actions.push({ "type": "Action.OpenUrl", "url": "https://www.haufe-akademie.de/" + entity.Veranstaltungskennung, "title": "Erfahren Sie mehr", }) return temp } function getCardsFromResult(result, session) { var cards = [] for (key in result) { cards.push(getAddaptiveCardFromEntity(result[key], session)) } return cards; } function getCarouselFromResult(result, session){ var cards = getCardsFromResult(result, session) // create reply with Carousel AttachmentLayout var reply if (!cards || (cards.length === 0)){ reply = "Ich konnte keine passenden Einträge für sie finden. Bitte versuchen sie es erneut." } else{ var cardText="" if (cards.length == 1) { cardText = "Ich habe einen eintrag gefunden, die Ihren Kriterien entsprechen:" } else { cardText = "Ich habe " + cards.length + " Einträge gefunden, die Ihren Kriterien entsprechen:" } reply = new builder.Message(session) .text(cardText)
.attachmentLayout(builder.AttachmentLayout.carousel) .attachments(cards); } return reply } module.exports.getCarouselFromResult=getCarouselFromResult
Deployment As already mentioned deployment is done through the Jenkins pipeline. In general the current
deployment strategy is to deploy the bot in an docker container and host that currently on a
Linux VM. Alternatives would be Azure Container Services or Azure Bot Service as hosting
environments. Due to the short time frame for the first sprint we leveraged this way as it was the
easiest for the project team.
Here is the dockerfile
FROM node:boron RUN mkdir -p /usr/src/app WORKDIR /usr/src/app COPY package.json /usr/src/app/ RUN npm install COPY app.js /usr/src/app ADD core /usr/src/app/core ADD handlers /usr/src/app/handlers ADD helpers /usr/src/app/helpers ADD actions /usr/src/app/actions EXPOSE 3978 CMD [ "npm", "start" ]
and the corresponding bash script
# !/bin/bash source .env set -e DOCKER_LOCAL=".docker/machine/" docker-machine -s ${DOCKER_LOCAL} ls #Checking if docker machine exists test_con=`docker-machine -s ${DOCKER_LOCAL} ls | grep ${VM_NAME}` #If docker machin doesn't exist create it if [ ${#test_con} -lt 2 ] ; then echo "----- Creating machine -----" docker-machine -s ${DOCKER_LOCAL} create --driver generic --generic-ip-address=${VM_IP} --generic-ssh-user=${VM_USER} --generic-ssh-key=${SSH_KEY_PATH} ${VM_NAME} fi export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://${VM_IP}:2376" export DOCKER_CERT_PATH="${DOCKER_LOCAL}/machines/${VM_NAME}" export DOCKER_MACHINE_NAME=${VM_NAME} #Build image on remote host docker build -t haufe-lexware/akachatbot . set +e docker stop $(docker ps -aq --filter name="akachatbot") docker rm -vf $(docker ps -aq --filter name="akachatbot") set -e
docker run -p 3978:3978 -e "MICROSOFT_APP_PASSWORD=${MICROSOFT_APP_PASSWORD}" -e "MICROSOFT_APP_ID=${MICROSOFT_APP_ID}" -e "LUIS_MODEL_URL=${LUIS_MODEL_URL}" -e "AZURE_SEARCH_URL=${AZURE_SEARCH_URL}" -e "AZURE_SEARCH_KEY=${AZURE_SEARCH_KEY}" -e "AZURE_SEARCH_INDEX=${AZURE_SEARCH_INDEX}" -d --name akachatbot haufe-lexware/akachatbot docker system prune -f
Conclusions We solved the problem to have a chat interface enabled for customers around the clock in
assisting them to find trainings and seminars by implementing a Bot using the Bot Framework
leveraging Language Understanding Intelligence Service and accessing the training data through
Azure Search to provide the results in a conversation with the customer through the chat interface
via the Bot.
With the use of the Bot Framework and LUIS it was pretty straightforward to create a base
architecture and a first implementation to enhance this bot application easily within the next
sprints. The current implementation took a couple of days from zero to a working version.
Additional resources This video shows the basic functionality of the bot. The bot itself just supports german, but the
video has english narration.
<iframe width="560" height="315" src="https://www.youtube.com/embed/X8SVBXhvzr8"
frameborder="0" allowfullscreen></iframe>
Team To provide a proof-of-concept implementation, a team worked together in May 2017, each
focusing on a different part of the chatbot implementation:
• Ioan-Bogdan Cimpoesu, Haufe Gruppe
• George Ganea, Haufe Gruppe
• Dariusz Parys, Microsoft
• Larissa Gruner, Haufe Akademie