Ola Bilińska 03.10.2019

How we built our first Google Assistant – Semilac case study

The popularity of voice assistants is growing day by day, just as the number of tools used to build them. As soon as Dialogflow started to support the Polish language, we decided to test it with our own Google Assistant. This led us to creating the first voice assistant for Semilac. If you’re curious how we did this, you’re in the right place.

History has a way of repeating itself. Just over a year ago, we cooperated with Semilac and I had the opportunity to launch one of my first chatbots. Today, I’m going to tell you how we created our first voice assistant for the same brand. I’ve been watching for over a year how the incredibly engaged Semilac community gets excited about new chatbot features. Having drawn conclusions from their feedback, we could go a step further because we knew what Semilac clients like and expect. Even the most in-depth and extensive research wouldn’t give us as much as being involved in the life of the brand and staying in touch with the target group.

However, it doesn’t mean that I immediately got down to work. Even though researching wasn’t that important and necessary in the case of this brand, I had to explore certain tools. Because this would allow me to find out how Semilac could use the voice assistant to meet the needs of users.

What is Google Assistant?
Google Assistant is a virtual voice assistant developed by Google. It’s available on mobile and smart home devices that are used by millions of households from all over the world. 

How to start my adventure with Google Assistant?

In the first step, I devoted some time to learn the following tools – DialogFlow and Google Actions. The former allows you to construct conversations with the help of NLP (Natural Language Processing). The latter lets you add these conversations to your Google Assistant.

What are Google Actions?
Actions on Google is a software framework that allows you to extend the functionality of Google Assistant, which is available on over 500 million devices, including smart speakers, phones, cars, TVs, headphones, watches, etc.

I’m not saying it was easy to learn the tools. Working on the KODA Bots platform, created by our developers who love programming as much as UX, is very comfortable. It was more difficult with Dialogflow because I’ve never worked with it before and I had to learn how to use it. I don’t want you to get me wrong – Google, of course, has highly skilled specialists. But Google Assistant, just like the platforms we’re currently using to build it, need some improvement.

Semilac is a Polish brand that is dynamically expanding to foreign markets. That’s why we immediately knew we wanted to make the Assistant bilingual. We started with a Polish version and a slightly simplified English version, which we’re planning to extend in the nearest future.

Together with our developers, we learned how to use both tools to be able to show our client what we can design. We decided to start with the basics, that is FAQs and inspirations, as they’ve already been working well inside the Semilac chatbot. What’s more, we added Google Maps to let clients find the nearest Semilac stores. We also included some extra features, like interesting facts about nails and the history of the brand.

Intencje w Asystencie Google
Intents inside the Google Assistant based on nail stylization inspirations

If you’re a woman, you have to admit how uncomfortable it is to use a phone and polish nails at the same time. The main goal of our project was to create a practical tool, with which girls could quickly find answers to questions without damaging their freshly painted nails. Have you ever had your gel wrinkle during nail polishing? Instead of typing your question (and damaging your manicure), it’s much easier to ask a voice assistant.

 

Whether we paint our nails at home or a nail studio, our hands are always occupied so we can’t surf the Internet to find the information we need. The bot frees the hands of our clients, responds to their questions with voice, and shortens the response time so that they don’t have to wait to get answers. Voice marketing is the future! That’s why brands are so eager to use voice supported tools to build relations and to maintain regular and “real” contact with clients. The challenge they have to face is to produce functional and smart solutions that will keep pace with the constantly and dynamically changing consumer reality.

Joanna Przybylska, Innovation Manager at Semilac

What will people actually say? 

This was the key question we asked ourselves before starting our research. People speak in a completely different way than they write (and believe me, we experienced it firsthand when we launched the Assistant). 

  • Spoken language is much more straightforward.
  • People use the voice assistant to find different information than in a Messenger chatbot (to which they’re usually directed from a Facebook fan page).

Despite its rising popularity, Google Assistant is still new when compared to Messenger, the most frequently used communication app in Poland. This affected not only on the scope of the content we added to the Assistant but also the way in which users can ask questions.

The challenge

Considering the differences between how people speak and write, transforming a textbot directly into a voicebot isn’t necessarily the best idea. So our approach was to “think with voice”. This perspective laid a sound foundation for our first Google Assistant project.  

Pro tip: A detailed and precise voice script helps improve user experience. The script should be simple, specific, and straightforward so that it could better match answers to questions asked by users. 

Putting the Google Assistant into practice

Inside Dialogflow, we defined a percent probability so that the system could determine whether users ask questions related to a specific topic. The lower is the percentage, the more likely it is that the Assistant assigns an answer to a question. But the risk of making a mistake also increases. Remember about this especially in the beginning, when the scope of the content you publish inside the Assistant isn’t yet that wide. Most probably, it’ll quickly turn out that the percentage of matching answers, which is set as default by Google, will be too low. And even if it doesn’t come to this during in-house tests (when we know how and what to look for), it’ll come to the surface during the first external tests.

Encje w Asystencie Google
Examples of entities inside Semilac’s Google Assistant

We started by adding dozens of questions, which the Semilac Assistant was able to answer. Thanks to this, the percentage of matching answers only slightly increased. Fortunately, after we had launched the tool and showed it to our users, we didn’t have to increase the percentage. Soon we’ll be able to let the Assistant match answers with more freedom.

After we gained some knowledge and chose a course of action, we could start adding content. We knew we had a lot of work to do when it came to choosing elements, as they were supposed to display not only simple text messages but also inspiration ideas with photos, nail stylization descriptions, and buttons that would redirect users to Semilac’s online store.

Remember that if you’re creating a Google Assistant that’s supposed to listen to users after given elements are displayed, every small dialog flow has to end with a question, which suggests users what they can do next. 

Pro tip: It’s also worth displaying suggestions in the form of buttons. They look similar to chatbot quick replies, except that we can’t choose the place to which users can be directed. So when we click on a button, the Assistant matches its content with its own database and displays the most probable element.

Why is everybody talking about chatbots?

Because brands who use them in business are getting results
Let's talk about how it can work for you

I usually used Google Assistant Simple Response instead of Default Text Response to enter text messages. It’s because Google Assistant Simple Response has a practical option called “Customize Audio Output”. Our Assistant has many long sentences with advisory content. So with the option to customize audio output, I could divide the text into parts that would be either displayed on the screen or read out loud. The content was similar but different in length.

An example of how we customized answers in the Semilac Google Assistant
An example of how we customized answers in the Semilac Google Assistant

We can define how Google Assistant selects answers to questions by entering Training phrases. Dialogflow’s Training tab is a very useful feature. It enables us not only to look through the conversation history between users and our Assistant but also to check whether it has properly reacted to given messages. 

  • If the Google Assistant’s reaction is correct, we mark the answer as green. 
  • If the Google Assistant’s reaction is incorrect, we can choose any other response that we’ve entered into the bot.
Trenowanie fraz w Asystencie Google
Training phrases in Google Assistant

Thanks to this, we don’t have to search for intents from our database and enter new phrases on our own. After the answers are marked as either correct or incorrect, the words users have spoken will be automatically moved to Training phrases. Trust me – when you have thousands of Google Assistant users, this saves an enormous amount of time. 

Because I added content both in Polish and English, I could test several features that aren’t yet available in the Polish version of Dialogflow. One of these features is small talk – a tab where we have several dozens of default questions and utterances, which, according to Google, are frequently used during conversations with the Assistant. This is the place where we should add sample dialogs that fit the image of our brand. As I’ve already mentioned – when people talk with a voice chatbot, they’re much more straightforward and they’re much more willing to test its “intelligence” than when they exchange regular text messages. So it’s worth adding responses to questions like “sing something” right at the beginning.

Small Talk w Asystencie Google
Small Talk inside the Google Assistant

Integrations with Google Assistant

Users can find nail stylization inspirations and the nearest Semilac stores. We also integrated the voicebot database with shops and nail stylizations than can be found on the brand website but are different from those that are displayed in the Semilac chatbot.

The Assistant is able to detect different types of stylizations (manicure, pedicure, or make-up) and collect data from the defined categories. We didn’t use any magic tricks here, we just adopted already available solutions. We had to precisely define various types of entities (the data that is essential to complete the user’s request) for stylization categories and their correspondent synonyms (for example, mani or pedi). Thanks to this, users receive selected stylizations from the chosen category.

We used the “share location” option when we integrated Semilac shop locations. To match the voicebot’s answer with a user’s query, we download the user’s coordinates through Google, compare them with the store location database, and determine the distance between the user and a given store.

Pro tip: It’s worth focusing on the analysis of variables before you move on to programming. There’s a simple reason for this. A Messenger chatbot has blocks that consist of different elements, like titles, URLs, images, and descriptions that may vary from those inside a voice Assistant when a text is either read or displayed. If you synchronize this data, creating elements in a voicebot will be much easier because you won’t have to make changes during the later stages of your project.

Google Assistant – focusing on development

The voicebot we designed for Semilac is available on Google Home devices as well as on smartphones. It was a difficult but also challenging process. The tools we used are still available in beta versions. Unfortunately, we experienced this firsthand when Google made updates and interrupted our plans. It was a challenging road but we’re very pleased with the results. The biggest reward is to see the satisfaction of users who are eager to test our tool. At the same time, their feedback gives us the opportunity to improve and update content. Although the Semilac Assistant is already equipped with quality content and integrated with external systems, it’s only the beginning of its journey. Richer with the new experience, with heads full of ideas, we planned what features to add in the upcoming months. So stay tuned because we’re just getting started 😉

 

Such an innovative approach is ingrained in the DNA of Semilac. We’ve already successfully launched a mobile app, chatbot, an AR effect on Facebook, and product personalization. So, in a sense, the Google voice assistant is a way of preparing ourselves for artificial intelligence, which we’re planning to use in the nearest future. 

Joanna Przybylska, Innovation Manager at Semilac

 

You can use the Semilac voice assistant on smartphones and tablets with the Android 5.0 version and higher that are compatible with Google Play. You can also use it on iOS devices if you install the Google Assistant app from the App Store. To start your conversation with the voicebot, just say „Talk to Semilac”.

Key conclusions we’ve drawn after releasing the Google Assistant

  • Using voice to control a text chatbot doesn’t simply work on the ON/OFF principle. If you’re converting a regular chatbot to a voicebot, follow the “voice first” rule.
  • We don’t speak the same way we write. So don’t use the same dialog because you’ll have to work twice as hard to adjust it later on. 
  • It’s absolutely necessary to make your product easy to use. Remember that less is more. This rule should be followed especially during the first stage of voicebot design.
  • It’s crucial to inform users during conversations about the features of your voicebot.  After every answer, you should inform people what they can do next.

Do you like this article? Share it!

Comments

Check other articles Oli

Ultimate chatbot guide Guide to chatbots world