Check our Ultimate Chatbot Guide
Technical Questions You Should Ask a Chatbot Company. Part I

Technical Questions You Should Ask a Chatbot Company. Part I

Michał Pawełczyk 21.03.2019
Michał Pawełczyk

From curiosities to effective communication tools – that’s how the perception of chatbots has changed. We can find them after scanning a QR code from a beer bottle, after transferring money in a mobile banking app, or even when playing an interactive game that is combined with a TV show (a little spoiler, more about it soon) ? In such cases, chatbot malfunctions can be very expensive. Every minute of silence can cause financial loss and reputational damage. Even the biggest brands can have technical problems. Although we can never avoid errors completely, we can limit them if we choose the right chatbot provider and ask questions about security.

Fault-tolerant Architecture

Just like online shops or websites, chatbots are computer programs that run on a server. Chatbots are quite easy to use – they’re like a power socket, to which we can plug different tools. For instance, if we plug Messenger, the chatbot will inform you about different user actions, such as sent messages. After the chatbot receives a user’s message, it creates a response and sends it back. Only as much, and yet so much. Unfortunately, many processes can fail along the way.


external API

The system has to process messages with the help of the OCR (Optical Text Recognition) module, download and save users’ sessions, and perform many other activities that depend on individual cases.

A well designed system will deal with partial errors. That’s why it’s so important to divide the chatbot platform into independent boxes that talk with each other (to insiders – yes, I’m talking about microservices architecture). If a given component fails, for example, the one that is responsible for submitting contest entries, the chatbot will still be able to properly respond to users.

This brings us to the first important question. Is the implemented solution fault-tolerant? If yes, then what faults is it tolerant to? What are the possible failure scenarios?

Performance is the Key to Success

Scalability is a complex subject that software companies have to deal with. Saying “It’s going to be fine” isn’t enough. You also need tests that simulate server load. They are carried out by defining the so-called use cases.

If we talk about chatbots, use cases can include button clicks or sent messages. Defining the right use cases is essential. Why? For example, you can easily overlook the fact that the communicator sends, besides notifications about new user activities, lots of event blocks (such as notifications about sent messages). And this can have a negative impact on the performance of your application.

The test should verify how the system responds to the constantly flowing stream of information about:

  • button clicks (1000 times),
  • received messages (1000), part of which got recognized by the NLP module,
  • notifications about sent messages (1200),
  • messages that have been read by users (1000).

Thanks to the simulation, we can learn, for example, about the chatbot response time, RAM usage, CPU load. With this knowledge, the chatbot provider can work on developing and improving the tool. At the same time, you gain a greater sense of security, knowing that the chatbot won’t let you down at crucial moments.

Now we know what load your chatbot can handle. Let’s add another circumstance to our scenario. What will happen if we have a load (such as the one from the example above), and at the same time, we send push notifications to 100 000 users? Will the chatbot response time remain the same or will it change?

Thanks to performance tests, we can answer these questions. That’s why you should always remember to mention about them when talking with the chatbot company.

Why is everybody talking about chatbots?
Because brands who use them in business are getting results
Let's talk

Monitoring in Case of Emergencies

It’s happened! For the last 10 minutes, the chatbot has been silent despite our performance tests and security strengthening. The Murphy’s Law works. Time is running out, people keep sending messages, yet there are no changes on the horizon.The IT department already knows about the problem and is working on the solution. It was immediately informed about the malfunction because our diagnostics tool found out what’s going on in the computer (the so-called health check). A few minutes later, everything is back to normal. It turns out that the server hard disk has failed, so no additional repairs are necessary. A public cloud has detected the error and moved the chatbot to another server. The error was easy to find and the only thing the administrator had to do was to check if everything works after restarting the system. Unfortunately, in reality, it rarely happens that the problem gets fixed so fast. More often, we have to look through computer crash records in order to diagnose the problem, and this takes time. That’s why it’s essential to have a properly configured system that warns about failures of specific computer modules.

Play it safe. Ask the chatbot provider whether he uses diagnostics tools and how much time does it take him to correct a critical error. This knowledge will be very helpful in case of any emergency.

A Chatbot is not Different from Computer Systems

IT companies have to deal with a lot of technical chatbot problems in order to make you feel comfortable and secure. That’s why you should pay special attention to such issues. But this is only the beginning.

Equally important are questions related to:

  • data security,
  • integration with other computer systems – after all, the chatbot is not a lonely island but a part of a larger whole,
  • admin dashboard requirements.

But more about it in part II ?