RiveScript's Original Niche

I wrote this article for the RiveScript Community Wiki, but am reposting it here for visibility.

I've been noticing more and more lately that people are using RiveScript to power Facebook Messenger chatbots, which adds a whole lot of complexity that RiveScript wasn't ready for. This article explains why RiveScript was designed the way that it is, what it's doing to support modern chatbots, and recommendations for how to design a modern chatbot.

Some of RiveScript's design decisions were driven by the original niche, or use case, that RiveScript was originally written to fill. This niche is the same as the one that SmarterChild filled in the early 2000's, as well as most of the A.L.I.C.E. Bots from around the same era.

Today, modern chatbots work differently than those that came before. Facebook Messenger bots, for example, expect to communicate with a web server, where a message from a user causes Facebook to ping a "web hook URL" on the bot author's server which returns a response for the user. RiveScript was not originally designed with this use case in mind, because the sort of bot that RiveScript was designed for didn't work in this way.

Classic Chatbots

I'll refer to the SmarterChild and Alicebot types as "Classic Chatbots" for the sake of this article.

Classic chatbots were usually clients to an instant messenger platform, such as AOL Instant Messenger or MSN Messenger. They could also be clients for IRC servers or other similar systems.

Classic bots would typically run as one process on a computer somewhere, much like an Instant Messenger client would run as one process for an end user to access the service. You'd run a program or open a terminal window to start the bot, and the bot would be a long-running task on that computer; when the program was killed by the owner or if it crashed, the bot would immediately disconnect from its instant messenger services.

In-Memory User Variables in RiveScript

This approach to running the classic chatbots influenced the way RiveScript handles user variables.

By default, RiveScript keeps all variables about the users who interact with it in system memory (RAM), i.e. by putting them into a dictionary variable. When the bot first starts up, its memory of users is blank, and as people chat with it, it can start temporarily remembering variables about the users, such as their name, age, or the recent replies that the bot had sent back to them.

RiveScript supports methods like get_uservars(), which "exports" a copy of RiveScript's memory to the programmer, so that they could store the variables on the hard disk or put them in a MySQL database for long term storage. And RiveScript supports methods like set_uservars(), which allows the programmer to take that stored long-term user data and put it back into RiveScript's active memory. This way, a RiveScript bot could export its data when it shuts down and then recall it again when it starts back up. The JavaScript implementation has an example that does just this.

This system of keeping temporary working memory during the lifetime of the single bot process and providing ways to export and import them were perfectly suited for running classic chatbots. The entire bot personality was controlled by one single process running on a single computer somewhere on the Internet, and it didn't require the user to muck about with MySQL or Redis or any other database engine that might have been an obstacle for new users who just wanted to run their own chatbot.

Modern Chatbots

I'll refer to the new wave of chatbots (for the likes of Facebook Messenger, Microsoft Bot Framework, and Twilio) as "modern chatbots."

With some modern chatbot frameworks, the bot maker builds a web service with a "web hook URL", that accepts an API request with a user message and returns an answer for the user. For Facebook Messenger, for example, a user might send a message to your chatbot, and then Facebook will ping your web hook to get the answer, and then send the answer to the original user.

This architecture completely changes the way that bots are written and deployed. Instead of a single process running on a single machine, your bot program now has to deal with all the moving parts that come with deploying web applications. Web apps are typically load balanced across multiple physical machines, and each machine might be running multiple copies of the app itself in different processes.

With this architecture, RiveScript's default in-memory storage for user variables starts to fall apart. Even if you only have one web server, but it's running four instances of the web app, each request coming in from Facebook only has a 25% chance of talking with the exact same process as the previous request. Suddenly RiveScript has difficulty remembering a user's name, or their previous messages, and huge parts of RiveScript stop working reliably, for example the %Previous command will fail if RiveScript can't recall its last reply sent to the user.

When you add multiple physical machines into the mix, the problem gets worse.

User Variable Session Drivers

In order to make RiveScript more robust to work in this new chatbot environment, some of the implementations have started adding support for alternative ways of storing user variables.

Instead of only keeping user variables in system RAM, this would allow you to put them directly into a Redis cache or a MySQL database. This is much closer to the way that normal web apps have already worked for decades; you can have as many servers and processes as you want, and if they all share data by communicating with a database, then they all stay in sync regardless of which particular process handles a request from a user.

The Python and Go ports of RiveScript already support pluggable session drivers, and there are Redis implementations already available for those, and SQL based drivers are coming soon. Developers can also implement their own drivers, if they don't want to wait for official ones or if they have any special use cases. The Java implementation supports this now, too (thanks, Marcel Overdijk!), and the Perl one may get support in the future if there is demand for it.

I left out the JavaScript port, as this one is going to be a lot harder than the rest. None of the RiveScript implementations were designed to work with asynchronous code, and that has worked out fine for most of them, but JavaScript is a heavily async programming language. Most session drivers that developers might want to use (e.g. MongoDB, MySQL) typically work asynchronously with a callback or promise system. More backstory and details about this in particular is available on the rivescript-js wiki. I'm still working on finding a way to support this; see the async label on its GitHub Issues.

Recommendations for Modern RiveScript Bots

If you're making a RiveScript chatbot for a modern bot framework, you're probably best off writing your bot in Go, Python or Java, and using an alternative session driver for user variables rather than the default in-memory driver.

If you're stuck with JavaScript, you may be able to alleviate the user variable problem by using the getUservars() and setUservars() functions out-of-band from RiveScript; for example, immediately after getting a reply from a user, export their variables and put them into your database. And immediately before you get a reply, pull them from your database and put them into RiveScript. This would keep the bot "up to date" with the user's variables regardless of which process or machine is handling any given request. Also, be sure to make liberal use of the clearUservars() function to reset temporary user memory from time to time; the bot doesn't need to hold onto this temporary data if it's already putting it into a database and retrieving it before it needs it again.

Summary

RiveScript was born out of a time when running chatbots meant that you only ran one program from one computer, and the program would sign in to various messenger services. RiveScript's design decisions mirror this era of chatbots. With the "new wave" of bots, chatbots now resemble web application servers rather than small client programs, and this has required some new work to evolve RiveScript to meet this new use case.