AI
Finite Mixture Model based on Dirichlet Distribution
This blog post is the second part of an article series on Dirichlet Process mixture models. In the previous article we had an overview of several Cluster Analysis techniques and we discussed some of the problems/limitations that rise by using them. Moreover we briefly presented the Dirichlet Process Mixture Models, we talked about why they […]
 May 12, 2014
 Vasilis Vryniotis
 . 4 Comments
This blog post is the second part of an article series on Dirichlet Process mixture models. In the previous article we had an overview of several Cluster Analysis techniques and we discussed some of the problems/limitations that rise by using them. Moreover we briefly presented the Dirichlet Process Mixture Models, we talked about why they are useful and we presented some of their applications.
Update: The Datumbox Machine Learning Framework is now opensource and free to download. Check out the package com.datumbox.framework.machinelearning.clustering to see the implementation of Dirichlet Process Mixture Models in Java.
The Dirichlet Process Mixture Models can be a bit hard to swallow at the beginning primarily because they are infinite mixture models with many different representations. Fortunately a good way to approach the subject is by starting from the Finite Mixture Models with Dirichlet Distribution and then moving to the infinite ones.
Consequently in this article I will briefly present some important distributions that we will need, we will use them to construct the Dirichlet Prior with Multinomial Likelihood model and then we will move to the Finite Mixture Model based on the Dirichlet Distribution.
1. Beta Distribution
The Beta distribution is a family of continuous distributions which is defined in the interval of [0,1]. It is parameterized by two positive parameters a and b and its form heavily depends upon the selection of those two parameters.
Figure 1: Beta Distribution for different a, b parameters
The Beta distribution is commonly used to model a distribution over probabilities and has the following probability density:
Equation 1: Beta PDF
Where Γ(x) is the gamma function and a, b the parameters of the distribution. Beta is commonly used as a distribution of probability values and gives us the likelihood that the modelled probability equals to a particular value P = p0. By its definition Beta distribution is able to model the probability of binary outcomes which take values true or false. The parameters a and b can be considered as the pseudocounts of success and failure respectively. Thus the Beta Distribution models the probability of success given a successes and b failures.
2. Dirichlet Distribution
The Dirichlet Distribution is the generalisation of Beta Distribution for multiple outcomes (or in other words it is used for events with multiple outcomes). It is parameterized with k parameters a_{i} which must be positive. Dirichlet Distribution equals to the Beta Distribution when the number of variables k = 2.
Figure 2: Dirichlet Distribution for various a_{i} parameters
The Dirichlet distribution is commonly used to model a distribution over probabilities and has the following probability density:
Equation 2: Dirichlet PDF
Where Γ(x) is the gamma function, the p_{i} take values in [0,1] and Σp_{i}=1. The Dirichlet distribution models the joint distribution of p_{i} and gives the likelihood of P_{1}=p_{1},P_{2}=p_{2},….,P_{k1}=p_{k1} with P_{k}=1 – ΣP_{i}. As in the case of Beta, the a_{i} parameters can be considered as pseudocounts of the appearances of each i event. The Dirichlet distribution is used to model the probability of k rival events occurring and is often denoted as Dirichlet(a).
3. Dirichlet Prior with Multinomial Likelihood
As mentioned earlier the Dirichlet distribution can be seen as a distribution over probability distributions. In cases where we want to model the probability of k events occurring, a Bayesian approach would be to use Multinomial Likelihood and Dirichlet Priors .
Below we can see the graphical model of such a model.
Figure 3: Graphical Model of Dirichlet Priors with Multinomial Likelihood
In the above graphical model, α is a k dimensional vector with the hyperparameters of Dirichlet priors, p is a k dimensional vector with the probability values and x_{i} is a scalar value from 1 to k which tells us which event has occurred. Finally we should note that the P follows the Dirichlet distribution parameterized with vector α and thus P ~ Dirichlet(α), while the x_{i} variables follow the Discrete distribution (Multinomial) parameterized with the p vector of probabilities. Similar hierarchical models can be used in document classification to represent the distributions of keyword frequencies for in different topics.
4. Finite Mixture Model with Dirichlet Distribution
By using Dirichlet Distribution we can construct a Finite Mixture Model which can be used to perform clustering. Let’s assume that we have the following model:
Equation 3: Finite Mixture Model with Dirichlet Distribution
The above model assumes the following: We have a dataset X with n observations and we want to perform cluster analysis on it. The k is a constant finite number which shows the number of clusters/components that we will use. The c_{i} variables store the cluster assignment of observation X_{i}, they take values from 1 to k and follow the Discrete Distribution with parameter p which are the mixture probabilities of the components. The F is the generative distribution of our X and it is parameterized with a parameter which depends on the cluster assignment of each observation. In total we have k unique parameters equal to the number of our clusters. The variable stores the parameters that parameterize the generative F Distribution and we assume that it follows a base G_{0} distribution. The p variable stores the mixture percentages for every one of the k clusters and follows the Dirichlet with parameters α/k. Finally the α is a k dimensional vector with the hyperparameters (pseudocounts) of Dirichlet distribution [2].
Figure 4: Graphical Model of Finite Mixture Model with Dirichlet Distribution
A simpler and less mathematical way to explain the model is the following. We assume that our data can be grouped in k clusters. Each cluster has its own parameters and those parameters are used to generate our data. The parameters are assumed to follow some distribution G_{0}. Each observation is represented with a vector x_{i} and a c_{i} value which indicates the cluster to which it belongs. Consequently the c_{i} can be seen as a variable which follows the Discrete Distribution with a parameter p which is nothing but the mixture probabilities, i.e. the probability of the occurrence of each cluster. Given that we handle our problem in a Bayesian way, we don’t treat the parameter p as a constant unknown vector. Instead we assume that the P follows Dirichlet which is parameterized by hyperparameters α/k.
5. Working with infinite k clusters
The previous mixture model allows us to perform unsupervised learning, follows a Bayesian approach and can be extended to have a hierarchical structure. Nevertheless it is a finite model because it uses a constant predefined k number of clusters. As a result it requires us to define the number of components before performing Cluster Analysis and as we discussed earlier in most applications this is unknown and can’t be easily estimated.
One way to resolve this is to imagine that k has a very large value which tends to infinity. In other words we can imagine the limit of this model when k tends to infinity. If this is the case, then we can see that despite that the number of clusters k is infinite, the actual number of clusters that are active (the ones that have at least one observation), can’t be larger than n (which is the total number of the observations in our dataset). In fact as we will see later, the number of active clusters will be significantly less than n and they will be proportional to .
Of course taking the limit of k to infinity is nontrivial. Several questions rise such as whether it is possible to take such a limit, how would this model look like and how can we construct and use such a model.
In the next article we will focus on exactly these questions: we will define the Dirichlet Process, we will present the various representations of DP and finally we will focus on the Chinese Restaurant Process which is an intuitive and efficient way to construct a Dirichlet Process.
I hope you found this post useful. If you did please take a moment to share the article on Facebook and Twitter. 🙂
Source: http://blog.datumbox.com/finitemixturemodelbasedondirichletdistribution/
AI
How does it know?! Some beginner chatbot tech for newbies.
Most people will know by now what a chatbot or conversational AI is. But how does one design and build an intelligent chatbot? Let’s investigate some essential concepts in bot design: intents, context, flows and pages.
I like using Google’s Dialogflow platform for my intelligent assistants. Dialogflow has a very accurate NLP engine at a cost structure that is extremely competitive. In Dialogflow there are roughly two ways to build the bot tech. One is through intents and context, the other is by means of flows and pages. Both of these design approaches have their own version of Dialogflow: “ES” and “CX”.
Dialogflow ES is the older version of the Dialogflow platform which works with intents, context and entities. Slot filling and fulfillment also help manage the conversation flow. Here are Google’s docs on these concepts: https://cloud.google.com/dialogflow/es/docs/concepts
Context is what distinguishes ES from CX. It’s a way to understand where the conversation is headed. Here’s a diagram that may help understand how context works. Each phrase that you type triggers an intent in Dialogflow. Each response by the bot happens after your message has triggered the most likely intent. It’s Dialogflow’s NLP engine that decides which intent best matches your message.
1. 8 Proven Ways to Use Chatbots for Marketing (with Real Examples)
2. How to Use Texthero to Prepare a Textbased Dataset for Your NLP Project
What’s funny is that even though you typed ‘yes’ in exactly the same way twice, the bot gave you different answers. There are two intents that have been programmed to respond to ‘yes’, but only one of them is selected. This is how we control the flow of a conversation by using context in Dialogflow ES.
Unfortunately the way we program context into a bot on Dialogflow ES is not supported by any visual tools like the diagram above. Instead we need to type this context in each intent without seeing the connection to other intents. This makes the creation of complex bots quite tedious and that’s why we map out the design of our bots in other tools before we start building in ES.
The newer Dialogflow CX allows for a more advanced way of managing the conversation. By adding flows and pages as additional control tools we can now visualize and control conversations easily within the CX platform.
This entire diagram is a ‘flow’ and the blue blocks are ‘pages’. This visualization shows how we create bots in Dialogflow CX. It’s immediately clear how the different pages are related and how the user will move between parts of the conversation. Visuals like this are completely absent in Dialogflow ES.
It then makes sense to use different flows for different conversation paths. A possible distinction in flows might be “ordering” (as seen here), “FAQs” and “promotions”. Structuring bots through flows and pages is a great way to handle complex bots and the visual UI in CX makes it even better.
At the time of writing (October 2020) Dialogflow CX only supports English NLP and its pricing model is surprisingly steep compared to ES. But bots are becoming critical tech for an increasing number of companies and the cost reductions and quality of conversations are enormous. Building and managing bots is in many cases an ongoing task rather than a single, roundedoff project. For these reasons it makes total sense to invest in a tool that can handle increasing complexity in an easytouse UI such as Dialogflow CX.
This article aims to give insight into the tech behind bot creation and Dialogflow is used merely as an example. To understand how I can help you build or manage your conversational assistant on the platform of your choice, please contact me on LinkedIn.
AI
Who is chatbot Eliza?
Between 1964 and 1966 Eliza was born, one of the very first conversational agents. Discover the whole story.
Between 1964 and 1966 Eliza was born, one of the very first conversational agents. Its creator, Joseph Weizenbaum was a researcher at the famous Artificial Intelligence Laboratory of the MIT (Massachusetts Institute of Technology). His goal was to enable a conversation between a computer and a human user. More precisely, the program simulates a conversation with a Rogérian psychoanalyst, whose method consists in reformulating the patient’s words to let him explore his thoughts himself.
1. 8 Proven Ways to Use Chatbots for Marketing (with Real Examples)
2. How to Use Texthero to Prepare a Textbased Dataset for Your NLP Project
The program was rather rudimentary at the time. It consists in recognizing key words or expressions and displaying in return questions constructed from these key words. When the program does not have an answer available, it displays a “I understand” that is quite effective, albeit laconic.
Weizenbaum explains that his primary intention was to show the superficiality of communication between a human and a machine. He was very surprised when he realized that many users were getting caught up in the game, completely forgetting that the program was without real intelligence and devoid of any feelings and emotions. He even said that his secretary would discreetly consult Eliza to solve his personal problems, forcing the researcher to unplug the program.
Conversing with a computer thinking it is a human being is one of the criteria of Turing’s famous test. Artificial intelligence is said to exist when a human cannot discern whether or not the interlocutor is human. Eliza, in this sense, passes the test brilliantly according to its users.
Eliza thus opened the way (or the voice!) to what has been called chatbots, an abbreviation of chatterbot, itself an abbreviation of chatter robot, literally “talking robot”.
Source: https://chatbotslife.com/whoischatbotelizabfeef79df804?source=rss—a49517e4c30b—4
AI
How to take S3 backups with DejaDup on Ubuntu 20.10
DejaDup is the default backup application for Gnome. It’s a GUI for duplicity, focuses on simplicity, supports incremental encrypted backups and up until recently supported a large number of cloud providers. Unfortunately as of version 42.0, all major cloud providers have been removed. Thus given that Ubuntu 20.10 ships with the specific version, any user […]
 October 18, 2020
 Vasilis Vryniotis
 . No comments
DejaDup is the default backup application for Gnome. It’s a GUI for duplicity, focuses on simplicity, supports incremental encrypted backups and up until recently supported a large number of cloud providers. Unfortunately as of version 42.0, all major cloud providers have been removed. Thus given that Ubuntu 20.10 ships with the specific version, any user who upgrades and has backups on Amazon S3 won’t be able to access them. In this blog post, we will provide a solution that will allow you to continue taking backups on AWS S3 using DejaDup.
The mandatory rant (feel free to skip)
The removal of the cloud providers should not come as a surprise. I’m not exactly sure which version of DejaDup deprecated them but it was around the release of Ubuntu 17.10 when they were all hidden as an option. So for 3 long years, people who had backups on Amazon S3, Google Cloud Storage, Openstack Swift, Rackspace etc could still use the deprecated feature and prepare for the inevitable removal.
So why complain you might ask? Well, first of all, when you update from an earlier version of Ubuntu to 20.10, you don’t really know that the all cloud providers are removed from DejaDup. Hence if something goes wrong during the update, you won’t be able to easily access your backups and restore your system.
Another big problem is the lack of storage options on the last version of DejaDup. They decided to change their policy and support only “consumertargeted cloud services” but currently they only support Google Drive. So they eliminated all the cost efficient options for mass storage and kept only one single very expensive option. I’m not really sure how this is good for the users of the application. Linux was always about having a choice (too much of it in many cases), so why not maintain multiple storage options to serve both the experience and inexperienced users? Thankfully because we are on Linux, we have option to fix this.
How to use Deja Dup v42+ with AWS S3
WARNING: I have not tested thoroughly the following setup so use it at your own risk. If the computer explodes in your face, you lose your data, or your spouse takes your kids and leaves you, don’t blame me.
Installing s3fs fuse
With that out of the way, let’s proceed to the fix. We will use s3fs fuse, a program that allows you to mount an S3 bucket via FUSE and effectively make it look like a local disk. Thankfully you don’t have to compile it from source as it’s on Ubuntu’s repos. To install it, type the following in your terminal:
sudo apt install s3fs
Setting up your AWS credentials file
Next, we need to configure your credentials. The s3fs supports two methods for authentication: an AWS credential file or a custom passwd file. In this tutorial we will use the first method but if you are interested for the latter feel free to view the s3fs documentation on Github. To setup your credentials make sure that the file ~/.aws/credentials contains your AWS access id and secret key. It should look like this:
[default]
aws_access_key_id=YOUR_ACCESS_KEY_ID
aws_secret_access_key=YOUR_SECRET_ACCESS_KEY
Mounting your bucket to your local filesystem
Once your have your credentials file you are ready to mount your backup bucket. If you don’t remember the bucket name you can find it by visiting your AWS account. To mount and unmount the bucket to/from a specific location type:
# mount
s3fs BUCKET_NAME /path/to/location
# unmount
fusermount u /path/to/location
Mounting the bucket like this is only temporary and will not persist across reboots. You can add it on /etc/fstab but I believe this only works with the passwd file. If you want to use your AWS credentials file an easy workaround it to create a shortcut in your Startup Applications Preferences.
Note that you can add a small 10 sec delay to ensure that the WiFi is connected before you try to mount the bucket. Internet access is obviously necessary for mounting it successfully. If you are behind VPNs or have other complex setups, you can also create a bash script that makes the necessary checks before you execute the mount command. Sky is the limit!
Configuring DejaDup
With the bucket mounted as a local drive, we can now easily configure DejaDup to use it. First of all we need to change the backend to local. This can be done either by using a program like dconfig or the console with the following command:
gsettings set org.gnome.DejaDup backend 'local'
Finally we open DejaDup, go to preferences and point the storage location to the directory that has your S3 backup files. Make sure you select the subdirectory that contains the backup files; this is typically a subdirectory in your mount point that has name equal to your computer’s hostname. Last but not least, make sure that the S3 mount directory is excluded from DejaDup! To do this, check the ignored folders in Preferences.
That’s it! Now go to your restore tab and DejaDup will be able to read your previous backups. You can also take new ones.
Gotchas
There are a few things to keep in mind in this setup:
 First of all, you must be connected on the internet when you mount the bucket. If you are not the bucket won’t be mounted. So, I advise you instead of just calling the mount command, to write a bash script that does the necessary checks before mounting (internet connection is on, firewall allows external requests etc).
 Taking backups like that seems slower than using the old native S3 support and it is likely to generate more network traffic (mind AWS traffic costs!). This is expected because DejaDup thinks it’s accessing the local filesystem, so there is no need for aggressive caching or minimization of operations that cause network traffic.
 You should expect stability issues. As we said earlier, DejaDup does not know it writes data over the wire so much of the functionalities that usually exist in such setups (such as retryonfail) are missing. And obviously if you lose connection midway of the backup you will have to delete it and start a new one to avoid corrupting your future backups.
 Finally keep in mind that this is a very experimental setup and if you really want to have a reliable solution, you should do your own research and select something that meets your needs.
If you have a recommendation for an OpenSource Backup solution that allows locally encrypted incremental backups, supports S3 and has an easy to use UI please leave a comment as I’m more than happy to give it a try.
Source: http://blog.datumbox.com/howtotakes3backupswithdejaduponubuntu2010/

AI1 week ago
7 Reasons Why is your business in need of a chatbot?

AI1 week ago
Future of Contact Centers Enabling Employees to Work from Home

AI1 week ago
5 Astonishing Statistics Associated With ELearning Industry

AI1 week ago
11 Simple Steps to Develop an Entertaining and Engaging Mobile Game App

AI1 week ago
This is How Chatbots Are Replacing Traditional Customer Support

AI1 week ago
Fast reinforcement learning through the composition of behaviours

AI1 week ago
Submit a Guest Post: Get Results with Posts on Other Blogs

AI5 days ago
5 Reasons Business Should Consider to Hire a Franchise Consultant