How to Develop a Text Recommendation Engine

Using TensorflowJS, TypeScript, ReactJS & MongoDB

Published in

Geek Culture

6 min readMay 2, 2021

Flow Diagram of a Serverless Recommendation Engine.

Recent advances in Deep Learning have made it possible to extend the power of Artificial Intelligence into web browsers. Introduced in 2018, TensorflowJS enables web developers to integrate state-of-the-art models in their applications. One of these models, the universal-sentence-encoder, uses advances in Natural Language Processing to transform words and sentences into mathematical vectors known as word embeddings.

Developed by seminal papers in deep learning, including Google’s word2vec and Stanford’s GloVe. Word embedding unlocks a variety of applications: mapping synonyms and antonyms, deciphering analogies, measuring biases. A text recommendation engine can be understood as a similarity optimization problem. In a vector space, this translates to finding the closest point for a set of initial conditions.

Outsourcing ML operations to the front-end with TensorflowJS has several benefits that can include: privacy, cost-effectiveness, speed, and ease. For example, Federate Learning is a distributed AI paradigm that shares the training and prediction tasks with deployed devices like mobile phones and IoT.

The following approach to build a text recommendation engine pairs recent advances in deep learning advances with geographical database queries to create a smooth user experience. It is particularly effective in data-scarce applications that require fast response times and cheap computation costs, like Social Networks.

1. Preparing our Dataset

Assuming an existing dataset of potential recommendations to serve already exists. The first step is to tokenize the texts into sentences. After that, the input is ready to be fed into TensorflowJS:

import { Tensor2D } from '@tensorflow/tfjs-node'
import { UniversalSentenceEncoder } from '@tensorflow-models/universal-sentence-encoder'const embed = async(sentences: string[]):Promise<Tensor2D> => {
    const model = await use.load()
    const embeddings = await model.embed(sentences)
    return embeddings
}

In the preceding code, Tensorflow and the sentence encoder model are imported. Then, the model is loaded, and the sentences are embedded into vectors in a single line of code. The result is a 2D Tensor composed of 512-dimensional vectors, each representing a sentence.

Finding the distance between two 512-dimensional points is a slow and expensive task. However, there is a long tradition of databases working with spatial relationships. In particular, MongoDB has support to create 2D indexes and perform queries on coordinate pairs. To accomplish this, we will use the ml-PCA node module to reduce the dimensionality of the word embeddings. First, we train the model:

import { PCA, IPCAModel } from 'ml-pca'
import { promises as fs } from 'fs'const trainPCA = async(embeddings:number[][]) => {
    const pca = new PCA(embeddings)
    await fs.writeFile(PCA_ROOT, JSON.stringify(pca))
}

And then we make the predictions:

import PCA_Model from PCA_ROOTconst reduceDimensions(embeddings:number[][]) => {
    const pca = PCA.load(PCA_Model as IPCAModel)
    const points = pca.predict([embeddings], {nComponents:2})
    return points
}

After saving the results to MongoDB, we have a clean dataset of recommendations ready to be served.

2. Handling the user’s input

To get the initial set of conditions from which to serve the recommendations, there are two options:

A history of choices made by the user.
Ask for text input from the user.

The first option is simple: first, we only need to find a function to average the location of the choices made by the user. Handling text input is not difficult either, but we would need to implement the same pipeline we used to process the recommendations. Regardless, we will be left with 2D coordinates from which to fetch the DB:

const makeRecommendations = async(x:number, y:number) => {
    const geoNear = {$geoNear: {near:[x,y], distanceField:'dist'}}
    const limit = { $limit: 100 }    // Get the top 100 Recommendations from the DB collection.
    const recs = await Recs.aggregate([geoNear, limit]).toArray()
    return recs
}

We are now ready to serve the recommendations, but we can do one more thing to improve the accuracy of our suggestions: reuse the original 512-dimensions word embeddings!

3. Serving the Recommendations

Using word embeddings to rank the top 100 recommendation is the best of two worlds: the speed of 2D indexes; and the accuracy of 512-dimensional vectors. But to implement it, we need to define and compute a distance between two points. In this example, I’m going to use the absolute value distance to sort the recommendations:

// Absolute Value Distance Computation.
const similarity = (userInput:number[], recommendation:number[])=> {
    if (userInput.length !== recommendation.length) return Infinity    // Iterate through each dimension and compute the difference.
    const delta = userInput.reduce((d, i, idx) => 
        d + Math.abs(i - recommendation[idx])
    , 0)    return delta
}const sortRecomendations = (userInput:number[], recs:iRec[]) => { 
    return recs.sort(({embeddings:a}, {embeddings:b}) => 
        similarity(userInput, a) > similarity(userInput, b) ? 1 : -1
    )
}

Then, use the absolute value distance to render sorted recommendations in a React application:

import { useEffect, useState } from 'react'const Recommendations = (recs: iRecs[], userInput:number) => {
    const [ sorted, setSorted ] = useState<iRecs[]>()
    useEffect(() => { 
        const sortedRecs = sortRecomendations(userInput, recs)
        setSorted(sortedRecs)
   }, [recs])    return sorted.map((r, k) => <Recommendation {...r} key={k} />
}

Finally, to close our Machine Learning pipeline, we need a system to evaluate and improve the accuracy of the results. I suggest sending anonymized events to a Product Intelligence tool, like Amplitude. These ideas can get you started to evaluate the model’s performance:

The correlation between the closeness of a suggestion and its click rate.
The further above a recommendation is presented, the more clicks it gets.
Measure differences in efficiency exist based on the time and date.
Segment users based on their searches and interests.

Review of the flow diagram for a Machine-Learning Text-Recommendation Engine’s Pipeline.

Conclusion: Potential Applications

Thank you for reading; as a bonus, I will present to you six ideas where you can apply this technology to build a product or business:

A Reading Recommendation Engine: similar to what Netflix does with movies; or to Google’s App. You could create an app or newsletter that serves reading recommendations based on the users’ interests.
Document Indexing: fields like Academia, Medicine, or Legal could benefit from the ability to classify and retrieve with speed relevant information based on non-trivial queries.
Searching Documentations: AWS docs comprise thousand of pages and hundreds of products. And AWS Kendra powers its search engine using Machine Learning. With this algorithm, you could create a competing, non-proprietary technology to search for answers in other documentation.
Understanding your users: if your website or app has a search component: you can guide your marketing efforts by understanding what your customers want based on what they search for.
Meaningful Visualization: trying to understand visually what happened after a meeting or a week’s worth of emails is tough. Imagine creating charts based on the word embeddings to summarize and measure non-numerical interactions.
Social Networks: build the next thriving, niche social network by recommending the best content, developing meaningful relationships among the users, and building smart automatized moderation tools.

I hope you enjoyed this story and find it useful. If you are interested in reading more about entrepreneurial AI in the front-end, please consider giving me a follow. As always your claps, and words of encouragement are highly appreciated.