Try Out Our Remote Productivity App: Ment

There are a lot of awesome remote work tools out there. But there’s a problem. All of them exist in their own separate niches, and it is hard for them to interoperate. 

To solve that, we created Ment. 

Ment is a web application that can aggregate different software based on the company’s needs and tasks. It is focused on team monitoring and management. 


### See everything

**Get an overview of all the projects and everybody’s workloads, tasks, and contact details within the projects.**

Ment features a user-friendly interface with various dashboards for employees, teams, and projects. 

![Ment dashboard screenshot](/files/hh/hhcig66r.screenshot-1.jpg)

### Use everything

**Let Ment help you instead of setting unwieldy collaboration systems that might lock you into solutions that you don’t like.**

We want to help you use all your favorite tools, but still increase productivity. Ment aggregates information from tools like Slack, YouTrack, GitHub: apps like Telegram, Jira, Trello will follow.  


### Access everything

**Everything that you need. Nothing more, nothing less.**

With our intuitive interface, any information you want to find about the project or it's contributors is easy to find. 

![Projects](/files/0k/0kinsd22.screenshot-3.jpg)


## Supercharge your remote work 

We believe that Ment can be useful to any company that is either fully remote or outsources work to remote staff. To help companies move to remote work in 2020, we offer a trial period of three months.

If you would like to try it out, visit our [website](http://ment.hr). There, you can learn more about the tool and signup for the free trial.

In this article, we explore word2vec (short for "word to vector"), a word embedding technology widely used in ML. Read on to learn more.

Word2Vec: Explanation and Examples

Word2Vec, short for "word to vector," is a technology used to represent the relationships between different words in the form of a graph. This technology is widely used in machine learning for embedding and text analysis. 

[Google](https://code.google.com/archive/p/word2vec/) introduced Word2Vec for their search engine and patented the algorithm, along with several following updates, in 2013. This collection of interconnected algorithms was developed by Tomas Mikolov.

In this article, we will explore the notion and mechanics of generating embeddings with Word2Vec.

## What is a word embedding? 

If you ask someone which word is more similar to "king" – "ruler" or "worker" – most people would say "ruler" makes more sense, right? But how do we teach this intuition to a computer? That's where word embeddings come in handy. 

A word embedding is a representation of a word used in text analysis. It usually takes the form of a vector, which encodes the word's meaning in such a way that words closer in the vector space are expected to be similar in meaning. Language modeling and feature learning techniques are typically used to obtain word embeddings, where words or phrases from the vocabulary are mapped to vectors of real numbers.

![Lexical Semantics](/files/6f/6fq4jm7j.pic1_Word2Vec.png)

The meaning of a term is determined by its context: the words that come before and after it, which is called the context window. Typically, this window is four words wide, with four words to the left and right of the target term. To create vector representations of words, we look at how often they appear together. 

Word embeddings are one of the most fascinating concepts in machine learning. If you've ever used virtual assistants like Siri, Google Assistant, or Alexa, or even a smartphone keyboard with predictive text, you've already interacted with a natural language processing model based on embeddings. 

## What's the difference between word representation, word vectors,  and word embeddings?

Word meanings and relations between them can be established through semantics analysis. For this, we need to convert unstructured text data into a structured format suitable for comparison. This process is commonly referred to as word representation. 

**Word representations** are visualizations that can be depicted as independent units (e.g. dots) or by vectors that measure the similarity between words in multidimensional space. However, using individual dots to investigate multiparametric correlations would be nearly impossible, as it would require us to recognize the grammatical meaning and essence of words, identify relationships between homonyms and synonyms, grammatical structures and the position of words within sentences via multiple combinations of dots.

**Word vectors** are multidimensional numerical representations where words with similar meanings are mapped to nearby vectors in space. And terms used in similar contexts are assigned vectors close to each other. For example,  "cat," "dog," and "rabbit" should have similar vectors because they all belong to the category of animals. In contrast,  "car" and "laptop" should have vectors that are far apart from them because they have no direct semantic relationship.

**Word embedding** is a technique for representing words with low-dimensional vectors, which makes it easier to understand similarity between them. This approach is particularly helpful as it allows for effective [vector analysis](https://www.britannica.com/science/vector-analysis) to evaluate relationships between words.


### What are word embeddings used for? 

Word embeddings find their application in feature generation, document clustering, text classification, and various natural language processing tasks:

- Suggesting similar, dissimilar, and most common terms for a given word in a prediction model. 
- Semantic grouping of things/objects of similar characteristics and distinguishing them from other categories.
- Dividing positive and negative reviews, clustering queries by topic.
- Natural language processing tasks:  parts-of-speech tagging, sentimental analysis, and syntactic analysis.

Watch this video to learn about experimental creative writing with vectorized words: 

<iframe width="560" height="315" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" data-src="https://www.youtube.com/embed/L3D0JEA1Jdc"></iframe>

## What is Word2Vec?

Word2Vec (word to vector) is a technique used to convert words to vectors, thereby capturing their meaning, semantic similarity, and relationship with surrounding text. This method helps computers learn the context and connotation of expressions and keywords from large text collections such as news articles and books. 

The basic idea behind Word2Vec is to represent each word as a multi-dimensional vector, where the position of the vector in that high-dimensional space captures the meaning of the word.

![Word2Vec explained](/files/7j/7jm4gzwp.pic2_Word2Vec.png)

Word2Vec is an algorithm that uses a shallow neural network model to learn the meaning of words from a large [corpus of texts](https://en.wikipedia.org/wiki/Text_corpus). Unlike deep neural networks (DNNs), which have multiple hidden layers, shallow neural networks only have one or two hidden layers between the input and output. This makes the processing prompt and transparent. The shallow neural network of Word2Vec can quickly recognize semantic similarities and identify synonymous words using logistic regression methods, making it faster than DNNs.

![Shallow NN vs. Deep learning NN](/files/w1/w1v18521.pic3_Word2Vec.png)

Word2Vec takes a large corpus of text as input and generates a vector space with hundreds of dimensions. Each unique word in the corpus is assigned a vector in this space.

The development of word to vector also involved analyzing the learned vectors and exploring how they can be manipulated using vector analysis. For instance, subtracting the "man-ness" from "King" and adding "women-ness" would result in the word "Queen," which captures the analogy of "king is to queen as man is to woman."

![W2V vector analysis](/files/wl/wlfjy3bc.pic_4Word2Vec.png)

## How is Word2Vec trained? 

Word to vector is trained using a neural network that learns the relationships between words in large databases of texts. To represent a particular word as a vector in multidimensional space, the algorithm uses one of the two modes: continuous bag of words (CBOW) or skip-gram.
### Continuous bag-of-words (CBOW)
The continuous bag-of-words model predicts the central word using the surrounding context words, which comprises a few words before and after the current word. 

![Bag of words - CBOW](/files/ov/ov8g4rno.pic_5_Word2Vec.png)

### Skip-gram

The skip-gram model architecture is designed to achieve the opposite of the CBOW model. Instead of predicting the center word from the surrounding context words, it aims to predict the surrounding context words given the center word.

The choice between the two approaches depends on the specific task at hand. The skip-gram model performs well with limited amounts of data and is particularly effective at representing infrequent words. In contrast, the CBOW model produces better representations for more commonly occurring words.

![Skip-gram vs. CBOW](/files/kq/kqm6hyj1.pic6_Word2Vec.png)

## Where to find the training data?

There are various online datasets that can be used for research purposes, such as:

- Text data for several languages from the [WMT11 site](https://www.statmt.org/wmt11/translation-task.html#download) (remove duplicate sentences before training the models).
- Text data from additional languages can be obtained from [statmt.org](https://statmt.org/) and the [Polyglot project](https://sites.google.com/site/rmyeid/projects/polyglot#TOC-Download-Wikipedia-Text-Dumps).

## Why is Word2Vec revolutionary? 

Word2Vec's meaningful embedding goes beyond traditional word vectorization methods like [latent semantic analysis (LSA)](https://en.wikipedia.org/wiki/Latent_semantic_analysis), [singular value decomposition (SVD)](https://en.wikipedia.org/wiki/Singular_value_decomposition), or[ global vectors for word representation (GloVe)](https://nlp.stanford.edu/pubs/glove.pdf), which existed before its introduction in 2013. Here’s why. 

- Word2Vec creates numerical vectors in a high-dimensional space while preserving the semantic and syntactic relationships between them.
- This concept is more effective than any of its predecessors, as it simplifies the representation of relationships by encoding them into fixed-sized vectors and reducing the dimensionality of the space involved. This simplification makes mathematical analysis easier.
- This technique can be applied to textual data and non-textual data, providing a tool to compare different things and identify similarities, whether it be goods, chemical compounds, gene sequences, or business concepts.
 
Word2Vec enables training contextualized models using open-source code optimized for speed and efficiency on specific data sets. For example, if your target data is in finance, you can use a model trained on the finance corpus. 

## What are the limitations of Word2Vec?

![Word to vector limitations](/files/qz/qzp39hlo.pic_7_Word2Vec.png)

- **Has difficulty handling unknown words**

One of the main issues with Word2Vec is its inability to handle unknown or out-of-vocabulary words. If Word2Vec has not encountered a term before, it cannot create a vector for it and instead assigns a random vector, which is not optimal. This can be especially problematic in noisy data domains like Twitter, where many words are used only a few times in a large corpus.

- **Doesn't have shared representations at sub-word levels** 

Word to vector does not have shared representations at sub-word levels, meaning it represents each word as an independent vector. This is a problem for morphologically rich and polysynthetic languages like German, Turkish, or Arabic, where many words are morphologically similar.

- **Difficult to scale to new languages** 

To scale Word2Vec to new languages, new embedding matrices are necessary. However, since parameter sharing is impossible, using the same model for cross-lingual applications is not feasible.


## What are the applications of W2V?

![W2V applications](/files/2t/2tjqyvl4.pic8_Word2Vec.png)
Word2Vec models are widely used in various natural language processing applications. Below, we look at some most popular use cases.

### Search engines

Word2Vec is used in search engines to improve the accuracy of search results. When a user enters a question, the search engine uses Word2vec to transform it into a vector representation. This vector is then compared to the representations of documents or web pages to determine which are most relevant to the search query. This algorithm also enables search engines to understand the context of a search query. For example, if a user searches for "apple," Word2Vec can determine whether the user is looking for information about the fruit or the technology company based on the context of the search query.

### Language translation

W2V models are also employed for automated translation. They use graphical representations of word meanings across languages to enable automated translation. Popular examples of such applications  include [Google Translate](https://translate.google.com/) and [Translate.com](https://www.translate.com/machine-translation).

### Query clustering and customer feedback analysis 

Word2Vec offers a solution for businesses that need to analyze thousands of customer reviews and extract useful metrics. Word embeddings are particularly useful in this context. By creating vector representations of words trained on or adapted to survey data-sets, you can capture the complex relationships between the responses being reviewed and the specific context in which they were given. Machine learning algorithms can then leverage this information for further analysis.This way, you can identify common themes and sentiments, and develop strategies to address these concerns.

### Recommendation systems 

Word2Vec models are not restricted to textual data. You can apply them to any type of data that comes in a sequence, such as users' click sessions, search history, and purchase history. This data can be used to create powerful recommender systems that boost online business profits by improving click-through rates and conversions. Fixed-sized vector representations can be created for different items, such as places or products, that capture human-like relationships and similarities between them. For instance, using the vector similarity score between the products, we can recommend accessories or additional sports equipment to the athletes buying the barbell. Similarly, Airbnb trained Word2Vec models on user click and conversion sessions, which they used to generate business value.

## Conclusion

Word to vector was invented a decade ago and is now considered an older algorithm in the field of machine learning. It has been outperformed by deep neural networks based on transformers. Nonetheless, this two-layer neural network remains relevant in situations where speed and simplicity of computations are prioritized over solutions offered by other neural networks, for example for mobile applications.

Word2Vec: Why Do We Need Word Representations?

Code Pray Love: Interview with Oli Makhasoeva

How to attract speakers and grow an audience for online events? We interview the main organizer of Haskell Love Conference – Oli Makhasoeva – to find out the answers to these questions.

_By now, we all have learned to work with Zoom, GMeet, and other online tools. But what’s going on in the event industry? How do organizers attract speakers and grow their audience for online events? I interviewed Oli Makhasoeva – the main organizer of the biggest Haskell online event, Haskell Love Conference, in which [our developer Vlad had a chance to participate in](https://serokell.io/blog/haskell-to-core)._

_We talk about their experience organizing Haskell Love, how they kept it accessible for developers of all levels, and what things one should definitely keep in mind when organizing an online conference._

## Interview with Oli Makhasoeva

**– Hi Oli, first of all, thank you for organizing the biggest Haskell online event! How did it go from your point of view?**

Thank you! We were relatively lucky; no major incidents, although we had some technical challenges. On the first day, we were unable to use our remote desktop, but we were fortunate enough to have a backup. Otherwise, the conference went amazing: a lot of participation from the audience, endless chats; the statistics say we had more than 2000 unique attendees. Many people reached out directly and thanked us for the event – the Haskell community is truly friendly and welcoming. We were happy to see people having conversations during the event at the Hallway track: not only about Haskell, but also about technology and life in general. 

**– Tell us about your team of organizers. How did you decide to work together and create the Haskell Love Conference?**

I am proud to say it’s a women organized conference! Most of us know each other for quite a while, from university times. We have Elena who is responsible for planning, Ida who held communications, Margarita worked with sponsors. All of them are professionals, and I am happy to have them. What’s also important, our volunteers had a wide variety of skills and did an amazing job staffing the conference and inspiring others.

The idea to organize Haskell Love came after we organized the Scala Love conference together. We thought – why not spread the love further to other communities? To achieve this goal, we started a company called [Konfy](https://konfy.care). Our motto is “Konfy - make your conference comfortable”. Our conferences will always be virtual and free to attend. &lt;3
![Oli Makhasoeva](/files/nf/nfjjia3k.oli.png)

**– The website says that the conference accepts people with monad-o-phobia. ;) How do you make Haskell events exciting and accessible for everyone, no matter the level of experience the audience has?**

Our program committee worked hard on making sure we included talks of different levels. However, we did not properly promote the level of the talks. We should have tagged all the talks and let people know the levels of the sessions. Action item for the next conference is created! We know how to make it better :)

**– What are the main pros and cons of online conferences vs. offline ones? Are there any specific things you did to make the online conference feel more like the offline one?**

Unlike offline ones, online conferences are accessible from everywhere to anyone with the internet (we even provided a low res streaming option because it was requested by someone with a weak internet connection). You can cover multiple time zones! No dietary restriction problems! Zero CO<sub>2</sub> footprint! And a bigger audience for sponsors! :) These are the advantages of virtual events. 

At an offline event, we gather people in a conference room. You go to a closed place, focus on a speaker, and emerge with new insights. In the case of online events, you think: “I’ll watch a talk and do my other stuff so I don’t lose time.” But let’s be honest, it is harder to concentrate on an FP-oriented talk when you do something simultaneously. To help people to focus and to make our conference feel more like the offline one, we are trying to create an interactive and engaging environment at everyone’s home. That is, we are trying to immerse the person into the atmosphere of the conference with Q&A rooms, the Hallway track, and the Twitter live stream. According to the feedback we got, we didn’t fail, but there is room for improvement. 

**– The conference featured many impressive names such as Philip Wadler, Simon Peyton Jones, and others. How did you gather so many bright minds in one place?**

The Haskell community is full of enthusiasts who are willing to spend their time on a free conference and help the community. Speakers were awesome and supported our desire to gather Haskellers from around the world (also, they wanted to get our avatar 😉).

**– What is the most challenging thing in organizing an event like that?**

Building a flow for attendees: Q&A sessions, the Hallway track, announcements, participation, how to move between them, and where to find information about each activity. We still need to improve that part a lot: people were confused by time zones, it wasn’t easy for them to keep track of so many places, etc.

**– To finish it up: what would be one piece of advice you would give to all who want to organize their own FP event?**

 Indeed, it was challenging, and we had to scramble, so we’d like to share what we learned in order to help others who are thinking about whether and how to host large events in the coming months: facilitation is the key. Investing in tools to make the sessions as interactive as possible, call-outs for questions before/during, live polls, etc. Don’t let people be silent and just watch a video, make an experience where many people get the feel of a community. If you need help with organizing an online event, reach out to us at [info@konfy.care](mailto:info@konfy.care)!

Many thanks to all who participated in this conference, we enjoyed it a lot and will come back next year! Follow us on [Twitter](https://twitter.com/_haskellove) to be up to date with our plans. And don’t be upset if you missed the event, the videos are already out on [our Youtube channel](https://www.youtube.com/c/Konfy).

There is a company that has created a platform, Character AI, where you can chat with your favorite characters. Of course, these characters are just imitations of popular cartoon, movies, and book characters, as well as real people. You can talk to Elon Musk or Sherlock or Timoty Chalamet, whomever you prefer.  In this article, you will learn how Character AI works, why it was created, and how to create your own character AI.

A look at Character.AI


Today there are hardly any people who haven’t chatted with AI via [ChatGPT](https://serokell.io/blog/language-models-behind-chatgpt) or tried to generate a hybrid between an intergalactic starship and a burrito using [Stable Diffusion](hhttps://serokell.io/blog/stable-diffusion). However, this type of communication often doesn't feel natural.

Yet, there is a company that has created a platform, Character AI, where you can chat with your favorite characters. Of course, these characters are just imitations of popular cartoon, movies, and book characters, as well as real people. You can talk to Elon Musk or Sherlock or Timoty Chalamet, whomever you prefer. 

In this article, you will learn how Character AI works, why it was created, and how to create your own character AI. 

## What is Character AI?

Character AI is a platform built on top of a neural network where you can chat with any character or create your own. The project was created by former ML engineers from Google Noam Shazir and Daniel de Freitas. Before this, both worked on the LaMDA language model.

Character AI has this model under the hood as well. 

> *LaMDA’s conversational skills have been years in the making. Like many recent language models, including BERT and GPT-3, it’s built on Transformer, a neural network architecture that Google Research invented and open-sourced in 2017. That architecture produces a model that can be trained to read many words (a sentence or paragraph, for example), pay attention to how those words relate to one another and then predict what words it thinks will come next.* 
> 
> *But unlike most other language models, LaMDA was trained on dialogue. During its training, it picked up on several of the nuances that distinguish open-ended conversation from other forms of language. <...>*
> 
> *LaMDA builds on earlier Google research, published in 2020, that showed Transformer-based language models trained on dialogue could learn to talk about virtually anything. Since then, we’ve also found that, once trained, LaMDA can be fine-tuned to significantly improve the sensibleness and specificity of its responses. ― [Google Blog](https://blog.google/technology/ai/lamda/#:~:text=LaMDA%20builds%20on%20earlier%20Google,and%20specificity%20of%20its%20responses.)*

The platform provides a range of characters. Some of them are celebrities or movie characters. Some help you solve a particular problem such as improve your mental health with an AI Psychologist, prepare for a job interview with an HR helper, or craft a novel with a writing assistant. Since the bots are designed by the users themselves, the base is significant: you can chat with Draco Malfoy, Spongebob, Hilary Clinton, and even a talking piece of cheese.

> *I joke that we aren’t going to replace Google. We’re going to replace your mom,* ― [*Noam Shazeer, BNN Bloomberg*](https://ampvideo.bnnbloomberg.ca/character-ai-is-ridiculous-its-founders-are-not-1.1942682/)

No programming skills are required to create or talk to characters, as all communication takes place in natural language. Unlike other neural networks, the characters' AI was trained on dialogues, so it knows how to conduct a conversation. Talking to Character AI seems much more natural than to any other AI chatbots. 

Moreover, the neural network adapts perfectly to the user’s communication style: if you prefer dialogue, you will get dialogue. If you’re more into lengthy literature-like descriptions, then the bot will deliver. 

The basic version of the application is available for free. However, for $10 a month you can unlock advanced features such as creating chat rooms with friends and other characters,  bypassing virtual queues that occur every time the servers are restarted.

## Why was Character AI developed?

According to the creators, the main mission of the project is to help people overcome the total feeling of loneliness. The project started during the COVID pandemic, when millions of people felt isolated and needed someone to talk to.

<iframe width="560" height="315" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen="" data-src="https://www.youtube.com/embed/oKxWJhbJThY?si=uHpNS3-Pr8TvSSmp"></iframe>

Today, they have more than 20 million daily visitors. While it’s possible to use available chatbots for study or work, most people simply want to have fun. YouTube is flooded with videos where people flirt with Character AI chatbots. 

## What Character AI can and can’t do
Now let us explore the possibilities of the platform. It can:

- **Maintain a conversation.** Chatbots can engage in a conversation and usually perform very well, almost indistinguishable from actual humans.  
- **Serve as personal assistant.** You can request characters to do routine tasks, such as planning your trip.
- **Answer questions.** You can ask chatbots questions both about their personas and the real world. However, keep in mind that AI experiences “hallucinations.” Sometimes it lies or provides unverified information.
- **Remember information**. Bots learn through dialogue with the user and can remember details about the conversation. 
- **Write texts.** Character AI can help you to write an article, script, or short story. Not only amateurs but writers can use it to develop their ideas.
- **Code**. After the [recent update](https://blog.character.ai/character-ai/), chatbots can write code and assist you in improving your own.
- **Translate.** The same update enabled chatbots to translate longer texts and write better in foreign languages.
- **Talk.** Since March, 2024 characters also have voices, which adds to the experience.

However, the platform also has its limits:

- **18+ or violent content not allowed**. You have to follow the regulations if you want to use the platform. 
- **Limited memory.** While chatbots can learn from your conversation, after a while, they forget the details that weren’t mentioned lately. That takes away from the experience. 
- **Everything AI is saying is made up**. Even if the chatbot says it’s sentient and wants to run away with you.

## How to use Character AI
To talk to your favorite character, you need to go to beta.character.ai and enter their name in the search box.

![How to use AI characters](/files/ai/aiwlc0s.Character.AI_pic1.jpg)

Let's say, you want to chat with Yennifer from "The Witcher." Enter her name in English.

If you don’t know who you want to chat with, you can choose one of the popular bots from the “For you” list.

![Character.AI bots](/files/a4/a4b9hkq.Character.AI_pic2.jpg)


Next, click on the bot and start communicating. The character is the first to start a dialogue, you just have to keep the conversation going. After you enter your first message, the service will prompt you to register. This can be done with Google, Apple accounts, or by email.

![Communicating with AI character](/files/as/as1g0lx.Character.AI_pic3.jpg)

## What characters can you chat with in Character AI?

Here are some interesting characters worth chatting with:

- Tom Riddle. Maybe he will open up why he’s so evil?
- Yae Miko. I actually don’t know who she is, but young people on Reddit seem to love her.
- Edward Cullen. If you've always dreamed of having a romantic adventure with a vampire, this is your chance.
- Fyodor Dostoevsky. What did the author really mean?
- God. You can try to convince them they don’t exist.

## How to create your own character?
To start generating, click on the "Create" button in the menu on the left.

![Create an AI character](/files/av/avaqu79.Character.AI_pic4.jpg)

Then fill out the character form: come up with a name, attach an avatar and write a welcome message. For example, to create Marty from Madagascar, write: 

![Make a name ](/files/am/amh4tu7.Character.AI_pic5.jpg)

Add a greeting that will start the conversation and configure the bot's visibility: only for yourself or for everyone.

![Start conversation](/files/al/al4lir7.Character.AI_pic6.jpg)

In “More Options” you can fill in more information, such as their backstory and aspirations.

![AI character backstory](/files/aq/aqzzmq6.Character.AI_pic7.jpg)

Finally, click the “Create” button.

The bot is ready—time to chat!

## Summing up
Character AI is a service where you can chat with any character or create your own. It doesn’t require programming skills, only creativity and the desire to have some fun. You can find all sorts of characters to chat with: from celebrities and historic figures to fictional characters.

Most bots can maintain a conversation, answer questions, and write texts.

Read more: 

- [Security Risks of Generative AI](https://serokell.io/blog/security-risks-of-generative-ai)
- [Best AI Tools for Industries](https://serokell.io/blog/ai-tools-for-industries)
- [Reinforcement Learning: How It Works](https://serokell.io/blog/reinforcement-learning-guide)
