Introduction to Sentence Transformers and NLP for Semantic Searching
Table of Contents
This workshop introduces the concept of sentence transformers and how to use them to build a text searching algorithm with semantic searching. We will also cover basic NLP techniques, such as removing stopwords and lemmatization for optimizing the algorithm.
This workshop is taught and developed in Google Colab, which is an online Jupyter Notebook editor. To access the workshop notebook, follow this link:
If you instead want to run the colab locally, follow the steps below to setup the Jupyter Notebook and install the proper pip requiements.
- (Optional) If you wish to use a virutal environment with python, run
python -m venv venvfollowed byvenv/Scripts/activateif on Windows andvenv/bin/activateon MacOS/Linux - Run
pip install -r requirements.txtin this directory. This will install the python requirements for the project. - Open the Jupyter Notebook
Sentence-Transformer-Workshop.ipynband choose either your python environment or the virtual environment if you made one.
- Introduction to basic text searching algorithms
- Theory behind semantic searching
- Utilizing sentence transformers for semantic searching
- Optimization with NLP techniques
- Hands on development in Google Colab
Distributed under the MIT License. See LICENSE for more information.
Tristan Pank @tristanpank
Workshop Link Link
- Data Science and Informatics for hosting the workshop.
- Marielle Doenges for peer reviewing the material.