How to instal & setup Spark NLP in high-compliance environments with no internet connection and download & load the pretrained models locally. Introduction Spark NLP is a Natural Language Processing (NLP) library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines…

Converting natural language questions to SQL queries on scale — Disclaimer: Text2SQL annotator in Spark NLP is deprecated in v3.x (March 2021)and not being used anymore. If you want to test this module, please use earlier versions of Spark NLP for Healthcare. The amount of data produced daily has been increasing exponentially since the start of the new millennia. Most…

The first end-to-end pretrained models and pipelines to detect Adverse Drug Reactions on scale with the help of Spark NLP and BioBert. Adverse Drug Reactions (ADRs) or Adverse Drug Events (ADEs) are potentially very dangerous to patients and are amongst the top causes of morbidity and mortality [1]. Many ADRs…

Training a SOTA multi-class text classifier with Bert and Universal Sentence Encoders in Spark NLP with just a few lines of code in less than 10 min. — Natural language processing (NLP) is a key component in many data science systems that must understand or reason about a text. Common use cases include text classification, question answering, paraphrasing or summarising, sentiment analysis, natural language BI, language modeling, and disambiguation.

Training a NER with BERT with a few lines of code in Spark NLP and getting SOTA accuracy. — NER is a subtask of information extraction that seeks to locate and classify named entities mentioned in unstructured text into pre-defined categories such as person names, organizations, locations, medical codes, time expressions, quantities, monetary values, percentages, etc.

In this series, we are going to write a separate article for each annotator in the Spark NLP library and this is the first one. In our first article, remember that we talked about certain types of columns that each Annotator accepts or outputs. So, what are we going to…

This is the second article in a series of blog posts to help Data Scientists and NLP practitioners learn the basics of Spark NLP library from scratch and easily integrate it into their workflows. This blogpost covers installation on MacOS and Linux machines, but I also shared the links below…

* This is the first article in a series of blog posts to help Data Scientists and NLP practitioners learn the basics of Spark NLP library from scratch and easily integrate it into their workflows. During this series, we will do our best to produce high-quality content and clear instructions…