I am trying to run the following code in my azure databricks workbook
import pyspark.ml.feature
from pyspark.ml.feature import Tokenizer,StopWordsRemover
tokenizer = Tokenizer()
However I am facing this error:
Py4JError: An error occurred while calling
None.org.apache.spark.ml.feature.Tokenizer. Trace:
py4j.security.Py4JSecurityException: Constructor public
org.apache.spark.ml.feature.Tokenizer(java.lang.String) is not
whitelisted.
Similar errors are coming up for StopWordsRemover and some other functions from pyspark.ml.feature too
Is there a work around to avoid this error so that I can use the same code?
New contributor
Ipsheet Agarwal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.