I have created a web-app using Django. I this web-app I want to add functionality to extract phrases from content. My code is working fine in development but not working in production. Using nltk
package I have created a function which gives returns a list of phrases from content. below is my code:
<code>import nltk
from typing import List
def extract_phrases(content: str) -> List:
# Tokenize text into sentences
sentences = nltk.sent_tokenize(content)
# Initialize list to store phrases
phrases = set()
# Iterate through each sentence
for sentence in sentences:
# Tokenize sentence into words
words = nltk.word_tokenize(sentence)
# Get part-of-speech tags for words
pos_tags = nltk.pos_tag(words)
# Initialize list to store phrases in current sentence
current_phrase = []
# Iterate through each word and its part-of-speech tag
for word, pos_tag in pos_tags:
# If the word is a noun or an adjective, add it to the current phrase
if pos_tag.startswith("NN") or pos_tag.startswith("JJ"):
current_phrase.append(word)
# If the word is not a noun or adjective and the current phrase is not empty,
# add the current phrase to the list of phrases and reset it
elif current_phrase:
phrases.add(" ".join(current_phrase))
current_phrase = []
# Add the last phrase from the current sentence (if any)
if current_phrase:
phrases.add(" ".join(current_phrase))
return list(phrases)
</code>
<code>import nltk
from typing import List
def extract_phrases(content: str) -> List:
# Tokenize text into sentences
sentences = nltk.sent_tokenize(content)
# Initialize list to store phrases
phrases = set()
# Iterate through each sentence
for sentence in sentences:
# Tokenize sentence into words
words = nltk.word_tokenize(sentence)
# Get part-of-speech tags for words
pos_tags = nltk.pos_tag(words)
# Initialize list to store phrases in current sentence
current_phrase = []
# Iterate through each word and its part-of-speech tag
for word, pos_tag in pos_tags:
# If the word is a noun or an adjective, add it to the current phrase
if pos_tag.startswith("NN") or pos_tag.startswith("JJ"):
current_phrase.append(word)
# If the word is not a noun or adjective and the current phrase is not empty,
# add the current phrase to the list of phrases and reset it
elif current_phrase:
phrases.add(" ".join(current_phrase))
current_phrase = []
# Add the last phrase from the current sentence (if any)
if current_phrase:
phrases.add(" ".join(current_phrase))
return list(phrases)
</code>
import nltk
from typing import List
def extract_phrases(content: str) -> List:
# Tokenize text into sentences
sentences = nltk.sent_tokenize(content)
# Initialize list to store phrases
phrases = set()
# Iterate through each sentence
for sentence in sentences:
# Tokenize sentence into words
words = nltk.word_tokenize(sentence)
# Get part-of-speech tags for words
pos_tags = nltk.pos_tag(words)
# Initialize list to store phrases in current sentence
current_phrase = []
# Iterate through each word and its part-of-speech tag
for word, pos_tag in pos_tags:
# If the word is a noun or an adjective, add it to the current phrase
if pos_tag.startswith("NN") or pos_tag.startswith("JJ"):
current_phrase.append(word)
# If the word is not a noun or adjective and the current phrase is not empty,
# add the current phrase to the list of phrases and reset it
elif current_phrase:
phrases.add(" ".join(current_phrase))
current_phrase = []
# Add the last phrase from the current sentence (if any)
if current_phrase:
phrases.add(" ".join(current_phrase))
return list(phrases)
The function and entire web-app properly working in development environment. But in production where the nltk module is used that line is not executing.
Note
I have activated my virtual environment and run following code also:
<code>import nltk
nltk.download('all')
</code>
<code>import nltk
nltk.download('all')
</code>
import nltk
nltk.download('all')