I am currently making a system for users to generate flashcards for Languages.
This involves adding information such as the definition, pronunciation, and example sentences for a word.
So far, I have had success by loading all of the information into a Python Dictionary, however as I add more features/words, I will run out of memory.
I am curious if I should either move to loading text files on the fly, or move the information to a MySQL database. The most important metric is lookup speed, and I know a lot of work has gone into making SQL databases performant, however I may be able to have better performance by specializing the text files for my use case (sort the files alphabetically, or create a hash lookup system)
The information stored will be static, and users can choose what information they want to include in the flashcards. Right now, the dataset is about 10GB in-memory. Adding more features and languages will take this to above 50GB
I’d appreciate any recommendations on possible approaches
Jarvis Coghlin is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.