I installed pytorch, transformers, and python-dotenv to run this script:

from transformers import pipeline
from dotenv import load_dotenv
import torch

import os
import json
# from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

load_dotenv()

device = "cuda" if torch.cuda.is_available() else "cpu"

# device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")

def summarize_definitions(definitions):
    # Use a pipeline as a high-level helper

    pipe = pipeline("text-generation", model="mistralai/Mistral-Nemo-Instruct-2407", device=device)


    cleaned_definitions = {}

    for i, (term, defs) in enumerate(definitions.items()):
        combined_defs = f"Please summarize these definitions into a JSON array of simple ideally 1-3 word definitions: {json.dumps(defs, indent=2)}"

        # Summarize the combined definitions

        messages = [
          {"role": "user", "content": combined_defs},
        ]
        summary = pipe(messages, min_length=1, max_new_tokens=1000)

        print(summary)

    return cleaned_definitions

def main():
    with open('import/language/tibetan/definitions.out.json', 'r', encoding='utf-8') as f:
        definitions = json.load(f)

    cleaned_definitions = summarize_definitions(definitions)
    print(cleaned_definitions)

if __name__ == "__main__":
    main()

When I run it, I just see this (it already previously installed 5 model chunks, each about 4-5GB):

$ python3 transform.py
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████| 5/5 [00:37<00:00,  7.48s/it]
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.

Then it just sits there.

I tried commenting back in this line:

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")

To see if I could use the GPU to speed things up? But it just said this and exited without any other content/messaging:

$ python3 transform.py
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████| 5/5 [00:39<00:00,  7.88s/it]
zsh: killed     python3 transform.py
(venv) $ /opt/homebrew/Cellar/[email protected]/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

Any ideas what I’m doing wrong? Is it supposed to take a long time to get the first output? Or what is supposed to happen? I am new to this AI stuff, though have been programming for a while.