Learning Objectives

• understand javascript resources (더 이상 물러날 곳이 없다…!)
  • https://github.com/shiffman/ML-for-Creative-Coding/tree/main/08-transformers-js

Notes

Work Desk

• playing around with code examples asking my question of the day………

  

  

  

• I looked into the list of models compatible with transformers.js that Dan shared, and the sentence similarity / text classification ones stood out for me.

  

  https://huggingface.co/nomic-ai/nomic-embed-text-v1.5

  • task prefixes: search_document, search_query, clustering, classification

  https://huggingface.co/jinaai/jina-reranker-v2-base-multilingual

  • ideas:
    • nomic embedder: put in corpus from a short story everyone’s familiar with (e.g. Alice in Wonderland?) and search from dialog - quiz if it’s what Alice said
    • jina reranker: give a sentence. prompt user to write that in another language they’re familiar with. retrieve the sentence with a language that has the closest semantic distance

• errors

  • nomic embedder: does not have pipeline sentence-similarity 🤔

    

    

    

    • when I changed the pipeline to text-classification, it says it’s unsupported.

  • jina reranker: won’t load model

    

    

    • this time, the model is not even recognized.

  • then I kept running other models one by one..

  • I got excited because I thought this worked: https://huggingface.co/Xenova/nllb-200-distilled-600M

    • but alas, it was a false alarm..

    • using this, I aim to 1) let the user write in an input with English, then get a phrase back in one random language(out of this list).

      

      • another iteration I’m considering is to load a generative model and 2) give fortune-cookie esque statements, one in English and one in another random language. I tried making the generation part using the https://editor.p5js.org/ml_4_cc/sketches/o7C5EfNtt but this one also failed to run 😭 and reasoning models are ridiculous..

        

        srsly?

Questions

• how to define the dtypes? And how to know to set what device?

    // Load the Transformers.js model pipeline with async/await
    let options = {
      device: "webgpu",
      dtype: "fp16",
      progress_callback: logProgress,
    };
  

• errors running example sketches: