Add a LanguageModel class that implements the Llama2 architecture via llama2.c and Emscripten

[gohai]

This makes it possible to do inference on toy-models (15M, 42M, 110M parameters) using the Llama2 architecture, as implemented in llama2.c.

The included emscripten build artifacts came from this tree: https://github.com/gohai/llama2.c-emscripten

TODO:

• CDN for models & tokenizer.bin
• not firm about naming of things ("LanguageModel"?, "manual*"?, "tokens"?)
• would be nice to come up with a Colab notebook to do training/fine-tuning of custom models

[shiffman]

Wow, this is amazing @gohai!

First, as we work on documenting this, a wonderful reference is Let Us Show You How GPT Works by @aatishb. Hi Aatish, tagging you mostly to say hi, but would of course welcome your input!

I added a few comments @gohai. In my experience with teaching the previous charRNN models with ml5.js, the main pain points were:

• Generating character by character or token by token was very confusing for students and should probably be reserved for more advanced use cases (though great to include in ml5.js if we can?)
• What students often want to do is train their own model, but I was never able to successfully pull this off with all the steps required to train in python then convert to JS. A clear and easy to use colab notebook could absolutely work though!

My other "worry" relates to treading into this territory and providing "out of the box" models. LLMs, as is well documented, have all sorts of bias and other issues. It would wonderful to collaborate with the team members who are working on educational materials and documentation to think about how we guide users to be aware of the limitations and possible dangers of certain models and datasets. The code of conduct can be a good reference as well for considering use cases and applications. This could be a great discussion for the full group! cc @sproutleaf

[shiffman]

Agreed it would be great to support preload().

Also, I wonder if we should require a "string" that references a specific model to load. Even though this is perhaps an extra, unecessary step it emphasizes to the user that this isn't magic, and requires an acknowledgement of the specific model they are using. Some options:

  lm = ml5.languageModel("TinyStories", onModelLoaded);
  lm = ml5.languageModel("TinyLlamas", onModelLoaded);

Referencing the model name is probably more important, but it's also important to emphasize the specific dataset that was used for training.

[shiffman]

I love the idea of an onToken() event, but for beginners it would be incredibly useful to have an API that mirrors the HuggingFace inference API. In this case, you pass a prompt and a desired number of tokens and you get the entire result back. Something along the lines of:

  let options = {
    temperature: 0.9,
    maxTokens: 100
  };
  lm.generate(prompt, options, gotText);

I might also consider just be the form prompt, maxTokens, callback and then a JS object with more properties if you wanted to set temperature and other more, maybe:

  lm.generate(prompt, 100, gotText);

and then:

  let options = {
    prompt: "How are you?",
    temperature: 0.9,
    maxTokens: 100
  };
  lm.generate(options, gotText);

Though maybe it's good to always have the prompt separatel

[gohai]

@aatishb I so love this article, particularly how it raises the transparency/blackbox-ism issues with OpenAI to a general audience! (Believe that's also at the core for why bother with toy models - not to suggest a similar type of performance is achievable in the browser, but to unpack this technology, and how it got built/trained, as widely as possibly.)

Thank you for taking the time to think through this, @shiffman!

• The suggestion of requiring the user to spell out the model name makes so much sense! (implemented this in 4eb89d9) I feel similarly about what you wrote about the dangers of whatever the "out-of-the-box" experience is going to be. Looking forward to working with the larger group on this!
• Following your suggestion, I changed the main callback to be on-completion rather than on-token and simplified the most basic example. (b544b24)
• I'll look into training of custom models. From scratch on the TinyStories dataset seems to take a "couple of hours" on four A100. This is probably a step to far for Colab 😄, but perhaps there will be improvements in the implementation. There are people working on implementing fine-tuning using LoRA, this might be doable!
• Preload now works (hackishly, d962efe)

[shiffman]

I like how simple it is to have just a single string that comes back, but it might match how other models in ml5.js work if we instead of wrap everything inside an object, for example:

function gotText(results) {
  console.log(results.text);
  console.log(results.words);
}

{
  prompt: "How are you",
  text: "I am doing fine.",
  words: ["I", "am", "doing", "fine"]
}

I could also imagine returning a prompt property. Sometimes students want to include the prompt in what they display back and sometimes they don't. Keeping track of it in a global variable can be awkward (especially if there are multiple calls to generate() so having it available in the results can be helpful!

[gohai]

@shiffman words and out are properties of the instance currently - would passing the instance as the result make sense, or would you feel a "result" object value is nicer, which the user is able to mutate as they like etc? (text over out for the literal output?) I'll add prompt, although currently its strictly the beginning of the response - more of a "prefix" really.
Thank you for your review! I'll try it out