custom Llama 7B gpt4all large language model AI for NVDA

[joshknnd1982]

Could you please train and create a custom Llama 7B quantized bin, large language model for NVDA? 7B is best because it uses the least ram so the most people could use it. Please train it on email discussions for all the NVDA email lists, addon development documentation, NVDA user guides and keyboard commands, keyboard shortcuts for office, web browsers, and other popular apps, large amounts of python code from various NVDA addons and any other stuff you can think of. Such a large language model NVDA expert should cut down on list traffic, help people use NVDA better and more effectively and efficiently. It would also cut down on tech support questions. Also add the ability for users to ask it how to do specific tasks in word, and other apps with NVDA. It could even write basic code snippets for making simple addons for relabeling buttons in apps and things and more including information on making new espeak variants and tips for espeak customizations. Lastly this llm will have to be made into a compatible quantized .bin file.

[fernando-jose-silva]

A feature I like about voice over for iphone that I think might have something to do with this is the real-time recognition of pictures and photos:
For example, putting your finger on a payment receipt on whatsapp that is in a photo and the screen reader reading what is written on the receipt is wonderful.
Putting your finger on a hycon and hearing the shape of this hycon is very interesting.
In the photo library, being able to have an idea of what's in the photo just by placing your finger on it is really cool.
Functionality can be added to nvda with add-ons such as cloud vision, what I would add is the possibility of seeing hycon shapes on the screen (configurable as on or off), and automatic recognition of photos in apps or files, and perhaps offline as works in voice over on many occasions.

[XLTechie]

"AI Features in NVDA" should be a discussion.

[joshknnd1982]

One other thing I thought of. gpt4all with its custom NVDA llm, the exe file lets you add various commands including providing prompts right away changing various parameters and so on. This means that you could do some really crazy stuff with this NVDA gpt AI model. Imagine an NVDA addon that will give the custom NVDA gpt access to the internet and updated github code. In other words gpt4all could be automated to do cool stuff with some python code. It could find updated NVDA topics online, give them to gpt4all NVDA custom llm as input go get info on some other app, give the gpt llm that input, remember both things, then take action based on the AI's response.

[joshknnd1982]

you may be interested in checking this out for a future NVDA goal.

https://github.com/Torantulino/Auto-GPT