is it worthy to use chat gpt for generating NVDA specific code to make solving issues more efficient? Let's maybe exchange tasks and questions for chat gpt which generate useful output.

[Adriani90]

The Cat GPT AI from Elon musk's company Open AI is incredibly advanced in performing dialogs and complex conversations. We could use that in the NVDA community to generate pieces of code but also for inspiration.

Here are some use cases that I see for the NVDA software:

• Better understand the code design and find code errors and mistakes more efficiently
• generate pieces of code for specific parts of a solution prototype
• discuss about improvements
• test if code is correct
• In an advanced stadium, we could give it a list of test criteria and then we could test if a code complies with them.

Happy to exchange here some thoughts and ideas.
@feerrenrut, @derekriemer, @josephsl, @michaelDCurran, @jcsteh, @codeofdusk, @seanbudd, @lukaszgo1, @LeonarddeR

[Adriani90]

Here is an example of output that I got from the chat gpt, this is really crazy. Well not perfect but by far not bad and maybe it has a lot of potential.

Question 1: write a code in python wich adds a new layer in NVDA scripts to change the script for the find dialog in NVDA which currently is nvda+ctrl+f.

Answer 1: Here is a sample code that you could use to add a new layer to the NVDA scripts to change the script for the find dialog in NVDA:
import globalPluginHandler
import ui
import api
import keyboardHandler

This class represents the new layer in the NVDA scripts

class FindDialogLayer(globalPluginHandler.GlobalPlugin):

This function is called when the user presses the new shortcut key combination

(nvda+alt+f in this example)

def script_find(self, gesture):

Open the find dialog

ui.find.open()

Set the focus to the search field in the find dialog

api.setFocus(api.getForegroundObject().children[0].children[1])

This function registers the new layer in the NVDA scripts

def registerNewLayer():

Create an instance of the new layer

newLayer = FindDialogLayer()

Register the new shortcut key combination (nvda+alt+f in this example)

and specify the script that should be executed when the user presses the keys

keyboardHandler.registerShortcut(newLayer, 'alt+f', 'find')

Call the registerNewLayer function to register the new layer in the NVDA scripts

registerNewLayer()
This code creates a new layer in the NVDA scripts called FindDialogLayer and adds a new shortcut key combination (nvda+alt+f in this example) that opens the find dialog and sets the focus to the search field in the dialog. You can modify this code to use a different shortcut key combination if you prefer.

Question 2: improve the code to add an entry in the input gestures dialog to let the users assign their own shortcut or to replace the current shortcut for the find dialog in NVDA

Answer 2: Here is an updated version of the code that adds a new layer to the NVDA scripts to change the script for the find dialog in NVDA and also adds an entry in the input gestures dialog to let the users assign their own shortcut or to replace the current shortcut for the find dialog:
import globalPluginHandler
import ui
import api
import keyboardHandler
import inputCore

This class represents the new layer in the NVDA scripts

class FindDialogLayer(globalPluginHandler.GlobalPlugin):

This function is called when the user presses the new shortcut key combination

(nvda+alt+f in this example)

def script_find(self, gesture):

Open the find dialog

ui.find.open()

Set the focus to the search field in the find dialog

api.setFocus(api.getForegroundObject().children[0].children[1])

This function registers the new layer in the NVDA scripts

def registerNewLayer():

Create an instance of the new layer

newLayer = FindDialogLayer()

Register the new shortcut key combination (nvda+alt+f in this example)

and specify the script that should be executed when the user presses the keys

keyboardHandler.registerShortcut(newLayer, 'alt+f', 'find')

Add an entry in the input gestures dialog to let the users assign their own

shortcut or to replace the current shortcut for the find dialog

inputCore.manager.registerGesture("find", "find")

Call the registerNewLayer function to register the new layer in the NVDA scripts

registerNewLayer()
This code includes the additional step of registering the new shortcut key combination in the input gestures dialog, which allows the user to customize the shortcut or to replace the default shortcut for the find dialog.

[Adriani90]

This example is also worth loking at:

Question 1: write a python code for NVDA to play a silence in the background to avoid the sound card from turning off when NVDA did not speak for longer than 10 seconds.
Answer 1: Here is an example of a python code that could be used to implement this feature in NVDA:
import time
from threading import Timer

silence_duration = 10 # duration in seconds after which silence should be played

def play_silence():
# Code to play a silence in the background
pass

def start_silence_timer():
Timer(silence_duration, play_silence).start()

def speak(text):
start_silence_timer()
# Code to speak the given text
pass

# Start the silence timer when the script is run
start_silence_timer()
In this code, the play_silence() function is used to play a silence in the background. This function can be implemented using a suitable audio library in python.
The start_silence_timer() function creates a timer that will call the play_silence() function after the specified duration of silence (in this case, 10 seconds). This function is called every time NVDA speaks, so that the timer is reset and a silence is not played while NVDA is speaking.
The speak() function is used to speak the given text. This function also calls the start_silence_timer() function to reset the timer before speaking the text.
The code starts the silence timer when the script is run, so that a silence will be played if NVDA does not speak for longer than the specified duration.

[josephsl]

Hi,

I think this might be useful in teaching people some parts of add-on development examples and possibilities, but I must point out that Chat GPT gave wrong answers to both questions/examples:

• Find command layer: wrong class to begin with, with recent scriptHandler.script decorator not in use to at least highlight how this can be used.
• Play silence: the AI (I must say) fundamentally misunderstood the question - it should not be playing silence after ten seconds - it could have scored better if it played it constantly.

I think it is important to also point out that Chat GPT is sensitive to prompt formulation. In other words, how humans ask questions can have an impact on the material the AI gathers to write its response. Not that the example prompts were bad - they were challenging and can produce million different answers.

My overall opinion is that Chat GPT can offer something for people to play with, but not at the level of offering its contribution to NVDA community at this time. Add-on development or writing NVDA pull requests involve human input in various stages - AI can help based on data it can fetch and run algorithms on, but we are dealing with development of a technology that strives for accessibility, something AI's may or may not grasp fully. In short, I tend to favor more human-led approach to software development.

Thanks.

[jcsteh]

ChatGPT has some amazing potential to help even advanced coders. With a few follow-up questions to refine the task, I was able to get it to quickly write an NVDA add-on for some clipboard related tasks with few (but still some) errors. However, even putting aside any potential licensing issues, human expertise is going to be required to screen for inaccuracy and ensure the most efficient, effective solution is being used. This is especially true for core contributions. I think that individual coders may choose to be inspired by ChatGPT as they wish, but what we don't want is a situation where non-coding contributors arbitrarily submit code generated by ChatGPT with no idea as to whether it even works or with no regard to efficiency, effectiveness or potential harm.

[Adriani90]

Yeah I fully agree with this, so that's why I actually wanted to start a discussion on this. I think the potential is there in which people might get atracted to submit code based on chatGPT output. The thing is that in a real scenario no one probably would recognize what comes from the chatGPT and what is really based on missing coding experience of the author. I think this AI can be trained really quickly with subsequent questions regarding code improvements and so on.

I think human interventions are very very important, so let's continue spread awareness for this point definitely. The efficiency gains you get in your workflow through this AI tool are impressive though.

[Adriani90]

cc: @Robert-J-H, @mltony, @ruifontes, @nvdaes, @AAClause, @JulienCochuyt

[Adriani90]

With Github Copilot X things will get really interesting.

https://github.com/features/preview/copilot-x
It seems it builds on Chat GPT4.