running with OpenAI GPT

[raywgs]

Hey,

Do I just replace the KEYS with my OpenAI API key for the run.sh to run using gpt-4? Or where do I configure that?

Thanks

[simran-arora]

Yes, you should include your key in the bash script

The keys get pulled in here, when the Manifest objects are being constructed:

evaporate/evaporate/utils.py

Line 138 in 83204a5

def get_manifest_sessions(MODELS, MODEL2URL=None, KEYS=[]):

[raywgs]

Hey simran, how do I run evaporate on my own dataset where I don't have a ground truth? I just want to test the performance on my own data that is a list of txt files.

[simran-arora]

Hi! the gold extractions are used to provide feedback in a couple places in the pipeline.

1. When the WS label model is being trained. You can just put in a dummy file or modify the code to not perform the scoring.

   evaporate/evaporate/profiler.py

   Line 160 in 83204a5

   args.gold_extractions_file,

2. In the run_profiler.py code you can comment out anything that's an evaluation method. For instance:

   evaporate/evaporate/run_profiler.py

   Line 284 in 83204a5

   results = evaluate_synthetic_main(

Hope that helps!

[raywgs]

So, I can use an empty json file in place for the gold extractions and still have evaporate run normally?