Queries on v2.0b1

[FateDecider]

Functionality questions:

1. If you measure more than two variables in a recipe, is it true that you can plot all of them on the right side of the Scanner window?
2. How does "Multiple Recipes" functionality work? Do they run in parallel or one after another?
3. Thank you and very nice work on implementing "Multiple Parameters". Does the "Multiple Parameters" functionality run like a nested loop?
4. I understand we can execute recipe steps without sweeping any parameters. Does this mean I can measure all the instrument variables (and save them in a log file) under the first parameter (which is blank) before performing the N-dimensional scan?
5. Thanks for implementing the functionality to pass variables between different devices. With the added functionalities, I think it is now possible to save data recorded in the recipe automatically,
   i.e. users can write a customized "Save" step (this would be an action in Driver.py) after the sub-parameter scan has been completed.
   With that in mind, I think it is also possible to dynamically change the saved file's title depending on the various scan parameter values. Do you think so?

Misc questions:

1. Does the slider function work for 'float' type variables?
2. In which GUI py source file can I adjust the font size of the GUI?
3. In Ver 1.2.1, I realized sometimes the Scanner mode might record data too soon when starting a new parameter sweep (especially during Continuous Scan mode). The problem went away after I added pause times in scan.py, e.g. 0.2 sec or so before starting new scans and between each scan parameter value.
4. How do users (effectively) use utility functions in Drivers_utilities.py? I have been using the GUI without writing any utility functions for my drivers.
5. In the plots for Scanner and Plotter, is it possible to export the Transformed data (e.g. FFT) into a CSV? That did not seem to work for me in ver 1.2.1.

Thank you again for this package! :) 👍

[Python-simulation]

Small disclaimer, most of your questions refer to branch V2_candidate (and older versions) that is not yet "official", it still need some review before being accepted to 'master' branch and then pypi.

Functionality answers:

1. You can plot as many variables as you want. If the variable is of type int, float, bool it will be added to the scan pd.DataFrame, and x and y axes can be selected with the combobox at the bottom of the figure (you can only plot one y vs x at a time if it is the question). If it is of type np.ndarray pd.DataFrame it will be plotted separately and accessible using the top combobox with name 'Scan' or variable name of array/dataframe and you will have one plot line for each recipe step (for each parameter value). If it is of type str, byte, tuple it will not be plotted but will be saved in a folder like array and dataframe.
2. Multiple recipes will be executed in the order given in the scan, so one after another.
3. Multiple parameters uses the function product from the package itertools so for each step, each parameter is set again. For example if you define param1=[1,2,3] and param2=[5,6,7], a scan will do a recipe by setting the parameters as follow: param1=1 param2=5, param1=1 param2=6, param1=1 param2=7, param1=2 param2=5, param1=2 param2=6 .... I may changes this in the future to only change param1 when its value change and not for each param2 changes (but without changing the order of execution).
4. It is possible to execute a recipe without parameter, it can be used to initialize and terminate an instrument (start laser, return to 0 voltage) or to do one measurement of some parameters. If you have one recipe with no parameter, then the data will be saved as usual but without the parameter value. If you do multiple recipes, then you will have a file per recipe. For example saving 'test_measurement.txt' will results in saving two files 'test_measurement_recipe_1.txt' and 'test_measurement_recipe_2.txt'.
5. Do you talk about $eval: to execute python code? In that case, yes you can set a custom path directly in the scan without coding it in the driver. For example $eval:r"C:\Users\username\data_for_voltage"+f"={dummy.constant()}V.txt" will be interpreted as "C:\Users\username\data_for_voltage=0.0V.txt". But I think this is not needed in most cases because ALL data measured during a scan are saved.
   
   It could however be useful if you need to save data directly in an instrument PC with a custom path.

Misc answers:

1. Slider work for any int or float that is both readable and writable
2. For figures (scan, monitor, plotter) it would be in function pyqtgraph_fig_ax in core/utilities.py. For the GUI itself, you can add QtWidgets.QApplication.setFont(font) to core/gui/__init__.py
   
   I never though of it before but it could be a feature to add in autolab_config.ini
3. Do you mean that the first step of a recipe is measured before the parameter has been set in the instrument but was set in the scan? For example you set volt=2 in parameter but measure volt=0 instead of 2 in step? If so, this could mean that the instrument doesn't wait that the parameter is set to release the command. All instruments should have the command '*OPC?' after any command to ask the end of the command before continuing the code -> use self.query('*OPC?') in the driver after a command (to not be confused with the command self.write("*WAI") that tell a instrument to wait internally the end of execution before proceeding to next command). But personally I always use a wait command as first step to avoid this problem that could also come from transient measurement if the change needs time to stabilize.
4. The _utilities.py file is only used for os shell commands that I personally don't use, don't bother with it for the gui :)
5. Plots and their options comes from the package pyqtgraph. I just looked at it and you can save the fft data in h5 format (csv says original data so can't work). To read back data you can do this: import h5py; f = h5py.File(r"my_file.h5", 'r'); dset = f["Export"]; import matplotlib.pyplot as plt; plt.plot(dset[0], dset[1]) with dset[0] being x of first plot, dset[1] y of first plot and then dset[2] for x of second plot.... Personally I prefer to analyze data in post-process and use autolab only to acquire data.

I hope I've answered all your questions, but don't hesitate to ask for more details or clarifications.

[FateDecider]

I see, thanks for the reply. I will try soon. I suppose v2 is now official?

By the way at which part of the code I should add *OPC commands?

[Python-simulation]

Strange, is your self.timeout really the self.controller.timeout of self.controller = rm.open_resource(address) ? If so, maybe your instrument can't use OPC correctly, search in its doc if it is mentioned. Otherwise, you can use a wait action from the driver system:

The file scan.py is not used by autolab, it is an artifact of a aborted version. I will maybe make use of it in the future but for now don't consider it.

[FateDecider]

I see, thanks. I added time.sleep(0.1) within Driver_VISA's read and it resolved everything.

Thanks for introducing the wait function. I realized now you could specify a variable parameter under action. Is there a way to have the status (green/orange) light enabled for action? Say my action is turn_on_motor, and my parameter is either on/off. I want to be able to check the on/off status under the action line, without having to define another boolean variable.

Also, I need advice for setting up configuration. I added a step to a recipe, export configuration; closed the scanner and reopened it. When I try to import the configuration file, I get an error like this:
"Impossible to load configuration file: Address Oscillating Delay.scan amplitude not found for step 1 (scan amplitude_2)"

[Python-simulation]

I'm glad it works for you but a quick word about time.sleep, you should not use it directly inside a driver because reading or setting a value should not wait on itself.
For example you want to set 10 instruments in a row and read only after that, it is useless to have a wait inside all set functions because it will slow down for no reason the execution of your script/scan, one wait function outside of them could do the same job if you want to read after.
Then, don't put it in read functions neither because if you monitor a variable, you will wait between each point for no reason, or if you measure 10 independent variables, you will also wait extra time for nothing.
So what I recommend you to do for your scan is to use the driver named system to add a wait step. That way you control when and how long you wait before a specific operation. What I usually do is to set a wait step from system as first step of my scan and reduce the time until I see an issue with the setting of the parameter/reading of the same parameter. By doing so, I reduce the delay at its minimum to avoid having unnecessary long scan. Then, I add some extra wait steps only between set and measure steps that I think should have some delay between them.

About status for actions. I think there is a misunderstanding on what the green/orange light represent. For a variable, orange mean that the value written in the edit line has been modified. Green mean that the value has been read. For example if you have a variable that is only writable and you change its value, you will always have an orange light. Setting it keeps the light orange but it can't be read even if the value has been correctly set.
For an action, changing the argument has no consequence on the action so having a orange light is not relevant. Because an action is not readable, a green light has no meaning.
I think that for an action, it is sufficient to see that no error has been raised to see if it worked.
Lastly, even if you would have a different way to know that an action has been performed, it is in no case a way to know the state of a something. Only a boolean that can be read will tell you for sure what is the current state of something.

For the error you get, it is a bug that I was not aware of. Having space in a variable name seems to not work during the config importation. For now, the only solution for you would be to change the name of this variable in your driver to remove the space. I will fixe this in the next update.

[FateDecider]

I see. The first issue I last described was pretty strange because no matter how long we waited (either specified as a recipe step or in a read function within Driver class), it would yield a reading that was last read. For example, I have variables A and B on the control panel. After I initiate the device, I first read variable A which gives the correct reading. Next, when I read variable B (no matter how long later I do this step), I will get the last reading (which is variable A) and I have to click 'read' twice to get the readout of variable B. This problem completely went away after I added a short sleep within Driver_VISA:

def read(self): # 5) Defines a read function
time.sleep(0.1)
rep = self.inst.read() # Receives a string 'rep' from the instrument and return it
return rep

Thanks for the configuration fix. It now works for me. I have a small feature suggestion. Could you also add 'variable unit' (perhaps beside 'variable type') under the set steps in a recipe? I plan to have a fixed configuration in that we only change the value of the set parameters within the scanner and not the control panel.

May you elaborate a bit more on what you have plotted on the right of the multiple_recipe.png? (https://autolab.readthedocs.io/en/latest/gui/scanning.html) Does the black/white intensity represent the data in the third dimension? Is it possible to view the values of the 2D plane of the 3D array you have measured?

[Python-simulation]

What you describe in your issue looks like something I experienced with my yenista_tunics laser. If I set the wavelength, then the power, and read the power value, I will first get the wavelength instead and need to read again to have the power. I didn't find a solution to it, so if your problem is similar and having a delay works, then you are lucky :)

Yes I fixed the configuration issue in V2.0b2_candidate. But I would still recommend to use variable without space as it avoid confusion when using $eval: feature. i.g. a variable named 'my variable' from a driver 'test' is accessible with $eval:test.myvariable() which is not intuitive.

If I understand correctly, you would like to see the unit of a step inside a recipe. I will think where it can be put. But you mention setting a parameter, and the unit of the parameter is already shown to the right side of the parameter name.

To test new features I used the driver dummy, that I strongly recommend to check out to see what you can do with your own driver. The plot represent the variable dummy.array_3D, which is a bad name according to a friend :), consisting of an array 200 by 200 with random float values. It represent what could be outputted by a camera, so yes the grey scale represent the third dimension.

I thought the slicing option was something to implement because I didn't succeed to make it work but it turns out it was a color issue (plotting white line on white background..). Once again a bug has been found thanks to you ;).
If you change the background color, you can click on ROI to have a rectangle on your image that plot a slice (I think it is a mean of all slice in one direction) bellow your image. Check pyqtgraph documentation if you need more informations.

Currently you can change the background/foreground color by changing/removing the line 55 in autolab/core/gui/__init__.py : pg.setConfigOptions(background='w', foreground="k") but I will later add an option in autolab_config.ini and fixe the color issue by setting the plot to the chosen default foreground color instead of the default white color.

Here is an example using dummy.Image (image taken from my black-hole project):