Plugin live discovery issues

[butler]

The on the fly discovery of plugins has quite a few issue and needs to be looked at carefully. Some findings so far:

1. Creating a new model with the new model editor usually shows up as: [plugin]name_user indicating there is a duplicate of that name in the namespace. Interestingly exiting and reloading SasView and the name is now just [plugin]name.
2. Further if the active model is a plugin the model drop down is cleared (but not the GUI). However now, even if the model is reselected, or if the new model is selected (or any other plugin) an error is thrown on the console:

   2017-02-27 00:12:09 : Traceback (most recent call last):
     File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
       self.compute(*args, **kwargs)
   TypeError: 'NoneType' object is not callable
   
   2017-02-27 00:12:22 : Traceback (most recent call last):
     File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
       self.compute(*args, **kwargs)
   TypeError: 'NoneType' object is not callable
   

This is all cleared by simply clicking on "Load Plugin Models" and all works fine again.

3. Deleting a model, even if the current active category on the fit page is something else (e.g. cylinder), when choosing plugins and then a model from plugins (the deleted model does not show up) the following error is thrown on the command line console:

   Traceback (most recent call last):
     File "C:\Users\Paul\git\sasview_new\src\sas\sasgui\perspectives\fitting\fitpage.py", line 1347, in _onparamEnter
       self.save_current_state()
     File "C:\Users\Paul\git\sasview_new\src\sas\sasgui\perspectives\fitting\basepage.py", line 805, in save_current_state
       self.state.model = self.model.clone()
     File "C:\Users\Paul\git\sasmodels\sasmodels\sasview_model.py", line 506, in clone
       return deepcopy(self)
     File "C:\Users\Paul\workspace\SasView\Python27\lib\copy.py", line 190, in deepcopy
       y = _reconstruct(x, rv, 1, memo)
     File "C:\Users\Paul\workspace\SasView\Python27\lib\copy.py", line 334, in _reconstruct
       state = deepcopy(state, memo)
     File "C:\Users\Paul\workspace\SasView\Python27\lib\copy.py", line 163, in deepcopy
       y = copier(x, memo)
     File "C:\Users\Paul\workspace\SasView\Python27\lib\copy.py", line 256, in _deepcopy_dict
       for key, value in x.iteritems():
   RuntimeError: dictionary changed size during iteration
   

while the following error is given to the user in a popup:

	2017-02-27 00:02:13 : Traceback (most recent call last):
  File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
    self.compute(*args, **kwargs)
TypeError: 'NoneType' object is not callable

	2017-02-27 00:02:14 : Traceback (most recent call last):
  File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
    self.compute(*args, **kwargs)
TypeError: 'NoneType' object is not callable

and the log file

2017-02-27 00:02:13,983 ERROR Traceback (most recent call last):
  File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
    self.compute(*args, **kwargs)
  File "C:\Users\Paul\git\sasview_new\src\sas\sasgui\perspectives\fitting\model_thread.py", line 197, in compute
    output[index] = self.model.evalDistribution(self.data.x[index])
  File "C:\Users\Paul\git\sasmodels\sasmodels\sasview_model.py", line 582, in evalDistribution
    return self.calculate_Iq(qdist)
  File "C:\Users\Paul\git\sasmodels\sasmodels\sasview_model.py", line 617, in calculate_Iq
    return self._calculate_Iq(qx, qy)
  File "C:\Users\Paul\git\sasmodels\sasmodels\sasview_model.py", line 638, in _calculate_Iq
    magnetic=is_magnetic)
  File "C:\Users\Paul\git\sasmodels\sasmodels\kernelpy.py", line 165, in __call__
    self.q_input.nq, call_details, values, cutoff)
  File "C:\Users\Paul\git\sasmodels\sasmodels\kernelpy.py", line 193, in _loops
    return scale*form() + background
  File "C:\Users\Paul\git\sasmodels\sasmodels\kernelpy.py", line 151, in <lambda>
    self._form = lambda: form(q, *kernel_args)
  File "C:\Users\Paul\.sasview\plugin_models\broad_peak_test.py", line 90, in Iq
    with errstate(divide='ignore'):
TypeError: 'NoneType' object is not callable

2017-02-27 00:02:13,996 ERROR 2017-02-27 00:02:13 : Traceback (most recent call last):
  File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
    self.compute(*args, **kwargs)
TypeError: 'NoneType' object is not callable

2017-02-27 00:02:13,997 ERROR Traceback (most recent call last):
  File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
    self.compute(*args, **kwargs)
  File "C:\Users\Paul\git\sasview_new\src\sas\sasgui\perspectives\fitting\model_thread.py", line 197, in compute
    output[index] = self.model.evalDistribution(self.data.x[index])
  File "C:\Users\Paul\git\sasmodels\sasmodels\sasview_model.py", line 582, in evalDistribution
    return self.calculate_Iq(qdist)
  File "C:\Users\Paul\git\sasmodels\sasmodels\sasview_model.py", line 617, in calculate_Iq
    return self._calculate_Iq(qx, qy)
  File "C:\Users\Paul\git\sasmodels\sasmodels\sasview_model.py", line 638, in _calculate_Iq
    magnetic=is_magnetic)
  File "C:\Users\Paul\git\sasmodels\sasmodels\kernelpy.py", line 165, in __call__
    self.q_input.nq, call_details, values, cutoff)
  File "C:\Users\Paul\git\sasmodels\sasmodels\kernelpy.py", line 193, in _loops
    return scale*form() + background
  File "C:\Users\Paul\git\sasmodels\sasmodels\kernelpy.py", line 151, in <lambda>
    self._form = lambda: form(q, *kernel_args)
  File "C:\Users\Paul\.sasview\plugin_models\broad_peak_test.py", line 90, in Iq
    with errstate(divide='ignore'):
TypeError: 'NoneType' object is not callable

2017-02-27 00:02:14,088 ERROR 2017-02-27 00:02:14 : Traceback (most recent call last):
  File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
    self.compute(*args, **kwargs)
TypeError: 'NoneType' object is not callable

Again, simply clicking on "Load Plugin Models" seems to clean everything up and it now works again.

There are many permutations, particularly since there seems to be 4 slightly different style of models which can behave differently and combinations category and model in dropdown and which and when load, delete, create new, and edit a model are performed.

It may be that a lot of it is due to the dictionary of plugins not being updated by the on the fly discovery which is why the "load Plugin models" fixes things? or is there something about deep copies holding things around for too long?

[butler]

Note: all of the above is using the ticket-756 branch which fixes problems with using "Load Plugin Models" as described in that ticket. These problems exist in master as well at the moment but get convolved with the load issue (which now resolves problems with the ticket 756 fixes).

[butler]

Ticket #863 was a first observation of this much more broad category of problem. Am copying the content of that ticket here and closing #863.

Load Plugin is used to force an update of plugin models that may have changed but somehow not triggered an auto update. This works fine if the category at the time is not Plugin Model. However if it is "Plugin Model" and a plugin is selected on the fit page, load will clear the model dropdown box, but does not clear the rest of the model GUI stuff on the fit page. If the user then tries to change a parameter and run the model (e.g. by hitting return) throws an error. If the user then goes to the model selection drop down and reselects the model all is well again.

2017-02-26 11:30:47 : Traceback (most recent call last):
File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
    self.compute(*args, **kwargs)
AttributeError: 'NoneType' object has no attribute 'pow'

2017-02-26 11:30:47 : Traceback (most recent call last):
File "C:\Users\Paul\git\sasview_new\src\sas\sascalc\data_util\calcthread.py", line 268, in _run
    self.compute(*args, **kwargs)
AttributeError: 'NoneType' object has no attribute 'pow'

Interesting factoid: there are 4 types of models: new style (e.g. copied from the built in), created using simple model editor (essentially new style), old style plugins, and the sum/multiply models which remain a mix of styles. This last type in fact seems to be quite happy even when the drop down is cleared - i.e. it does not throw the error and in fact continues to work fine when parameters are changed as if it is loaded at some deeper level? maybe deep copy issue?

Possible solutions:

1. Rather than reset the selected model of Plugin Model, the code could reload whatever model had been selected and keep that model in the dropdown.
2. Finish the reset operation. Basically this is behaving similarly to changing category which clears the model selection box AND clears all the model GUI elements on the page. So just finish the job and clear out all those elements.

[butler]

From Paul Kienzle

Use formfactorbox.GetValue/SetValue rather than formfactorbox.GetLabel/SetLabel in fitting.py(321) update_custom_combo.

This was implemented in a branch but does not fix 2 though it does eliminate the problem of erasing the model in the drop-down. Needs more testing.

[wojciech]

Model reloading invoked from code doesn't seem to be working.

In fact when changing from new style to old style models "Load Plugin Models" doesn't work either.

This issue requires more investigation.

[butler]

this is rather annoying but have agreed to move to 4.2 due to lack of resources to address in a timely fashion for 4.1 release.

[butler]

Just noticed Wojciech's comment from this morning. I assume this is a mac thing as it mostly works on the PC though without branch-865 you have to choose a new model as I recall to get things to work again. I will test the current branch against the current master to see if there is value in pulling the current state of the branch before I actually move the ticket.

[butler]

Testing shows no advantage to merging at this time so moving to 4.2

[wojciech]

Tested on OSX 10.11. Issue 1 - "user" suffix is stil appended to newly created models. Otherwise it works fine!

[wojciech]

An error occured when tried to exit sasview after manipulation with plugins:

2017-05-16 13:54:25,474 ERROR gui_manager.clean_plugin_models:
10026   [Errno 30] Read-only file system: '/Volumes/SasView-4.1-MacOSX/SasView 4.1.app/Contents/Resources/plugin_models/__init__.py'

I need to reproduce it

[wojciech]

When I deleted old-style model (polynomial5) and then tried to remove model cretaed with editor I encountered following error:

Delete Error: 
Could not delete the file; Check if in use.

The model wasn't in use and now I cannot reproduce it…

[wojciech]

This may be unrealted issue but when editing model in "Advanced Plugin Editor" "Check Model" needs to be run to update model. "Saving" model is not sufficent.

[wojciech]

The problem with not being able to close sasview appears after deleting new style model (created with model editor) and then switching to old-style model (polynomial5 in my case) in drop down menu.

[butler]

Having agreed that this is still much better that it was we will leave these changes in 4.1.1 (and copy to master) but move this ticket to 4.2 — It may be that once we clear out all the old sasview model vestiges the remaining problems disappear.

[pkienzle]

Replying to wojciech:

Tested on OSX 10.11. Issue 1 - "user" suffix is stil appended to newly created models. Otherwise it works fine!

This is now fixed in the ticket-853-fit-gui-to-calc branch.

[pkienzle]

The branch ticket-853-fit-gui-to-calc (#853) includes changes to the GUI so that models are only reloaded if they are changed, and so that changed models preserve their parameters on update.

[butler]

after discussion with Paul and Wojciech it seems that this ticket is essentially done. To the extent that any issues remain a new ticket could be submitted or more likely we can wait till the new GUI. Now closing this.