4.2.0 plugin is does not load earlier project properly

[smk78]

Sabrina Gaertner here is trying to finish off some analysis she started a couple of years ago using SasView 4.1.0. Project file attached.

4.2.0 (release or Jenkins builds) does load the project, but with some caveats.

The project requires a lamellar+gaussian_peak plugin. If you create this in 4.2.0 (ie, new architecture) then load the project, all of the FitPage parameters remain at their defaults.

If, instead, you use the same plugin created in 4.1.0 (previous architecture), then all of the FitPage parameters are updated with the values in the project file: the desired behaviour!

On intermittent occasions (it is difficult to pin down the exact circumstances because of all the things we were trying), we got this error:

2018-09-27 13:59:35,467 : ERROR : sas.sasgui.guiframe.gui_statusbar (gui_statusbar.pyc:80) :: 2018-09-27 13:59:35 : Error: This model state has missing or outdated information.
Traceback (most recent call last):
  File "sas\sasgui\perspectives\fitting\fitpage.pyc", line 2036, in reset_page
  File "sas\sasgui\perspectives\fitting\basepage.pyc", line 1257, in reset_page_helper
  File "sas\sasgui\perspectives\fitting\basepage.pyc", line 2343, in _set_dipers_Param
  File "sas\sasgui\perspectives\fitting\basepage.pyc", line 2430, in _on_select_Disp
  File "sas\sasgui\perspectives\fitting\fitpage.pyc", line 603, in _set_sizer_dispersion
  File "sas\sasgui\perspectives\fitting\basepage.pyc", line 2421, in _reset_dispersity
  File "sas\sasgui\perspectives\fitting\basepage.pyc", line 843, in save_current_state
  File "sasmodels\sasview_model.pyc", line 520, in clone
  File "copy.pyc", line 190, in deepcopy
  File "copy.pyc", line 334, in _reconstruct
  File "copy.pyc", line 163, in deepcopy
  File "copy.pyc", line 256, in _deepcopy_dict
RuntimeError: dictionary changed size during iteration

Is this indicating that there is a problem with the lookup table of old parameters to new ones?

This bug seems to have some parallels with #1183, and since that is presently a blocker I'm making this a blocker also.

[smk78]

4.1.0 project file (15 FitPages?)

[smk78]

Plugin created in 4.1.0

[smk78]

Plugin created in 4.2.0

[butler]

The "intermittent" error is one I have also seen while working with plugins in various ways, but only very "intermittently" and thus not reported as I could not say what to do to cause the error. So if anybody can tell by looking at the error what is going on that would be great… but don't think it is a lookup table issue given I don't think I have been working with old to new that much?

On the main point, I assume that it is the *old* project file being loaded with a new plugin that is failing? or does this happen with a project file saved with the same plugin?

[smk78]

The project loads with either plugin, but the saved fit parameters only get put into the FitPages if you use the old model plugin.

[pkienzle]

Parameters in old model:

p1_scale
p1_background
p1_thickness
p1_sld
p1_sld_solvent
p2_scale
p2_background
p2_peak_pos
p2_sigma

Parameters in new model:

scale
background
A_scale
A_thickness
A_sld
A_sld_solvent
B_scale
B_peak_pos
B_sigma

The save file format does not record that it was a mixture model, only that it was a plugin of some arbitrary name. The file loader in page state doesn't know how to convert parameters from an old-style mixture model to the new style. It could probably guess, renaming p1_ to A_, p2_ to B_, etc, and setting scale to 1, and background to p1_background+p2_background. Any additional transformations, such as changes in the parameter name for the underlying models between 3.x and 4.x, would require more work, guessing the name of the models that went into the mixture, transforming their parameters, then combining the parameter sets. It gets more complicated if it is a sum of sum.

For now you could edit the svs file and rename the parameters by hand (global search and replace for p1 => A and p2 => B gets you most of the way there), or with a simple script if there are a lot of files.

[butler]

Thinking about this and talking to @krzywon as well as @pkienzle this is probably something that cannot actually be fixed. Some specific cases may be possible but not simple. Basically if the plugin model is not kept along with the project file it will get tricky.

However, if the plugin was not kept with the project file, besides the fix that @pkienzle mentions, the easiest would probably be to remake the plugin in the SasView version that created the project. All the versions are available and it is pretty quick to download and install then create the sum model.

Thus I don't see that it is a blocker for 4.2 (if it is 4.2 will not be going out any time soon) so recommend we move to 4.3 at which point we need to have a discussion about how complicated and convoluted structures of code we should or not aim for. What cases Do we want to handle? Should we be thinking about this as part of the 5.0 projects? i.e. do we start storing the code used in the project if it is a plugin for example?

[smk78]

That’s interesting. I’m sure I added a comment and closed this ticket earlier… must have forgotten to click submit. Doh!

Anyway, what it said was that I’ve updated Known Issues in the Release Notes with @pkienzle’s workaround. Suggest therefore we close this ticket.

[butler]

I agree. trying to make a partial fix this is going to be messy at best. Probably best to add this to the many other issues to consider when thinking about refactoring Save/Load projects in 5.x. Do we need a ticket for that?

Meanwhile closing the ticket as recommended by @smk78