Changeset 63ddc03 in sasview


Ignore:
Timestamp:
Apr 9, 2018 12:00:41 PM (7 years ago)
Author:
krzywon
Branches:
master, magnetic_scatt, release-4.2.2, ticket-1009, ticket-1094-headless, ticket-1242-2d-resolution, ticket-1243, ticket-1249, unittest-saveload
Children:
0863065
Parents:
4a8d55c (diff), b09aee8 (diff)
Note: this is a merge changeset, the changes displayed below correspond to the merge itself.
Use the (diff) links above to see all the changes relative to each parent.
Message:

Merge branch 'master' into ticket-1068-2

Files:
6 added
9 edited
1 moved

Legend:

Unmodified
Added
Removed

  • src/sas/sasgui/perspectives/calculator/model_editor.py

    r2469df7 rc6dfb9f  
    391391        self._msg_box.SetLabel(msg) 
    392392        self._msg_box.SetForegroundColour(color) 
     393        self._set_model_list() 
    393394        if self.parent.parent is not None: 
    394395            from sas.sasgui.guiframe.events import StatusEvent 
    … …  
    432433        if len(main_list) > 1: 
    433434            main_list.sort() 
     435        self.model1.Clear() 
     436        self.model2.Clear() 
    434437        for idx in range(len(main_list)): 
    435438            self.model1.Append(str(main_list[idx]), idx) 

  • src/sas/sasgui/perspectives/fitting/fitpage.py

    rbfeb823 ra7c6f38  
    365365        # StaticText for chi2, N(for fitting), Npts + Log/linear spacing 
    366366        self.tcChi = BGTextCtrl(self, wx.ID_ANY, "-", size=(75, 20), style=0) 
    367         self.tcChi.SetToolTipString("Chi2/Npts(Fit)") 
     367        self.tcChi.SetToolTipString("Chi2/DOF (DOF=Npts-Npar fitted)") 
    368368        self.Npts_fit = BGTextCtrl(self, wx.ID_ANY, "-", size=(75, 20), style=0) 
    369369        self.Npts_fit.SetToolTipString( 
    … …  
    391391        self.points_sizer.Add(self.pointsbox) 
    392392 
    393         box_description_1 = wx.StaticText(self, wx.ID_ANY, '   Chi2/Npts') 
     393        box_description_1 = wx.StaticText(self, wx.ID_ANY, 'Reduced Chi2') 
    394394        box_description_2 = wx.StaticText(self, wx.ID_ANY, 'Npts(Fit)') 
    395395 

  • src/sas/sasgui/perspectives/fitting/media/fitting_help.rst

    r5005ae0 r47ace50  
    426426See :ref:`Assessing_Fit_Quality`. 
    427427 
    428 The objective of model-fitting is to find a *physically-plausible* model, and set 
    429 of model parameters, that generate a theory that reproduces the experimental data 
    430 and gives residual values as close to zero as possible. 
     428The objective of model-fitting is to find a *physically-plausible* model, and 
     429set of model parameters, that generate a theory that reproduces the experimental 
     430data and minimizes the values of the residuals. 
    431431 
    432432Change the default values of the model parameters by hand until the theory line 
    433 starts to represent the experimental data. Then uncheck the tick boxes alongside 
    434 all parameters *except* the 'background' and the 'scale'. Click the *Fit* button. 
    435 SasView will optimise the values of the 'background' and 'scale' and also display 
    436 the corresponding uncertainties on the optimised values. 
    437  
    438 *NB: If no uncertainty is shown it generally means that the model is not very* 
    439 *dependent on the corresponding parameter (or that one or more parameters are* 
    440 *'correlated').* 
    441  
    442 In the bottom left corner of the *Fit Page* is a box displaying the normalised value 
    443 of the statistical $\chi^2$ parameter returned by the optimiser. 
     433starts to represent the experimental data. Then check the tick boxes alongside 
     434the 'background' and 'scale' parameters. Click the *Fit* button. SasView 
     435will optimise the values of the 'background' and 'scale' and also display the 
     436corresponding uncertainties on the optimised values. 
     437 
     438.. note:: 
     439   If the uncertainty on a fitted parameter is unrealistically large, or if it 
     440   displays as NaN, the model is most likely a poor representation of the data, 
     441   the parameter in question is highly correlated with one or more of the other 
     442   fitted parameters, or the model is relatively insensitive to the value of 
     443   that particular parameter. 
     444 
     445In the bottom left corner of the *Fit Page* is a box displaying a normalised 
     446value of the statistical $\chi^2$ parameter (the reduced $\chi^2$, 
     447See :ref:`Assessing_Fit_Quality`) returned by the optimiser. 
    444448 
    445449Now check the box for another model parameter and click *Fit* again. Repeat this 
    446 process until most or all parameters are checked and have been optimised. As the 
    447 fit of the theory to the experimental data improves the value of 'chi2/Npts' will 
    448 decrease. A good model fit should easily produce values of 'chi2/Npts' that are 
    449 close to one, and certainly <100. See :ref:`Assessing_Fit_Quality`. 
     450process until all relevant parameters are checked and have been optimised. As 
     451the fit of the theory to the experimental data improves, the value of 'Reduced 
     452Chi2' will decrease. A good model fit should produce values of Reduced Chi2 
     453close to one, and certainly << 100. See :ref:`Assessing_Fit_Quality`. 
    450454 
    451455SasView has a number of different optimisers (see the section :ref:`Fitting_Options`). 
    … …  
    462466*the Data Explorer is checked (see the section* :ref:`Loading_data` *).* 
    463467 
    464 This mode is an extension of the :ref:`Single_Fit_Mode` that fits two or more data 
    465 sets *to the same model* simultaneously. If necessary it is possible to constrain 
    466 fit parameters between data sets (eg, to fix a background level, or radius, etc). 
     468This mode is an extension of the :ref:`Single_Fit_Mode` that allows for some 
     469relatively extensive constraints between fitted parameters in a single *FitPage* 
     470or between several *FitPage*'s (eg, to constrain all fitted parameters to be the 
     471same in a contrast series of *FitPages* except for the solvent sld parameter, 
     472constrain the length to be twice that of the radius in a single *FitPage*, 
     473fix the radius of the sphere in one *FitPage* to be the same as the radius of 
     474the cylinder in a second *FitPage*, etc). 
    467475 
    468476If the data to be fit are in multiple files, load each file, then select each file 
    … …  
    501509next to *Add Constraint?* in the *Fit Constraints* box. 
    502510 
     511To constrain all identically named parameters to fit *simultaneously* to the 
     512same value across all the *Fitpages* use the *Easy Setup* drop-down buttons in 
     513the *Const & Simul Fit* page. 
     514 
    503515*NB: You can only constrain parameters that are set to refine.* 
     516 
     517Constraints will generally be of the form 
     518 
     519  Mi Parameter1 = Mj.Parameter1 
     520 
     521however the text box after the '=' sign can be used to adjust this 
     522relationship; for example 
     523 
     524  Mi Parameter1 = scalar \* Mj.Parameter1 
     525 
     526A 'free-form' constraint box is also provided. 
     527 
     528Many constraints can be entered for a single fit. 
    504529 
    505530When ready, click the *Fit* button on the *Const & Simul Fit* page, NOT the *Fit* 
    506531button on the individual *FitPage*'s. 
    507532 
    508 Simultaneous Fits without Constraints 
    509 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
    510  
    511533The results of the model-fitting will be returned to each of the individual 
    512534*FitPage*'s. 
    513535 
    514 Note that the chi2/Npts value returned is the SUM of the chi2/Npts of each fit. To 
    515 see the chi2/Npts value for a specific *FitPage*, click the *Compute* button at the 
    516 bottom of that *FitPage* to recalculate. Also see :ref:`Assessing_Fit_Quality`. 
    517  
    518 Simultaneous Fits with Constraints 
    519 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
    520  
    521 Use the *Easy Setup* drop-down buttons in the *Const & Simul Fit* page to set 
    522 up constraints between *FitPage*'s. 
    523  
    524 Constraints will generally be of the form 
    525  
    526   Mi Parameter1 = Mj.Parameter1 
    527  
    528 however the text box after the '=' sign can be used to adjust this 
    529 relationship; for example 
    530  
    531   Mi Parameter1 = scalar \* Mj.Parameter1 
    532  
    533 A 'free-form' constraint box is also provided. 
    534  
    535 Many constraints can be entered for a single fit. 
    536  
    537 The results of the model-fitting will be returned to each of the individual 
    538 *FitPage*'s. 
    539  
    540 Note that the chi2/Npts value returned is the SUM of the chi2/Npts of each fit. To 
    541 see the chi2/Npts value for a specific *FitPage*, click the *Compute* button at the 
    542 bottom of that *FitPage* to recalculate. Also see :ref:`Assessing_Fit_Quality`. 
     536Note that the Reduced Chi2 value returned is the SUM of the Reduced Chi2 of 
     537each fit. To see the Reduced Chi2 value for a specific *FitPage*, click the  
     538*Compute* button at the bottom of that *FitPage* to recalculate. Note that in 
     539doing so the degrees of freedom will be set to Npts. 
     540See :ref:`Assessing_Fit_Quality`.  Moreover in the case of constraints the 
     541degrees of freedom are less than one might think due to those constraints. 
    543542 
    544543.. ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ 
    … …  
    766765.. ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ 
    767766 
    768 .. note::  This help document was last changed by Paul Butler, 10 September 
    769    2017 
     767.*Document History* 
     768 
     769| 2017-09-10 Paul Butler 
     770| 2017-09-15 Steve King 
     771| 2018-03-05 Paul Butler 

  • src/sas/sasgui/perspectives/fitting/media/residuals_help.rst

    r99ded31 r84ac3f1  
    2727 
    2828$\chi^2$ is a statistical parameter that quantifies the differences between 
    29 an observed data set and an expected dataset (or 'theory'). 
    30  
    31 When showing the a model with the data, *SasView* displays this parameter 
    32 normalized to the number of data points, $N_\mathrm{pts}$ such that 
     29an observed data set and an expected dataset (or 'theory') calculated as 
    3330 
    3431.. math:: 
    3532 
    36   \chi^2_N 
    37   =  \sum[(Y_i - \mathrm{theory}_i)^2 / \mathrm{error}_i^2] / N_\mathrm{pts} 
     33  \chi^2 
     34  =  \sum[(Y_i - \mathrm{theory}_i)^2 / \mathrm{error}_i^2] 
    3835 
    39 When performing a fit, *SasView* instead displays the reduced $\chi^2_R$, 
    40 which takes into account the number of fitting parameters $N_\mathrm{par}$ 
    41 (to calculate the number of 'degrees of freedom'). This is computed as 
     36Fitting typically minimizes the value of $\chi^2$.  For assessing the quality of 
     37the model and its "fit" however, *SasView* displays the traditional reduced 
     38$\chi^2_R$ which normalizes this parameter by dividing it by the number of 
     39degrees of freedom (or DOF). The DOF is the number of data points being 
     40considered, $N_\mathrm{pts}$, reduced by the number of free (i.e. fitted) 
     41parameters, $N_\mathrm{par}$. Note that model parameters that are kept fixed do 
     42*not* contribute to the DOF (they are not "free"). This reduced value is then 
     43given as 
    4244 
    4345.. math:: 
    … …  
    4749  / [N_\mathrm{pts} - N_\mathrm{par}] 
    4850 
    49 The normalized $\chi^2_N$ and the reduced $\chi^2_R$ are very close to each 
    50 other when $N_\mathrm{pts} \gg N_\mathrm{par}$. 
     51Note that this means the displayed value will vary depending on the number of 
     52parameters used in the fit. In particular, when doing a calculation without a 
     53fit (e.g. manually changing a parameter) the DOF will now equal $N_\mathrm{pts}$ 
     54and the $\chi^2_R$ will be the smallest possible for that combination of model, 
     55data set, and set of parameter values. 
     56 
     57When $N_\mathrm{pts} \gg N_\mathrm{par}$ as it should for proper fitting, the 
     58value of the reduced $\chi^2_R$ will not change very much. 
    5159 
    5260For a good fit, $\chi^2_R$ tends to 1. 
    … …  
    9098| 2015-06-08 Steve King 
    9199| 2017-09-28 Paul Kienzle 
     100| 2018-03-04 Paul Butler 

  • test/sascalculator/test/utest_sas_gen.py

    r5bb05a4 r39a018b  
    6060        Test that the calculator calculates. 
    6161        """ 
    62         f = self.omfloader.read("A_Raw_Example-1.omf") 
     62        f = self.omfloader.read(find("A_Raw_Example-1.omf")) 
    6363        omf2sld = sas_gen.OMF2SLD() 
    6464        omf2sld.set_data(f) 

  • src/sas/sascalc/dataloader/file_reader_base_class.py

    ra58b5a0 r4a8d55c  
    3131FIELDS_2D = ('data', 'qx_data', 'qy_data', 'q_data', 'err_data', 
    3232                 'dqx_data', 'dqy_data', 'mask') 
    33  
     33DEPRECATION_MESSAGE = ("\rThe extension of this file suggests the data set migh" 
     34                       "t not be fully reduced. Support for the reader associat" 
     35                       "ed with this file type has been removed. An attempt to " 
     36                       "load the file was made, but, should it be successful, " 
     37                       "SasView cannot guarantee the accuracy of the data.") 
    3438 
    3539class FileReader(object): 
    … …  
    4044    # List of allowed extensions 
    4145    ext = ['.txt'] 
     46    # Deprecated extensions 
     47    deprecated_extensions = ['.asc', '.nxs'] 
    4248    # Bypass extension check and try to load anyway 
    4349    allow_all = False 
    … …  
    8793                    if not self.f_open.closed: 
    8894                        self.f_open.close() 
     95                    if any(filepath.lower().endswith(ext) for ext in 
     96                           self.deprecated_extensions): 
     97                        self.handle_error_message(DEPRECATION_MESSAGE) 
    8998                    if len(self.output) > 0: 
    9099                        # Sort the data that's been loaded 
    … …  
    146155        else: 
    147156            logger.warning(msg) 
     157            raise NoKnownLoaderException(msg) 
    148158 
    149159    def send_to_output(self): 

  • src/sas/sascalc/dataloader/loader.py

    rdc8d1c2 r4a8d55c  
    9090            ascii_loader = ascii_reader.Reader() 
    9191            return ascii_loader.read(path) 
     92        except NoKnownLoaderException: 
     93            pass  # Try the Cansas XML reader 
    9294        except DefaultReaderException: 
    9395            pass  # Loader specific error to try the cansas XML reader 
    … …  
    100102            cansas_loader = cansas_reader.Reader() 
    101103            return cansas_loader.read(path) 
     104        except NoKnownLoaderException: 
     105            pass  # Try the NXcanSAS reader 
    102106        except DefaultReaderException: 
    103107            pass  # Loader specific error to try the NXcanSAS reader 

  • src/sas/sasgui/guiframe/local_perspectives/data_loader/data_loader.py

    r20fa5fe r2924532  
    185185            try: 
    186186                message = "Loading {}...\n".format(p_file) 
    187                 self.load_update(output=output, message=message, info="info") 
     187                self.load_update(message=message, info="info") 
    188188                temp = self.loader.load(p_file, format) 
    189189                if not isinstance(temp, list): 
    … …  
    201201                        else: 
    202202                            file_errors[basename] = [error_message] 
    203                         self.load_update(output=output, 
    204                             message=error_message, info="warning") 
    205  
    206                 self.load_update(output=output, 
    207                 message="Loaded {}\n".format(p_file), 
    208                 info="info") 
     203 
     204                self.load_update(message="Loaded {}\n".format(p_file), 
     205                                 info="info") 
    209206 
    210207            except NoKnownLoaderException as e: 
    211208                exception_occurred = True 
    212                 logger.error(e.message) 
    213  
    214209                error_message = "Loading data failed!\n" + e.message 
    215                 self.load_update(output=None, message=e.message, info="warning") 
     210                self.load_complete(output=None, 
     211                                   message=error_message, 
     212                                   info="warning") 
    216213 
    217214            except Exception as e: 
    218215                exception_occurred = True 
    219                 logger.error(e.message) 
    220  
    221216                file_err = "The Data file you selected could not be " 
    222217                file_err += "loaded.\nMake sure the content of your file" 
    … …  
    225220                file_err += " following:\n" 
    226221                file_err += e.message 
    227                 file_errors[basename] = [file_err] 
     222                self.load_complete(output=None, 
     223                                   message=file_err, 
     224                                   info="error") 
    228225 
    229226        if len(file_errors) > 0: 
    230227            error_message = "" 
    231228            for filename, error_array in file_errors.iteritems(): 
    232                 error_message += "The following errors occured whilst " 
     229                error_message += "The following issues were found whilst " 
    233230                error_message += "loading {}:\n".format(filename) 
    234231                for message in error_array: 
    235232                    error_message += message + "\n" 
    236                 error_message += "\n" 
    237             if not exception_occurred: # Some data loaded but with errors 
    238                 self.load_update(output=output, message=error_message, info="error") 
    239  
    240         if not exception_occurred: # Everything loaded as expected 
     233                error_message = error_message[:-1] 
     234            self.load_complete(output=output, 
     235                               message=error_message, 
     236                               info="error") 
     237 
     238        elif not exception_occurred: # Everything loaded as expected 
    241239            self.load_complete(output=output, message="Loading data complete!", 
    242240                               info="info") 
    … …  
    244242            self.load_complete(output=None, message=error_message, info="error") 
    245243 
    246  
    247     def load_update(self, output=None, message="", info="warning"): 
     244    def load_update(self, message="", info="warning"): 
    248245        """ 
    249246        print update on the status bar 
    250247        """ 
    251248        if message != "": 
    252             wx.PostEvent(self.parent, StatusEvent(status=message, info=info, 
     249            wx.PostEvent(self.parent, StatusEvent(status=message, 
     250                                                  info=info, 
    253251                                                  type="progress")) 
    254252 
    … …  
    257255         post message to status bar and return list of data 
    258256        """ 
    259         wx.PostEvent(self.parent, StatusEvent(status=message, info=info, 
     257        wx.PostEvent(self.parent, StatusEvent(status=message, 
     258                                              info=info, 
    260259                                              type="stop")) 
    261260        if output is not None: 

  • test/sasdataloader/test/utest_generic_file_reader_class.py

    rf53d684 r4a8d55c  
    88import numpy as np 
    99 
    10 from sas.sascalc.dataloader.data_info import DataInfo, plottable_1D 
     10from sas.sascalc.dataloader.data_info import DataInfo, plottable_1D, Data1D 
     11from sas.sascalc.dataloader.loader import Loader 
     12from sas.sascalc.dataloader.loader_exceptions import NoKnownLoaderException 
    1113from sas.sascalc.dataloader.file_reader_base_class import FileReader 
    1214 
    … …  
    2426        self.bad_file = find("ACB123.txt") 
    2527        self.good_file = find("123ABC.txt") 
     28        self.generic_reader = Loader() 
     29        self.deprecated_file_type = find("FEB18012.ASC") 
    2630 
    2731    def test_bad_file_path(self): 
    28         output = self.reader.read(self.bad_file) 
    29         self.assertEqual(output, []) 
     32        self.assertRaises(NoKnownLoaderException, self.reader.read, 
     33                          self.bad_file) 
    3034 
    3135    def test_good_file_path(self): 
    … …  
    3640        self.assertEqual(len(output), 1) 
    3741        self.assertEqual(output[0].meta_data["blah"], '123ABC exists!') 
     42 
     43    def test_old_file_types(self): 
     44        f = self.generic_reader.load(self.deprecated_file_type) 
     45        last_f = f[0] 
     46        if hasattr(last_f, "errors"): 
     47            self.assertEquals(len(last_f.errors), 1) 
     48        else: 
     49            self.fail("Errors did not propogate to the file properly.") 
     50 
     51    def test_same_file_unknown_extensions(self): 
     52        # Five files, all with the same content, but different file extensions 
     53        no_ext = find("test_data//TestExtensions") 
     54        not_xml = find("test_data//TestExtensions.notxml") 
     55        # Deprecated extensions 
     56        asc_dep = find("test_data//TestExtensions.asc") 
     57        nxs_dep = find("test_data//TestExtensions.nxs") 
     58        # Native extension as a baseline 
     59        xml_native = find("test_data//TestExtensions.xml") 
     60        # Load the files and check contents 
     61        no_ext_load = self.generic_reader.load(no_ext) 
     62        asc_load = self.generic_reader.load(asc_dep) 
     63        nxs_load = self.generic_reader.load(nxs_dep) 
     64        not_xml_load = self.generic_reader.load(not_xml) 
     65        xml_load = self.generic_reader.load(xml_native) 
     66        self.check_unknown_extension(no_ext_load[0]) 
     67        self.check_unknown_extension(asc_load[0]) 
     68        self.check_unknown_extension(nxs_load[0]) 
     69        self.check_unknown_extension(not_xml_load[0]) 
     70        self.check_unknown_extension(xml_load[0]) 
     71        # Be sure the deprecation warning is passed with the file 
     72        self.assertEquals(len(asc_load[0].errors), 1) 
     73        self.assertEquals(len(nxs_load[0].errors), 1) 
     74 
     75    def check_unknown_extension(self, data): 
     76        self.assertTrue(isinstance(data, Data1D)) 
     77        self.assertEquals(len(data.x), 138) 
     78        self.assertEquals(data.sample.ID, "TK49 c10_SANS") 
     79        self.assertEquals(data.meta_data["loader"], "CanSAS XML 1D") 
    3880 
    3981    def tearDown(self): 
Note: See TracChangeset for help on using the changeset viewer.