Changeset b76cf0d in sasview for src/sas/sascalc/dataloader

Timestamp:

Sep 15, 2016 8:33:57 AM (8 years ago)

Author:

Piotr Rozyczko <rozyczko@…>

Branches:

ESS_GUI, ESS_GUI_Docs, ESS_GUI_batch_fitting, ESS_GUI_bumps_abstraction, ESS_GUI_iss1116, ESS_GUI_iss879, ESS_GUI_iss959, ESS_GUI_opencl, ESS_GUI_ordering, ESS_GUI_sync_sascalc

Children:

7743c09

Parents:

6d8f692

git-author:

Lewis O'Driscoll <lewis.o'driscoll@…> (08/18/16 10:19:47)

git-committer:

Piotr Rozyczko <rozyczko@…> (09/15/16 08:33:57)

Message:

Improve efficiency of CanSAS 2.0 reader

When it came to reading the Q and I data values, the reader was looping
through every single data point in the hdf5 file and appending it to an
array. This caused problems when loading very large files, rendering
SasView? unresponsive. Now, the reader saves the entire array from the
file, instead of looping through each point.

File:

• src/sas/sascalc/dataloader/readers/cansas_reader_HDF5.py (modified) (3 diffs)

src/sas/sascalc/dataloader/readers/cansas_reader_HDF5.py

@@ -62 +62 @@
         :return: List of Data1D/2D objects and/or a list of errors.
         """
-
         ## Reinitialize the class when loading a new data file to reset all class variables
         self.reset_class_variables()
@@ -136 +135 @@
                 ## If this is a dataset, store the data appropriately
                 data_set = data[key][:]
+                unit = self._get_unit(value)
+
+                ## I and Q Data
+                if key == u'I':
+                    if type(self.current_dataset) is plottable_2D:
+                        self.current_dataset.data = data_set.flatten()
+                        self.current_dataset.zaxis("Intensity", unit)
+                    else:
+                        self.current_dataset.y = data_set.flatten()
+                        self.current_dataset.yaxis("Intensity", unit)
+                    continue
+                elif key == u'Idev':
+                    if type(self.current_dataset) is plottable_2D:
+                        self.current_dataset.err_data = data_set.flatten()
+                    else:
+                        self.current_dataset.dy = data_set.flatten()
+                    continue
+                elif key == u'Q':
+                    self.current_dataset.xaxis("Q", unit)
+                    if type(self.current_dataset) is plottable_2D:
+                        self.current_dataset.q = data_set.flatten()
+                    else:
+                        self.current_dataset.x = data_set.flatten()
+                    continue
+                elif key == u'Qy':
+                    self.current_dataset.yaxis("Q_y", unit)
+                    self.current_dataset.qy_data = data_set.flatten()
+                    continue
+                elif key == u'Qydev':
+                    self.current_dataset.dqy_data = data_set.flatten()
+                    continue
+                elif key == u'Qx':
+                    self.current_dataset.xaxis("Q_x", unit)
+                    self.current_dataset.qx_data = data_set.flatten()
+                    continue
+                elif key == u'Qxdev':
+                    self.current_dataset.dqx_data = data_set.flatten()
+                    continue
+                elif key == u'Mask':
+                    self.current_dataset.mask = data_set.flatten()
+                    continue
 
                 for data_point in data_set:
                     ## Top Level Meta Data
-                    unit = self._get_unit(value)
                     if key == u'definition':
                         self.current_datainfo.meta_data['reader'] = data_point
@@ -148 +187 @@
                     elif key == u'SASnote':
                         self.current_datainfo.notes.append(data_point)
-
-                    ## I and Q Data
-                    elif key == u'I':
-                        if type(self.current_dataset) is plottable_2D:
-                            self.current_dataset.data = np.append(self.current_dataset.data, data_point)
-                            self.current_dataset.zaxis("Intensity", unit)
-                        else:
-                            self.current_dataset.y = np.append(self.current_dataset.y, data_point)
-                            self.current_dataset.yaxis("Intensity", unit)
-                    elif key == u'Idev':
-                        if type(self.current_dataset) is plottable_2D:
-                            self.current_dataset.err_data = np.append(self.current_dataset.err_data, data_point)
-                        else:
-                            self.current_dataset.dy = np.append(self.current_dataset.dy, data_point)
-                    elif key == u'Q':
-                        self.current_dataset.xaxis("Q", unit)
-                        if type(self.current_dataset) is plottable_2D:
-                            self.current_dataset.q = np.append(self.current_dataset.q, data_point)
-                        else:
-                            self.current_dataset.x = np.append(self.current_dataset.x, data_point)
-                    elif key == u'Qy':
-                        self.current_dataset.yaxis("Q_y", unit)
-                        self.current_dataset.qy_data = np.append(self.current_dataset.qy_data, data_point)
-                    elif key == u'Qydev':
-                        self.current_dataset.dqy_data = np.append(self.current_dataset.dqy_data, data_point)
-                    elif key == u'Qx':
-                        self.current_dataset.xaxis("Q_x", unit)
-                        self.current_dataset.qx_data = np.append(self.current_dataset.qx_data, data_point)
-                    elif key == u'Qxdev':
-                        self.current_dataset.dqx_data = np.append(self.current_dataset.dqx_data, data_point)
-                    elif key == u'Mask':
-                        self.current_dataset.mask = np.append(self.current_dataset.mask, data_point)
 
                     ## Sample Information