Changeset d65fb2f in sasview for src/sas

Timestamp:

Aug 19, 2016 6:57:38 AM (8 years ago)

Author:

lewis

Branches:

master, ESS_GUI, ESS_GUI_Docs, ESS_GUI_batch_fitting, ESS_GUI_bumps_abstraction, ESS_GUI_iss1116, ESS_GUI_iss879, ESS_GUI_iss959, ESS_GUI_opencl, ESS_GUI_ordering, ESS_GUI_sync_sascalc, costrafo411, magnetic_scatt, release-4.1.1, release-4.1.2, release-4.2.2, release_4.0.1, ticket-1009, ticket-1094-headless, ticket-1242-2d-resolution, ticket-1243, ticket-1249, ticket885, unittest-saveload

Children:

a854f21

Parents:

cde6e52

git-author:

Lewis O'Driscoll <lewis.o'driscoll@…> (08/18/16 08:19:47)

git-committer:

Lewis O'Driscoll <lewis.o'driscoll@…> (08/19/16 06:57:38)

Message:

Improve efficiency of CanSAS 2.0 reader

When it came to reading the Q and I data values, the reader was looping
through every single data point in the hdf5 file and appending it to an
array. This caused problems when loading very large files, rendering
SasView? unresponsive. Now, the reader saves the entire array from the
file, instead of looping through each point.

File:

• src/sas/sascalc/dataloader/readers/cansas_reader_HDF5.py (modified) (3 diffs)

src/sas/sascalc/dataloader/readers/cansas_reader_HDF5.py

@@ -62 +62 @@
         :return: List of Data1D/2D objects and/or a list of errors.
         """
-
         ## Reinitialize the class when loading a new data file to reset all class variables
         self.reset_class_variables()
@@ -136 +135 @@
                 ## If this is a dataset, store the data appropriately
                 data_set = data[key][:]
+                unit = self._get_unit(value)
+
+                ## I and Q Data
+                if key == u'I':
+                    if type(self.current_dataset) is plottable_2D:
+                        self.current_dataset.data = data_set.flatten()
+                        self.current_dataset.zaxis("Intensity", unit)
+                    else:
+                        self.current_dataset.y = data_set.flatten()
+                        self.current_dataset.yaxis("Intensity", unit)
+                    continue
+                elif key == u'Idev':
+                    if type(self.current_dataset) is plottable_2D:
+                        self.current_dataset.err_data = data_set.flatten()
+                    else:
+                        self.current_dataset.dy = data_set.flatten()
+                    continue
+                elif key == u'Q':
+                    self.current_dataset.xaxis("Q", unit)
+                    if type(self.current_dataset) is plottable_2D:
+                        self.current_dataset.q = data_set.flatten()
+                    else:
+                        self.current_dataset.x = data_set.flatten()
+                    continue
+                elif key == u'Qy':
+                    self.current_dataset.yaxis("Q_y", unit)
+                    self.current_dataset.qy_data = data_set.flatten()
+                    continue
+                elif key == u'Qydev':
+                    self.current_dataset.dqy_data = data_set.flatten()
+                    continue
+                elif key == u'Qx':
+                    self.current_dataset.xaxis("Q_x", unit)
+                    self.current_dataset.qx_data = data_set.flatten()
+                    continue
+                elif key == u'Qxdev':
+                    self.current_dataset.dqx_data = data_set.flatten()
+                    continue
+                elif key == u'Mask':
+                    self.current_dataset.mask = data_set.flatten()
+                    continue
 
                 for data_point in data_set:
                     ## Top Level Meta Data
-                    unit = self._get_unit(value)
                     if key == u'definition':
                         self.current_datainfo.meta_data['reader'] = data_point
@@ -148 +187 @@
                     elif key == u'SASnote':
                         self.current_datainfo.notes.append(data_point)
-
-                    ## I and Q Data
-                    elif key == u'I':
-                        if type(self.current_dataset) is plottable_2D:
-                            self.current_dataset.data = np.append(self.current_dataset.data, data_point)
-                            self.current_dataset.zaxis("Intensity", unit)
-                        else:
-                            self.current_dataset.y = np.append(self.current_dataset.y, data_point)
-                            self.current_dataset.yaxis("Intensity", unit)
-                    elif key == u'Idev':
-                        if type(self.current_dataset) is plottable_2D:
-                            self.current_dataset.err_data = np.append(self.current_dataset.err_data, data_point)
-                        else:
-                            self.current_dataset.dy = np.append(self.current_dataset.dy, data_point)
-                    elif key == u'Q':
-                        self.current_dataset.xaxis("Q", unit)
-                        if type(self.current_dataset) is plottable_2D:
-                            self.current_dataset.q = np.append(self.current_dataset.q, data_point)
-                        else:
-                            self.current_dataset.x = np.append(self.current_dataset.x, data_point)
-                    elif key == u'Qy':
-                        self.current_dataset.yaxis("Q_y", unit)
-                        self.current_dataset.qy_data = np.append(self.current_dataset.qy_data, data_point)
-                    elif key == u'Qydev':
-                        self.current_dataset.dqy_data = np.append(self.current_dataset.dqy_data, data_point)
-                    elif key == u'Qx':
-                        self.current_dataset.xaxis("Q_x", unit)
-                        self.current_dataset.qx_data = np.append(self.current_dataset.qx_data, data_point)
-                    elif key == u'Qxdev':
-                        self.current_dataset.dqx_data = np.append(self.current_dataset.dqx_data, data_point)
-                    elif key == u'Mask':
-                        self.current_dataset.mask = np.append(self.current_dataset.mask, data_point)
 
                     ## Sample Information