Changes in / [4e9f227:786685e] in sasview
- Files:
-
- 11 edited
Legend:
- Unmodified
- Added
- Removed
-
sansview/README.txt
r10be315 r7a67f4da 4 4 SasView 3.0.0 5 5 6 - Implemented Polarization and magnetic SANS parameters in some models. 7 - New Tool: Generic SANS / PDB reader & atomic scattering calculator 8 - New Tool: Image viewer 6 - The GUI look and feel has been refactored to be more familiar for Windows 7 users by using MDI frames. Graph windows are also now free-floating 8 - Four new models have been added: CoreShellEllipsoidXTModel, 9 RectangularPrismModel, RectangularHollowPrismModel and 10 RectangularHollowPrismInfThinWallsModel 11 - The data loader now supports ILL DAT data files and reads the full meta 12 information from canSAS file formats 9 13 - Redefined angles of angular parameters for anisotropic models 10 - Use MDI frames in GUI 11 12 - Previous Versions: 13 14 - Changed the application name to 'SasView'. 15 - Model category can be modified via (Category Manager). 16 - Improved Grid/Batch window functionality. 17 - Added a new tool; Data operation for addition, subtraction, multiplication, division, and combination of two data sets. 18 - Extended Sum Model Editor to Summation and Multiplication Editor. 19 - Better compiling error captures in the 'New' model editor. 20 - More outputs (e.g., Rg, background, or rod diameter) on LinearFit in certain axis scales. 21 - Added DataInfo for data in the DataExplorer and plots 22 - Added Maximize/Restore button in the title bar of the graphs 23 - The 'x' button now works as deleting the graph 24 - Added a hide button in the toolbar of the graph panel 25 - Fixed some bugs and improved some behaviors in the batch/grid panel. 26 - Edit SUM Model from the menubar can now generate more than one sum model. 27 - Report can now be saved as a pdf format on WIN and MAC. 28 - Multiprocessor support(Windows) 29 - Simple custom model editor 30 - Advanced model editor 31 - Sum model editing panel 32 - 3D graphic (for 2d data/results) and legend moves with mouse 33 - New Tool: density to vol fraction calculator and Python file editor 34 - Batch Fit included. 35 - More Graph modifications. 36 - More options for the fit weighting. 37 - Added a Python (PyCrust) shell in the 'Tool' menu. 38 - The 'Startup Setting' in the View menu can remember the last data file folder. 39 - Updated the resolution computation for the gravitational effect and added TOF estimation capability. 40 - Fixed the problem of displaying the fit results in the wrong parameter (with ParkMC FitEngine). 41 - Pr Inversion: Fixed a missing Rg output 42 - Startup Setting: fixed a problem with DataExplorer ON/OFF 43 - Fixed a bug w/ 2D color map dialog 44 - Minor feature added: Enable to load a data folder from the command line 45 - Much easier graphical user interface 46 - Optimized for the speed and accuracy of the computations 47 - Added Many shape, polymer, and other models including SphericalSLD, OnionExponetialShell, and even ReflectivityModels 48 - Added Data Explorer to manage, plot, delete, or setup for computation 49 - Added Instrumental resolution estimator (as a Tool) 50 - Customizable Startup appearance 51 - More functionalities on plot panels 52 - Combined Modeling and Fitting 53 - Save/open a SasView project or SasView analysis (subproject) 54 - Start the SasView application from a data file by double-clicking or from command line 55 - Easy manipulation of data and plot 56 - Provides Normalized residual plot 57 - Added useful key-combinations to copy (fitpage), paste (fitpage), change fit-tolerance, etc. 58 - Report 59 - 2D masked circular averaging 60 - 2D smearing calculation now uses dQ_parrellel and dQ_perpendicular 61 - Improved the speed of loading 2D data. 62 - Improved the speed of P*S calculations. 63 - Added 2D smearing calculation for dQx and dQy given along the x-y axes. 64 - MAC release 65 - Implemented the invariant, volume fraction, and specific surface area computation. 66 - Implemented the scattering length density calculator. 67 - Re-structured 2D calculation to 2D reduced(Q) data rather than raw pixel data 68 - Capable of Mask enhanced 2D calculation and fitting. 69 - Added a 2D mask data editor. 70 - Added inputs for the slit and pinhole resolution. 71 - Added a slit size calculator. 72 - Support more format options to save a graph. 73 - Enable to display multiple data sets in one graph by loading a data into the graph. 74 - Added a tool bar in a plot panel. 75 - Implemented P(r) inversion (Indirect Fourier transformations). 76 - Improved fitting and model calculation speed by a factor of ten. 77 - Supporting many more model functions. 78 - Supporting P(Q)*S(Q) for most of the shape based form factors. 79 - Added more distribution functions for the polydispersion calculations. 80 - Added a bookmark feature so that the results of the calculation can be recalled later. 81 - Q range reset button is added. 82 - Added a color bar in 2D data plots. 83 - Added a model function detail button for an easy access to the model help from the fitting panel. 84 - Simultaneous fit of a number of different sets of data with/without the constraints. 85 - Loading and displaying 1D and 2D data of various formats. 86 - 1D and 2D data fitting using Scipy or Park (a MC fitting optimizer) fit engine. 87 - 2D data manipulation and modeling. 88 - Supporting a number of standard model and model-independent functions including form factor and structure factor functions and their multiplications. 89 - Plug-in mechanism for data readers. 90 - Easy pop-up menu by mouse clicking on a given plot. 91 - Users arrange the various windows. 92 - Supporting varius 2D averaging methods : Circular, sectorslicer, annulus, boxsum, boxQx and boxQy. 93 - User defined Qrange (Qmin and Qmax) for both 1D and 2D data for fitting and modeling. 94 - The user can toggle between a number of different scales on all plots. 95 - Support saving data in the formats of ASCII and xml. 14 - A number of minor features have been added such as permitting a log 15 distribution of points when using a model to simulate data, and the 16 addition of a Kratky plot option to the linear plots 17 - a number of bugs have also been fixed 18 - BETA: Magnetic contrast supporting full polarization analysis has been 19 implemented for some spherical and cylindrical models 20 - BETA: Two new tools have been added: 21 * A generic scattering calculator which takes an atomic, magnetic or 22 SLD distribution in space and generates the appropriate 2D 23 scattering pattern. In some cases the orientationally averaged 24 (powder) 1D scattering can also be computed. Supported formats 25 include: SLD or text, PDB, and OMF magnetic moment distribution 26 file. 27 * An image viewer/converter for data in image format; this reads in 28 an image file and will attempt to convert the image pixels to 29 data. Supported formats include: TIFF, TIF, PNG, BMP, JPG 30 31 - New in Version 2.2.1 32 - Minor patch to support CanSAS XML v1.1 file format 33 34 - New in Version 2.2.0 35 - Application name changed to SasView 36 - New fully customizable Category Manager added for better management of 37 increasing number of models 38 - Improved the Grid Window functionality in the batch fitting mode 39 - Added a simpler Graph/Plot modification interface 40 - Added a new 'Data Operation' tool for addition, subtraction, multiplication, 41 division, of two data sets. 42 - The 'Sum Model' editor was extended and renamed 'Summation and Multiplication' 43 editor 44 - Added more plot symbols options for 1d plots 45 - Added improved trapping of compiling errors to the 'New model editor' 46 - Added some intelligent outputs (e.g., Rg, background, or rod diameter depending on the choice of axis scale of the plot) to the linear fits 47 - Added more models 48 49 - Feature set from Previous Versions: 50 51 - Perspectives Available 52 - Invariant calculator: Calculates the invariant, volume fraction, and 53 specific surface area. 54 - P(r) inversion calculator: Indirect Fourier transformation method. 55 - Fitting: the tool used for modeling and fitting 1D and 2D data to 56 analytical model functions 57 - Tools: provides a number of useful supplementary tools such as SLD 58 calculation 59 60 - Fitting 61 - Includes a large number of model functions, both form factors and structure factors. 62 - Support P(Q)*S(Q) for form factors that flag they can be so multiplied. 63 - Supports Gaussian, lognormal, Shulz, rectangular and custom distribution 64 functions for models that need to include polydispersity or for orientational 65 distributions if appropriate. 66 - Anisotropic shapes and magnetic moment modeling in 2D allow for a non-uniform 67 distribution of orientations of a given axis leading to modeling and fitting 68 capabilities of non azimuthaly symmetric data. 69 - User can choose to weight fits or not. If using weights, the user can choose 70 the error bar on each point if provided in the file, the square root 71 of the intensity or the intensity itself. 72 - Instrumental resolution smearing of model or fits is provided with several 73 options: read the resolution/point fromt he file. Input a pinhole resolution 74 or a slit resolution. 75 - Users can define the Qrange (Qmin and Qmax) for both 1D and 2D data for 76 fitting and modeling, but not graphically. The range can be reset to the 77 defaults (limits of q in data set for a fit) with the reset button. 78 - A mask can be applied to 2D calculation and fitting. 79 - Normalized residual plots are provided with every fit. 80 - Model function help available through detail button or from the fitting panel. 81 - Simultaneous/(advanced)constrained fitting allows for fitting a single 82 data set or several different sets simultaneously with the application 83 of advanced constraints relating fit parameters to functions of other 84 parameters (including from a different set). For example thickness of 85 shell = sin(30) times the length. 86 - Models that are the sum of two other models can be easily generated through the 87 SUM Model menubar item. 88 - New Python models can be added on the fly by creating an appropriate Python 89 file in the model plugin directory. Two tools are provided to help: 90 An easy to use custom model editor allows the quick generation of new Python 91 models by supplying only the parameters and their default value (box 1) 92 and the mathematical function of the model (box 2) and generating the 93 necessary *.py file. A separate advanced model editor provides a full Python 94 file editor. Either way once saved the model becomes immediately available 95 to the application. 96 - A batch fittng capability allows for the analysis of a series of data sets to 97 a sinble model and provides the results in a tabular form suitable for saving 98 or plotting the evolution of the fit parameters with error bars (from within 99 the application). 100 101 - Tools 102 - A scattering length density calculator,including some X-ray information 103 is provided. 104 - A density to vol. fraction converter is provided 105 - In application access to a Python shell/editor (PyCrust) is provided 106 - An instrument resolution calculator, including possible gravitational and 107 TOF effects is provided 108 - A slit size calculator optimized for Anton Paar Saxess is provided. 109 - A kiessig fringe thickness calculator is provided 110 - Plots and plot management 111 - A 3D graphing option (for 2d data/results) is provided with the view 112 controlled by the mouse 113 - 2D plots are shown with an intensity color bar. 2D Color map can be user 114 adjusted. 115 - Supports output of plot to a variety of graphic formats. Supported formats 116 include: png, eps, emf, jpg/jpeg, pdf, ps, tif/tiff, rawRGBbitmap(raw, rgba), 117 and scalable vector graphic (svg/svgz) 118 - Supports ouput of data in plot (1 or 2D) to limited data formats 119 - Multiple data sets can be loaded into a single graph for viewing (but a fit 120 plot can currently only have a single plot). 121 - Extensive context sensitive plot/fitting/manipulation options are available 122 through a right mouse click pop-up menu on plots. 123 - Data management 124 - Supports 2 + column 1D ASCII data, NIST 1D and 2D data, and canSAS data 125 via plug-in mechanism which can easily allow other readers as appropriate. 126 - 2D data is expected in Q space but for historical reasons accepts the 127 NIST 2D raw pixel format and will do conversion internally. 128 - The full data and metadata available to SasView is viewable in ASCII via 129 right clicking on a data set and choosing Data Info in the DataExplorer 130 or on the plots 131 - Supports loading a single file, multiple files, or a whole folder 132 - An optional Data Explorer is provided (default) which simplifies managing, 133 plotting, deleting, or setup for computation. Most functions however do 134 not require access to the explorer/manager and can be accessed through 135 right click menus and the toolbar. The data explorer can be re-started 136 from the menu bar. 137 - Data manipulation 138 - Support various 2D averaging methods : Circular, sectors, annular, 139 boxsum, boxQx and boxQy. 140 - A 2D data maks editor is provided 141 - 2D mask can be applied to the circular averaging. 142 - Miscellaneous features 143 - limited reports can be generated in pdf format 144 - Provides multiprocessor support(Windows only) 145 - Limited startup customization currently includes default startup 146 data folder and choice of default starting with data manager 147 - Limited support for saving(opening) a SasView project or a SasView analysis 148 (subproject) is provided. 149 - SasView can be launched and loaded with a file of interesty by double-clicking 150 on that file (recognized extension) 151 - A data file or data folder can be passed to SasView when launched from 152 the command line. 153 - Limited bookmarking capability to later recall the results of a fit calculation 154 is provided. 155 - Extensive help is provided through context sensitive mouse roll-over, 156 information bar (at the bottom of the panel), the console menu, and 157 access to the help files in several different ways. 158 96 159 97 160 98 161 2- Downloading and Installing 99 162 100 *** Note: If you have EXE or ZIP SasView installer, you don't need any of the following. 101 102 2.1- System Requirements: 103 - Python version >= 2.5 and < 3.0 should be running on the system 104 - We currently use Python 2.6 105 106 2.2- Installing from source: 107 - Get the code from https://sasviewproject.svn.sourceforge.net/svnroot/sansviewproject/releases/sasview-x.x.x 108 - run 'python setup.py install' under the 'sasview-x.x.x' folder 109 - run 'python sasview.py' under the 'sasview' folder. 110 - The following modules are required (version numbers are what are used in the release build): 111 - wxPython 2.8.12.1 (NOTE: do NOT use version 2.9) 112 - matplotlib 1.1.0 (NOTE: Mac build is using version 1.0.1) 113 - SciPy 0.10.1 114 - pisa 3.0.27 (NOTE Mac Version uses 3.0.33 BUT -- DO NOT USE ver 3.0.33 on windows: it will not work!) 115 - setuptools 0.6c11 116 117 (The following three are easily installed using easy_install) 118 - lxml 2.3.0 119 - numpy 1.6.1 (NOTE: Mac build is using version 1.5.1) 120 - periodictable 1.3.0 121 122 (The following are additional dependencies for Mac) 123 - py2app 124 125 (The following are additional dependencies for Windows) 126 - comtypes 0.6.2 (for PDF support on windows systems) 127 - pywin32 build 217 (to read ms office) 128 - pyPdf 1.13 129 - html5lib 0.95 130 - reportlab 2.5 131 - pyparsing 1.5.5 (required for periodictable and bundling) 132 - PIL 1.1.7 (Python Image Library) 133 - py2exe 0.6.9 (WIN) 134 - svn 135 - MinGW w/ gcc version 4.6.1 (WIN) 136 - Innosetup (WIN). 137 138 (On Windows, the following site has all the dependencies nicely packaged) 139 http://www.lfd.uci.edu/~gohlke/pythonlibs/ 163 *** Note 1: Much more information is available at www.sasview.org under links. 164 Look in the 'For Developers' section and particularly the wiki at 165 www.sasview.org/trac/wiki. 166 *** Note 2: If you have EXE or ZIP SasView installer, you won't need any of 167 the following. 168 169 2.1- System Requirements: 170 - Python version >= 2.5 and < 3.0 should be running on the system 171 - We currently use Python 2.6 172 173 2.2- Installing from source: 174 - Get the code from sourceforge at https://svn.code.sf.net/p/sasview/code/ 175 for trunk end with code/trunk for a this release version end in 176 code/releases/sasview-3.0.0 177 - run 'python setup.py install' under the 'sasview-x.x.x' folder 178 - run 'python sasview.py' under the 'sasview' folder. 179 - The following modules are required (version numbers are what are used in the release build): 180 - wxPython 2.8.12.1 (NOTE: do NOT use version 2.9) 181 - matplotlib 1.1.0 182 - SciPy 0.10.1 (NOTE: Mac build uses 0.10.0) 183 - pisa 3.0.27 (DO NOT USE ver 3.0.33 on windows: it will not work!) 184 - setuptools 0.6c11 185 186 (The following three are easily installed using easy_install) 187 - lxml 2.3.0.0 (NOTE: Mac build uses 3.1.0.0) 188 - numpy 1.6.1 (NOTE: Mac build uses version 1.6.2) 189 - periodictable 1.3.0 190 191 (The following are additional dependencies for Mac) 192 - py2app 0.7.1 193 194 (The following are additional dependencies for Windows) 195 - comtypes 0.6.2 (for PDF support on windows systems) 196 - pywin32 build 217 (to read ms office) 197 - html5lib 0.95-dev 198 - reportlab 2.5 (NOTE: Mab build uses 2.6 though should not be needed) 199 - pyparsing 1.5.5 (required for periodictable and bundling 200 NOTE: is ALSO present on Mac Build) 201 - PIL 1.1.7 (Python Image Library - NOTE: is also present on Mac build) 202 - py2exe 0.6.9 (WIN) 203 - vcredist_x86.exe (version 9.0.21022.8 -microsoft visual C 2008 204 re-distributable) 205 - subversion -1.6.0 (<1.6.1) 206 - MinGW w/ gcc version 4.6.1 (WIN) 207 - Innosetup (WIN - isetup 5.4.2-unicode). 208 209 (On Windows, the following site has all the dependencies nicely packaged) 210 http://www.lfd.uci.edu/~gohlke/pythonlibs/ 140 211 141 212 142 213 3- Known Issues 143 214 144 3.1- All systems: 145 - very old computers may not be able to run 146 147 3.2- Windows: 148 - None 215 3.1- All systems: 216 - very old computers may not be able to run 217 - polydispersity on multiple parameters included in a simultaneous/ 218 constrained fit will likely not be correct 219 - Save project does not store the state of all the windows 220 221 3.2- Windows: 222 - None 149 223 150 151 - None 224 3.3- MAC: 225 - multiprocessing does not currently work on MAC OS 152 226 153 3.4- Linux: 154 - None 155 156 4- Troubleshooting 157 158 - None 227 3.4- Linux: 228 - None 229 230 4- SasView website 231 232 - www.sasview.org. This main project site is the gateway to all 233 information about the sasview project. It includes information 234 about the project, a FAQ page and links to all developer and user 235 information, tools and resources. 159 236 160 237 5- Frequently Asked Questions 161 238 162 - None 239 - www.sasview.org/faq.html 163 240 164 241 6- Installer download website 165 242 166 - http://sourceforge.net/projects/sansviewproject/files/ 243 - Latest release Version 244 - http://sourceforge.net/projects/sasview/files/ 245 - Latest developer Windows or Ubuntu build 246 - http://build.sasview.org/ 247 - Latest developer Mac OS build 248 - http://download.mantidproject.org/jenkins/view/All/job/sasview_snowleopard_32bit/ 249 250 -
src/sans/dataloader/readers/cansas_constants.py
r9e2bc6c rac5b69d 3 3 the cansas_reader.py file to read in any version of the cansas format. 4 4 """ 5 class cansasConstants: 6 7 ns = '' 5 class CansasConstants: 6 """ 7 The base class to define where all of the data is to be saved by 8 cansas_reader.py. 9 """ 10 11 names = '' 8 12 format = '' 9 13 10 14 11 15 def __init__(self): 12 self.n s = self.CANSAS_NS16 self.names = self.CANSAS_NS 13 17 self.format = self.CANSAS_FORMAT 14 18 15 19 16 def _iterate_namespace(self, namespace):20 def iterate_namespace(self, namespace): 17 21 """ 18 22 Method to iterate through a cansas constants tree based on a list of … … 23 27 """ 24 28 # The current level to look through in cansas_constants. 25 return_me = currentLevel()29 return_me = CurrentLevel() 26 30 return_me.current_level = self.CANSAS_FORMAT.get("SASentry") 27 31 # Defaults for variable and datatype … … 30 34 return_me.ns_optional = True 31 35 for name in namespace: 32 if name != "SASentry": 33 return_me.current_level = \ 34 return_me.current_level.get("children").get(name, "") 35 if return_me.current_level == "": 36 try: 37 if name != "SASentry": 36 38 return_me.current_level = \ 37 return_me.current_level.get("<any>", "") 38 cl_variable = return_me.current_level.get("variable", "") 39 cl_datatype = return_me.current_level.get("storeas", "") 40 cl_units_optional = \ 41 return_me.current_level.get("units_required", "") 42 # Where are how to store the variable for the given namespace 43 # CANSAS_CONSTANTS tree is hierarchical, so is no value, inherit 44 return_me.ns_variable = cl_variable if cl_variable != "" else \ 45 return_me.ns_variable 46 return_me.ns_datatype = cl_datatype if cl_datatype != "" else \ 47 return_me.ns_datatype 48 return_me.ns_optional = cl_units_optional if \ 49 cl_units_optional != return_me.ns_optional \ 50 else return_me.ns_optional 39 return_me.current_level.get("children").get(name, "") 40 if return_me.current_level == "": 41 return_me.current_level = \ 42 return_me.current_level.get("<any>", "") 43 cl_variable = return_me.current_level.get("variable", "") 44 cl_datatype = return_me.current_level.get("storeas", "") 45 cl_units_optional = \ 46 return_me.current_level.get("units_required", "") 47 # Where are how to store the variable for the given namespace 48 # CANSAS_CONSTANTS tree is hierarchical, so is no value, inherit 49 return_me.ns_variable = cl_variable if cl_variable != "" else \ 50 return_me.ns_variable 51 return_me.ns_datatype = cl_datatype if cl_datatype != "" else \ 52 return_me.ns_datatype 53 return_me.ns_optional = cl_units_optional if \ 54 cl_units_optional != return_me.ns_optional \ 55 else return_me.ns_optional 56 except AttributeError: 57 return_me.ns_variable = "{0}.meta_data[\"{2}\"] = \"{1}\"" 58 return_me.ns_datatype = "content" 59 return_me.ns_optional = True 51 60 return return_me 52 61 … … 454 463 } 455 464 SASINSTR_SRC = { 456 "attributes" : {"name" : {"variable" : "{0}.source.name = \"{1}\""}}, 465 "attributes" : {"name" : {"variable" : \ 466 "{0}.source.name = \"{1}\""}}, 457 467 "variable" : None, 458 468 "children" : { 459 "radiation" : {"variable" : "{0}.source.radiation = \"{1}\""}, 469 "radiation" : {"variable" : \ 470 "{0}.source.radiation = \"{1}\""}, 460 471 "beam_size" : SASINSTR_SRC_BEAMSIZE, 461 472 "beam_shape" : {"variable" : \ … … 647 658 "unit" : "pixel_size_unit", 648 659 "attributes" : { 649 "unit" : "{0}.pixel_size_unit = \"{1}\"", 660 "unit" : \ 661 "{0}.pixel_size_unit = \"{1}\"", 650 662 "storeas" : "content" 651 663 } … … 655 667 "unit" : "pixel_size_unit", 656 668 "attributes" : { 657 "unit" : "{0}.pixel_size_unit = \"{1}\"", 669 "unit" : \ 670 "{0}.pixel_size_unit = \"{1}\"", 658 671 "storeas" : "content" 659 672 } … … 663 676 "unit" : "pixel_size_unit", 664 677 "attributes" : { 665 "unit" : "{0}.pixel_size_unit = \"{1}\"", 678 "unit" : \ 679 "{0}.pixel_size_unit = \"{1}\"", 666 680 "storeas" : "content" 667 681 } … … 740 754 } 741 755 742 class currentLevel: 756 class CurrentLevel: 757 """ 758 A helper class to hold information on where you are in the constants tree 759 """ 743 760 744 761 current_level = '' … … 748 765 749 766 def __init__(self): 750 self.current_level = ''767 self.current_level = {} 751 768 self.ns_variable = '' 752 769 self.ns_datatype = "content" -
src/sans/dataloader/readers/cansas_reader.py
r2e3b055 rbb1b892 18 18 import sys 19 19 import datetime 20 import inspect 21 # For saving individual sections of data 20 22 from sans.dataloader.data_info import Data1D 21 23 from sans.dataloader.data_info import Collimation … … 24 26 from sans.dataloader.data_info import Process 25 27 from sans.dataloader.data_info import Aperture 28 # Both imports used. Do not remove either. 29 from xml.dom.minidom import parseString 26 30 import sans.dataloader.readers.xml_reader as xml_reader 27 from sans.dataloader.readers.cansas_constants import cansasConstants 31 from sans.dataloader.readers.xml_reader import XMLreader 32 from sans.dataloader.readers.cansas_constants import CansasConstants 28 33 29 34 _ZERO = 1e-16 … … 36 41 HAS_CONVERTER = False 37 42 38 constants = cansasConstants()39 CANSAS_FORMAT = constants.format40 CANSAS_NS = constants.ns43 CONSTANTS = CansasConstants() 44 CANSAS_FORMAT = CONSTANTS.format 45 CANSAS_NS = CONSTANTS.names 41 46 ALLOW_ALL = True 42 47 43 48 49 # minidom used in functions called by outside classes 50 import xml.dom.minidom 51 # DO NOT REMOVE 52 # Called by outside packages: 53 # sans.perspectives.invariant.invariant_state 54 # sans.perspectives.fitting.pagestate 44 55 def get_content(location, node): 45 56 """ … … 51 62 :return: Element, or None 52 63 """ 53 nodes = node.xpath(location, namespaces={'ns': CANSAS_NS}) 64 nodes = node.xpath(location, 65 namespaces={'ns': CANSAS_NS.get("1.0").get("ns")}) 54 66 55 67 if len(nodes) > 0: … … 59 71 60 72 61 def get_float(location, node): 62 """ 63 Get the content of a node as a float 64 65 :param location: xpath location 66 :param node: node to start at 67 """ 68 nodes = node.xpath(location, namespaces={'ns': CANSAS_NS}) 69 70 value = None 71 attr = {} 72 if len(nodes) > 0: 73 try: 74 value = float(nodes[0].text) 75 except: 76 # Could not pass, skip and return None 77 msg = "cansas_reader.get_float: could not " 78 msg += " convert '%s' to float" % nodes[0].text 79 logging.error(msg) 80 if nodes[0].get('unit') is not None: 81 attr['unit'] = nodes[0].get('unit') 82 return value, attr 83 84 85 # This is called by sans.perspectives.fitting.pagestate.py 86 # Do not remove 73 # DO NOT REMOVE 74 # Called by outside packages: 75 # sans.perspectives.fitting.pagestate 87 76 def write_node(doc, parent, name, value, attr={}): 88 77 """ … … 105 94 106 95 107 class Reader( ):96 class Reader(XMLreader): 108 97 """ 109 98 Class to load cansas 1D XML files … … 114 103 ##CanSAS version - defaults to version 1.0 115 104 cansas_version = "1.0" 116 ##Data reader 117 # TODO: make the reader extend the XMLreader class? 118 reader = xml_reader.XMLreader() 105 106 logging = [] 119 107 errors = [] 120 108 121 109 type_name = "canSAS" 122 123 110 ## Wildcards 124 type = ["XML files (*.xml)|*.xml" ]111 type = ["XML files (*.xml)|*.xml", "SasView Save Files (*.svs)|*.svs"] 125 112 ## List of allowed extensions 126 ext = ['.xml', '.XML' ]113 ext = ['.xml', '.XML', '.svs', '.SVS'] 127 114 128 115 ## Flag to bypass extension check … … 133 120 self.errors = [] 134 121 135 def is_cansas(self ):122 def is_cansas(self, ext="xml"): 136 123 """ 137 124 Checks to see if the xml file is a CanSAS file 138 125 """ 139 if self. reader.validateXML():126 if self.validate_xml(): 140 127 name = "{http://www.w3.org/2001/XMLSchema-instance}schemaLocation" 141 value = self. reader.xmlroot.get(name)142 if (CANSAS_NS.get(self.cansas_version).get("ns") == \143 value.rsplit(" ")[0] ):128 value = self.xmlroot.get(name) 129 if CANSAS_NS.get(self.cansas_version).get("ns") == \ 130 value.rsplit(" ")[0]: 144 131 return True 132 if ext == "svs": 133 return True 145 134 return False 146 135 … … 152 141 """ 153 142 # X - Q value; Y - Intensity (Abs) 154 x = numpy.empty(0)155 y = numpy.empty(0)156 dx = numpy.empty(0)157 dy = numpy.empty(0)143 x_vals = numpy.empty(0) 144 y_vals = numpy.empty(0) 145 dx_vals = numpy.empty(0) 146 dy_vals = numpy.empty(0) 158 147 dxl = numpy.empty(0) 159 148 dxw = numpy.empty(0) … … 162 151 output = [] 163 152 # ns - Namespace hierarchy for current xml object 164 ns = []153 ns_list = [] 165 154 166 155 # Check that the file exists … … 168 157 basename = os.path.basename(xml) 169 158 _, extension = os.path.splitext(basename) 170 # If the fi eltype is not allowed, return nothing159 # If the file type is not allowed, return nothing 171 160 if extension in self.ext or self.allow_all: 161 # Get the file location of 172 162 base_name = xml_reader.__file__ 173 163 base_name = base_name.replace("\\","/") … … 175 165 176 166 # Load in xml file and get the cansas version from the header 177 self. reader.setXMLFile(xml)178 root = self. reader.xmlroot167 self.set_xml_file(xml) 168 root = self.xmlroot 179 169 if root is None: 180 170 root = {} … … 187 177 188 178 # Link a schema to the XML file. 189 self. reader.setSchema(schema_path)179 self.set_schema(schema_path) 190 180 191 181 # Try to load the file, but raise an error if unable to. 192 182 # Check the file matches the XML schema 193 183 try: 194 if self.is_cansas( ):184 if self.is_cansas(extension): 195 185 # Get each SASentry from XML file and add it to a list. 196 186 entry_list = root.xpath('/ns:SASroot/ns:SASentry', 197 187 namespaces={'ns': cansas_defaults.get("ns")}) 198 ns .append("SASentry")188 ns_list.append("SASentry") 199 189 200 190 # If multiple files, modify the name for each is unique … … 204 194 # Parse each SASentry item 205 195 for entry in entry_list: 206 # Define a new Data1D object with zeroes for x and y 207 data1d = Data1D(x,y,dx,dy) 196 # Define a new Data1D object with zeroes for 197 # x_vals and y_vals 198 data1d = Data1D(x_vals, y_vals, dx_vals, dy_vals) 208 199 data1d.dxl = dxl 209 200 data1d.dxw = dxw … … 220 211 221 212 # Get all preprocessing events and encoding 222 self. reader.setProcessingInstructions()213 self.set_processing_instructions() 223 214 data1d.meta_data[PREPROCESS] = \ 224 self. reader.processingInstructions215 self.processing_instructions 225 216 226 217 # Parse the XML file 227 218 return_value, extras = \ 228 self._parse_entry(entry, ns , data1d)219 self._parse_entry(entry, ns_list, data1d) 229 220 del extras[:] 230 221 … … 251 242 output.append(return_value) 252 243 else: 253 value = self. reader.findInvalidXML()244 value = self.find_invalid_xml() 254 245 output.append("Invalid XML at: {0}".format(value)) 255 246 except: 256 247 # If the file does not match the schema, raise this error 257 raise RuntimeError, "%s cannot be read \increment" % xml248 raise RuntimeError, "%s cannot be read" % xml 258 249 return output 259 250 # Return a list of parsed entries that dataloader can manage 260 251 return None 261 252 262 def _create_unique_key(self, dictionary, name, i= 0):253 def _create_unique_key(self, dictionary, name, numb = 0): 263 254 """ 264 255 Create a unique key value for any dictionary to prevent overwriting … … 267 258 :param dictionary: A dictionary with any number of entries 268 259 :param name: The index of the item to be added to dictionary 269 :param i: The number to be appended to the name, starts at 0260 :param numb: The number to be appended to the name, starts at 0 270 261 """ 271 262 if dictionary.get(name) is not None: 272 i+= 1263 numb += 1 273 264 name = name.split("_")[0] 274 name += "_{0}".format( i)275 name = self._create_unique_key(dictionary, name, i)265 name += "_{0}".format(numb) 266 name = self._create_unique_key(dictionary, name, numb) 276 267 return name 277 268 … … 284 275 285 276 :param new_current_level: cansas_constants level as returned by 286 _iterate_namespace277 iterate_namespace 287 278 :param attr: The attributes of the node 288 279 :param data1d: Where the values will be saved … … 318 309 return 319 310 except: 320 err_msg = "CanSAS reader: could not convert the units" 311 err_msg = \ 312 "CanSAS reader: could not convert the units" 321 313 self.errors.append(err_msg) 322 314 return … … 343 335 return node_value, value_unit 344 336 345 def _parse_entry(self, dom, n s, data1d, extras =[]):337 def _parse_entry(self, dom, names=["SASentry"], data1d=None, extras=[]): 346 338 """ 347 339 Parse a SASEntry - new recursive method for parsing the dom of … … 349 341 and extra nodes to be read in simultaneously. 350 342 351 :param dom: dom object with a namespace base of n s352 :param n s: A list of element names that lead up to the dom object343 :param dom: dom object with a namespace base of names 344 :param names: A list of element names that lead up to the dom object 353 345 :param data1d: The data1d object that will be modified 354 346 :param extras: Any values that should go into meta_data when data1d 355 347 is not a Data1D object 356 348 """ 357 349 358 350 # A portion of every namespace entry 351 if data1d == None: 352 x_vals = numpy.empty(0) 353 y_vals = numpy.empty(0) 354 dx_vals = numpy.empty(0) 355 dy_vals = numpy.empty(0) 356 dxl = numpy.empty(0) 357 dxw = numpy.empty(0) 358 data1d = Data1D(x_vals, y_vals, dx_vals, dy_vals) 359 data1d.dxl = dxl 360 data1d.dxw = dxw 361 359 362 base_ns = "{0}{1}{2}".format("{", \ 360 363 CANSAS_NS.get(self.cansas_version).get("ns"), "}") … … 366 369 for node in dom: 367 370 try: 368 # Get the element name and set the current n s level371 # Get the element name and set the current names level 369 372 tagname = node.tag.replace(base_ns, "") 370 373 tagname_original = tagname 371 ns.append(tagname) 374 if tagname == "fitting_plug_in" or tagname == "pr_inversion" or\ 375 tagname == "invariant": 376 continue 377 names.append(tagname) 372 378 attr = node.attrib 373 379 children = node.getchildren() 380 if len(children) == 0: 381 children = None 374 382 save_data1d = data1d 375 383 … … 410 418 411 419 # Get where to store content 412 cs_values = constants._iterate_namespace(ns)420 cs_values = CONSTANTS.iterate_namespace(names) 413 421 # If the element is a child element, recurse 414 422 if children is not None: 415 423 # Returned value is new Data1D object with all previous and 416 424 # new values in it. 417 data1d, extras = self._parse_entry(node, ns, data1d, extras) 425 data1d, extras = self._parse_entry(node, 426 names, data1d, extras) 418 427 419 428 #Get the information from the node … … 488 497 pass 489 498 490 except TypeError :499 except TypeError as e: 491 500 pass 492 501 except Exception as e: … … 509 518 else: 510 519 save_data1d = data1d 511 data1d = save_data1d 512 # Remove tagname from ns to restore original base 513 ns.remove(tagname_original) 514 520 if tagname_original == "fitting_plug_in" or \ 521 tagname_original == "invariant" or \ 522 tagname_original == "pr_inversion": 523 pass 524 else: 525 data1d = save_data1d 526 # Remove tagname from names to restore original base 527 names.remove(tagname_original) 515 528 return data1d, extras 516 529 530 517 531 def _to_xml_doc(self, datainfo): 518 532 """ … … 521 535 :param datainfo: Data1D object 522 536 """ 523 524 537 if not issubclass(datainfo.__class__, Data1D): 525 538 raise RuntimeError, "The cansas writer expects a Data1D instance" 526 539 527 540 # Get PIs and create root element 528 pis = self.reader.return_processing_instructions() 529 doc = self.reader.create_tree(pis[0]) 530 i = 1 531 for i in range(1,len(pis) - 1): 532 doc = self.reader.append(pis[i], doc) 541 pis = self.return_processing_instructions() 542 if len(pis) > 0: 543 pi_tree = self.create_tree(pis[0]) 544 i = 1 545 for i in range(1,len(pis) - 1): 546 pi_tree = self.append(pis[i], pi_tree) 547 pi_string = self.to_string(pi_tree) 548 else: 549 pi_string = "" 533 550 534 551 # Define namespaces and create SASroot object … … 540 557 else: 541 558 url = "http://svn.smallangles.net/svn/canSAS/1dwg/trunk/" 542 schema Location = "{0} {1}cansas1d.xsd".format(ns, url)543 attrib = {"{" + xsi + "}schemaLocation" : schema Location,559 schema_location = "{0} {1}cansas1d.xsd".format(ns, url) 560 attrib = {"{" + xsi + "}schemaLocation" : schema_location, 544 561 "version" : version} 545 562 nsmap = {'xsi' : xsi, None: ns} 546 563 547 main_node = self. reader.create_element("{" + ns + "}SASroot", \564 main_node = self.create_element("{" + ns + "}SASroot", \ 548 565 attrib = attrib, \ 549 566 nsmap = nsmap) 550 567 551 568 # Create ElementTree, append SASroot and apply processing instructions 552 base_string = self.reader.toString(doc) + \ 553 self.reader.toString(main_node) 554 base_element = self.reader.create_element_from_string(base_string) 555 doc = self.reader.create_tree(base_element) 569 base_string = pi_string + self.to_string(main_node) 570 base_element = self.create_element_from_string(base_string) 571 doc = self.create_tree(base_element) 556 572 557 573 # Create SASentry Element 558 entry_node = self. reader.create_element("SASentry")574 entry_node = self.create_element("SASentry") 559 575 root = doc.getroot() 560 576 root.append(entry_node) … … 576 592 577 593 # Data info 578 node = self. reader.create_element("SASdata")579 self. reader.append(node, entry_node)594 node = self.create_element("SASdata") 595 self.append(node, entry_node) 580 596 581 597 for i in range(len(datainfo.x)): 582 pt = self. reader.create_element("Idata")598 pt = self.create_element("Idata") 583 599 node.append(pt) 584 600 self.write_node(pt, "Q", datainfo.x[i], {'unit': datainfo.x_unit}) … … 598 614 self.write_node(pt, "dQl", datainfo.dxl[i], 599 615 {'unit': datainfo.x_unit}) 600 616 601 617 # Transmission Spectrum Info 602 618 for i in range(len(datainfo.trans_spectrum)): 603 619 spectrum = datainfo.trans_spectrum[i] 604 node = self. reader.create_element("SAStransmission_spectrum",620 node = self.create_element("SAStransmission_spectrum", 605 621 {"name" : spectrum.name}) 606 self. reader.append(node, entry_node)622 self.append(node, entry_node) 607 623 if isinstance(spectrum.timestamp, datetime.datetime): 608 624 node.setAttribute("timestamp", spectrum.timestamp) 609 625 for i in range(len(spectrum.wavelength)): 610 pt = self. reader.create_element("Tdata")626 pt = self.create_element("Tdata") 611 627 node.append(pt) 612 628 self.write_node(pt, "Lambda", spectrum.wavelength[i], … … 621 637 622 638 # Sample info 623 sample = self. reader.create_element("SASsample")639 sample = self.create_element("SASsample") 624 640 if datainfo.sample.name is not None: 625 self. reader.write_attribute(sample,641 self.write_attribute(sample, 626 642 "name", 627 643 str(datainfo.sample.name)) 628 self. reader.append(sample, entry_node)644 self.append(sample, entry_node) 629 645 self.write_node(sample, "ID", str(datainfo.sample.ID)) 630 646 self.write_node(sample, "thickness", datainfo.sample.thickness, … … 634 650 {"unit": datainfo.sample.temperature_unit}) 635 651 636 pos = self. reader.create_element("position")652 pos = self.create_element("position") 637 653 written = self.write_node(pos, 638 654 "x", … … 648 664 {"unit": datainfo.sample.position_unit}) 649 665 if written == True: 650 self. reader.append(pos, sample)651 652 ori = self. reader.create_element("orientation")666 self.append(pos, sample) 667 668 ori = self.create_element("orientation") 653 669 written = self.write_node(ori, "roll", 654 670 datainfo.sample.orientation.x, … … 661 677 {"unit": datainfo.sample.orientation_unit}) 662 678 if written == True: 663 self. reader.append(ori, sample)679 self.append(ori, sample) 664 680 665 681 for item in datainfo.sample.details: … … 667 683 668 684 # Instrument info 669 instr = self. reader.create_element("SASinstrument")670 self. reader.append(instr, entry_node)685 instr = self.create_element("SASinstrument") 686 self.append(instr, entry_node) 671 687 672 688 self.write_node(instr, "name", datainfo.instrument) 673 689 674 690 # Source 675 source = self. reader.create_element("SASsource")691 source = self.create_element("SASsource") 676 692 if datainfo.source.name is not None: 677 self. reader.write_attribute(source,693 self.write_attribute(source, 678 694 "name", 679 695 str(datainfo.source.name)) 680 self. reader.append(source, instr)696 self.append(source, instr) 681 697 if datainfo.source.radiation == None or datainfo.source.radiation == '': 682 698 datainfo.source.radiation = "neutron" 683 699 self.write_node(source, "radiation", datainfo.source.radiation) 684 700 685 size = self. reader.create_element("beam_size")701 size = self.create_element("beam_size") 686 702 if datainfo.source.beam_size_name is not None: 687 self. reader.write_attribute(size,703 self.write_attribute(size, 688 704 "name", 689 705 str(datainfo.source.beam_size_name)) … … 697 713 {"unit": datainfo.source.beam_size_unit}) 698 714 if written == True: 699 self. reader.append(size, source)715 self.append(size, source) 700 716 701 717 self.write_node(source, "beam_shape", datainfo.source.beam_shape) … … 718 734 datainfo.collimation.append(coll) 719 735 for item in datainfo.collimation: 720 coll = self. reader.create_element("SAScollimation")736 coll = self.create_element("SAScollimation") 721 737 if item.name is not None: 722 self. reader.write_attribute(coll, "name", str(item.name))723 self. reader.append(coll, instr)738 self.write_attribute(coll, "name", str(item.name)) 739 self.append(coll, instr) 724 740 725 741 self.write_node(coll, "length", item.length, … … 727 743 728 744 for apert in item.aperture: 729 ap = self. reader.create_element("aperture")745 ap = self.create_element("aperture") 730 746 if apert.name is not None: 731 self. reader.write_attribute(ap, "name", str(apert.name))747 self.write_attribute(ap, "name", str(apert.name)) 732 748 if apert.type is not None: 733 self. reader.write_attribute(ap, "type", str(apert.type))734 self. reader.append(ap, coll)735 736 size = self. reader.create_element("size")749 self.write_attribute(ap, "type", str(apert.type)) 750 self.append(ap, coll) 751 752 size = self.create_element("size") 737 753 if apert.size_name is not None: 738 self. reader.write_attribute(size,754 self.write_attribute(size, 739 755 "name", 740 756 str(apert.size_name)) … … 746 762 {"unit": apert.size_unit}) 747 763 if written == True: 748 self. reader.append(size, ap)764 self.append(size, ap) 749 765 750 766 self.write_node(ap, "distance", apert.distance, … … 758 774 759 775 for item in datainfo.detector: 760 det = self. reader.create_element("SASdetector")776 det = self.create_element("SASdetector") 761 777 written = self.write_node(det, "name", item.name) 762 778 written = written | self.write_node(det, "SDD", item.distance, 763 779 {"unit": item.distance_unit}) 764 780 if written == True: 765 self. reader.append(det, instr)781 self.append(det, instr) 766 782 767 off = self. reader.create_element("offset")783 off = self.create_element("offset") 768 784 written = self.write_node(off, "x", item.offset.x, 769 785 {"unit": item.offset_unit}) … … 773 789 {"unit": item.offset_unit}) 774 790 if written == True: 775 self. reader.append(off, det)776 777 ori = self. reader.create_element("orientation")791 self.append(off, det) 792 793 ori = self.create_element("orientation") 778 794 written = self.write_node(ori, "roll", item.orientation.x, 779 795 {"unit": item.orientation_unit}) … … 785 801 {"unit": item.orientation_unit}) 786 802 if written == True: 787 self. reader.append(ori, det)803 self.append(ori, det) 788 804 789 center = self. reader.create_element("beam_center")805 center = self.create_element("beam_center") 790 806 written = self.write_node(center, "x", item.beam_center.x, 791 807 {"unit": item.beam_center_unit}) … … 797 813 {"unit": item.beam_center_unit}) 798 814 if written == True: 799 self. reader.append(center, det)800 801 pix = self. reader.create_element("pixel_size")815 self.append(center, det) 816 817 pix = self.create_element("pixel_size") 802 818 written = self.write_node(pix, "x", item.pixel_size.x, 803 819 {"unit": item.pixel_size_unit}) … … 810 826 {"unit": item.slit_length_unit}) 811 827 if written == True: 812 self. reader.append(pix, det)828 self.append(pix, det) 813 829 814 830 # Processes info 815 831 for item in datainfo.process: 816 node = self. reader.create_element("SASprocess")817 self. reader.append(node, entry_node)832 node = self.create_element("SASprocess") 833 self.append(node, entry_node) 818 834 819 835 self.write_node(node, "name", item.name) … … 831 847 # Note info 832 848 if len(datainfo.notes) == 0: 833 node = self. reader.create_element("SASnote")834 self. reader.append(node, entry_node)849 node = self.create_element("SASnote") 850 self.append(node, entry_node) 835 851 else: 836 852 for item in datainfo.notes: 837 node = self. reader.create_element("SASnote")838 self. reader.write_text(node, item)839 self. reader.append(node, entry_node)853 node = self.create_element("SASnote") 854 self.write_text(node, item) 855 self.append(node, entry_node) 840 856 841 857 842 858 # Return the document, and the SASentry node associated with 843 # the data we just wrote 859 # the data we just wrote 860 # If the calling function was not the cansas reader, return a minidom 861 # object rather than an lxml object. 862 863 frm = inspect.stack()[1] 864 mod_name = frm[1].replace("\\", "/").replace(".pyc", "") 865 mod_name = mod_name.replace(".py", "") 866 mod = mod_name.split("/readers/") 867 mod_name = mod[1] 868 if mod_name != "cansas_reader": 869 string = self.to_string(doc, pp=False) 870 doc = parseString(string) 871 node_name = entry_node.tag 872 node_list = doc.getElementsByTagName(node_name) 873 entry_node = node_list.item(0) 874 844 875 return doc, entry_node 845 876 846 877 847 def write_node(self, parent, name, value, attr= {}):878 def write_node(self, parent, name, value, attr=None): 848 879 """ 849 880 :param doc: document DOM … … 856 887 """ 857 888 if value is not None: 858 parent = self. reader.ebuilder(parent, name, value, attr)889 parent = self.ebuilder(parent, name, value, attr) 859 890 return True 860 891 return False … … 872 903 # Write the file 873 904 fd = open(filename, 'w') 874 if self. reader.encoding == None:875 self. reader.encoding = "UTF-8"876 doc.write(fd, encoding=self. reader.encoding,905 if self.encoding == None: 906 self.encoding = "UTF-8" 907 doc.write(fd, encoding=self.encoding, 877 908 pretty_print=True, xml_declaration=True) 878 909 fd.close() 879 910 911 912 # DO NOT REMOVE - used in saving and loading panel states. 913 def _store_float(self, location, node, variable, storage, optional=True): 914 """ 915 Get the content of a xpath location and store 916 the result. Check that the units are compatible 917 with the destination. The value is expected to 918 be a float. 919 920 The xpath location might or might not exist. 921 If it does not exist, nothing is done 922 923 :param location: xpath location to fetch 924 :param node: node to read the data from 925 :param variable: name of the data member to store it in [string] 926 :param storage: data object that has the 'variable' data member 927 :param optional: if True, no exception will be raised 928 if unit conversion can't be done 929 930 :raise ValueError: raised when the units are not recognized 931 """ 932 entry = get_content(location, node) 933 try: 934 value = float(entry.text) 935 except: 936 value = None 937 938 if value is not None: 939 # If the entry has units, check to see that they are 940 # compatible with what we currently have in the data object 941 units = entry.get('unit') 942 if units is not None: 943 toks = variable.split('.') 944 local_unit = None 945 exec "local_unit = storage.%s_unit" % toks[0] 946 if local_unit != None and units.lower() != local_unit.lower(): 947 if HAS_CONVERTER == True: 948 try: 949 conv = Converter(units) 950 exec "storage.%s = %g" % (variable, 951 conv(value, units=local_unit)) 952 except: 953 exc_type, exc_value, exc_traceback = sys.exc_info() 954 err_mess = "CanSAS reader: could not convert" 955 err_mess += " %s unit [%s]; expecting [%s]\n %s" \ 956 % (variable, units, local_unit, exc_value) 957 self.errors.append(err_mess) 958 if optional: 959 logging.info(err_mess) 960 else: 961 raise ValueError, err_mess 962 else: 963 err_mess = "CanSAS reader: unrecognized %s unit [%s];"\ 964 % (variable, units) 965 err_mess += " expecting [%s]" % local_unit 966 self.errors.append(err_mess) 967 if optional: 968 logging.info(err_mess) 969 else: 970 raise ValueError, err_mess 971 else: 972 exec "storage.%s = value" % variable 973 else: 974 exec "storage.%s = value" % variable 975 976 977 # DO NOT REMOVE - used in saving and loading panel states. 978 def _store_content(self, location, node, variable, storage): 979 """ 980 Get the content of a xpath location and store 981 the result. The value is treated as a string. 982 983 The xpath location might or might not exist. 984 If it does not exist, nothing is done 985 986 :param location: xpath location to fetch 987 :param node: node to read the data from 988 :param variable: name of the data member to store it in [string] 989 :param storage: data object that has the 'variable' data member 990 991 :return: return a list of errors 992 """ 993 entry = get_content(location, node) 994 if entry is not None and entry.text is not None: 995 exec "storage.%s = entry.text.strip()" % variable -
src/sans/dataloader/readers/xml_reader.py
r2e3b055 rac5b69d 17 17 from lxml import etree 18 18 from lxml.builder import E 19 parser = etree.ETCompatXMLParser(remove_comments=True, remove_pis=False) 19 20 PARSER = etree.ETCompatXMLParser(remove_comments=True, remove_pis=False) 20 21 21 22 class XMLreader(): 23 """ 24 Generic XML read and write class. Mostly helper functions. 25 Makes reading/writing XML a bit easier than calling lxml libraries directly. 26 27 :Dependencies: 28 This class requires lxml 2.3 or higher. 29 """ 22 30 23 31 xml = None … … 27 35 schemadoc = None 28 36 encoding = None 29 processing Instructions = None37 processing_instructions = None 30 38 31 39 def __init__(self, xml = None, schema = None, root = None): 32 40 self.xml = xml 33 41 self.schema = schema 34 self.processing Instructions = {}42 self.processing_instructions = {} 35 43 if xml is not None: 36 self.set XMLFile(xml, root)44 self.set_xml_file(xml, root) 37 45 else: 38 46 self.xmldoc = None 39 47 self.xmlroot = None 40 48 if schema is not None: 41 self.set Schema(schema)49 self.set_schema(schema) 42 50 else: 43 51 self.schemadoc = None … … 47 55 Read in an XML file into memory and return an lxml dictionary 48 56 """ 49 if self.validate XML():50 self.xmldoc = etree.parse(self.xml, parser = parser)57 if self.validate_xml(): 58 self.xmldoc = etree.parse(self.xml, parser = PARSER) 51 59 else: 52 raise etree. ValidationError(self, self.findInvalidXML())60 raise etree.XMLSchemaValidateError(self, self.find_invalid_xml()) 53 61 return self.xmldoc 54 62 55 def set XMLFile(self, xml, root = None):63 def set_xml_file(self, xml, root = None): 56 64 """ 57 65 Set the XML file and parse … … 59 67 try: 60 68 self.xml = xml 61 self.xmldoc = etree.parse(self.xml, parser = parser)69 self.xmldoc = etree.parse(self.xml, parser = PARSER) 62 70 self.xmlroot = self.xmldoc.getroot() 63 71 except Exception: … … 66 74 self.xmlroot = None 67 75 68 def set Schema(self, schema):76 def set_schema(self, schema): 69 77 """ 70 78 Set the schema file and parse … … 72 80 try: 73 81 self.schema = schema 74 self.schemadoc = etree.parse(self.schema, parser = parser)82 self.schemadoc = etree.parse(self.schema, parser = PARSER) 75 83 except Exception: 76 84 self.schema = None 77 85 self.schemadoc = None 78 86 79 def validate XML(self):87 def validate_xml(self): 80 88 """ 81 89 Checks to see if the XML file meets the schema … … 83 91 valid = True 84 92 if self.schema is not None: 85 self.parse SchemaAndDoc()86 schema Check = etree.XMLSchema(self.schemadoc)87 valid = schema Check.validate(self.xmldoc)93 self.parse_schema_and_doc() 94 schema_check = etree.XMLSchema(self.schemadoc) 95 valid = schema_check.validate(self.xmldoc) 88 96 return valid 89 97 90 def find InvalidXML(self):98 def find_invalid_xml(self): 91 99 """ 92 100 Finds the first offending element that should not be present in XML file 93 101 """ 94 first Error = ""95 self.parse SchemaAndDoc()102 first_error = "" 103 self.parse_schema_and_doc() 96 104 schema = etree.XMLSchema(self.schemadoc) 97 105 try: 98 first Error = schema.assertValid(self.xmldoc)106 first_error = schema.assertValid(self.xmldoc) 99 107 except etree.DocumentInvalid as e: 100 first Error = str(e)101 return first Error102 103 def parse SchemaAndDoc(self):108 first_error = str(e) 109 return first_error 110 111 def parse_schema_and_doc(self): 104 112 """ 105 113 Creates a dictionary of the parsed schema and xml files. 106 114 """ 107 self.set XMLFile(self.xml)108 self.set Schema(self.schema)109 110 def to String(self, elem, pp=False, encoding=None):115 self.set_xml_file(self.xml) 116 self.set_schema(self.schema) 117 118 def to_string(self, elem, pp=False, encoding=None): 111 119 """ 112 120 Converts an etree element into a string … … 129 137 return dic 130 138 131 def set ProcessingInstructions(self):139 def set_processing_instructions(self): 132 140 """ 133 141 Take out all processing instructions and create a dictionary from them … … 135 143 """ 136 144 dic = {} 137 p i= self.xmlroot.getprevious()138 while p iis not None:139 pi_string = self.to String(pi)145 proc_instr = self.xmlroot.getprevious() 146 while proc_instr is not None: 147 pi_string = self.to_string(proc_instr) 140 148 if "?>\n<?" in pi_string: 141 149 pi_string = pi_string.split("?>\n<?") … … 145 153 for item in pi_string: 146 154 dic = self.break_processing_instructions(item, dic) 147 p i = pi.getprevious()155 proc_instr = proc_instr.getprevious() 148 156 if 'xml' in dic: 149 self.set Encoding(dic['xml'])157 self.set_encoding(dic['xml']) 150 158 del dic['xml'] 151 self.processing Instructions = dic152 153 def set Encoding(self, attr_str):159 self.processing_instructions = dic 160 161 def set_encoding(self, attr_str): 154 162 """ 155 163 Find the encoding in the xml declaration and save it as a string … … 169 177 self.encoding = None 170 178 171 def _create_unique_key(self, dictionary, name, i= 0):179 def _create_unique_key(self, dictionary, name, numb = 0): 172 180 """ 173 181 Create a unique key value for any dictionary to prevent overwriting … … 176 184 :param dictionary: A dictionary with any number of entries 177 185 :param name: The index of the item to be added to dictionary 178 :param i: The number to be appended to the name, starts at 0186 :param numb: The number to be appended to the name, starts at 0 179 187 """ 180 188 if dictionary.get(name) is not None: 181 i+= 1189 numb += 1 182 190 name = name.split("_")[0] 183 name += "_{0}".format( i)184 name = self._create_unique_key(dictionary, name, i)191 name += "_{0}".format(numb) 192 name = self._create_unique_key(dictionary, name, numb) 185 193 return name 186 194 … … 193 201 return etree.ElementTree(root) 194 202 195 def create_element_from_string(self, s):203 def create_element_from_string(self, xml_string): 196 204 """ 197 205 Create an element from an XML string 198 206 199 :param s: A string of xml200 """ 201 return etree.fromstring( s)202 203 def create_element(self, name, attrib= {}, nsmap=None):207 :param xml_string: A string of xml 208 """ 209 return etree.fromstring(xml_string) 210 211 def create_element(self, name, attrib=None, nsmap=None): 204 212 """ 205 213 Create an XML element for writing to file … … 207 215 :param name: The name of the element to be created 208 216 """ 217 if attrib == None: 218 attrib = {} 209 219 return etree.Element(name, attrib, nsmap) 210 220 … … 237 247 """ 238 248 pi_list = [] 239 for key in self.processingInstructions: 240 value = self.processingInstructions.get(key) 241 pi = etree.ProcessingInstruction(key, value) 242 pi_list.append(pi) 249 if self.processing_instructions is not None: 250 for key in self.processing_instructions: 251 value = self.processing_instructions.get(key) 252 pi_item = etree.ProcessingInstruction(key, value) 253 pi_list.append(pi_item) 243 254 return pi_list 244 255 … … 253 264 return tree 254 265 255 def ebuilder(self, parent, elementname, text=None, attrib={}): 266 def ebuilder(self, parent, elementname, text=None, attrib=None): 267 """ 268 Use lxml E builder class with arbitrary inputs. 269 270 :param parnet: The parent element to append a child to 271 :param elementname: The name of the child in string form 272 :param text: The element text 273 :param attrib: A dictionary of attribute names to attribute values 274 """ 256 275 text = str(text) 276 if attrib == None: 277 attrib = {} 257 278 elem = E(elementname, attrib, text) 258 279 parent = parent.append(elem) -
src/sans/guiframe/gui_manager.py
r98816c43 ra3b635b 776 776 # Append item from plugin under menu file if necessary 777 777 self._populate_file_menu() 778 779 778 780 if not wx.VERSION_STRING >= '3.0.0.0': 779 781 self.SetMenuBar(self._menubar) … … 1349 1351 if config._do_aboutbox: 1350 1352 self._help_menu.AppendSeparator() 1351 id = wx.NewId() 1352 self._help_menu.Append(id, '&About', 'Software information') 1353 wx.EVT_MENU(self, id, self._onAbout) 1353 self._help_menu.Append(wx.ID_ABOUT, '&About', 'Software information') 1354 wx.EVT_MENU(self, wx.ID_ABOUT, self._onAbout) 1354 1355 1355 1356 # Checking for updates … … 1491 1492 self._file_menu.AppendSeparator() 1492 1493 1493 def _add_menu_file(self):1494 """1495 add menu file1496 """1497 1498 # File menu1499 self._file_menu = wx.Menu()1500 1494 style1 = self.__gui_style & GUIFRAME.MULTIPLE_APPLICATIONS 1501 1495 if OPEN_SAVE_MENU: … … 1526 1520 'Save state of the current active analysis panel') 1527 1521 wx.EVT_MENU(self, id, self._on_save_application) 1522 if not sys.platform =='darwin': 1528 1523 self._file_menu.AppendSeparator() 1529 1530 id = wx.NewId() 1531 self._file_menu.Append(id, '&Quit', 'Exit') 1532 wx.EVT_MENU(self, id, self.Close) 1524 id = wx.NewId() 1525 self._file_menu.Append(id, '&Quit', 'Exit') 1526 wx.EVT_MENU(self, id, self.Close) 1527 1528 def _add_menu_file(self): 1529 """ 1530 add menu file 1531 """ 1532 # File menu 1533 self._file_menu = wx.Menu() 1533 1534 # Add sub menus 1534 1535 self._menubar.Append(self._file_menu, '&File') … … 1947 1948 if temp is not None: 1948 1949 doc = temp 1949 1950 1950 1951 # Write the XML document 1951 1952 extens = APPLICATION_STATE_EXTENSION … … 1961 1962 except: 1962 1963 msg = "Error occurred while saving: " 1963 msg += "To save, at lea t one application panel "1964 msg += "To save, at least one application panel " 1964 1965 msg += "should have a data set.." 1965 1966 wx.PostEvent(self, StatusEvent(status=msg)) -
src/sans/perspectives/fitting/pagestate.py
reddb6ec rac5b69d 20 20 21 21 import xml.dom.minidom 22 from xml.dom.minidom import parseString 22 23 from lxml import etree 23 24 … … 672 673 # We are appending to an existing document 673 674 newdoc = doc 674 top_element = newdoc.createElement(FITTING_NODE_NAME) 675 try: 676 top_element = newdoc.createElement(FITTING_NODE_NAME) 677 except: 678 string = etree.tostring(doc, pretty_print=True) 679 newdoc = parseString(string) 680 top_element = newdoc.createElement(FITTING_NODE_NAME) 675 681 if entry_node is None: 676 682 newdoc.documentElement.appendChild(top_element) 677 683 else: 678 entry_node.appendChild(top_element) 679 684 try: 685 entry_node.appendChild(top_element) 686 except: 687 node_name = entry_node.tag 688 node_list = newdoc.getElementsByTagName(node_name) 689 entry_node = node_list.item(0) 690 entry_node.appendChild(top_element) 691 680 692 attr = newdoc.createAttribute("version") 681 693 attr.nodeValue = '1.0' … … 750 762 for item in list_of_state_parameters: 751 763 element = newdoc.createElement(item[0]) 752 com = "self._toXML_helper( list=self.%s,"764 com = "self._toXML_helper(thelist=self.%s," 753 765 com += " element=element, newdoc=newdoc)" 754 766 exec com % item[1] … … 762 774 return None 763 775 else: 764 return newdoc .toprettyxml()776 return newdoc 765 777 766 778 def _fromXML_helper(self, node, list): … … 1252 1264 state = PageState() 1253 1265 state.fromXML(node=nodes[0]) 1266 1254 1267 except: 1255 1268 logging.info("XML document does not contain fitting information.\n %s" % sys.exc_value) … … 1257 1270 return state 1258 1271 1259 def _parse_ entry(self, dom):1272 def _parse_save_state_entry(self, dom): 1260 1273 """ 1261 1274 Parse a SASentry … … 1268 1281 node = dom.xpath('ns:data_class', namespaces={'ns': CANSAS_NS}) 1269 1282 if not node or node[0].text.lstrip().rstrip() != "Data2D": 1270 return CansasReader._parse_entry(self, dom) 1283 return_value, _ = self._parse_entry(dom) 1284 numpy.trim_zeros(return_value.x) 1285 numpy.trim_zeros(return_value.y) 1286 numpy.trim_zeros(return_value.dy) 1287 size_dx = return_value.dx.size 1288 size_dxl = return_value.dxl.size 1289 size_dxw = return_value.dxw.size 1290 if size_dxl == 0 and size_dxw == 0: 1291 return_value.dxl = None 1292 return_value.dxw = None 1293 numpy.trim_zeros(return_value.dx) 1294 elif size_dx == 0: 1295 return_value.dx = None 1296 size_dx = size_dxl 1297 numpy.trim_zeros(return_value.dxl) 1298 numpy.trim_zeros(return_value.dxw) 1299 1300 return return_value, _ 1271 1301 1272 1302 #Parse 2D … … 1539 1569 for entry in entry_list: 1540 1570 try: 1541 sas_entry = self._parse_entry(entry)1571 sas_entry, _ = self._parse_save_state_entry(entry) 1542 1572 except: 1543 1573 raise … … 1608 1638 # Sanity check 1609 1639 if self.cansas == True: 1610 1611 1640 # Add fitting information to the XML document 1612 1641 doc = self.write_toXML(datainfo, fitstate) 1613 1642 # Write the XML document 1614 fd = open(filename, 'w')1615 fd.write(doc.toprettyxml())1616 fd.close()1617 1643 else: 1618 fitstate.toXML(file=filename) 1644 doc = fitstate.toXML(file=filename) 1645 1646 # Save the document no matter the type 1647 fd = open(filename, 'w') 1648 fd.write(doc.toprettyxml()) 1649 fd.close() 1619 1650 1620 1651 def write_toXML(self, datainfo=None, state=None): … … 1638 1669 state.data.run = [str(state.data.name)] 1639 1670 state.data.run_name[0] = state.data.name 1640 1671 1641 1672 if issubclass(state.data.__class__, 1642 1673 sans.dataloader.data_info.Data1D): … … 1648 1679 1649 1680 if state is not None: 1650 state.toXML(doc=doc, file=data.filename, entry_node=sasentry)1651 1681 doc = state.toXML(doc=doc, file=data.filename, entry_node=sasentry) 1682 1652 1683 return doc 1653 1684 -
src/sans/perspectives/invariant/invariant_panel.py
r5777106 r92a2ecd 218 218 if new_doc is not None: 219 219 if doc is not None and hasattr(doc, "firstChild"): 220 child = new_doc.firstChild.firstChild 221 doc.firstChild.appendChild(child) 220 child = new_doc.getElementsByTagName("SASentry") 221 for item in child: 222 doc.firstChild.appendChild(item) 222 223 else: 223 224 doc = new_doc -
src/sans/perspectives/invariant/invariant_state.py
r51f14603 r92a2ecd 254 254 : param entry_node: XML node within the XML document at which we will append the data [optional] 255 255 """ 256 # TODO: Get this to work 256 257 from xml.dom.minidom import getDOMImplementation 257 258 import time … … 273 274 else: 274 275 entry_node.appendChild(top_element) 275 276 276 277 attr = newdoc.createAttribute("version") 277 278 attr.nodeValue = '1.0' … … 352 353 return None 353 354 else: 354 return newdoc .toprettyxml()355 return newdoc 355 356 356 357 def fromXML(self, file=None, node=None): … … 742 743 for entry in entry_list: 743 744 744 sas_entry = self._parse_entry(entry)745 sas_entry, _ = self._parse_entry(entry) 745 746 invstate = self._parse_state(entry) 746 747 … … 809 810 # Add the invariant information to the XML document 810 811 if state is not None: 811 state.toXML(datainfo.name,doc=doc, entry_node=sasentry)812 doc = state.toXML(datainfo.name,doc=doc, entry_node=sasentry) 812 813 return doc 813 814 -
src/sans/perspectives/pr/inversion_panel.py
r5777106 r92a2ecd 255 255 if new_doc is not None: 256 256 if doc is not None and hasattr(doc, "firstChild"): 257 child = new_doc.firstChild.firstChild 258 doc.firstChild.appendChild(child) 257 child = new_doc.getElementsByTagName("SASentry") 258 for item in child: 259 doc.firstChild.appendChild(item) 259 260 else: 260 261 doc = new_doc -
src/sans/perspectives/pr/inversion_state.py
r5777106 r92a2ecd 148 148 149 149 """ 150 #TODO: Get this to work 150 151 from xml.dom.minidom import getDOMImplementation 151 152 … … 221 222 return None 222 223 else: 223 return newdoc .toprettyxml()224 return newdoc 224 225 225 226 def fromXML(self, file=None, node=None): … … 470 471 471 472 for entry in entry_list: 472 sas_entry = self._parse_entry(entry)473 sas_entry, _ = self._parse_entry(entry) 473 474 prstate = self._parse_prstate(entry) 474 475 #prstate could be None when .svs file is loaded … … 504 505 # Sanity check 505 506 if self.cansas == True: 506 doc = self.write_toXML(datainfo, prstate)507 doc = self.write_toXML(datainfo, prstate) 507 508 # Write the XML document 508 509 fd = open(filename, 'w') … … 527 528 # Create basic XML document 528 529 doc, sasentry = self._to_xml_doc(datainfo) 529 530 530 531 # Add the invariant information to the XML document 531 532 if state is not None: 532 state.toXML(doc=doc, entry_node=sasentry)533 doc = state.toXML(doc=doc, entry_node=sasentry) 533 534 534 535 return doc -
test/sansdataloader/test/utest_cansas.py
r2e3b055 rac5b69d 5 5 warnings.simplefilter("ignore") 6 6 7 import unittest8 import numpy9 7 import sans.dataloader.readers.cansas_reader as cansas 10 8 from sans.dataloader.loader import Loader … … 12 10 from sans.dataloader.readers.xml_reader import XMLreader 13 11 from sans.dataloader.readers.cansas_reader import Reader 14 from sans.dataloader.readers.cansas_constants import cansasConstants12 from sans.dataloader.readers.cansas_constants import CansasConstants 15 13 16 14 import os … … 18 16 import urllib2 19 17 import StringIO 18 import pylint as pylint 19 import unittest 20 import numpy 20 21 21 22 from lxml import etree 22 23 from xml.dom import minidom 23 24 24 CANSAS_FORMAT = cansasConstants.CANSAS_FORMAT25 CANSAS_NS = cansasConstants.CANSAS_NS25 CANSAS_FORMAT = CansasConstants.CANSAS_FORMAT 26 CANSAS_NS = CansasConstants.CANSAS_NS 26 27 27 28 class cansas_reader(unittest.TestCase): … … 72 73 def test_real_xml(self): 73 74 reader = XMLreader(self.xml_valid, self.schema_1_0) 74 valid = reader.validate XML()75 valid = reader.validate_xml() 75 76 if valid: 76 77 self.assertTrue(valid) … … 101 102 filename = "isis_1_1_write_test.xml" 102 103 xmlreader = XMLreader(self.isis_1_1, self.schema_1_1) 103 valid = xmlreader.validate XML()104 xmlreader.set ProcessingInstructions()104 valid = xmlreader.validate_xml() 105 xmlreader.set_processing_instructions() 105 106 self.assertTrue(valid) 106 107 fo = open(self.isis_1_1) … … 126 127 def test_double_trans_spectra(self): 127 128 xmlreader = XMLreader(self.isis_1_1_doubletrans, self.schema_1_1) 128 self.assertTrue(xmlreader.validate XML())129 self.assertTrue(xmlreader.validate_xml()) 129 130 reader = Loader() 130 131 data = reader.load(self.isis_1_1_doubletrans) … … 146 147 valid = [] 147 148 reader1 = XMLreader(self.xml_valid, self.schema_1_0) 148 self.assertTrue(reader1.validate XML())149 self.assertTrue(reader1.validate_xml()) 149 150 reader2 = XMLreader(self.xml_invalid, self.schema_1_0) 150 self.assertFalse(reader2.validate XML())151 self.assertFalse(reader2.validate_xml()) 151 152 reader3 = XMLreader(self.xml_valid, self.schema_1_1) 152 self.assertFalse(reader3.validate XML())153 self.assertFalse(reader3.validate_xml()) 153 154 reader4 = XMLreader(self.xml_invalid, self.schema_1_1) 154 self.assertFalse(reader4.validate XML())155 self.assertFalse(reader4.validate_xml()) 155 156 reader5 = XMLreader(self.isis_1_0, self.schema_1_0) 156 self.assertTrue(reader5.validate XML())157 self.assertTrue(reader5.validate_xml()) 157 158 reader6 = XMLreader(self.isis_1_1, self.schema_1_1) 158 self.assertTrue(reader6.validate XML())159 self.assertTrue(reader6.validate_xml()) 159 160 reader7 = XMLreader(self.isis_1_1, self.schema_1_0) 160 self.assertFalse(reader7.validate XML())161 self.assertFalse(reader7.validate_xml()) 161 162 162 163 163 164 def test_old_cansas_files(self): 164 165 reader1 = XMLreader(self.cansas1d, self.schema_1_0) 165 self.assertTrue(reader1.validate XML())166 self.assertTrue(reader1.validate_xml()) 166 167 file_loader = Loader() 167 168 file1 = file_loader.load(self.cansas1d) 168 169 reader2 = XMLreader(self.cansas1d_units, self.schema_1_0) 169 self.assertTrue(reader2.validate XML())170 self.assertTrue(reader2.validate_xml()) 170 171 reader3 = XMLreader(self.cansas1d_badunits, self.schema_1_0) 171 self.assertTrue(reader3.validate XML())172 self.assertTrue(reader3.validate_xml()) 172 173 reader4 = XMLreader(self.cansas1d_slit, self.schema_1_0) 173 self.assertTrue(reader4.validate XML())174 self.assertTrue(reader4.validate_xml()) 174 175 175 176 … … 177 178 filename = "isis_1_0_write_test.xml" 178 179 xmlreader = XMLreader(self.isis_1_0, self.schema_1_0) 179 valid = xmlreader.validate XML()180 valid = xmlreader.validate_xml() 180 181 self.assertTrue(valid) 181 182 reader_generic = Loader() … … 191 192 written_data = return_data[0] 192 193 xmlwrite = XMLreader(filename, self.schema_1_0) 193 valid = xmlreader.validate XML()194 valid = xmlreader.validate_xml() 194 195 self.assertTrue(valid) 195 196 self._check_data(written_data) … … 198 199 def test_processing_instructions(self): 199 200 reader = XMLreader(self.isis_1_1, self.schema_1_1) 200 valid = reader.validate XML()201 valid = reader.validate_xml() 201 202 if valid: 202 203 ## find the processing instructions and make into a dictionary 203 dic = self.get ProcessingInstructions(reader)204 dic = self.get_processing_instructions(reader) 204 205 self.assertTrue(dic == {'xml-stylesheet': 'type="text/xsl" href="cansas1d.xsl" '}) 205 206 … … 208 209 209 210 ## take the processing instructions and put them back in 210 xmldoc = self.set ProcessingInstructions(xmldoc, dic)211 xmldoc = self.set_processing_instructions(xmldoc, dic) 211 212 xml_output = xmldoc.toprettyxml() 212 213 213 214 214 def set ProcessingInstructions(self, minidomObject, dic):215 xmlroot = minidom Object.firstChild215 def set_processing_instructions(self, minidom_object, dic): 216 xmlroot = minidom_object.firstChild 216 217 for item in dic: 217 pi = minidom Object.createProcessingInstruction(item, dic[item])218 minidom Object.insertBefore(pi, xmlroot)219 return minidom Object220 221 222 def get ProcessingInstructions(self, XMLreaderObject):218 pi = minidom_object.createProcessingInstruction(item, dic[item]) 219 minidom_object.insertBefore(pi, xmlroot) 220 return minidom_object 221 222 223 def get_processing_instructions(self, xml_reader_object): 223 224 dict = {} 224 pi = XMLreaderObject.xmlroot.getprevious()225 pi = xml_reader_object.xmlroot.getprevious() 225 226 i = 0 226 227 while pi is not None:
Note: See TracChangeset
for help on using the changeset viewer.