Changes in / [786685e:4e9f227] in sasview
- Files:
-
- 11 edited
Legend:
- Unmodified
- Added
- Removed
-
sansview/README.txt
r7a67f4da r10be315 4 4 SasView 3.0.0 5 5 6 - The GUI look and feel has been refactored to be more familiar for Windows 7 users by using MDI frames. Graph windows are also now free-floating 8 - Four new models have been added: CoreShellEllipsoidXTModel, 9 RectangularPrismModel, RectangularHollowPrismModel and 10 RectangularHollowPrismInfThinWallsModel 11 - The data loader now supports ILL DAT data files and reads the full meta 12 information from canSAS file formats 6 - Implemented Polarization and magnetic SANS parameters in some models. 7 - New Tool: Generic SANS / PDB reader & atomic scattering calculator 8 - New Tool: Image viewer 13 9 - Redefined angles of angular parameters for anisotropic models 14 - A number of minor features have been added such as permitting a log 15 distribution of points when using a model to simulate data, and the 16 addition of a Kratky plot option to the linear plots 17 - a number of bugs have also been fixed 18 - BETA: Magnetic contrast supporting full polarization analysis has been 19 implemented for some spherical and cylindrical models 20 - BETA: Two new tools have been added: 21 * A generic scattering calculator which takes an atomic, magnetic or 22 SLD distribution in space and generates the appropriate 2D 23 scattering pattern. In some cases the orientationally averaged 24 (powder) 1D scattering can also be computed. Supported formats 25 include: SLD or text, PDB, and OMF magnetic moment distribution 26 file. 27 * An image viewer/converter for data in image format; this reads in 28 an image file and will attempt to convert the image pixels to 29 data. Supported formats include: TIFF, TIF, PNG, BMP, JPG 10 - Use MDI frames in GUI 30 11 31 - New in Version 2.2.1 32 - Minor patch to support CanSAS XML v1.1 file format 33 34 - New in Version 2.2.0 35 - Application name changed to SasView 36 - New fully customizable Category Manager added for better management of 37 increasing number of models 38 - Improved the Grid Window functionality in the batch fitting mode 39 - Added a simpler Graph/Plot modification interface 40 - Added a new 'Data Operation' tool for addition, subtraction, multiplication, 41 division, of two data sets. 42 - The 'Sum Model' editor was extended and renamed 'Summation and Multiplication' 43 editor 44 - Added more plot symbols options for 1d plots 45 - Added improved trapping of compiling errors to the 'New model editor' 46 - Added some intelligent outputs (e.g., Rg, background, or rod diameter depending on the choice of axis scale of the plot) to the linear fits 47 - Added more models 48 49 - Feature set from Previous Versions: 50 51 - Perspectives Available 52 - Invariant calculator: Calculates the invariant, volume fraction, and 53 specific surface area. 54 - P(r) inversion calculator: Indirect Fourier transformation method. 55 - Fitting: the tool used for modeling and fitting 1D and 2D data to 56 analytical model functions 57 - Tools: provides a number of useful supplementary tools such as SLD 58 calculation 59 60 - Fitting 61 - Includes a large number of model functions, both form factors and structure factors. 62 - Support P(Q)*S(Q) for form factors that flag they can be so multiplied. 63 - Supports Gaussian, lognormal, Shulz, rectangular and custom distribution 64 functions for models that need to include polydispersity or for orientational 65 distributions if appropriate. 66 - Anisotropic shapes and magnetic moment modeling in 2D allow for a non-uniform 67 distribution of orientations of a given axis leading to modeling and fitting 68 capabilities of non azimuthaly symmetric data. 69 - User can choose to weight fits or not. If using weights, the user can choose 70 the error bar on each point if provided in the file, the square root 71 of the intensity or the intensity itself. 72 - Instrumental resolution smearing of model or fits is provided with several 73 options: read the resolution/point fromt he file. Input a pinhole resolution 74 or a slit resolution. 75 - Users can define the Qrange (Qmin and Qmax) for both 1D and 2D data for 76 fitting and modeling, but not graphically. The range can be reset to the 77 defaults (limits of q in data set for a fit) with the reset button. 78 - A mask can be applied to 2D calculation and fitting. 79 - Normalized residual plots are provided with every fit. 80 - Model function help available through detail button or from the fitting panel. 81 - Simultaneous/(advanced)constrained fitting allows for fitting a single 82 data set or several different sets simultaneously with the application 83 of advanced constraints relating fit parameters to functions of other 84 parameters (including from a different set). For example thickness of 85 shell = sin(30) times the length. 86 - Models that are the sum of two other models can be easily generated through the 87 SUM Model menubar item. 88 - New Python models can be added on the fly by creating an appropriate Python 89 file in the model plugin directory. Two tools are provided to help: 90 An easy to use custom model editor allows the quick generation of new Python 91 models by supplying only the parameters and their default value (box 1) 92 and the mathematical function of the model (box 2) and generating the 93 necessary *.py file. A separate advanced model editor provides a full Python 94 file editor. Either way once saved the model becomes immediately available 95 to the application. 96 - A batch fittng capability allows for the analysis of a series of data sets to 97 a sinble model and provides the results in a tabular form suitable for saving 98 or plotting the evolution of the fit parameters with error bars (from within 99 the application). 100 101 - Tools 102 - A scattering length density calculator,including some X-ray information 103 is provided. 104 - A density to vol. fraction converter is provided 105 - In application access to a Python shell/editor (PyCrust) is provided 106 - An instrument resolution calculator, including possible gravitational and 107 TOF effects is provided 108 - A slit size calculator optimized for Anton Paar Saxess is provided. 109 - A kiessig fringe thickness calculator is provided 110 - Plots and plot management 111 - A 3D graphing option (for 2d data/results) is provided with the view 112 controlled by the mouse 113 - 2D plots are shown with an intensity color bar. 2D Color map can be user 114 adjusted. 115 - Supports output of plot to a variety of graphic formats. Supported formats 116 include: png, eps, emf, jpg/jpeg, pdf, ps, tif/tiff, rawRGBbitmap(raw, rgba), 117 and scalable vector graphic (svg/svgz) 118 - Supports ouput of data in plot (1 or 2D) to limited data formats 119 - Multiple data sets can be loaded into a single graph for viewing (but a fit 120 plot can currently only have a single plot). 121 - Extensive context sensitive plot/fitting/manipulation options are available 122 through a right mouse click pop-up menu on plots. 123 - Data management 124 - Supports 2 + column 1D ASCII data, NIST 1D and 2D data, and canSAS data 125 via plug-in mechanism which can easily allow other readers as appropriate. 126 - 2D data is expected in Q space but for historical reasons accepts the 127 NIST 2D raw pixel format and will do conversion internally. 128 - The full data and metadata available to SasView is viewable in ASCII via 129 right clicking on a data set and choosing Data Info in the DataExplorer 130 or on the plots 131 - Supports loading a single file, multiple files, or a whole folder 132 - An optional Data Explorer is provided (default) which simplifies managing, 133 plotting, deleting, or setup for computation. Most functions however do 134 not require access to the explorer/manager and can be accessed through 135 right click menus and the toolbar. The data explorer can be re-started 136 from the menu bar. 137 - Data manipulation 138 - Support various 2D averaging methods : Circular, sectors, annular, 139 boxsum, boxQx and boxQy. 140 - A 2D data maks editor is provided 141 - 2D mask can be applied to the circular averaging. 142 - Miscellaneous features 143 - limited reports can be generated in pdf format 144 - Provides multiprocessor support(Windows only) 145 - Limited startup customization currently includes default startup 146 data folder and choice of default starting with data manager 147 - Limited support for saving(opening) a SasView project or a SasView analysis 148 (subproject) is provided. 149 - SasView can be launched and loaded with a file of interesty by double-clicking 150 on that file (recognized extension) 151 - A data file or data folder can be passed to SasView when launched from 152 the command line. 153 - Limited bookmarking capability to later recall the results of a fit calculation 154 is provided. 155 - Extensive help is provided through context sensitive mouse roll-over, 156 information bar (at the bottom of the panel), the console menu, and 157 access to the help files in several different ways. 158 12 - Previous Versions: 13 14 - Changed the application name to 'SasView'. 15 - Model category can be modified via (Category Manager). 16 - Improved Grid/Batch window functionality. 17 - Added a new tool; Data operation for addition, subtraction, multiplication, division, and combination of two data sets. 18 - Extended Sum Model Editor to Summation and Multiplication Editor. 19 - Better compiling error captures in the 'New' model editor. 20 - More outputs (e.g., Rg, background, or rod diameter) on LinearFit in certain axis scales. 21 - Added DataInfo for data in the DataExplorer and plots 22 - Added Maximize/Restore button in the title bar of the graphs 23 - The 'x' button now works as deleting the graph 24 - Added a hide button in the toolbar of the graph panel 25 - Fixed some bugs and improved some behaviors in the batch/grid panel. 26 - Edit SUM Model from the menubar can now generate more than one sum model. 27 - Report can now be saved as a pdf format on WIN and MAC. 28 - Multiprocessor support(Windows) 29 - Simple custom model editor 30 - Advanced model editor 31 - Sum model editing panel 32 - 3D graphic (for 2d data/results) and legend moves with mouse 33 - New Tool: density to vol fraction calculator and Python file editor 34 - Batch Fit included. 35 - More Graph modifications. 36 - More options for the fit weighting. 37 - Added a Python (PyCrust) shell in the 'Tool' menu. 38 - The 'Startup Setting' in the View menu can remember the last data file folder. 39 - Updated the resolution computation for the gravitational effect and added TOF estimation capability. 40 - Fixed the problem of displaying the fit results in the wrong parameter (with ParkMC FitEngine). 41 - Pr Inversion: Fixed a missing Rg output 42 - Startup Setting: fixed a problem with DataExplorer ON/OFF 43 - Fixed a bug w/ 2D color map dialog 44 - Minor feature added: Enable to load a data folder from the command line 45 - Much easier graphical user interface 46 - Optimized for the speed and accuracy of the computations 47 - Added Many shape, polymer, and other models including SphericalSLD, OnionExponetialShell, and even ReflectivityModels 48 - Added Data Explorer to manage, plot, delete, or setup for computation 49 - Added Instrumental resolution estimator (as a Tool) 50 - Customizable Startup appearance 51 - More functionalities on plot panels 52 - Combined Modeling and Fitting 53 - Save/open a SasView project or SasView analysis (subproject) 54 - Start the SasView application from a data file by double-clicking or from command line 55 - Easy manipulation of data and plot 56 - Provides Normalized residual plot 57 - Added useful key-combinations to copy (fitpage), paste (fitpage), change fit-tolerance, etc. 58 - Report 59 - 2D masked circular averaging 60 - 2D smearing calculation now uses dQ_parrellel and dQ_perpendicular 61 - Improved the speed of loading 2D data. 62 - Improved the speed of P*S calculations. 63 - Added 2D smearing calculation for dQx and dQy given along the x-y axes. 64 - MAC release 65 - Implemented the invariant, volume fraction, and specific surface area computation. 66 - Implemented the scattering length density calculator. 67 - Re-structured 2D calculation to 2D reduced(Q) data rather than raw pixel data 68 - Capable of Mask enhanced 2D calculation and fitting. 69 - Added a 2D mask data editor. 70 - Added inputs for the slit and pinhole resolution. 71 - Added a slit size calculator. 72 - Support more format options to save a graph. 73 - Enable to display multiple data sets in one graph by loading a data into the graph. 74 - Added a tool bar in a plot panel. 75 - Implemented P(r) inversion (Indirect Fourier transformations). 76 - Improved fitting and model calculation speed by a factor of ten. 77 - Supporting many more model functions. 78 - Supporting P(Q)*S(Q) for most of the shape based form factors. 79 - Added more distribution functions for the polydispersion calculations. 80 - Added a bookmark feature so that the results of the calculation can be recalled later. 81 - Q range reset button is added. 82 - Added a color bar in 2D data plots. 83 - Added a model function detail button for an easy access to the model help from the fitting panel. 84 - Simultaneous fit of a number of different sets of data with/without the constraints. 85 - Loading and displaying 1D and 2D data of various formats. 86 - 1D and 2D data fitting using Scipy or Park (a MC fitting optimizer) fit engine. 87 - 2D data manipulation and modeling. 88 - Supporting a number of standard model and model-independent functions including form factor and structure factor functions and their multiplications. 89 - Plug-in mechanism for data readers. 90 - Easy pop-up menu by mouse clicking on a given plot. 91 - Users arrange the various windows. 92 - Supporting varius 2D averaging methods : Circular, sectorslicer, annulus, boxsum, boxQx and boxQy. 93 - User defined Qrange (Qmin and Qmax) for both 1D and 2D data for fitting and modeling. 94 - The user can toggle between a number of different scales on all plots. 95 - Support saving data in the formats of ASCII and xml. 159 96 160 97 161 98 2- Downloading and Installing 162 99 163 *** Note 1: Much more information is available at www.sasview.org under links.164 Look in the 'For Developers' section and particularly the wiki at165 www.sasview.org/trac/wiki.166 *** Note 2: If you have EXE or ZIP SasView installer, you won't need any of 167 the following. 100 *** Note: If you have EXE or ZIP SasView installer, you don't need any of the following. 101 102 2.1- System Requirements: 103 - Python version >= 2.5 and < 3.0 should be running on the system 104 - We currently use Python 2.6 168 105 169 2.1- System Requirements: 170 - Python version >= 2.5 and < 3.0 should be running on the system 171 - We currently use Python 2.6 106 2.2- Installing from source: 107 - Get the code from https://sasviewproject.svn.sourceforge.net/svnroot/sansviewproject/releases/sasview-x.x.x 108 - run 'python setup.py install' under the 'sasview-x.x.x' folder 109 - run 'python sasview.py' under the 'sasview' folder. 110 - The following modules are required (version numbers are what are used in the release build): 111 - wxPython 2.8.12.1 (NOTE: do NOT use version 2.9) 112 - matplotlib 1.1.0 (NOTE: Mac build is using version 1.0.1) 113 - SciPy 0.10.1 114 - pisa 3.0.27 (NOTE Mac Version uses 3.0.33 BUT -- DO NOT USE ver 3.0.33 on windows: it will not work!) 115 - setuptools 0.6c11 172 116 173 2.2- Installing from source: 174 - Get the code from sourceforge at https://svn.code.sf.net/p/sasview/code/ 175 for trunk end with code/trunk for a this release version end in 176 code/releases/sasview-3.0.0 177 - run 'python setup.py install' under the 'sasview-x.x.x' folder 178 - run 'python sasview.py' under the 'sasview' folder. 179 - The following modules are required (version numbers are what are used in the release build): 180 - wxPython 2.8.12.1 (NOTE: do NOT use version 2.9) 181 - matplotlib 1.1.0 182 - SciPy 0.10.1 (NOTE: Mac build uses 0.10.0) 183 - pisa 3.0.27 (DO NOT USE ver 3.0.33 on windows: it will not work!) 184 - setuptools 0.6c11 117 (The following three are easily installed using easy_install) 118 - lxml 2.3.0 119 - numpy 1.6.1 (NOTE: Mac build is using version 1.5.1) 120 - periodictable 1.3.0 185 121 186 (The following three are easily installed using easy_install) 187 - lxml 2.3.0.0 (NOTE: Mac build uses 3.1.0.0) 188 - numpy 1.6.1 (NOTE: Mac build uses version 1.6.2) 189 - periodictable 1.3.0 190 191 (The following are additional dependencies for Mac) 192 - py2app 0.7.1 193 194 (The following are additional dependencies for Windows) 195 - comtypes 0.6.2 (for PDF support on windows systems) 196 - pywin32 build 217 (to read ms office) 197 - html5lib 0.95-dev 198 - reportlab 2.5 (NOTE: Mab build uses 2.6 though should not be needed) 199 - pyparsing 1.5.5 (required for periodictable and bundling 200 NOTE: is ALSO present on Mac Build) 201 - PIL 1.1.7 (Python Image Library - NOTE: is also present on Mac build) 202 - py2exe 0.6.9 (WIN) 203 - vcredist_x86.exe (version 9.0.21022.8 -microsoft visual C 2008 204 re-distributable) 205 - subversion -1.6.0 (<1.6.1) 206 - MinGW w/ gcc version 4.6.1 (WIN) 207 - Innosetup (WIN - isetup 5.4.2-unicode). 208 209 (On Windows, the following site has all the dependencies nicely packaged) 210 http://www.lfd.uci.edu/~gohlke/pythonlibs/ 122 (The following are additional dependencies for Mac) 123 - py2app 124 125 (The following are additional dependencies for Windows) 126 - comtypes 0.6.2 (for PDF support on windows systems) 127 - pywin32 build 217 (to read ms office) 128 - pyPdf 1.13 129 - html5lib 0.95 130 - reportlab 2.5 131 - pyparsing 1.5.5 (required for periodictable and bundling) 132 - PIL 1.1.7 (Python Image Library) 133 - py2exe 0.6.9 (WIN) 134 - svn 135 - MinGW w/ gcc version 4.6.1 (WIN) 136 - Innosetup (WIN). 137 138 (On Windows, the following site has all the dependencies nicely packaged) 139 http://www.lfd.uci.edu/~gohlke/pythonlibs/ 211 140 212 141 213 142 3- Known Issues 214 143 215 3.1- All systems: 216 - very old computers may not be able to run 217 - polydispersity on multiple parameters included in a simultaneous/ 218 constrained fit will likely not be correct 219 - Save project does not store the state of all the windows 144 3.1- All systems: 145 - very old computers may not be able to run 220 146 221 222 147 3.2- Windows: 148 - None 223 149 224 225 - multiprocessing does not currently work on MAC OS 150 3.3- MAC: 151 - None 226 152 227 228 153 3.4- Linux: 154 - None 229 155 230 4- SasView website156 4- Troubleshooting 231 157 232 - www.sasview.org. This main project site is the gateway to all 233 information about the sasview project. It includes information 234 about the project, a FAQ page and links to all developer and user 235 information, tools and resources. 158 - None 236 159 237 160 5- Frequently Asked Questions 238 161 239 - www.sasview.org/faq.html 162 - None 240 163 241 164 6- Installer download website 242 165 243 - Latest release Version 244 - http://sourceforge.net/projects/sasview/files/ 245 - Latest developer Windows or Ubuntu build 246 - http://build.sasview.org/ 247 - Latest developer Mac OS build 248 - http://download.mantidproject.org/jenkins/view/All/job/sasview_snowleopard_32bit/ 249 250 166 - http://sourceforge.net/projects/sansviewproject/files/ -
src/sans/dataloader/readers/cansas_constants.py
rac5b69d r9e2bc6c 3 3 the cansas_reader.py file to read in any version of the cansas format. 4 4 """ 5 class CansasConstants: 6 """ 7 The base class to define where all of the data is to be saved by 8 cansas_reader.py. 9 """ 10 11 names = '' 5 class cansasConstants: 6 7 ns = '' 12 8 format = '' 13 9 14 10 15 11 def __init__(self): 16 self.n ames = self.CANSAS_NS12 self.ns = self.CANSAS_NS 17 13 self.format = self.CANSAS_FORMAT 18 14 19 15 20 def iterate_namespace(self, namespace):16 def _iterate_namespace(self, namespace): 21 17 """ 22 18 Method to iterate through a cansas constants tree based on a list of … … 27 23 """ 28 24 # The current level to look through in cansas_constants. 29 return_me = CurrentLevel()25 return_me = currentLevel() 30 26 return_me.current_level = self.CANSAS_FORMAT.get("SASentry") 31 27 # Defaults for variable and datatype … … 34 30 return_me.ns_optional = True 35 31 for name in namespace: 36 try: 37 if name != "SASentry": 32 if name != "SASentry": 33 return_me.current_level = \ 34 return_me.current_level.get("children").get(name, "") 35 if return_me.current_level == "": 38 36 return_me.current_level = \ 39 return_me.current_level.get("children").get(name, "") 40 if return_me.current_level == "": 41 return_me.current_level = \ 42 return_me.current_level.get("<any>", "") 43 cl_variable = return_me.current_level.get("variable", "") 44 cl_datatype = return_me.current_level.get("storeas", "") 45 cl_units_optional = \ 46 return_me.current_level.get("units_required", "") 47 # Where are how to store the variable for the given namespace 48 # CANSAS_CONSTANTS tree is hierarchical, so is no value, inherit 49 return_me.ns_variable = cl_variable if cl_variable != "" else \ 50 return_me.ns_variable 51 return_me.ns_datatype = cl_datatype if cl_datatype != "" else \ 52 return_me.ns_datatype 53 return_me.ns_optional = cl_units_optional if \ 54 cl_units_optional != return_me.ns_optional \ 55 else return_me.ns_optional 56 except AttributeError: 57 return_me.ns_variable = "{0}.meta_data[\"{2}\"] = \"{1}\"" 58 return_me.ns_datatype = "content" 59 return_me.ns_optional = True 37 return_me.current_level.get("<any>", "") 38 cl_variable = return_me.current_level.get("variable", "") 39 cl_datatype = return_me.current_level.get("storeas", "") 40 cl_units_optional = \ 41 return_me.current_level.get("units_required", "") 42 # Where are how to store the variable for the given namespace 43 # CANSAS_CONSTANTS tree is hierarchical, so is no value, inherit 44 return_me.ns_variable = cl_variable if cl_variable != "" else \ 45 return_me.ns_variable 46 return_me.ns_datatype = cl_datatype if cl_datatype != "" else \ 47 return_me.ns_datatype 48 return_me.ns_optional = cl_units_optional if \ 49 cl_units_optional != return_me.ns_optional \ 50 else return_me.ns_optional 60 51 return return_me 61 52 … … 463 454 } 464 455 SASINSTR_SRC = { 465 "attributes" : {"name" : {"variable" : \ 466 "{0}.source.name = \"{1}\""}}, 456 "attributes" : {"name" : {"variable" : "{0}.source.name = \"{1}\""}}, 467 457 "variable" : None, 468 458 "children" : { 469 "radiation" : {"variable" : \ 470 "{0}.source.radiation = \"{1}\""}, 459 "radiation" : {"variable" : "{0}.source.radiation = \"{1}\""}, 471 460 "beam_size" : SASINSTR_SRC_BEAMSIZE, 472 461 "beam_shape" : {"variable" : \ … … 658 647 "unit" : "pixel_size_unit", 659 648 "attributes" : { 660 "unit" : \ 661 "{0}.pixel_size_unit = \"{1}\"", 649 "unit" : "{0}.pixel_size_unit = \"{1}\"", 662 650 "storeas" : "content" 663 651 } … … 667 655 "unit" : "pixel_size_unit", 668 656 "attributes" : { 669 "unit" : \ 670 "{0}.pixel_size_unit = \"{1}\"", 657 "unit" : "{0}.pixel_size_unit = \"{1}\"", 671 658 "storeas" : "content" 672 659 } … … 676 663 "unit" : "pixel_size_unit", 677 664 "attributes" : { 678 "unit" : \ 679 "{0}.pixel_size_unit = \"{1}\"", 665 "unit" : "{0}.pixel_size_unit = \"{1}\"", 680 666 "storeas" : "content" 681 667 } … … 754 740 } 755 741 756 class CurrentLevel: 757 """ 758 A helper class to hold information on where you are in the constants tree 759 """ 742 class currentLevel: 760 743 761 744 current_level = '' … … 765 748 766 749 def __init__(self): 767 self.current_level = {}750 self.current_level = '' 768 751 self.ns_variable = '' 769 752 self.ns_datatype = "content" -
src/sans/dataloader/readers/cansas_reader.py
rbb1b892 r2e3b055 18 18 import sys 19 19 import datetime 20 import inspect21 # For saving individual sections of data22 20 from sans.dataloader.data_info import Data1D 23 21 from sans.dataloader.data_info import Collimation … … 26 24 from sans.dataloader.data_info import Process 27 25 from sans.dataloader.data_info import Aperture 28 # Both imports used. Do not remove either.29 from xml.dom.minidom import parseString30 26 import sans.dataloader.readers.xml_reader as xml_reader 31 from sans.dataloader.readers.xml_reader import XMLreader 32 from sans.dataloader.readers.cansas_constants import CansasConstants 27 from sans.dataloader.readers.cansas_constants import cansasConstants 33 28 34 29 _ZERO = 1e-16 … … 41 36 HAS_CONVERTER = False 42 37 43 CONSTANTS = CansasConstants()44 CANSAS_FORMAT = CONSTANTS.format45 CANSAS_NS = CONSTANTS.names38 constants = cansasConstants() 39 CANSAS_FORMAT = constants.format 40 CANSAS_NS = constants.ns 46 41 ALLOW_ALL = True 47 42 48 43 49 # minidom used in functions called by outside classes50 import xml.dom.minidom51 # DO NOT REMOVE52 # Called by outside packages:53 # sans.perspectives.invariant.invariant_state54 # sans.perspectives.fitting.pagestate55 44 def get_content(location, node): 56 45 """ … … 62 51 :return: Element, or None 63 52 """ 64 nodes = node.xpath(location, 65 namespaces={'ns': CANSAS_NS.get("1.0").get("ns")}) 53 nodes = node.xpath(location, namespaces={'ns': CANSAS_NS}) 66 54 67 55 if len(nodes) > 0: … … 71 59 72 60 73 # DO NOT REMOVE 74 # Called by outside packages: 75 # sans.perspectives.fitting.pagestate 61 def get_float(location, node): 62 """ 63 Get the content of a node as a float 64 65 :param location: xpath location 66 :param node: node to start at 67 """ 68 nodes = node.xpath(location, namespaces={'ns': CANSAS_NS}) 69 70 value = None 71 attr = {} 72 if len(nodes) > 0: 73 try: 74 value = float(nodes[0].text) 75 except: 76 # Could not pass, skip and return None 77 msg = "cansas_reader.get_float: could not " 78 msg += " convert '%s' to float" % nodes[0].text 79 logging.error(msg) 80 if nodes[0].get('unit') is not None: 81 attr['unit'] = nodes[0].get('unit') 82 return value, attr 83 84 85 # This is called by sans.perspectives.fitting.pagestate.py 86 # Do not remove 76 87 def write_node(doc, parent, name, value, attr={}): 77 88 """ … … 94 105 95 106 96 class Reader( XMLreader):107 class Reader(): 97 108 """ 98 109 Class to load cansas 1D XML files … … 103 114 ##CanSAS version - defaults to version 1.0 104 115 cansas_version = "1.0" 105 106 logging = [] 116 ##Data reader 117 # TODO: make the reader extend the XMLreader class? 118 reader = xml_reader.XMLreader() 107 119 errors = [] 108 120 109 121 type_name = "canSAS" 122 110 123 ## Wildcards 111 type = ["XML files (*.xml)|*.xml" , "SasView Save Files (*.svs)|*.svs"]124 type = ["XML files (*.xml)|*.xml"] 112 125 ## List of allowed extensions 113 ext = ['.xml', '.XML' , '.svs', '.SVS']126 ext = ['.xml', '.XML'] 114 127 115 128 ## Flag to bypass extension check … … 120 133 self.errors = [] 121 134 122 def is_cansas(self , ext="xml"):135 def is_cansas(self): 123 136 """ 124 137 Checks to see if the xml file is a CanSAS file 125 138 """ 126 if self. validate_xml():139 if self.reader.validateXML(): 127 140 name = "{http://www.w3.org/2001/XMLSchema-instance}schemaLocation" 128 value = self. xmlroot.get(name)129 if CANSAS_NS.get(self.cansas_version).get("ns") == \130 value.rsplit(" ")[0] :141 value = self.reader.xmlroot.get(name) 142 if (CANSAS_NS.get(self.cansas_version).get("ns") == \ 143 value.rsplit(" ")[0]): 131 144 return True 132 if ext == "svs":133 return True134 145 return False 135 146 … … 141 152 """ 142 153 # X - Q value; Y - Intensity (Abs) 143 x _vals= numpy.empty(0)144 y _vals= numpy.empty(0)145 dx _vals= numpy.empty(0)146 dy _vals= numpy.empty(0)154 x = numpy.empty(0) 155 y = numpy.empty(0) 156 dx = numpy.empty(0) 157 dy = numpy.empty(0) 147 158 dxl = numpy.empty(0) 148 159 dxw = numpy.empty(0) … … 151 162 output = [] 152 163 # ns - Namespace hierarchy for current xml object 153 ns _list= []164 ns = [] 154 165 155 166 # Check that the file exists … … 157 168 basename = os.path.basename(xml) 158 169 _, extension = os.path.splitext(basename) 159 # If the fi letype is not allowed, return nothing170 # If the fiel type is not allowed, return nothing 160 171 if extension in self.ext or self.allow_all: 161 # Get the file location of162 172 base_name = xml_reader.__file__ 163 173 base_name = base_name.replace("\\","/") … … 165 175 166 176 # Load in xml file and get the cansas version from the header 167 self. set_xml_file(xml)168 root = self. xmlroot177 self.reader.setXMLFile(xml) 178 root = self.reader.xmlroot 169 179 if root is None: 170 180 root = {} … … 177 187 178 188 # Link a schema to the XML file. 179 self. set_schema(schema_path)189 self.reader.setSchema(schema_path) 180 190 181 191 # Try to load the file, but raise an error if unable to. 182 192 # Check the file matches the XML schema 183 193 try: 184 if self.is_cansas( extension):194 if self.is_cansas(): 185 195 # Get each SASentry from XML file and add it to a list. 186 196 entry_list = root.xpath('/ns:SASroot/ns:SASentry', 187 197 namespaces={'ns': cansas_defaults.get("ns")}) 188 ns _list.append("SASentry")198 ns.append("SASentry") 189 199 190 200 # If multiple files, modify the name for each is unique … … 194 204 # Parse each SASentry item 195 205 for entry in entry_list: 196 # Define a new Data1D object with zeroes for 197 # x_vals and y_vals 198 data1d = Data1D(x_vals, y_vals, dx_vals, dy_vals) 206 # Define a new Data1D object with zeroes for x and y 207 data1d = Data1D(x,y,dx,dy) 199 208 data1d.dxl = dxl 200 209 data1d.dxw = dxw … … 211 220 212 221 # Get all preprocessing events and encoding 213 self. set_processing_instructions()222 self.reader.setProcessingInstructions() 214 223 data1d.meta_data[PREPROCESS] = \ 215 self. processing_instructions224 self.reader.processingInstructions 216 225 217 226 # Parse the XML file 218 227 return_value, extras = \ 219 self._parse_entry(entry, ns _list, data1d)228 self._parse_entry(entry, ns, data1d) 220 229 del extras[:] 221 230 … … 242 251 output.append(return_value) 243 252 else: 244 value = self. find_invalid_xml()253 value = self.reader.findInvalidXML() 245 254 output.append("Invalid XML at: {0}".format(value)) 246 255 except: 247 256 # If the file does not match the schema, raise this error 248 raise RuntimeError, "%s cannot be read " % xml257 raise RuntimeError, "%s cannot be read \increment" % xml 249 258 return output 250 259 # Return a list of parsed entries that dataloader can manage 251 260 return None 252 261 253 def _create_unique_key(self, dictionary, name, numb= 0):262 def _create_unique_key(self, dictionary, name, i = 0): 254 263 """ 255 264 Create a unique key value for any dictionary to prevent overwriting … … 258 267 :param dictionary: A dictionary with any number of entries 259 268 :param name: The index of the item to be added to dictionary 260 :param numb: The number to be appended to the name, starts at 0269 :param i: The number to be appended to the name, starts at 0 261 270 """ 262 271 if dictionary.get(name) is not None: 263 numb+= 1272 i += 1 264 273 name = name.split("_")[0] 265 name += "_{0}".format( numb)266 name = self._create_unique_key(dictionary, name, numb)274 name += "_{0}".format(i) 275 name = self._create_unique_key(dictionary, name, i) 267 276 return name 268 277 … … 275 284 276 285 :param new_current_level: cansas_constants level as returned by 277 iterate_namespace286 _iterate_namespace 278 287 :param attr: The attributes of the node 279 288 :param data1d: Where the values will be saved … … 309 318 return 310 319 except: 311 err_msg = \ 312 "CanSAS reader: could not convert the units" 320 err_msg = "CanSAS reader: could not convert the units" 313 321 self.errors.append(err_msg) 314 322 return … … 335 343 return node_value, value_unit 336 344 337 def _parse_entry(self, dom, n ames=["SASentry"], data1d=None, extras=[]):345 def _parse_entry(self, dom, ns, data1d, extras = []): 338 346 """ 339 347 Parse a SASEntry - new recursive method for parsing the dom of … … 341 349 and extra nodes to be read in simultaneously. 342 350 343 :param dom: dom object with a namespace base of n ames344 :param n ames: A list of element names that lead up to the dom object351 :param dom: dom object with a namespace base of ns 352 :param ns: A list of element names that lead up to the dom object 345 353 :param data1d: The data1d object that will be modified 346 354 :param extras: Any values that should go into meta_data when data1d 347 355 is not a Data1D object 348 356 """ 349 357 350 358 # A portion of every namespace entry 351 if data1d == None:352 x_vals = numpy.empty(0)353 y_vals = numpy.empty(0)354 dx_vals = numpy.empty(0)355 dy_vals = numpy.empty(0)356 dxl = numpy.empty(0)357 dxw = numpy.empty(0)358 data1d = Data1D(x_vals, y_vals, dx_vals, dy_vals)359 data1d.dxl = dxl360 data1d.dxw = dxw361 362 359 base_ns = "{0}{1}{2}".format("{", \ 363 360 CANSAS_NS.get(self.cansas_version).get("ns"), "}") … … 369 366 for node in dom: 370 367 try: 371 # Get the element name and set the current n ames level368 # Get the element name and set the current ns level 372 369 tagname = node.tag.replace(base_ns, "") 373 370 tagname_original = tagname 374 if tagname == "fitting_plug_in" or tagname == "pr_inversion" or\ 375 tagname == "invariant": 376 continue 377 names.append(tagname) 371 ns.append(tagname) 378 372 attr = node.attrib 379 373 children = node.getchildren() 380 if len(children) == 0:381 children = None382 374 save_data1d = data1d 383 375 … … 418 410 419 411 # Get where to store content 420 cs_values = CONSTANTS.iterate_namespace(names)412 cs_values = constants._iterate_namespace(ns) 421 413 # If the element is a child element, recurse 422 414 if children is not None: 423 415 # Returned value is new Data1D object with all previous and 424 416 # new values in it. 425 data1d, extras = self._parse_entry(node, 426 names, data1d, extras) 417 data1d, extras = self._parse_entry(node, ns, data1d, extras) 427 418 428 419 #Get the information from the node … … 497 488 pass 498 489 499 except TypeError as e:490 except TypeError: 500 491 pass 501 492 except Exception as e: … … 518 509 else: 519 510 save_data1d = data1d 520 if tagname_original == "fitting_plug_in" or \ 521 tagname_original == "invariant" or \ 522 tagname_original == "pr_inversion": 523 pass 524 else: 525 data1d = save_data1d 526 # Remove tagname from names to restore original base 527 names.remove(tagname_original) 511 data1d = save_data1d 512 # Remove tagname from ns to restore original base 513 ns.remove(tagname_original) 514 528 515 return data1d, extras 529 516 530 531 517 def _to_xml_doc(self, datainfo): 532 518 """ … … 535 521 :param datainfo: Data1D object 536 522 """ 523 537 524 if not issubclass(datainfo.__class__, Data1D): 538 525 raise RuntimeError, "The cansas writer expects a Data1D instance" 539 526 540 527 # Get PIs and create root element 541 pis = self.return_processing_instructions() 542 if len(pis) > 0: 543 pi_tree = self.create_tree(pis[0]) 544 i = 1 545 for i in range(1,len(pis) - 1): 546 pi_tree = self.append(pis[i], pi_tree) 547 pi_string = self.to_string(pi_tree) 548 else: 549 pi_string = "" 528 pis = self.reader.return_processing_instructions() 529 doc = self.reader.create_tree(pis[0]) 530 i = 1 531 for i in range(1,len(pis) - 1): 532 doc = self.reader.append(pis[i], doc) 550 533 551 534 # Define namespaces and create SASroot object … … 557 540 else: 558 541 url = "http://svn.smallangles.net/svn/canSAS/1dwg/trunk/" 559 schema _location = "{0} {1}cansas1d.xsd".format(ns, url)560 attrib = {"{" + xsi + "}schemaLocation" : schema _location,542 schemaLocation = "{0} {1}cansas1d.xsd".format(ns, url) 543 attrib = {"{" + xsi + "}schemaLocation" : schemaLocation, 561 544 "version" : version} 562 545 nsmap = {'xsi' : xsi, None: ns} 563 546 564 main_node = self. create_element("{" + ns + "}SASroot", \547 main_node = self.reader.create_element("{" + ns + "}SASroot", \ 565 548 attrib = attrib, \ 566 549 nsmap = nsmap) 567 550 568 551 # Create ElementTree, append SASroot and apply processing instructions 569 base_string = pi_string + self.to_string(main_node) 570 base_element = self.create_element_from_string(base_string) 571 doc = self.create_tree(base_element) 552 base_string = self.reader.toString(doc) + \ 553 self.reader.toString(main_node) 554 base_element = self.reader.create_element_from_string(base_string) 555 doc = self.reader.create_tree(base_element) 572 556 573 557 # Create SASentry Element 574 entry_node = self. create_element("SASentry")558 entry_node = self.reader.create_element("SASentry") 575 559 root = doc.getroot() 576 560 root.append(entry_node) … … 592 576 593 577 # Data info 594 node = self. create_element("SASdata")595 self. append(node, entry_node)578 node = self.reader.create_element("SASdata") 579 self.reader.append(node, entry_node) 596 580 597 581 for i in range(len(datainfo.x)): 598 pt = self. create_element("Idata")582 pt = self.reader.create_element("Idata") 599 583 node.append(pt) 600 584 self.write_node(pt, "Q", datainfo.x[i], {'unit': datainfo.x_unit}) … … 614 598 self.write_node(pt, "dQl", datainfo.dxl[i], 615 599 {'unit': datainfo.x_unit}) 616 600 617 601 # Transmission Spectrum Info 618 602 for i in range(len(datainfo.trans_spectrum)): 619 603 spectrum = datainfo.trans_spectrum[i] 620 node = self. create_element("SAStransmission_spectrum",604 node = self.reader.create_element("SAStransmission_spectrum", 621 605 {"name" : spectrum.name}) 622 self. append(node, entry_node)606 self.reader.append(node, entry_node) 623 607 if isinstance(spectrum.timestamp, datetime.datetime): 624 608 node.setAttribute("timestamp", spectrum.timestamp) 625 609 for i in range(len(spectrum.wavelength)): 626 pt = self. create_element("Tdata")610 pt = self.reader.create_element("Tdata") 627 611 node.append(pt) 628 612 self.write_node(pt, "Lambda", spectrum.wavelength[i], … … 637 621 638 622 # Sample info 639 sample = self. create_element("SASsample")623 sample = self.reader.create_element("SASsample") 640 624 if datainfo.sample.name is not None: 641 self. write_attribute(sample,625 self.reader.write_attribute(sample, 642 626 "name", 643 627 str(datainfo.sample.name)) 644 self. append(sample, entry_node)628 self.reader.append(sample, entry_node) 645 629 self.write_node(sample, "ID", str(datainfo.sample.ID)) 646 630 self.write_node(sample, "thickness", datainfo.sample.thickness, … … 650 634 {"unit": datainfo.sample.temperature_unit}) 651 635 652 pos = self. create_element("position")636 pos = self.reader.create_element("position") 653 637 written = self.write_node(pos, 654 638 "x", … … 664 648 {"unit": datainfo.sample.position_unit}) 665 649 if written == True: 666 self. append(pos, sample)667 668 ori = self. create_element("orientation")650 self.reader.append(pos, sample) 651 652 ori = self.reader.create_element("orientation") 669 653 written = self.write_node(ori, "roll", 670 654 datainfo.sample.orientation.x, … … 677 661 {"unit": datainfo.sample.orientation_unit}) 678 662 if written == True: 679 self. append(ori, sample)663 self.reader.append(ori, sample) 680 664 681 665 for item in datainfo.sample.details: … … 683 667 684 668 # Instrument info 685 instr = self. create_element("SASinstrument")686 self. append(instr, entry_node)669 instr = self.reader.create_element("SASinstrument") 670 self.reader.append(instr, entry_node) 687 671 688 672 self.write_node(instr, "name", datainfo.instrument) 689 673 690 674 # Source 691 source = self. create_element("SASsource")675 source = self.reader.create_element("SASsource") 692 676 if datainfo.source.name is not None: 693 self. write_attribute(source,677 self.reader.write_attribute(source, 694 678 "name", 695 679 str(datainfo.source.name)) 696 self. append(source, instr)680 self.reader.append(source, instr) 697 681 if datainfo.source.radiation == None or datainfo.source.radiation == '': 698 682 datainfo.source.radiation = "neutron" 699 683 self.write_node(source, "radiation", datainfo.source.radiation) 700 684 701 size = self. create_element("beam_size")685 size = self.reader.create_element("beam_size") 702 686 if datainfo.source.beam_size_name is not None: 703 self. write_attribute(size,687 self.reader.write_attribute(size, 704 688 "name", 705 689 str(datainfo.source.beam_size_name)) … … 713 697 {"unit": datainfo.source.beam_size_unit}) 714 698 if written == True: 715 self. append(size, source)699 self.reader.append(size, source) 716 700 717 701 self.write_node(source, "beam_shape", datainfo.source.beam_shape) … … 734 718 datainfo.collimation.append(coll) 735 719 for item in datainfo.collimation: 736 coll = self. create_element("SAScollimation")720 coll = self.reader.create_element("SAScollimation") 737 721 if item.name is not None: 738 self. write_attribute(coll, "name", str(item.name))739 self. append(coll, instr)722 self.reader.write_attribute(coll, "name", str(item.name)) 723 self.reader.append(coll, instr) 740 724 741 725 self.write_node(coll, "length", item.length, … … 743 727 744 728 for apert in item.aperture: 745 ap = self. create_element("aperture")729 ap = self.reader.create_element("aperture") 746 730 if apert.name is not None: 747 self. write_attribute(ap, "name", str(apert.name))731 self.reader.write_attribute(ap, "name", str(apert.name)) 748 732 if apert.type is not None: 749 self. write_attribute(ap, "type", str(apert.type))750 self. append(ap, coll)751 752 size = self. create_element("size")733 self.reader.write_attribute(ap, "type", str(apert.type)) 734 self.reader.append(ap, coll) 735 736 size = self.reader.create_element("size") 753 737 if apert.size_name is not None: 754 self. write_attribute(size,738 self.reader.write_attribute(size, 755 739 "name", 756 740 str(apert.size_name)) … … 762 746 {"unit": apert.size_unit}) 763 747 if written == True: 764 self. append(size, ap)748 self.reader.append(size, ap) 765 749 766 750 self.write_node(ap, "distance", apert.distance, … … 774 758 775 759 for item in datainfo.detector: 776 det = self. create_element("SASdetector")760 det = self.reader.create_element("SASdetector") 777 761 written = self.write_node(det, "name", item.name) 778 762 written = written | self.write_node(det, "SDD", item.distance, 779 763 {"unit": item.distance_unit}) 780 764 if written == True: 781 self. append(det, instr)765 self.reader.append(det, instr) 782 766 783 off = self. create_element("offset")767 off = self.reader.create_element("offset") 784 768 written = self.write_node(off, "x", item.offset.x, 785 769 {"unit": item.offset_unit}) … … 789 773 {"unit": item.offset_unit}) 790 774 if written == True: 791 self. append(off, det)792 793 ori = self. create_element("orientation")775 self.reader.append(off, det) 776 777 ori = self.reader.create_element("orientation") 794 778 written = self.write_node(ori, "roll", item.orientation.x, 795 779 {"unit": item.orientation_unit}) … … 801 785 {"unit": item.orientation_unit}) 802 786 if written == True: 803 self. append(ori, det)787 self.reader.append(ori, det) 804 788 805 center = self. create_element("beam_center")789 center = self.reader.create_element("beam_center") 806 790 written = self.write_node(center, "x", item.beam_center.x, 807 791 {"unit": item.beam_center_unit}) … … 813 797 {"unit": item.beam_center_unit}) 814 798 if written == True: 815 self. append(center, det)816 817 pix = self. create_element("pixel_size")799 self.reader.append(center, det) 800 801 pix = self.reader.create_element("pixel_size") 818 802 written = self.write_node(pix, "x", item.pixel_size.x, 819 803 {"unit": item.pixel_size_unit}) … … 826 810 {"unit": item.slit_length_unit}) 827 811 if written == True: 828 self. append(pix, det)812 self.reader.append(pix, det) 829 813 830 814 # Processes info 831 815 for item in datainfo.process: 832 node = self. create_element("SASprocess")833 self. append(node, entry_node)816 node = self.reader.create_element("SASprocess") 817 self.reader.append(node, entry_node) 834 818 835 819 self.write_node(node, "name", item.name) … … 847 831 # Note info 848 832 if len(datainfo.notes) == 0: 849 node = self. create_element("SASnote")850 self. append(node, entry_node)833 node = self.reader.create_element("SASnote") 834 self.reader.append(node, entry_node) 851 835 else: 852 836 for item in datainfo.notes: 853 node = self. create_element("SASnote")854 self. write_text(node, item)855 self. append(node, entry_node)837 node = self.reader.create_element("SASnote") 838 self.reader.write_text(node, item) 839 self.reader.append(node, entry_node) 856 840 857 841 858 842 # Return the document, and the SASentry node associated with 859 # the data we just wrote 860 # If the calling function was not the cansas reader, return a minidom 861 # object rather than an lxml object. 862 863 frm = inspect.stack()[1] 864 mod_name = frm[1].replace("\\", "/").replace(".pyc", "") 865 mod_name = mod_name.replace(".py", "") 866 mod = mod_name.split("/readers/") 867 mod_name = mod[1] 868 if mod_name != "cansas_reader": 869 string = self.to_string(doc, pp=False) 870 doc = parseString(string) 871 node_name = entry_node.tag 872 node_list = doc.getElementsByTagName(node_name) 873 entry_node = node_list.item(0) 874 843 # the data we just wrote 875 844 return doc, entry_node 876 845 877 846 878 def write_node(self, parent, name, value, attr= None):847 def write_node(self, parent, name, value, attr={}): 879 848 """ 880 849 :param doc: document DOM … … 887 856 """ 888 857 if value is not None: 889 parent = self. ebuilder(parent, name, value, attr)858 parent = self.reader.ebuilder(parent, name, value, attr) 890 859 return True 891 860 return False … … 903 872 # Write the file 904 873 fd = open(filename, 'w') 905 if self. encoding == None:906 self. encoding = "UTF-8"907 doc.write(fd, encoding=self. encoding,874 if self.reader.encoding == None: 875 self.reader.encoding = "UTF-8" 876 doc.write(fd, encoding=self.reader.encoding, 908 877 pretty_print=True, xml_declaration=True) 909 878 fd.close() 910 879 911 912 # DO NOT REMOVE - used in saving and loading panel states.913 def _store_float(self, location, node, variable, storage, optional=True):914 """915 Get the content of a xpath location and store916 the result. Check that the units are compatible917 with the destination. The value is expected to918 be a float.919 920 The xpath location might or might not exist.921 If it does not exist, nothing is done922 923 :param location: xpath location to fetch924 :param node: node to read the data from925 :param variable: name of the data member to store it in [string]926 :param storage: data object that has the 'variable' data member927 :param optional: if True, no exception will be raised928 if unit conversion can't be done929 930 :raise ValueError: raised when the units are not recognized931 """932 entry = get_content(location, node)933 try:934 value = float(entry.text)935 except:936 value = None937 938 if value is not None:939 # If the entry has units, check to see that they are940 # compatible with what we currently have in the data object941 units = entry.get('unit')942 if units is not None:943 toks = variable.split('.')944 local_unit = None945 exec "local_unit = storage.%s_unit" % toks[0]946 if local_unit != None and units.lower() != local_unit.lower():947 if HAS_CONVERTER == True:948 try:949 conv = Converter(units)950 exec "storage.%s = %g" % (variable,951 conv(value, units=local_unit))952 except:953 exc_type, exc_value, exc_traceback = sys.exc_info()954 err_mess = "CanSAS reader: could not convert"955 err_mess += " %s unit [%s]; expecting [%s]\n %s" \956 % (variable, units, local_unit, exc_value)957 self.errors.append(err_mess)958 if optional:959 logging.info(err_mess)960 else:961 raise ValueError, err_mess962 else:963 err_mess = "CanSAS reader: unrecognized %s unit [%s];"\964 % (variable, units)965 err_mess += " expecting [%s]" % local_unit966 self.errors.append(err_mess)967 if optional:968 logging.info(err_mess)969 else:970 raise ValueError, err_mess971 else:972 exec "storage.%s = value" % variable973 else:974 exec "storage.%s = value" % variable975 976 977 # DO NOT REMOVE - used in saving and loading panel states.978 def _store_content(self, location, node, variable, storage):979 """980 Get the content of a xpath location and store981 the result. The value is treated as a string.982 983 The xpath location might or might not exist.984 If it does not exist, nothing is done985 986 :param location: xpath location to fetch987 :param node: node to read the data from988 :param variable: name of the data member to store it in [string]989 :param storage: data object that has the 'variable' data member990 991 :return: return a list of errors992 """993 entry = get_content(location, node)994 if entry is not None and entry.text is not None:995 exec "storage.%s = entry.text.strip()" % variable -
src/sans/dataloader/readers/xml_reader.py
rac5b69d r2e3b055 17 17 from lxml import etree 18 18 from lxml.builder import E 19 20 PARSER = etree.ETCompatXMLParser(remove_comments=True, remove_pis=False) 19 parser = etree.ETCompatXMLParser(remove_comments=True, remove_pis=False) 21 20 22 21 class XMLreader(): 23 """24 Generic XML read and write class. Mostly helper functions.25 Makes reading/writing XML a bit easier than calling lxml libraries directly.26 27 :Dependencies:28 This class requires lxml 2.3 or higher.29 """30 22 31 23 xml = None … … 35 27 schemadoc = None 36 28 encoding = None 37 processing _instructions = None29 processingInstructions = None 38 30 39 31 def __init__(self, xml = None, schema = None, root = None): 40 32 self.xml = xml 41 33 self.schema = schema 42 self.processing _instructions = {}34 self.processingInstructions = {} 43 35 if xml is not None: 44 self.set _xml_file(xml, root)36 self.setXMLFile(xml, root) 45 37 else: 46 38 self.xmldoc = None 47 39 self.xmlroot = None 48 40 if schema is not None: 49 self.set _schema(schema)41 self.setSchema(schema) 50 42 else: 51 43 self.schemadoc = None … … 55 47 Read in an XML file into memory and return an lxml dictionary 56 48 """ 57 if self.validate _xml():58 self.xmldoc = etree.parse(self.xml, parser = PARSER)49 if self.validateXML(): 50 self.xmldoc = etree.parse(self.xml, parser = parser) 59 51 else: 60 raise etree. XMLSchemaValidateError(self, self.find_invalid_xml())52 raise etree.ValidationError(self, self.findInvalidXML()) 61 53 return self.xmldoc 62 54 63 def set _xml_file(self, xml, root = None):55 def setXMLFile(self, xml, root = None): 64 56 """ 65 57 Set the XML file and parse … … 67 59 try: 68 60 self.xml = xml 69 self.xmldoc = etree.parse(self.xml, parser = PARSER)61 self.xmldoc = etree.parse(self.xml, parser = parser) 70 62 self.xmlroot = self.xmldoc.getroot() 71 63 except Exception: … … 74 66 self.xmlroot = None 75 67 76 def set _schema(self, schema):68 def setSchema(self, schema): 77 69 """ 78 70 Set the schema file and parse … … 80 72 try: 81 73 self.schema = schema 82 self.schemadoc = etree.parse(self.schema, parser = PARSER)74 self.schemadoc = etree.parse(self.schema, parser = parser) 83 75 except Exception: 84 76 self.schema = None 85 77 self.schemadoc = None 86 78 87 def validate _xml(self):79 def validateXML(self): 88 80 """ 89 81 Checks to see if the XML file meets the schema … … 91 83 valid = True 92 84 if self.schema is not None: 93 self.parse _schema_and_doc()94 schema _check = etree.XMLSchema(self.schemadoc)95 valid = schema _check.validate(self.xmldoc)85 self.parseSchemaAndDoc() 86 schemaCheck = etree.XMLSchema(self.schemadoc) 87 valid = schemaCheck.validate(self.xmldoc) 96 88 return valid 97 89 98 def find _invalid_xml(self):90 def findInvalidXML(self): 99 91 """ 100 92 Finds the first offending element that should not be present in XML file 101 93 """ 102 first _error = ""103 self.parse _schema_and_doc()94 firstError = "" 95 self.parseSchemaAndDoc() 104 96 schema = etree.XMLSchema(self.schemadoc) 105 97 try: 106 first _error = schema.assertValid(self.xmldoc)98 firstError = schema.assertValid(self.xmldoc) 107 99 except etree.DocumentInvalid as e: 108 first _error = str(e)109 return first _error110 111 def parse _schema_and_doc(self):100 firstError = str(e) 101 return firstError 102 103 def parseSchemaAndDoc(self): 112 104 """ 113 105 Creates a dictionary of the parsed schema and xml files. 114 106 """ 115 self.set _xml_file(self.xml)116 self.set _schema(self.schema)117 118 def to _string(self, elem, pp=False, encoding=None):107 self.setXMLFile(self.xml) 108 self.setSchema(self.schema) 109 110 def toString(self, elem, pp=False, encoding=None): 119 111 """ 120 112 Converts an etree element into a string … … 137 129 return dic 138 130 139 def set _processing_instructions(self):131 def setProcessingInstructions(self): 140 132 """ 141 133 Take out all processing instructions and create a dictionary from them … … 143 135 """ 144 136 dic = {} 145 p roc_instr= self.xmlroot.getprevious()146 while p roc_instris not None:147 pi_string = self.to _string(proc_instr)137 pi = self.xmlroot.getprevious() 138 while pi is not None: 139 pi_string = self.toString(pi) 148 140 if "?>\n<?" in pi_string: 149 141 pi_string = pi_string.split("?>\n<?") … … 153 145 for item in pi_string: 154 146 dic = self.break_processing_instructions(item, dic) 155 p roc_instr = proc_instr.getprevious()147 pi = pi.getprevious() 156 148 if 'xml' in dic: 157 self.set _encoding(dic['xml'])149 self.setEncoding(dic['xml']) 158 150 del dic['xml'] 159 self.processing _instructions = dic160 161 def set _encoding(self, attr_str):151 self.processingInstructions = dic 152 153 def setEncoding(self, attr_str): 162 154 """ 163 155 Find the encoding in the xml declaration and save it as a string … … 177 169 self.encoding = None 178 170 179 def _create_unique_key(self, dictionary, name, numb= 0):171 def _create_unique_key(self, dictionary, name, i = 0): 180 172 """ 181 173 Create a unique key value for any dictionary to prevent overwriting … … 184 176 :param dictionary: A dictionary with any number of entries 185 177 :param name: The index of the item to be added to dictionary 186 :param numb: The number to be appended to the name, starts at 0178 :param i: The number to be appended to the name, starts at 0 187 179 """ 188 180 if dictionary.get(name) is not None: 189 numb+= 1181 i += 1 190 182 name = name.split("_")[0] 191 name += "_{0}".format( numb)192 name = self._create_unique_key(dictionary, name, numb)183 name += "_{0}".format(i) 184 name = self._create_unique_key(dictionary, name, i) 193 185 return name 194 186 … … 201 193 return etree.ElementTree(root) 202 194 203 def create_element_from_string(self, xml_string):195 def create_element_from_string(self, s): 204 196 """ 205 197 Create an element from an XML string 206 198 207 :param xml_string: A string of xml208 """ 209 return etree.fromstring( xml_string)210 211 def create_element(self, name, attrib= None, nsmap=None):199 :param s: A string of xml 200 """ 201 return etree.fromstring(s) 202 203 def create_element(self, name, attrib={}, nsmap=None): 212 204 """ 213 205 Create an XML element for writing to file … … 215 207 :param name: The name of the element to be created 216 208 """ 217 if attrib == None:218 attrib = {}219 209 return etree.Element(name, attrib, nsmap) 220 210 … … 247 237 """ 248 238 pi_list = [] 249 if self.processing_instructions is not None: 250 for key in self.processing_instructions: 251 value = self.processing_instructions.get(key) 252 pi_item = etree.ProcessingInstruction(key, value) 253 pi_list.append(pi_item) 239 for key in self.processingInstructions: 240 value = self.processingInstructions.get(key) 241 pi = etree.ProcessingInstruction(key, value) 242 pi_list.append(pi) 254 243 return pi_list 255 244 … … 264 253 return tree 265 254 266 def ebuilder(self, parent, elementname, text=None, attrib=None): 267 """ 268 Use lxml E builder class with arbitrary inputs. 269 270 :param parnet: The parent element to append a child to 271 :param elementname: The name of the child in string form 272 :param text: The element text 273 :param attrib: A dictionary of attribute names to attribute values 274 """ 255 def ebuilder(self, parent, elementname, text=None, attrib={}): 275 256 text = str(text) 276 if attrib == None:277 attrib = {}278 257 elem = E(elementname, attrib, text) 279 258 parent = parent.append(elem) -
src/sans/guiframe/gui_manager.py
ra3b635b r98816c43 776 776 # Append item from plugin under menu file if necessary 777 777 self._populate_file_menu() 778 779 780 778 if not wx.VERSION_STRING >= '3.0.0.0': 781 779 self.SetMenuBar(self._menubar) … … 1351 1349 if config._do_aboutbox: 1352 1350 self._help_menu.AppendSeparator() 1353 self._help_menu.Append(wx.ID_ABOUT, '&About', 'Software information') 1354 wx.EVT_MENU(self, wx.ID_ABOUT, self._onAbout) 1351 id = wx.NewId() 1352 self._help_menu.Append(id, '&About', 'Software information') 1353 wx.EVT_MENU(self, id, self._onAbout) 1355 1354 1356 1355 # Checking for updates … … 1492 1491 self._file_menu.AppendSeparator() 1493 1492 1493 def _add_menu_file(self): 1494 """ 1495 add menu file 1496 """ 1497 1498 # File menu 1499 self._file_menu = wx.Menu() 1494 1500 style1 = self.__gui_style & GUIFRAME.MULTIPLE_APPLICATIONS 1495 1501 if OPEN_SAVE_MENU: … … 1520 1526 'Save state of the current active analysis panel') 1521 1527 wx.EVT_MENU(self, id, self._on_save_application) 1522 if not sys.platform =='darwin':1523 1528 self._file_menu.AppendSeparator() 1524 id = wx.NewId() 1525 self._file_menu.Append(id, '&Quit', 'Exit') 1526 wx.EVT_MENU(self, id, self.Close) 1527 1528 def _add_menu_file(self): 1529 """ 1530 add menu file 1531 """ 1532 # File menu 1533 self._file_menu = wx.Menu() 1529 1530 id = wx.NewId() 1531 self._file_menu.Append(id, '&Quit', 'Exit') 1532 wx.EVT_MENU(self, id, self.Close) 1534 1533 # Add sub menus 1535 1534 self._menubar.Append(self._file_menu, '&File') … … 1948 1947 if temp is not None: 1949 1948 doc = temp 1950 1949 1951 1950 # Write the XML document 1952 1951 extens = APPLICATION_STATE_EXTENSION … … 1962 1961 except: 1963 1962 msg = "Error occurred while saving: " 1964 msg += "To save, at lea st one application panel "1963 msg += "To save, at leat one application panel " 1965 1964 msg += "should have a data set.." 1966 1965 wx.PostEvent(self, StatusEvent(status=msg)) -
src/sans/perspectives/fitting/pagestate.py
rac5b69d reddb6ec 20 20 21 21 import xml.dom.minidom 22 from xml.dom.minidom import parseString23 22 from lxml import etree 24 23 … … 673 672 # We are appending to an existing document 674 673 newdoc = doc 675 try: 676 top_element = newdoc.createElement(FITTING_NODE_NAME) 677 except: 678 string = etree.tostring(doc, pretty_print=True) 679 newdoc = parseString(string) 680 top_element = newdoc.createElement(FITTING_NODE_NAME) 674 top_element = newdoc.createElement(FITTING_NODE_NAME) 681 675 if entry_node is None: 682 676 newdoc.documentElement.appendChild(top_element) 683 677 else: 684 try: 685 entry_node.appendChild(top_element) 686 except: 687 node_name = entry_node.tag 688 node_list = newdoc.getElementsByTagName(node_name) 689 entry_node = node_list.item(0) 690 entry_node.appendChild(top_element) 691 678 entry_node.appendChild(top_element) 679 692 680 attr = newdoc.createAttribute("version") 693 681 attr.nodeValue = '1.0' … … 762 750 for item in list_of_state_parameters: 763 751 element = newdoc.createElement(item[0]) 764 com = "self._toXML_helper( thelist=self.%s,"752 com = "self._toXML_helper(list=self.%s," 765 753 com += " element=element, newdoc=newdoc)" 766 754 exec com % item[1] … … 774 762 return None 775 763 else: 776 return newdoc 764 return newdoc.toprettyxml() 777 765 778 766 def _fromXML_helper(self, node, list): … … 1264 1252 state = PageState() 1265 1253 state.fromXML(node=nodes[0]) 1266 1267 1254 except: 1268 1255 logging.info("XML document does not contain fitting information.\n %s" % sys.exc_value) … … 1270 1257 return state 1271 1258 1272 def _parse_ save_state_entry(self, dom):1259 def _parse_entry(self, dom): 1273 1260 """ 1274 1261 Parse a SASentry … … 1281 1268 node = dom.xpath('ns:data_class', namespaces={'ns': CANSAS_NS}) 1282 1269 if not node or node[0].text.lstrip().rstrip() != "Data2D": 1283 return_value, _ = self._parse_entry(dom) 1284 numpy.trim_zeros(return_value.x) 1285 numpy.trim_zeros(return_value.y) 1286 numpy.trim_zeros(return_value.dy) 1287 size_dx = return_value.dx.size 1288 size_dxl = return_value.dxl.size 1289 size_dxw = return_value.dxw.size 1290 if size_dxl == 0 and size_dxw == 0: 1291 return_value.dxl = None 1292 return_value.dxw = None 1293 numpy.trim_zeros(return_value.dx) 1294 elif size_dx == 0: 1295 return_value.dx = None 1296 size_dx = size_dxl 1297 numpy.trim_zeros(return_value.dxl) 1298 numpy.trim_zeros(return_value.dxw) 1299 1300 return return_value, _ 1270 return CansasReader._parse_entry(self, dom) 1301 1271 1302 1272 #Parse 2D … … 1569 1539 for entry in entry_list: 1570 1540 try: 1571 sas_entry , _ = self._parse_save_state_entry(entry)1541 sas_entry = self._parse_entry(entry) 1572 1542 except: 1573 1543 raise … … 1638 1608 # Sanity check 1639 1609 if self.cansas == True: 1610 1640 1611 # Add fitting information to the XML document 1641 1612 doc = self.write_toXML(datainfo, fitstate) 1642 1613 # Write the XML document 1614 fd = open(filename, 'w') 1615 fd.write(doc.toprettyxml()) 1616 fd.close() 1643 1617 else: 1644 doc = fitstate.toXML(file=filename) 1645 1646 # Save the document no matter the type 1647 fd = open(filename, 'w') 1648 fd.write(doc.toprettyxml()) 1649 fd.close() 1618 fitstate.toXML(file=filename) 1650 1619 1651 1620 def write_toXML(self, datainfo=None, state=None): … … 1669 1638 state.data.run = [str(state.data.name)] 1670 1639 state.data.run_name[0] = state.data.name 1671 1640 1672 1641 if issubclass(state.data.__class__, 1673 1642 sans.dataloader.data_info.Data1D): … … 1679 1648 1680 1649 if state is not None: 1681 doc =state.toXML(doc=doc, file=data.filename, entry_node=sasentry)1682 1650 state.toXML(doc=doc, file=data.filename, entry_node=sasentry) 1651 1683 1652 return doc 1684 1653 -
src/sans/perspectives/invariant/invariant_panel.py
r92a2ecd r5777106 218 218 if new_doc is not None: 219 219 if doc is not None and hasattr(doc, "firstChild"): 220 child = new_doc.getElementsByTagName("SASentry") 221 for item in child: 222 doc.firstChild.appendChild(item) 220 child = new_doc.firstChild.firstChild 221 doc.firstChild.appendChild(child) 223 222 else: 224 223 doc = new_doc -
src/sans/perspectives/invariant/invariant_state.py
r92a2ecd r51f14603 254 254 : param entry_node: XML node within the XML document at which we will append the data [optional] 255 255 """ 256 # TODO: Get this to work257 256 from xml.dom.minidom import getDOMImplementation 258 257 import time … … 274 273 else: 275 274 entry_node.appendChild(top_element) 276 275 277 276 attr = newdoc.createAttribute("version") 278 277 attr.nodeValue = '1.0' … … 353 352 return None 354 353 else: 355 return newdoc 354 return newdoc.toprettyxml() 356 355 357 356 def fromXML(self, file=None, node=None): … … 743 742 for entry in entry_list: 744 743 745 sas_entry , _= self._parse_entry(entry)744 sas_entry = self._parse_entry(entry) 746 745 invstate = self._parse_state(entry) 747 746 … … 810 809 # Add the invariant information to the XML document 811 810 if state is not None: 812 doc =state.toXML(datainfo.name,doc=doc, entry_node=sasentry)811 state.toXML(datainfo.name,doc=doc, entry_node=sasentry) 813 812 return doc 814 813 -
src/sans/perspectives/pr/inversion_panel.py
r92a2ecd r5777106 255 255 if new_doc is not None: 256 256 if doc is not None and hasattr(doc, "firstChild"): 257 child = new_doc.getElementsByTagName("SASentry") 258 for item in child: 259 doc.firstChild.appendChild(item) 257 child = new_doc.firstChild.firstChild 258 doc.firstChild.appendChild(child) 260 259 else: 261 260 doc = new_doc -
src/sans/perspectives/pr/inversion_state.py
r92a2ecd r5777106 148 148 149 149 """ 150 #TODO: Get this to work151 150 from xml.dom.minidom import getDOMImplementation 152 151 … … 222 221 return None 223 222 else: 224 return newdoc 223 return newdoc.toprettyxml() 225 224 226 225 def fromXML(self, file=None, node=None): … … 471 470 472 471 for entry in entry_list: 473 sas_entry , _= self._parse_entry(entry)472 sas_entry = self._parse_entry(entry) 474 473 prstate = self._parse_prstate(entry) 475 474 #prstate could be None when .svs file is loaded … … 505 504 # Sanity check 506 505 if self.cansas == True: 507 doc = 506 doc =self.write_toXML(datainfo, prstate) 508 507 # Write the XML document 509 508 fd = open(filename, 'w') … … 528 527 # Create basic XML document 529 528 doc, sasentry = self._to_xml_doc(datainfo) 530 529 531 530 # Add the invariant information to the XML document 532 531 if state is not None: 533 doc =state.toXML(doc=doc, entry_node=sasentry)532 state.toXML(doc=doc, entry_node=sasentry) 534 533 535 534 return doc -
test/sansdataloader/test/utest_cansas.py
rac5b69d r2e3b055 5 5 warnings.simplefilter("ignore") 6 6 7 import unittest 8 import numpy 7 9 import sans.dataloader.readers.cansas_reader as cansas 8 10 from sans.dataloader.loader import Loader … … 10 12 from sans.dataloader.readers.xml_reader import XMLreader 11 13 from sans.dataloader.readers.cansas_reader import Reader 12 from sans.dataloader.readers.cansas_constants import CansasConstants14 from sans.dataloader.readers.cansas_constants import cansasConstants 13 15 14 16 import os … … 16 18 import urllib2 17 19 import StringIO 18 import pylint as pylint19 import unittest20 import numpy21 20 22 21 from lxml import etree 23 22 from xml.dom import minidom 24 23 25 CANSAS_FORMAT = CansasConstants.CANSAS_FORMAT26 CANSAS_NS = CansasConstants.CANSAS_NS24 CANSAS_FORMAT = cansasConstants.CANSAS_FORMAT 25 CANSAS_NS = cansasConstants.CANSAS_NS 27 26 28 27 class cansas_reader(unittest.TestCase): … … 73 72 def test_real_xml(self): 74 73 reader = XMLreader(self.xml_valid, self.schema_1_0) 75 valid = reader.validate _xml()74 valid = reader.validateXML() 76 75 if valid: 77 76 self.assertTrue(valid) … … 102 101 filename = "isis_1_1_write_test.xml" 103 102 xmlreader = XMLreader(self.isis_1_1, self.schema_1_1) 104 valid = xmlreader.validate _xml()105 xmlreader.set _processing_instructions()103 valid = xmlreader.validateXML() 104 xmlreader.setProcessingInstructions() 106 105 self.assertTrue(valid) 107 106 fo = open(self.isis_1_1) … … 127 126 def test_double_trans_spectra(self): 128 127 xmlreader = XMLreader(self.isis_1_1_doubletrans, self.schema_1_1) 129 self.assertTrue(xmlreader.validate _xml())128 self.assertTrue(xmlreader.validateXML()) 130 129 reader = Loader() 131 130 data = reader.load(self.isis_1_1_doubletrans) … … 147 146 valid = [] 148 147 reader1 = XMLreader(self.xml_valid, self.schema_1_0) 149 self.assertTrue(reader1.validate _xml())148 self.assertTrue(reader1.validateXML()) 150 149 reader2 = XMLreader(self.xml_invalid, self.schema_1_0) 151 self.assertFalse(reader2.validate _xml())150 self.assertFalse(reader2.validateXML()) 152 151 reader3 = XMLreader(self.xml_valid, self.schema_1_1) 153 self.assertFalse(reader3.validate _xml())152 self.assertFalse(reader3.validateXML()) 154 153 reader4 = XMLreader(self.xml_invalid, self.schema_1_1) 155 self.assertFalse(reader4.validate _xml())154 self.assertFalse(reader4.validateXML()) 156 155 reader5 = XMLreader(self.isis_1_0, self.schema_1_0) 157 self.assertTrue(reader5.validate _xml())156 self.assertTrue(reader5.validateXML()) 158 157 reader6 = XMLreader(self.isis_1_1, self.schema_1_1) 159 self.assertTrue(reader6.validate _xml())158 self.assertTrue(reader6.validateXML()) 160 159 reader7 = XMLreader(self.isis_1_1, self.schema_1_0) 161 self.assertFalse(reader7.validate _xml())160 self.assertFalse(reader7.validateXML()) 162 161 163 162 164 163 def test_old_cansas_files(self): 165 164 reader1 = XMLreader(self.cansas1d, self.schema_1_0) 166 self.assertTrue(reader1.validate _xml())165 self.assertTrue(reader1.validateXML()) 167 166 file_loader = Loader() 168 167 file1 = file_loader.load(self.cansas1d) 169 168 reader2 = XMLreader(self.cansas1d_units, self.schema_1_0) 170 self.assertTrue(reader2.validate _xml())169 self.assertTrue(reader2.validateXML()) 171 170 reader3 = XMLreader(self.cansas1d_badunits, self.schema_1_0) 172 self.assertTrue(reader3.validate _xml())171 self.assertTrue(reader3.validateXML()) 173 172 reader4 = XMLreader(self.cansas1d_slit, self.schema_1_0) 174 self.assertTrue(reader4.validate _xml())173 self.assertTrue(reader4.validateXML()) 175 174 176 175 … … 178 177 filename = "isis_1_0_write_test.xml" 179 178 xmlreader = XMLreader(self.isis_1_0, self.schema_1_0) 180 valid = xmlreader.validate _xml()179 valid = xmlreader.validateXML() 181 180 self.assertTrue(valid) 182 181 reader_generic = Loader() … … 192 191 written_data = return_data[0] 193 192 xmlwrite = XMLreader(filename, self.schema_1_0) 194 valid = xmlreader.validate _xml()193 valid = xmlreader.validateXML() 195 194 self.assertTrue(valid) 196 195 self._check_data(written_data) … … 199 198 def test_processing_instructions(self): 200 199 reader = XMLreader(self.isis_1_1, self.schema_1_1) 201 valid = reader.validate _xml()200 valid = reader.validateXML() 202 201 if valid: 203 202 ## find the processing instructions and make into a dictionary 204 dic = self.get _processing_instructions(reader)203 dic = self.getProcessingInstructions(reader) 205 204 self.assertTrue(dic == {'xml-stylesheet': 'type="text/xsl" href="cansas1d.xsl" '}) 206 205 … … 209 208 210 209 ## take the processing instructions and put them back in 211 xmldoc = self.set _processing_instructions(xmldoc, dic)210 xmldoc = self.setProcessingInstructions(xmldoc, dic) 212 211 xml_output = xmldoc.toprettyxml() 213 212 214 213 215 def set _processing_instructions(self, minidom_object, dic):216 xmlroot = minidom _object.firstChild214 def setProcessingInstructions(self, minidomObject, dic): 215 xmlroot = minidomObject.firstChild 217 216 for item in dic: 218 pi = minidom _object.createProcessingInstruction(item, dic[item])219 minidom _object.insertBefore(pi, xmlroot)220 return minidom _object221 222 223 def get _processing_instructions(self, xml_reader_object):217 pi = minidomObject.createProcessingInstruction(item, dic[item]) 218 minidomObject.insertBefore(pi, xmlroot) 219 return minidomObject 220 221 222 def getProcessingInstructions(self, XMLreaderObject): 224 223 dict = {} 225 pi = xml_reader_object.xmlroot.getprevious()224 pi = XMLreaderObject.xmlroot.getprevious() 226 225 i = 0 227 226 while pi is not None:
Note: See TracChangeset
for help on using the changeset viewer.