INTRODUCTION

This wiki page was originally written by our MOST experienced developers, but has subsequently been edited by our LEAST experienced developer who felt some instructions could have been clearer, and learnt one or two things that were missing altogether! But they succeeded in converting a model that passed testing, so there is no reason why should not be able to do the same.

In the new sasmodels package there are three ways of writing models:

• As a pure python model : Example - ​broadpeak.py
• As a python model with embedded C : Example - ​sphere.py
• As a python wrapper with separate C code : Example - ​cylinder.py, ​cylinder.c

The documentation for the sasmodels package can be found at ​http://www.sasview.org/sasmodels/

Before following the conversion instructions below, it is worth reading the documentation at ​http://www.sasview.org/sasmodels/api/generate.html#module-sasmodels.generate which describes the structure of a model.

CREATE NEW MODEL FILES.

If you have not already done so, download the sasmodels package from the repo at ​https://github.com/SasView/sasmodels.

In the sasmodels/models directory, copy the appropriate files (using the examples above as templates) to mymodel.py (and mymodel.c, etc) as required, where "mymodel" is the NEW name for the model you are converting/creating. Please follow these new naming rules:

• No capitalization and thus no CamelCase. If necessary use underscore to separate (i.e. barbell not BarBell or broad_peak not BroadPeak)
• Remove “model” from the name (i.e. barbell not BarBellModel)

EDIT NEW MODEL FILES

The model interface definition is in the .py file. This file contains:

• MODEL DOCUMENTATION
• doc string. The .py starts with an r (for raw) and three sets of quotes to start the doc string and ends with a second set of 3 quotes. This is where the FULL documentation for the model goes (to be picked up by Sphinx). Paste the model documentation from the appropriate section of model_functions.rst (found at ​src/sas/models/media/model_functions.rst) here. DO NOT use the old html documentation. For example, for the BarBellModel, search for BarBellModel in the rst file then copy everything after the *2.x.x. ModelNameModel**. Then edit the result as follows:
  • the *2.x.x.x Model** section with its brief description of the model becomes the model title; it feels odd that the documentation string does not start with name and brief description but it is automatically added by the model loader.
  • replace *2.x.x.x. Definition** with just Definition followed by 10 dashes on the line below.
  • wherever possible, try to replace images of math functions with Latex equivalents. You can use the live demo Mathjax page (​http://www.mathjax.org/) to make sure the equation looks as expected. Also a lot of the Latex code can be taken from (or edited) from the PDF document created by Paul Kienzle: ​http://sasview.org/attachment/wiki/SasModels%20Work%20Package/Equations.docx.pdf.
  • remove the table of parameters which will get autogenerated.
  • copy non-math figures from the sasview/src/sas/models/media/img subdirectory to the sasmodels/sasmodels/models/img subdirectory and rename them to something sensible. For example, image183.jpg became power_law_1d.jpg.
  • figure captions should be indented relative to the ..image:: tag. This allows us to, for example, change the font of the caption through CSS.
  • remove the figure number since this will be generated automatically.
  • make sure all references to model parameters are appropriately renamed, and that the relationship between equation variables and model parameters are obvious.
• name = "mymodel". This is the name of the model that is shown to the user. Use the name of the file as described above in 1. There will be less confusion if the model name matches the file name, but they could be different.
• title = "short description". This is a new string to be used for a tooltip, for example. It can probably be pulled directly from the model_functions.rst file.
• description = """doc string""". Cut and paste the self.description string from the (old) SasView model header in ​src/sas/models/include. This gives a brief description of the equation and the parameters without the need to read the entire model documentation. Make sure the parameter names match the current model definition.
• PARAMETERS
  • parameters = [["name", "units", default, [min,max], "type", "tooltip"],…]. This is where the parameters get defined. The syntax should be obvious from the default. Copy the parameters from the model header file ​src/sas/models/include.
  • the order of the parameters in the definition will be the order of the parameters in the user interface and the order of the parameters in Iq(), Iqxy() and form_volume().
  • VERY IMPORTANT: We are trying to make the model parameters more consistent between models. So solvent_sld, for example, should have exactly the same name in every model. The current list of new parameters is:
    • radius = radius
    • sld = scattering length density of particle
    • solvent_sld = scattering length density of matrix
    • cor_length = correlation length
    • exp = exponent (example: porod_exp)
    • peak_pos = q_peak or q0 etc
    • theta = axis_theta, phi = axis_phi
    • vol_frac = volume fraction of particle (example: lg_vol_frac)
    • add more as you generate them!
    • NOTE: There is no need to specify 'scale' or 'background', these are implicit to all models.
  • lower and upper limits can be any number, or -inf or inf.
    • add limits where required the (old) sasview models don't usually define them.
    • the limits will show up as the default limits for the fit making it easy, for example, to force the radius to always be greater than zero.
  • "type" can take 3 values at this time: “”, “volume”, or “orientation”.
    • "volume" parameters are passed to Iq(), Iqxy(), and form_volume(), and have polydispersity loops generated automatically.
    • "orientation" parameters are only passed to Iqxy().
  • "units" is the string that is displayed to the user in the user interface. This is not latex markup. If you want the manual to show fancy markup, do the following:
    • check RST_UNITS variable near the top of ​sasmodels/generate.py for units with special markup
    • if you don't see the units you need there, make a new entry in the RST_UNITS table
    • use the macros defined in ​doc/rst_prolog, or add your own if needed
    • if the markup is trivial, or if the it is really a one-off, then you can put the restructured text commands into RST_UNITS directly (I'm not sure if $latex$ works in this context)
    • if there is enough demand, we can set units to "display string|markup string" and have the model loader/documentation generator select the correct version for whomever is asking
• THE COMPUTATION
  • Pure Python
    • def Iq (this section does not exist for c models. It is replaced by the list of c files to call)
      • Iq.vectorized = True or False. This is used to tag the definition as accepting a Q vector, or having to compute each Q individually.
        • use functions from numpy rather than math
        • don't use if statements. Instead do tricks like "a = x*(q>0) + y*(q⇐0)" which sets a to x if q is positive or y if q is zero or negative
        • return np.NaN if the parameters are not valid (e.g., cap_radius < radius for barbell)
    • def Iqxy. If this model does not have an oriented form just set it equal to the square root of the sum of Iq(x) squared and Iq(y) squared.
    • ​src/sas/models contains the pure python models, mixed together with the python + C models.
  • Python + C
    • model Iq, Iqxy are defined in ​src/sas/models/c_extension/c_models
    • supporting scattering calculations are defined in ​src/sas/models/c_extension/libigor
    • special functions are defined in ​src/sas/models/c_extension/cephes. With model definitions you are limited to the simple math functions defined in ​opencl
    • common functions should be moved into their own file, such as lib/J1.c for the bessel function J1.
    • in place of the above methods point to the C source files needed. This will be a list of any lib files needed, followed by the model.c file, where model is the same name as given to the python model.py. For example:

          source = ["lib/J1.c", "lib/gauss76.c", "my_model.c"]
      

    • the c file must contain the form_volume, Iq, Iqxy methods. These and any other functions defined (e.g. the _cyl() helper function) must be defined as doubles in the first lines of the file.
    • return NAN if the parameters are not valid (e.g., cap_radius < radius for barbell)
  • NOTE: for certain models, such as those that can be multiplied by a structure factor, the ER attribute should be set to the Equivalent Radius (of a sphere).
  • NOTE: for certain models, namely core-shell type models, the VR attribute returns the volume ratio for the core-shell.
• demo is a dictionary containing the value of each parameter as given in the rst documentation. Make sure to enter the appropriate values from that documentation. This will then be used to generate the example curve in said documentation.
• include polydispersity parameters (e.g., radius_pd=0.2, radius_pd_n=9) so that you can compare the polydispersity calculations against those from sasview; use -mono on the compare.sh command line to see the direct calculation without polydispersity.
• TRANSITION CHECKS
  • oldname is the name for this model used in SasView. Make sure to put the correct original name. This is required for the transition to allow compatibility and to test that the models are equivalent.
  • oldpars is the name of the parameters as used in SasView. For each parameter give the name of the parameter as used in the original SasView code version of this model. Again this is required for the transition.
• UNIT TEST(S)
  • THESE ARE VERY IMPORTANT. Include at least one test for each model and PLEASE make sure that the answer value is correct (i.e. not a random number).
  • test is a list of lists. Each list is one test and contains, in order:
    • a dictionary of parameter values. This can be {} using the default parameters, or filled with some parameters that will be different from the default ({‘radius’:10.0, ‘sld’:4}). Unlisted parameters will be given the default values.
    • the input q value or tuple of qx, qy values.
    • the output Iq or Iqxy expected of the model for the parameters and q value given.
    • inputs and outputs can also be lists if you have several values to test for the same model parameters.
  • for ER and VR, give the inputs as 'ER' and 'VR' respectively.
  • to summarize:

        tests = [
            [ {parameters}, q, I(q)],
            [ {parameters}, [q], [I(q)] ],
            [ {parameters}, [q1, q2, ...], [I(q1), I(q2), ...]],
            [ {parameters}, (qx, qy), I(qx, Iqy)],
            [ {parameters}, [(qx1, qy1), (qx2, qy2), ...],
                            [I(qx1,qy1), I(qx2,qy2), ...]],
            [ {parameters}, 'ER', ER(pars) ],
            [ {parameters}, 'VR', VR(pars) ],
        ]
    

where

    parameters = {'k1': v1, 'k2': v2, ...}

TEST YOUR NEW/NEWLY-CONVERTED MODEL

Test your new model by running compare.py to verify that the converted model is giving the same results as it did in SasView prior to conversion. In order to do this, you need either a version of SasView installed in your python path, or a locally-built SasView in a checked-out directory at the same level as the checked-out sasmodels repository; i.e. you have the file structure:

  drive:\some_folder

contains

  \bumps
  \periodictable
  \sasview
  \sasmodels

If using a locally-built SasView, you need to add drive:\some_folder\sasview to your PYTHONPATH environment variable, or add drive:\some_folder to your User PATH environment variable and edit \sasmodels\compare.sh so that the global SASVIEW points to some_folder:

  SASVIEW=(../some_folder/build/lib.*)

Also remember, if using a locally-built SasView, that if you have done a pull from the repo you will need to re-build before continuing!

The first thing to test is that you are getting the same answer as SasView for the 1D version of the model. This is done with:

./compare.sh modelname -1d

This will result in some comparison metrics between the OpenCL implementation (if installed - it should revert to using ctypes if no OpenCL is installed) by way of a plot of the two calculations and a comparison plot; ie, three plots. If you do not see three plots, see the note on OpenCL issues at the bottom of the page.

If the model has 2D orientational calculation, then you should additionally test with:

./compare.sh modelname -2d

Brief help for the comparison script can be obtained by just running ./compare.sh

If you are on Windows, just cd into the /sasmodel folder and omit the './'

If you have the error

ValueError: Model does not contain parameter …

then a parameter is sasmodels is not a parameter in sasview. Scale and background are the usual culprits, though polydispersity parameters can also be a problem. You will need to update sasmodels.convert.revert_model and sasmodels.convert.constrain_new_to_old to handle the differences.

Run a number of random parameter sets through your model to make sure that your default values aren't special:

./multi_compare.sh modelname 200 -1d100 0 double > modelname-1d.csv

and for 2d calculations:

./multi_compare.sh modelname 20 -2d32 0 double > modelname-2d.csv

Particularly for sasview with polydispersity, the 2D multi_compare may be much too slow, and you will want to use the following instead:

./multi_compare.sh modelname 20 -2d32 mono double > modelname-2d.csv

The output is a comma separated value text which you can view with a spreadsheet program. Broken values (those which differ by more than 1e-14) are shown along with a random seed. You can reproduce these parameters in compare.sh using -random=SEED.

For the random models,

• sld will be in(-0.5,10.5),
• angles (theta, phi, psi) will be in (-180,180),
• angular dispersion will be in (0,45),
• polydispersity will be in (0,1)
• other values will be in (0, 2*v) where v is the value of the parameter in demo.

Dispersion parameters n, sigma and type will be unchanged from demo so that run times are predictable.

Now run the unit tests that you have added:

python -m sasmodels.model_test modelname

CHECK THE DOCS

To build the docs you will need:

• sphinx

  conda install sphinx
  

• gnu make (e.g., from ​http://gnuwin32.sourceforge.net/packages.html, or maybe there is an anaconda mingw package)

Make sure C:\Program Files (x86)\!GnuWin32\bin and C:\Python27 are on your path.

Build the docs by changing path to /sasmodels/doc and typing "make html".

Navigate to path/to/sasmodels/doc/_build/html in your file browser and open index.html. You will need to use firefox, chrome or safari since internet explorer doesn't seem to support mathjax from a local file.

FINALLY

Once compare and the unit test(s) pass properly and everything is done, commit your new model to the repo and then edit the models table at http://trac.sasview.org/wiki/ListofModels to indicate that the conversion is complete.

OPENCL ISSUES If the compare.sh modelname -1d test only generates one plot instead of three, or the compare.sh modelname -2d test complains that there is "No module named pyopencl", then you may not have OpenCL installed or set-up correctly. There are two parts to satisfying this:

1. Make sure you have an OpenCL driver installed that is compatible with your graphics card:
   • GeForce? (Nvidia): ​http://www.nvidia.com/Download/index.aspx?lang=en-us
   • Radeon/FirePro? (AMD): ​http://support.amd.com/en-us/kb-articles/Pages/OpenCL2-Driver.aspx
   • Iris/HD (Intel): ​https://software.intel.com/en-us/articles/opencl-drivers

2. Make sure you have pyopencl installed:
   • Windows binaries are available at ​http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyopencl
   • Note: the link above distributes packages as 'wheels' (.whl).
   • To install a wheel (this example being for a 32-bit Python 2.7 installation with no additional packages), cd to the download folder and then:

     python -m pip install pyopencl-2015.1-cp27-none-win32.whl