SasView Code Camp-III

(Last edited: 13-Jan-2015)

Introduction

The third SasView Code Camp was held from February 11th to 20th 2015, hosted by the European Spallation Source and held at the ESS Data Management and Software Centre in Copenhagen.

Local arrangements were made by Torben Nielsen and Thomas Larsen with details on the planning page.

Initial goals set for the Code Camp

  • Release 3.1 preparation with the minimum requirements:
    • New Models interface
    • Wx 3.0 and new documentation in installer version
    • Fix model documentation format to include implementation and testing
    • BUMPS cleaned up and PARK removed
    • Major bugs fixed
    • Some new models added?
    • IF POSSIBLE
      • ticket #23 and #262 addressed (acknowledgement and bug report in GUI)
      • Address tickets #239 and #240 (licensing)
  • Non Release based Goals:
    • Easy developer set up - Ticket #219
    • Create first draft of priority roadmap (from Monday Discussion) ticket #300
    • Report on code camp activities
    • IF POSSIBLE
      • Unit testing working properly - ticket #301
      • Paper started - ticket #299
      • Advances on other projects which could involve providing a design or a report on options etc.

Participants

Paul Butler (NIST) Steve King (ISIS) Andrew Jackson (ESS)
Jeff Krzywon (NIST) Peter Parker (ISIS) Torben Nielsen (ESS)
Paul Kienzle (NIST) Richard Heenan (ISIS) Jurrian Bakker (DUT)
David Mannicke (ANSTO) Miguel Gonzales (ILL) Wim Bouwman (DUT)

Photos

Check back soon

Progress and Outcomes

The Code Camp gathered 10 developers working full time for over a week on the SasView code base. Further, the team was joined in the latter half of the camp by 2 new developers (Wim Bouwman and Jurrian Bakker from TUD) interested in adding support for SESANS data to the SasView package.

The Camp began with an introduction by Paul Butler outlining the goals for the week. Three major themes were identified as critical to the long term viability of SasView as a collaborative, community project: good and well maintained documentation, both user and developer, low barrier to entry for new developers, and ease of including new models both for developers and users. A schedule and series of work packages were formulated to guide the work.

Besides addressing feature requests and bug fixes, the first task was to migrate the SasView code repository from SourceForge, which has been problematic of late, to Github. This was successfully accomplished. Migration included redirecting host names, editing trac and web pages, moving web pages hosting from UTK to github as well as updating all the developer machines. In the process the final phase of renaming and reorganizing the folders in the SasView tree was completed. The official build servers and Jenkins site were moved to the DMSC servers and the development finally moved from using the old Wx 2.8 to using Wx 3.0.

The move to Wx 3.0 enabled the work on documentation begun at Code Camp II, addressing the first theme above, to be completed with a porting of all the non-model documentation to RST files that can be built by Sphinx. The use of Sphinx also allows the new documentation to be integrated into the GUI as well as be delivered to the web. The opportunity was take to simplify that part of the GUI code in the process. Most of the non-model documentation was also carefully edited and updated. This restructuring means that both the user documentation, and the developer documentation, (provided in the docstrings included in the code), are automatically generated and can be pushed to the website with each build. Further simplification is expected when the new model structure discussed below is completed. Thus the task of editing and updating user documentation is now more manageable and should allow for better and more timely upkeep as we move forward. Meanwhile the ease of access to the developer documentation should encourage better code documentation and starts addressing the second theme of lowering the barrier to entry for new developers.

With respect to that second theme, getting a developer environment properly installed has historically been a huge barrier, both for “amateur” developers lacking experience in installing the plethora of required packages and getting them running together, and for more professional programmers who may have a number of the wrong versions of required packages already installed. To address both issues, as well as others, such as ensuring that new required packages get propagated easily to developers and build machines, the Anaconda scientific python distribution was explored. This looks like a promising solution and significant progress was made in implementing it.

In order to address the third theme, a very large effort to completely restructure the models interface, move models into an independent package and support GPU computing was begun and, while not finished, progressed far enough that a code branch integrating the new sasmodels package into SasView was created. Approximately a quarter of the existing models were moved to the new package during the code camp. The next steps are to move the remaining models, integrate magnetic models, ensure full test coverage, and generate a build with the new package for beta testing. This work was much more ambitious than originally envisaged, but it provides major advantages and it was determined that it was better done as one integrated effort rather than piece by piece. When complete, the learning curve for developers adding a new model should drop from many days to a few hours while implementing a new model should also drop from a day or more to a few hours at most. User supplied models will have full access to the SasView infrastructure (in particular the polydispersity algorithms) and will be able to be supplied in C as well as python. As an added bonus, SasView will gain transparent access to multiprocessing and GPUs on many computers.

The SESANS project, a completely new feature, championed by the Delft group, progressed remarkably rapidly, partly aided by the new models interface. Delft format data is now recognized by, and can be loaded into, SasView. Further, the current sasmodels can be used (by running through a Hankel transform routine), in conjunction with the BUMPS optimization engine, to fit the data. Thus SESANS data can now be analyzed with SasView using the command line using any models which have been ported to the new structure. Adding support for analytical SESANS models and fully integrating with the GUI will be the next steps. A report of SESANS activities at the camp is attached

Finally, the group spent a half day discussing the long term needs, desires and dreams for the future of SasView. Questions discussed ranged from technologies to use and where they are currently headed, to immediate requests from instrument scientists at facilities, to key new features, and to visions for pipelining, web applications and submission of large calculations to clusters. Priorities were also touched upon. The goal is to use that discussion to develop the first roadmap document for the project, an exercise currently underway.

Next Steps

At the end of the camp, the progress on all of the work packages was assessed and it was decided that the migration of models to the new, separate, models package should not hold up a 3.1 release which has other significant enhancements over 3.0. Thus the release plan is for a 3.1 release incorporating the new documentation, the move to wx3.0, BUMPS as the only optimiser and bug fixes. There will then be a 4.0 release later in the year making use of the new models package which will enable simpler adding of new models and GPU acceleration.

The management team is currently formulating a roadmap document which will be circulated for comment amongst the developers before a public version is released.

The developers will continue to migrate models to the new sasmodels package and the necessary work to integrate it with the building of SasView will be performed.

Tickets Created and Addressed During the Code Camp

Work Package: Reporting Enhancements (1 match)

Ticket Resolution Summary Owner Reporter
#331 fixed Remove all references to DANSE from user feedback messages smk78

Work Package: SESANS Integration (10 matches)

Ticket Resolution Summary Owner Reporter
#352 SESANS with finite acceptance angles pkienzle
#351 SESANS for anistropic scattering pkienzle
#350 resolution for SESANS model pkienzle
#346 SESANS into GUI Design ajj pkienzle
#345 fixed BUMPS fit of SESANS pkienzle
#344 fixed Hankel transform pkienzle
#343 duplicate SESANS data loader pkienzle
#342 analytical models for sesans pkienzle
#341 fixed add SESANS reader butler
#340 fixed add support for sesans to sasmodels pkienzle

Work Package: SasModels Redesign (11 matches)

Ticket Resolution Summary Owner Reporter
#361 fixed Check all models for order of parameters being passed richardh ajj
#359 obsolete Be able to use new models from sasmodels alongside sas.models models ajj ajj
#330 fixed allow inf/-inf for model limits pkienzle
#319 fixed sasmodels unit testing ajj
#317 fixed Packaging of sasmodels (build scripts etc) trnielsen ajj
#316 wontfix Autogenerate sasmodel "shims" ajj
#315 fixed Be able to set choice of device for OpenCL ajj
#314 fixed Default to GPU when using OpenCL ajj
#313 wontfix Python polydispersity ajj
#312 fixed Problem with slit smearing butler
#90 obsolete Provide a way for plugin models to use the built-in polydispersity functions stephen.king@…

Work Package: SasView Admin (12 matches)

Ticket Resolution Summary Owner Reporter
#333 fixed write and submit abstract for SAS 2015 mathieu butler
#328 Create SasView organanization and resolve ownership issues butler
#321 fixed Remove references to sourceforge ajj
#311 fixed Obtain DOI for Sasview releases ajj ajj
#310 fixed fix links on webpages butler
#309 fixed change trac access to trac.sasview.org butler
#308 fixed change CNAME for www.sasview.org to point to github page butler butler
#306 fixed Set up build jobs for bumps and periodictable in Jenkins pkienzle
#305 fixed migrate repository to github ajj ajj
#304 fixed Migrate mailing lists away from sourceforge ajj
#303 Integrate user and developer websites ajj ajj
#262 fixed Provide acknowledgement text from UI smk78

Work Package: SasView Bug Fixing (17 matches)

Ticket Resolution Summary Owner Reporter
#358 fixed Breaking Averaging Unit Tests krzywon krzywon
#357 fixed Broken SciPyFit Unit Tests krzywon krzywon
#354 fixed Fix menu bar on generic scattering calculator tool butler
#349 fixed Box Sum slicer parameter boxes are no longer editable on Windows smk78
#336 fixed Refactor report dialog code mathieu
#335 fixed Report print preview is broken mathieu mathieu
#334 "Paste" functionality broken mathieu
#332 fixed fix normalization for angular dispersion pkienzle pkienzle
#329 fixed fit panel not defaulting to min and max from model definition pkienzle
#325 fixed find and remove rogue print statement butler butler
#320 fixed Error boxes in Fit panel appear and disappear randomly gonzalezm gonzalezm
#276 fixed Provide better error message when CanSAS file has invalid units smk78
#269 fixed Failure to assign model when using attached data file krzywon ajj
#265 fixed Investigate why Image Viewer tool is not loading .TIF or .BMP smk78
#264 fixed Remove .TIF extension form Load Data file extension filter smk78
#263 fixed Error in report output for fixed parameters Richard Heenan
#234 fixed Add warning about using the dataset drop down on Batch page smk78

Work Package: SasView Documentation (11 matches)

Ticket Resolution Summary Owner Reporter
#360 obsolete Several model file docstrings are incorrectly formatted smk78
#356 fixed Proof read Sphinx help docs smk78 smk78
#355 fixed Finish porting old html help docs to rst smk smk78
#339 fixed get models Detail button pointing to proper sphinx documentation butler butler
#338 fixed Remove help_panel files and calls thereto butler butler
#337 fixed Fix broken help on load image butler butler
#327 fixed Need to update wiki pages butler butler
#326 fixed clean up Jae-Hie's old website butler butler
#323 fixed Fix access to online documentation butler
#318 fixed Integration of sasmodels documentation with SasView smk78 ajj
#297 fixed Non Model Documentation needs to be ported to ReST butler

Work Package: SasView Framework Enhancements (2 matches)

Ticket Resolution Summary Owner Reporter
#322 Need to streamline requiring new version of packages butler
#218 fixed Finish code reorg including removal of all mention of sans krzywon butler

Work Package: SasView GUI Enhancements (1 match)

Ticket Resolution Summary Owner Reporter
#348 fixed control order in which fit parameters appear in gui richardh richardh

Work Package: SasView GUI Redesign (1 match)

Ticket Resolution Summary Owner Reporter
#353 fixed Add help button to panels butler

Work Package: SasView QA and testing (4 matches)

Ticket Resolution Summary Owner Reporter
#347 fixed lamellarFFHG investigate polydisp integration richardh richardh
#324 fixed Set logging to capture warnings ajj
#307 fixed Edit Pylint preferences butler
#302 fixed Replace pisa with xhtml2pdf mathieu ajj

Last modified 7 weeks ago Last modified on Oct 29, 2017 1:30:03 PM

Attachments (2)

Download all attachments as: .zip