SasView Code Camp V

Introduction

The fifth SasView Code Camp was held at the Spallation Neutron Source, Oak Ridge, Tennessee from 4th to 11th October, 2016

Local arrangements were made by Mathieu Doucet with details on the planning page.

Initial Goals

The management team met on Monday afternoon just prior to the Code Camp to work on a proposed set of goals for the Camp based on the published roadmap but informed by the many developments and activities that have taken place since the last official update. For example the ESS contribution through its SINE2020 funding to work on refactoring the GUI, an activity targeted for 2019 in the roadmap, makes moot some of the smaller GUI enhancements envisaged earlier. Further, the final release of SasView version 4.0, completing the work from the previous Code Camp in Delft, was released just prior to the code camp. The plan was presented to the developers and after a brief disscussion the primary goals for the code camp, which will be translated to the next update to the roadmap, were agreed:

  • TBA from list of tickets
  • Add a new corfunc perspective
  • Added file converter to support multifile data
  • Integrate SESANS into the SasView GUI
  • Fix Save Project when constrained fits are used
  • Continue work on GUI refactoring (and the clean separation of UI from computational code).
  • Work on increasing model coverage looking at non-overlap with SASfit models.
  • Begin work refactoring/clean-up of Batch fitting (to include batch operations on roi such as box sum and slices).
  • Finish model documentation review and formatting
  • Add missing documentation and documentation of new functionality
  • Work on improving infrastructure (build systems, 64 bit/Anaconda on all platforms of build machines, trac, licensing, etc)

These tasks build on the work done for release 4.0 and are aimed at a 4.1 release in late 2016/early 2017. The major items for this release are the integration of SESANS analysis, improvements to the Save Project functionality, inclusion of some models from SasFit, enhanced documentation and improved reliability.

Participants

Mathieu Doucet (ORNL) Andrew Jackson (ESS) Paul Butler (NIST) Steve King (STFC)
Ricardo Ferraz Leal (ORNL) Piotr Rozyczko (ESS) Jeff Krzywon (NIST) Richard Heenan (STFC)
Wim Bouwman (Delft) Wojciech Potrzebowski (ESS) Dirk Honecker (ILL) David Mannicke (ANSTO)
Jurrian Bakker (Delft)

Progress and Outcomes

This code camp brought 13 developers together to work for 7 days on the SasView and sasmodels code, documentation, and development infrastructure. One new developer, Dirk Honecker from ILL, joined the project.

As usual the camp began with introductions and updates from all the partner labs. NIST indicated that SasView usage has grown significantly among its users and staff, a trend that should only continue with the new features available in release of 4.0. This collaborative community approach to analysis software development is key to NIST’s ability to provide maximum data analysis support for the long term and the hope is to expand the number of staff able to contribute beyond the current 3 developers as time goes on. ANSTO have been focused on delivering data reduction for two new beam lines - Kookaburra and Bilby - but they want to be able to support their users who analyse their data using SasView. ISIS were happy to report that they had somehow managed to hire a summer undergrad to work on the code this summer who was extraordinarily productive providing many fixes and new features to the project. Several people asked about the possibility of hiring him again and it was suggested that was not out of the realm of possibility. From the ILL, Dirk Honecker introduced himself and said that his primary interest was in getting magnetic and oriented models working properly. SNS has users who use SasView and the primary interest from the facility is in incorporating sasmodels into their web-based data work flow, requiring some work on the sasmodels API. Delft has done some work on the SESANS project but not progressed as much as it would have liked on the code since last code camp. Their hope is that they can get an initial access to modelling SESANS data from within the SasView GUI at this code camp. Finally, ESS have continued the work laid out in the SINE2020 project - the first new specific funding for SasView since the UTK NSF grant ended mid 2012 - focusing on GUI redesign for SasView, SasFit integration work and performance enhancement of sasmodels.

While not slated for deployment in the next release, the work on GUI refactoring and SASfit model integration, tasks receiving the attention of the full time SasView developers at ESS, continued at this code camp in preparation for release 5.0. As part of that effort, some review discussions were undertaken and a few other developers began to help with those tasks both to speed up the development to meet our target release but more importantly to start learning the new code base for a smooth transition as the time approaches.

To focus the work for the code camp, the management team made proposed we target late 2016 for a v4.1 release of the features discussed and agreed. In order to achieve this, the development team was placed in groups focusing on:

  • New GUI
  • SESANS analysis integration
  • Sasmodels bug fixing and enhancement
  • SasView bug fixing and enhancement
  • Documentation & Training
  • Developer Infrastructure

By the end of the code camp, most of the main goals had been achieved. SESANS functionality was working in the GUI - both transforming SANS models automatically and calculating SESANS function directly. The save project functionality had been improved and repaired. The models from SASFit that do not currently overlap with sasmodels had been identified and work started on shipping them to the sasmodels marketplace for users to download. Refactoring of oriented cylinder-type models had begun to improve the behaviour when using orientation distributions. The documentation was broadly reviewed, then corrected and significantly expanded and enhanced to keep pace with the large number of new features and improved existing ones. As can be seen from the list of tickets that were closed or worked on during the code camp, there was a lot of work on stability improvements and bug fixing.

The work that had been done by Lewis O'Driscoll at ISIS over the summer was integrated into the main code base, adding a Correlation Function perspective, and a data converter for file formats that put Q and I in separate files. Perhaps the most exciting addition however is the sasmodels marketplace that Lewis wrote which is now live and has the potential of changing the way new models are developed and shared.

At the same time, several infrastructure enhancements needed to support the efficient development and deployment of SasView were made: some of the build servers were upgraded to deploy with scipy 0.18 (which proved to be more difficult than anticipated); a test build server for a 64bit windows build was set up and it was agreed that 4.1 will be the last 32bit Windows release; the Trac server was properly integrated with Github so that code browsing works and commit messages can add comments to and close Trac tickets. The developers all became comfortable with a branch-work-merge/pull request workflow, rather than committing all changes to master. This approach will hopefully allow us to make more frequent releases as we can choose to not include code in master that is not yet stable enough.

Overall, very good progress was made towards the 4.1 release goals.

Code Stats

Over the camp 85 tickets were closed.

Over the period of the camp, ten developers contributed to the !SasView code repository. There were 113 commits to all branches and master had 344 files changed with 79,668 additions and 39,499 deletions.

There were contributions from 10 developers to the sasmodels code repository. There were 52 commits to all branches and master had 1 pull request and a total of 46 commits. 57 files changed with 1,462 additions and 550 deletions.

Next Steps

All the developers will continue to work on the tickets that are assigned to them. The aim is for anything that is going to make it into 4.1 to be ready for merging with master by the end of October or mid-November at latest. It is interesting to not the ever increasing productivity and efficiency of the code camps, an indication of the growth and maturation of the development community.

Photos

All photos from the code camp on Flickr

https://c2.staticflickr.com/6/5751/30181537281_e0590738ec_n.jpg https://c5.staticflickr.com/9/8419/30232050876_82ae34ae08_n.jpg https://c8.staticflickr.com/9/8560/30267014415_7605539c24_n.jpg

Ticket Summary

Tickets Closed

Milestone: SansView 2.1.1 (1 match)

Ticket Resolution Summary Owner Reporter
#70 fixed Hayter Penfold model gives incorrect result butlerpd

Milestone: SasView 4.0.0 (10 matches)

Ticket Resolution Summary Owner Reporter
#623 fixed Data Info should display shape of 2D data lewis
#622 obsolete Data2D stores no information about the shape/dimensionality of the data lewis
#615 fixed loading invariant analysis doesn't butler
#585 fixed Need to develop "marketplace" for new models lewis butler
#447 fixed Saving plot as Raw RGBA format generates an image-less file smk78 smk78
#445 fixed Re-position of Add Text in a plot panel is not possible lewis butler
#368 fixed clean up ImageViewer interface butler butler
#293 fixed Add polymer micelle model butler
#259 fixed Unselect all data does not include sub-datasets lewis ajj
#245 fixed Allow user to set plot range lewis mathieu

Milestone: SasView 4.1.0 (65 matches)

Ticket Resolution Summary Owner Reporter
#755 duplicate S(q) models should not have scale and background parameters added ajj ajj
#752 duplicate "compute" button still not working richardh
#751 fixed load_models now ignores old sasview 3 custom models mathieu richardh
#749 fixed Also allow the Python Shell/Editor to be invoked from the Fitting > Edit Custom Models sub-menu smk78 richardh
#748 duplicate plug-in model unit tests from python shell not always working? pkienzle richardh
#744 fixed Old-style custom models don't load wojciech
#739 duplicate Load project issues krzywon richardh
#737 fixed Fix Report Results krzywon smk78
#736 fixed Review new File Converter documentation smk78 smk78
#735 fixed Review new Corfunc documentation smk78 smk78
#734 fixed Review new GPU and OpenCL documentation smk78 smk78
#733 wontfix Check orientational distribution dirk dirk
#732 fixed Let a custom model class name differ from its model name mathieu mathieu
#730 fixed Theories not deleted when before loading project file krzywon krzywon
#722 invalid Fitting Range and Checked Fit Parameters Not Saved krzywon
#720 fixed Incorporate instructions for running the SESANS GUI smk78 smk78
#718 fixed Magentism crashes SasView on Mac wojciech wojciech
#714 fixed cannot set custom smearing without data mathieu pkienzle
#712 fixed Clean up test data folder smk78 smk78
#710 fixed Update the test data documentation smk78 smk78
#709 fixed Update data_formats.rst when File Converter branch merged smk78 smk78
#706 fixed testing custom models - issues? pkienzle richardh
#705 fixed No about menu item on Mac ajj ajj
#704 fixed Add documentation on Hankel transformation for SESANS wimbouwman wimbouwman
#703 fixed Make the Python Shell Check Model error/info popup window bigger piotr smk78
#701 fixed Update documentation of data formats to Include SESANS wimbouwman wimbouwman
#700 fixed Add ANSTO logo to website footer smk78 smk78
#698 fixed Remove Select All parameters checkbox from FitPage mathieu smk78
#697 fixed Change default for parameter selection boxes on FitPage mathieu smk78
#696 fixed Investigate core shell bicelle model butler richardh
#694 fixed Compute button does nothing when there is data in the fit page ajj ajj
#693 fixed Update Acknowledgement window smk78 smk78
#692 fixed Update About window smk78 smk78
#690 fixed Update acknowledgement text in local_config.py when corfunc branch is merged smk78 smk78
#689 fixed Generate Zenodo DOI's for 4.0-beta and 4.0 release smk78
#687 fixed Rescale smeared to unsmeared model mathieu mathieu
#684 fixed Merge file converter branch ajj smk78
#683 fixed Merge corfunc branch ajj smk78
#682 fixed copy paste parameters should also copy constraints piotr butler
#678 wontfix Hard crash when running complex model on GPU pkienzle
#675 fixed Reparameterize hollow_cylinder richardh butler
#674 fixed custom model name and id must be the same mathieu pkienzle
#673 fixed custom models override builtin models of the same name mathieu pkienzle
#666 fixed OpenCL errors on exit wojciech pkienzle
#664 fixed custom models not appearing with structure factor button mathieu pkienzle
#662 fixed Allow editing of C models from SasView GUI tools piotr butler
#656 fixed Add link to model marketplace in SasView mathieu lewis
#652 fixed Logging is broken when "error while determining build number" is emitted wojciech smk78
#638 fixed Instrumental smearing not set automatically based on data mathieu ajj
#637 fixed Add timestamp to console log entries piotr ajj
#629 fixed Save Analysis option greyed out piotr lewis
#624 fixed custom models reloaded too often wojciech pkienzle
#616 fixed Problem with Generic Scattering Calculator buttons butler butler
#581 fixed min value on flat background should not be zero but -inf richardh richardh
#576 fixed updated custom models not being recompiled wojciech pkienzle
#553 fixed be_polyelectrolyte model docs richardh richardh
#535 fixed Change category management to load from models rather than default json file mathieu ajj
#512 fixed Add Menu Help to User Documentation smk78 butler
#500 fixed Support loading of data files from Anton Parr SAXSess krzywon ajj
#493 fixed Data Operations should allow for reasonable numerical fluctuations in q mathieu butler
#419 fixed Move GIT revision number to build script wojciech krzywon
#415 duplicate Save Project and Save Analysis don't open on double clicking krzywon krzywon
#288 fixed Problem with multishell model - glitches at high Q ajj ajj
#240 fixed Fix License.txt ajj butler
#10 fixed Save Analysis doesn't work for combined models jhjcho ajjackson

Milestone: Admin Tasks (2 matches)

Ticket Resolution Summary Owner Reporter
#711 fixed github integration not working on sasmodels ajj butler
#699 fixed Push new documentation to the website ajj smk78

Milestone: SasView Next Release +1 (5 matches)

Ticket Resolution Summary Owner Reporter
#605 obsolete swollen sphere parameterization pkienzle
#508 fixed Support loading of a wider variety of data files krzywon smk78
#503 wontfix Problems with Proxies OS X 10.9 ajj John White
#412 wontfix Do we use libxml2? smk78
#250 wontfix On Mac with WX 3.0 can kill fitting perspective and can't get it back ajj ajj

Milestone: SasView WishList (1 match)

Ticket Resolution Summary Owner Reporter
#183 fixed Create loader for 1D data in seperate single column files smk78

Milestone: sasmodels WishList (1 match)

Ticket Resolution Summary Owner Reporter
#723 fixed Spinodal Model dirk dirk

Last modified 6 months ago Last modified on Feb 24, 2017 11:05:46 PM