Changes between Version 54 and Version 55 of CodeCampIII

Timestamp:: Mar 24, 2015 9:59:51 AM (9 years ago)
Author:: smk78
Comment:: —

Legend:

: Unmodified
: Added
: Removed
: Modified

CodeCampIII

-                      v54
+                      v55
 == Introduction ==
 The third SasView Code Camp III was held from February 11th to 20th 2015, hosted by the [http://www.europeanspallationsource.se European Spallation Source] and held at the [http://europeanspallationsource.se/data-management-and-software ESS Data Management and Software Centre] in Copenhagen.
+The third [http://www.sasview.org/ SasView] Code Camp was held from February 11th to 20th 2015, hosted by the [http://www.europeanspallationsource.se European Spallation Source] and held at the [http://europeanspallationsource.se/data-management-and-software ESS Data Management and Software Centre] in Copenhagen.
 Local arrangements were made by Torben Nielsen and Thomas Larsen with details on the [wiki:CodeCampIIIPlanning planning page].
 == Initial goals set for the Code Camp ==
+* Release 3.1 with minimum requirements:
+* Release 3.1 preparation with the minimum requirements:
   * New Models interface
   * Wx 3.0 and new documentation in installer version
 …
   * BUMPS cleaned up and PARK removed
   * Major bugs fixed
   * some new models added?
+  * Some new models added?
   * IF POSSIBLE
     * ticket #23 and #262 addressed (acknowledgement and bug report in GUI)
 …
 == Photos ==
+Check back soon
 == Progress and Outcomes ==
 The code camp gathered 10 developers working full time for over a week on the !SasView code base.  Further, the team was joined in the latter half of the camp by 2 new developers (Wim Bouwman and Jurriaan Bakker from TUD) interested in adding support for SESANS data to the !SasView package.
+The Code Camp gathered 10 developers working full time for over a week on the !SasView code base.  Further, the team was joined in the latter half of the camp by 2 new developers (Wim Bouwman and Jurrian Bakker from TUD) interested in adding support for SESANS data to the !SasView package.
 The camp began with an introduction by Paul Butler outlining the goals for the week. Three major themes were identified as critical to the long term viability of !SasView as a collaborative, community project: good and well maintained documentation, both user and developer, low barrier to entry for new developers, and ease of including new models both for developers and users. A [wiki:CodeCampIIIScheduleAndWork schedule and series of work packages] were formulated to guide the work.
+The Camp began with an introduction by Paul Butler outlining the goals for the week. Three major themes were identified as critical to the long term viability of !SasView as a collaborative, community project: good and well maintained documentation, both user and developer, low barrier to entry for new developers, and ease of including new models both for developers and users. A [wiki:CodeCampIIIScheduleAndWork schedule and series of work packages] were formulated to guide the work.
 Besides addressing feature requests and bug fixes, the first task was to migrate the !SasView code repository from !SourceForge, which has been problematic of late, to Github. This was successfully accomplished. Migration included redirecting host names, editing trac and web pages, moving web pages hosting from UTK to github as well as updating all the developer machines.  In the process the final phase of renaming and reorganizing the folders in the !SasView tree was completed.  The official build servers and Jenkins site were moved to the DMSC servers and the development finally moved from using the old wx 2.8 to using wx 3.0.
+Besides addressing feature requests and bug fixes, the first task was to migrate the !SasView code repository from !SourceForge, which has been problematic of late, to Github. This was successfully accomplished. Migration included redirecting host names, editing trac and web pages, moving web pages hosting from UTK to github as well as updating all the developer machines. In the process the final phase of renaming and reorganizing the folders in the !SasView tree was completed.  The official build servers and Jenkins site were moved to the DMSC servers and the development finally moved from using the old Wx 2.8 to using Wx 3.0.
 The move to wx 3.0 enabled the work on documentation begun at Code Camp II, addressing the first theme above, to be completed with a porting of all the non-model documentation to RST files that can be built by sphinx. The use of sphinx allows the new documentation to be integrated into the GUI as well as be delivered to the web. The opportunity was take to simplify that part of the GUI code in the process. Most of the non-model documentation was also carefully edited and updated.  This restructuring means that both the user documentation, and the developer documentation, (provided in the docstrings included in the code), are automatically generated and can be pushed to the website with each build.   Further simplification is expected when the new model structure discussed below is completed. Thus the task of editing and updating user documentation is now more manageable and should allow for better and more timely upkeep as we move forward.  Meanwhile the ease of access to the developer documentation should encourage better code documentation and starts addressing the second theme of lowering the barrier to entry for new developers.
+The move to Wx 3.0 enabled the work on documentation begun at Code Camp II, addressing the first theme above, to be completed with a porting of all the non-model documentation to RST files that can be built by Sphinx. The use of Sphinx also allows the new documentation to be integrated into the GUI as well as be delivered to the web. The opportunity was take to simplify that part of the GUI code in the process. Most of the non-model documentation was also carefully edited and updated.  This restructuring means that both the user documentation, and the developer documentation, (provided in the docstrings included in the code), are automatically generated and can be pushed to the website with each build. Further simplification is expected when the new model structure discussed below is completed. Thus the task of editing and updating user documentation is now more manageable and should allow for better and more timely upkeep as we move forward. Meanwhile the ease of access to the developer documentation should encourage better code documentation and starts addressing the second theme of lowering the barrier to entry for new developers.
 With respect to that second theme, getting a developer environment properly installed has historically been a huge barrier, both for “amateur” developers lacking experience in installing the plethora of required packages and getting them running together, and for more professional programmers who may have a number of the wrong versions of required packages already installed.  To address both issues as well as others, such as ensuring that new required packages get propagated easily to developers and build machines, the Anaconda scientific python distribution was explored. This looks like a promising solution and significant progress was made in implementing it.
+With respect to that second theme, getting a developer environment properly installed has historically been a huge barrier, both for “amateur” developers lacking experience in installing the plethora of required packages and getting them running together, and for more professional programmers who may have a number of the wrong versions of required packages already installed. To address both issues, as well as others, such as ensuring that new required packages get propagated easily to developers and build machines, the Anaconda scientific python distribution was explored. This looks like a promising solution and significant progress was made in implementing it.
 In order to address the third theme, a very large effort to completely restructure the models interface, move models into an independent package and support GPU computing was begun and while not finished progressed far enough that a code branch integrating the new sasmodels package into !SasView was created. Approximately a quarter of the existing models were moved to the new package during the code camp. The next steps are to move the remaining models, integrate magnetic models, ensure full test coverage and generate a build with the new package for beta testing. This work was more ambitious than originally envisaged, but it provides major advantages and it was determined that it was better done as one integrated effort rather than piece by piece.  When complete, the learning curve for developers adding a new model should drop from many days to a few hours while implementing a new model should also drop from a day or more to a few hours at most.  User supplied models will have full access to the !SasView infrastructure (in particular the polydispersity) and will be able to be supplied in C as well as python. As an added bonus, !SasView will gain transparent access to multiprocessing and GPUs on many computers.
+In order to address the third theme, a very large effort to completely restructure the models interface, move models into an independent package and support GPU computing was begun and, while not finished, progressed far enough that a code branch integrating the new sasmodels package into !SasView was created. Approximately a quarter of the existing models were moved to the new package during the code camp. The next steps are to move the remaining models, integrate magnetic models, ensure full test coverage, and generate a build with the new package for beta testing. This work was much more ambitious than originally envisaged, but it provides major advantages and it was determined that it was better done as one integrated effort rather than piece by piece. When complete, the learning curve for developers adding a new model should drop from many days to a few hours while implementing a new model should also drop from a day or more to a few hours at most. User supplied models will have full access to the !SasView infrastructure (in particular the polydispersity algorithms) and will be able to be supplied in C as well as python. As an added bonus, !SasView will gain transparent access to multiprocessing and GPUs on many computers.
 The SESANS project, a completely new feature, championed by the Delft group, progressed remarkably rapidly partly aided by the new models interface.  The data is now recognized by and can be loaded into !SasView.  Further, the current sasmodels can be used (by running through a Henkel transform routine), in conjunction with BUMPS optimization engine, to fit the data. Thus SESANS data can now be analyzed with !SasView using the command line using any models which have been ported to the new structure.  Adding support for analytical SESANS models and Fully integrating with the GUI will be the next steps.
+The SESANS project, a completely new feature, championed by the Delft group, progressed remarkably rapidly, partly aided by the new models interface. Delft format data is now recognized by, and can be loaded into, !SasView. Further, the current sasmodels can be used (by running through a Hankel transform routine), in conjunction with the BUMPS optimization engine, to fit the data. Thus SESANS data can now be analyzed with !SasView using the command line using any models which have been ported to the new structure. Adding support for analytical SESANS models and fully integrating with the GUI will be the next steps.
+Finally, the group spent a half day discussing the long term needs, desires and dreams for the future of !SasView.  Questions discussed ranged from technologies to use and where they are currently headed,  to immediate requests from instrument scientists at facilities, to key new features, and to visions for pipelining, web applications and  submission of large calculations to clusters.  Priorities were also touched upon. The goal is to use that discussion to develop the first roadmap document for the project, an exercise currently underway.
+Finally, the group spent a half day discussing the long term needs, desires and dreams for the future of !SasView.  Questions discussed ranged from technologies to use and where they are currently headed, to immediate requests from instrument scientists at facilities, to key new features, and to visions for pipelining, web applications and  submission of large calculations to clusters.  Priorities were also touched upon. The goal is to use that discussion to develop the first roadmap document for the project, an exercise currently underway.
 == Next Steps ==
 At the end of the camp, the progress on all of the work packages was assessed and it was decided that the migration of models to the new, separate, models package should not hold up a 3.1 release which has other significant enhancements over 3.0. Thus the release plan is for a 3.1 release incorporating the new documentation, the move to wx3.0, bumps as the only optimiser and bug fixes. There will then be a 4.0 release later in the year making use of the new models package which will enable simpler adding of new models and GPU acceleration.
+At the end of the camp, the progress on all of the work packages was assessed and it was decided that the migration of models to the new, separate, models package should not hold up a 3.1 release which has other significant enhancements over 3.0. Thus the release plan is for a 3.1 release incorporating the new documentation, the move to wx3.0, BUMPS as the only optimiser and bug fixes. There will then be a 4.0 release later in the year making use of the new models package which will enable simpler adding of new models and GPU acceleration.
 The management team is formulating a roadmap document which will be circulated for comment amongst the developers before a public version is released.
+The management team is currently formulating a roadmap document which will be circulated for comment amongst the developers before a public version is released.
 The developers will continue to migrate models to the new sasmodels package and the necessary work to integrate it with the building of !SasView will be performed.
 …
 == Tickets Created and Addressed During the Code Camp ==
 [[TicketQuery(time=2015-02-11..2015-02-21&or&changetime=2015-02-11..2015-02-21&group=workpackage,desc=1,format=table,col=resolution|summary|owner|reporter)]]