Loading Folder behavior oddity if folder contains non data file

[butler]

When loading and entire folder with Load Folder or in fact using load file and selecting all the files indiscriminately, The first non data file raises an error which could be a different kind of SasView file (a .svs file for example - and the helpful error tells you how to load it) or something not recognized at all (giving a different error message). The two issues however are:

• The next non data file does not pop up an error.
• Further all the GOOD files following that first non data file are listed in the pop up box even though they, along with all the valid data files are in fact loaded. This is very confusing to the user as it suggests that something is wrong with those files as well.

[Lewis O'Driscoll <lewis.o'drisco]

In [cba0c7737464448bab917b2585f4518e4428ed9a]:

Show more useful message when loading folder fails (closes #614)

[butler]

The fix now does allow all the non-loading data to be reported with an appropriate error message, not just the first one. However, build 436 on my windows machine now not only lists those files but also lists all the files that in fact do get loaded with the standard error message that they didn't get loaded attached to each. The pop up should lists all the non-loading data with appropriate messages but not list any of the files that do get loaded.

[Lewis O'Driscoll <lewis.o'drisco]

In [70308f6c0db5fd4e37863a2fff66dd448c954d56]:

Fix error message shown when loading folder fails (fixes #614)

[butler]

There appears to be some rather convoluted logic going on. The latest fix does seem to stop the worst problem of having a popup indicating things had not loaded when in fact they did load.

However the behavior seems a bit erratic. Using some folders the pop up now works as intended, only listing files that failed to load and stating so an why. On the other hand using the test folder with some extra non data files thrown ins does not produce the pop up at all! It DOES however correctly identify in the console screen which files loaded and which failed to load and why. But even within the console screen there are some random behaviors where sometimes the errors are flagged in red preceded by a red x icon while in other cases the errors show up in normal black text preceded by the normal blue i (information) icon…. or sometimes no icon at all. Further ALL files show as loading — it seems that the error gets generated in a different stream and shows up sometime later in the list of information on the console - so loading x then at some random point later in the message stream … failed to load x.

Given that at least now the messages are correct and the fact that this may require some serious work, am moving it to the next release

[Lewis O'Driscoll <lewis.o'drisco]

In [a674d0b7da37ed36693004d2d51e0941ed38842f]:

Re-write error handling code in data_loader (fixes #614)

[smk78]

Also, if the folder contains any non-I(Q) data ASCII files that just happen to have space or tab delimited numbers in the opening lines then these get loaded as data (the first pair of numbers as X & Y). So examples of this include BSL/OTOKO 'header' files, eg: this CORFUNC file had the extension .CF1

     201       1       1       1       0       0       0       0       0       0
C:\Temp\X2

where it took the 201 & 1 as X & Y, or perhaps a temporary parameter file such as this one which had the extension .txt:

0.200000E+03
0.100000E+01
0.522096E+00  0.211909E-01  0.396440E-02  0.101775E-02  0.307366E-03

where it took the 0.522096E+00 0.211909E-01 as X & Y.

I suspect the ASCII reader is just a little too embracing. Maybe if it checked for two consecutive lines with pairs of numbers, or something like that? But tricky.

[butler]

This should actually not happen — ASCII reader has ALWAYS looked for X (where I think X is 6 now) rows with EXACTLY the same number of columns of numbers only. Not perfect but as close as one can get. Either there are more lines or a new bug has been introduced I think.

Can we have an example offending data file attached to this ticket?

[smk78]

Just re-checked in ESS W7 Build 466, this ASCII issue still exists. Am appending three examples files: X27000.CF1, extract.txt & New Text Document.txt

[smk78]

OTOKO format header file (ASCII)

[smk78]

Corfunc parameter file (ASCII)

[smk78]

Made up bogus data file (ASCII)

[smk78]

Uh-oh… Issue also exists in SasView? 3.1.2!

[smk78]

And SasView 2.2.1… so it's a very long-standing bug!

[butler]

This is essentially fixed. The new behavior identified by Steve King is completely separate (a failure of the ASCII reader not the folder reader). I have taken the relevant information and attachments and created a new ticket for that with an appropriate title.