Load bunch of files and concatenating waves

Ann
Posts: 8
Joined: 2018-01-07
Location: United States
Screen Shot 2018-01-09 at 12.31.05 PM.png

Hello everybody,

I have multiple files like s_0NNN_r_……_1.dat where NNN indicates the series number. Each Series has almost 2500 files. After reading each _1.dat files in a series, I am having waves: ProtonEnergyFor_SpecAmp, ProtonTime_SpecAmp, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp and at last my target is to concatenate all the corresponding waves successively after reading each file to get the following 1D concatenated waves: ProtonEnergyFor_SpecAmpNew, ProtonTime_SpecAmpNew, ProtonEnergyFor_PreAmpNew, ProtonTime_PreAmpNew. My code is working well and giving me the correct number points in the concatenated waves ONLY for s_0NNN_r_000000_1.dat, s_0NNN_r_000001_1.dat, …files (I tested it separately), but when I am trying to run the code over all files, it is getting stuck. I am getting error messages like this: error: expected wave names. I am attaching the code and the screenshot of the errors. Can you have a look at these and get me out of the situation, please?

I tried to modify and utilize the code according to my use from this link:http://www.igorexchange.com/node/6409

Thank you very much.

#pragma TextEncoding = "MacRoman"
#pragma rtGlobals=3		// Use modern global access method and strict wave access.
 
 
//At first read a single file and then concatenate the data from each file
//Use the following command in the commmand window to read all the files
//LoadAndConcatAllFiles("")
 
//**************************
//READING A SINGLE FILE//
//*************************
 
Menu "Load Waves"
     "Load One File.....", LoadOneNeutronDatFile("","")
     "Load and Concatenate All Files in Folder.....", LoadAndConcatAllFiles("")
     "Make Histogram for Proton Energy and Proton Time...", MakeHist()
     "Make Two Dim Hisotgram....", MakeTwoDimHist()
End
 
static strConstant FileNameExtension=".dat"
 
//LoadOneNeutronDatFile(pathName, fileName)
//Creates the following wave: TimeStampForSpecAmp, TimeStampForPreAmp, ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp     
 
 
Function LoadOneNeutronDatFile(pathName, fileName)
         String pathName     //Name of an Igor symbolic path or ""
         String fileName     //Name of file or full path to file
 
         //First we have to get a valid reference to a file 
         if (strlen(pathName) == 0 ||strlen(fileName) == 0)
                   //Display dialog looking for file
                   variable refNum
                   Open/D=2/R/F=FileNameExtension/P=$pathName refNum as fileName 
                   filename=S_fileName                   //S_filename is set by Open/D 
                   If (strlen(fileName)==0)              //User Cancelled?
                       Print "LoadOneNeutronDatFile was Cancelled"
                                    return -1
                   endif
          endif
 
 
          //String fullpath = S_filename    
 
          Variable i=0, j=0, k
          Variable Vnam1, Vnam2, NumEvent, SpecMax, PreAmpMax, SpecTiMax, PreAmpTiMax, SpecAmpAvg, PreAmpAvg, Numerator, Denominator, ProtonEnergy
 
 
 
          //String fullpath = S_filename                          
          Open/P=$pathName/R refNum as fileName // fullpath
          Printf "Reading from file \"%s\". Number of Event is: ", fileName//fullpath            // full path is basically the s_filename      
 
 
          FStatus refNum              // Used to read the file size
          NumEvent = V_logEOF/8208    // Gives the total Number of Events where 8208 is the total byte of a single event
          Print  NumEvent
 
 
          //Making Waves
 
          Make/O/N=(NumEvent) TimeStampForSpecAmp
          Make/O/N=(NumEvent) TimeStampForPreAmp
 
          Make/O/N=(NumEvent) ProtonEnergyFor_SpecAmp
          Make/O/N=(Numevent) ProtonTime_SpecAmp
 
          Make/O/N=(NumEvent) ProtonEnergyFor_PreAmp
          Make/O/N=(NumEvent) ProtonTime_PreAmp
 
 
          for (k=0; k<NumEvent; k+=1)    // Nested loop for reading all events in a single file  
                 Numerator = 0; Denominator = 0
 
                 //*******START OF ONE EVENT*******            
                 SpecMax = -inf; PreAmpMax = -inf  //Assuming the maximum value of SpecMax and PreAmpMax at the beginning to run the loop    
 
 
                 //Reading 8 byte time stamp for SpecAmp start
                 Fbinread/B=3/F=5 refNum, Vnam1
                 TimeStampForSpecAmp[k] = Vnam1
 
                 //Reading 2 byte for Spec Amp
                 for (j=0; j<2048; j+=1)
                     Fbinread/B=3/F=2 refNum, Vnam2
                          if (Vnam2 > Specmax)
                                Specmax = Vnam2
                                SpecTiMax = j*10^-7
                          endif
                     Numerator = Numerator + Vnam2                    
                  endfor
                  //SpecAmpAvg = Numerator / Denominator
                  SpecAmpAvg = Numerator / 2048                  
                  ProtonEnergy = Specmax - SpecAmpAvg
                  ProtonEnergyFor_SpecAmp[k] = ProtonEnergy
                  ProtonTime_SpecAmp[k] = SpecTimax
 
 
 
                  //Reading 8 byte time stamp for PreAmp start
                  Fbinread/B=3/F=5 refNum, Vnam1
                  TimeStampForPreAmp[k] = Vnam1
 
                  //Reading 2 byte for PreAmp
                  for (j = 0; j < 2048; j+=1)
                      Fbinread/B=3/F=2 refNum, Vnam2
                            if (Vnam2 > PreAmpmax)
                                   PreAmpmax = Vnam2
                                   PreAmpTimax = j*10^-7
                            endif
                       Numerator = Numerator + Vnam2
                   endfor
                   PreAmpAvg = Numerator / 2048
                   ProtonEnergy = PreAmpmax - PreAmpAvg
                   ProtonenergyFor_PreAmp[k] = ProtonEnergy
                   ProtonTime_PreAmp[k] = PreAmpTiMax 
 
                   //*********End OF ONE EVENT**********  
           endfor 
    Close refNum
    return 0  //Signifies Success
End
 
 
//*************************************//
//READING ALL FILES IN SERIES 340//
//************************************//    
 
 Function  LoadAndConcatAllFiles(pathName) 
           String pathName           //Name of symbolic path or " " to get dialog
           String fileName
           Variable index=0 
 
 
           Wave/D/Z ProtonEnergyFor_SpecAmp, ProtonTime_SpecAmp, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
           if (!WaveExists(ProtonEnergyFor_SpecAmp))
                //Create the output waves because the code below concatenates
                Make/O/N=0/D ProtonEnergyFor_SpecAmp, ProtonTime_SpecAmp, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
           endif
 
 
           if (strlen(pathName) == 0)       //If no path specified, create one
                NewPath/O temporaryPath     //This will put up a dialog
                if (V_flag !=0)             //User cancelled
                           return -1
                endif
                pathName = "temporaryPath"
            endif
 
 
           Variable result
           do                         //Loop through each file in folder
                         filename = IndexedFile($pathName, index, FileNameExtension)
                         if (strlen(fileName) == 0)      //No more files?
                                   break                 //Break out of loop
                         endif
 
                         //Load the new data into a temporary data folder
                         String TempDFName = "TempDataForLoadAndConcat"
                         NewDataFolder/O/S $TempDFName
 
 
 
                        String dfName = ParseFilePath(0, fileName, ":", 1, 0)
                        //Print ParseFilePath(0, fileName, ":, 1, 0)
                        dfName = RemoveEnding(dfName, ".dat")
 
                        Variable LengthOfString = strlen(dfName)  // Determining the length of the ..
                        String FirstPart = dfName[3, 5]           // To read only 340 Series
                        String LastPart = dfName[LengthOfString - 1, LengthOfString - 1]
                        if (Stringmatch(FirstPart, "*340*") && stringmatch(LastPart, "1"))
                                 result = LoadOneNeutronDatFile(pathName, fileName)         
                        endif
 
                        if (result != 0)
                                 string message
                                 sprintf message, "Ann error occured while loading the file \"%s\". Aborting the load.\r", fileName
                                 Print message
                                 DoAlert 0, message
                                 KillDataFolder $TempDFName
                                 break
                        endif
 
 
                        // Create wave references for the waves loaded into the temporary data folder
                        Wave ProtonEnergyFor_SpecAmpNew = :ProtonEnergyFor_SpecAmp
                        Wave ProtonTime_SpecAmpNew = :ProtonTime_SpecAmp
                        Wave ProtonEnergyFor_PreAmpNew = :ProtonEnergyFor_PreAmp
                        Wave ProtonTime_PreAmpNew = :ProtonTime_PreAmp
 
 
                        SetDataFolder ::     // Back to parent data folder
 
                        Wave ProtonEnergyFor_SpecAmp, ProtonTime_SpecAmp, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
 
                        Concatenate /NP {ProtonEnergyFor_SpecAmpNew}, ProtonEnergyFor_SpecAmp
                        Concatenate /NP {ProtonTime_SpecAmpNew}, ProtonTime_SpecAmp
                        Concatenate /NP {ProtonEnergyFor_PreAmpNew}, ProtonEnergyFor_PreAmp
                        Concatenate /NP {ProtonTime_PreAmpNew}, ProtonTime_PreAmp
 
                        KillDataFolder $TempDFName
 
                        Printf "Loaded file %d: \"%s\"\r", index, filename
 
                        index += 1
 
           while (1)
                        if (Exists("temporaryPath"))           // Kill temp path if exists
                             killPath temporaryPath
                        endif
 
                        Wave ProtonEnergyFor_SpecAmp, ProtonTime_SpecAmp, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
                        Sort ProtonEnergyFor_SpecAmp, ProtonTime_SpecAmp, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
 
                        return 0
End
 
 
 
 
//***********************************************//                        
 //MAKING HISTOGRAM
//******************************************
 
 
//Function MakeHist()
 
      //HISTOGRAM FOR PROTON ENERGY FOR SPECAMP
      Make/N=100/O ProtonEnergyFor_SpecAmpNew_Hist; DelayUpdate
      Histogram/C/B={0,300,100} ProtonEnergyFor_SpecAmpNew,  ProtonEnergyFor_SpecAmpNew_Hist;DelayUpdate  //binstatr, binwidth, num of bins 
      //Histogram/B=1 NogueiraWave1,NogueiraWave1_Hist;DelayUpdate
      Display ProtonEnergyFor_SpecAmpNew_Hist
      Modifygraph mode=5
      ModifyGraph standoff=0
      Label bottom "proton energy (a.u.)"
      Label left "frequency of proton energ"
      TextBox/C/N=text0/A=MC "proton energy histogram"
 
 
      //HISTOGRAM FOR PROTON TIME
 
 
      Make/N=100/O ProtonTime_SpecAmpNew_Hist;DelayUpdate
      Histogram/C/B={0,1.2e-6,100} ProtonTimeNew_SpecAmp, ProtonTime_SpecAmpNew_Hist; DelayUpdate  
      //Histogram/B=1 NogueiraWave1,NogueiraWave1_Hist;DelayUpdate
      Display ProtonTime_SpecAmpNew_Hist
      Modifygraph mode=5
      ModifyGraph standoff=0	
      Label left "frequency of proton time"
      Label bottom "proton time (s)"
      TextBox/C/N=text0/A=MC "histogram for proton time"
 
End
 
 
//********************************
//MAKING GOINT HISTOGRAM
//********************************
 
 
//Function TWODIMHIST()
       Wave ProtonEnergyFor_SpecAmpNew, ProtonTime_SpecAmpNew  
       //make /o/n=(25,25) myHist
       //setscale x, 0,3000,NogueiraWave1 
       //setscale y,0,10e-6,NogueiraWave2
       JointHistogram/BINS={100,100 } ProtonEnergyFor_SpecAmpNew, ProtonTime_SpecAmpNew 
       //JointHistogram/BINS={100,100} NogueiraWave1, NogueiraWave2
 
       NewImage M_JointHistogram
 
end
 
 
 
 
 
 
 
 
 
 
 
 
 

AttachmentSize
NDABL3.ipf11.73 KB

[ last edited January 11, 2018 - 09:02 ]
sjr51
Posts: 254
Joined: 2012-12-16
Location: United Kingdom

Looking at your code and your description I think you need to change your concatenate commands:

                        Concatenate /NP {ProtonEnergyFor_SpecAmp}, ProtonEnergyFor_SpecAmpNew
                        Concatenate /NP {ProtonTime_SpecAmp}, ProtonTime_SpecAmpNew
                        Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpNew
                        Concatenate /NP {ProtonTime_PreAmp}, ProtonTime_PreAmpNew

The second part is the destwave.


Ann
Posts: 8
Joined: 2018-01-07
Location: United States

sjr51 wrote:
Looking at your code and your description I think you need to change your concatenate commands:
                        Concatenate /NP {ProtonEnergyFor_SpecAmp}, ProtonEnergyFor_SpecAmpNew
                        Concatenate /NP {ProtonTime_SpecAmp}, ProtonTime_SpecAmpNew
                        Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpNew
                        Concatenate /NP {ProtonTime_PreAmp}, ProtonTime_PreAmpNew

The second part is the destwave.

Thank you very much for your reply. I switched the source and destination wave position as you said:

              Concatenate /NP {ProtonEnergyFor_SpecAmp}, ProtonEnergyFor_SpecAmpNew
                        Concatenate /NP {ProtonTime_SpecAmp}, ProtonTime_SpecAmpNew
                        Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpNew
                        Concatenate /NP {ProtonTime_PreAmp}, ProtonTime_PreAmpNew

But the problem is not fixed. Now the code is running well but I am getting the concatenated waves without any values. I am attaching a screenshot of it. Could you have a look of this and let me know if you have any ideas? Thanks.

AttachmentSize
Screen Shot 2018-01-09 at 4.14.34 PM.png138.4 KB

jjweimer
jjweimer's picture
Posts: 1296
Joined: 2007-08-14
Location: United States

I believe the suggestion is to write this ...

Concatenate /NP {ProtonEnergyFor_SpecAmp},  theConcatenatedWave
Concatenate /NP {ProtonTime_SpecAmp},  theConcatenatedWave
Concatenate /NP {ProtonEnergyFor_PreAmp}, theConcatenatedWave
Concatenate /NP {ProtonTime_PreAmp},  theConcatenatedWave

... where theConcatenatedWave is the wave that should contain the concatenated results.

Alternatively, you need to specify a different designation like this ...

Concatenate /NP {ProtonEnergyFor_SpecAmp},  ProtonEnergyFor_SpecAmpConcatenated
Concatenate /NP {ProtonTime_SpecAmp},  ProtonTime_SpecAmpConcatenated
Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpConcatenated
Concatenate /NP {ProtonTime_PreAmp},  ProtonTime_PreAmpConcatenated

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH


[ last edited January 9, 2018 - 14:41 ]
Ann
Posts: 8
Joined: 2018-01-07
Location: United States

jjweimer wrote:
I believe the suggestion is to write this ...

Concatenate /NP {ProtonEnergyFor_SpecAmp},  theConcatenatedWave
Concatenate /NP {ProtonTime_SpecAmp},  theConcatenatedWave
Concatenate /NP {ProtonEnergyFor_PreAmp}, theConcatenatedWave
Concatenate /NP {ProtonTime_PreAmp},  theConcatenatedWave

... where theConcatenatedWave is the wave that should contain the concatenated results.

Alternatively, you need to specify a different designation like this ...

Concatenate /NP {ProtonEnergyFor_SpecAmp},  ProtonEnergyFor_SpecAmpConcatenated
Concatenate /NP {ProtonTime_SpecAmp},  ProtonTime_SpecAmpConcatenated
Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpConcatenated
Concatenate /NP {ProtonTime_PreAmp},  ProtonTime_PreAmpConcatenated

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

[quote]

Well, I used a different designation but the problem is not fixed. If I change like that, the code again doesn't run. I am working on it for hours to get the expected concatenated waves with correct length but each time having some issues. Anyways, thanks a lot.


sjr51
Posts: 254
Joined: 2012-12-16
Location: United Kingdom

I just looked again at the code and how it works. Forget my earlier answer. I see now how it is supposed to run (I think).

When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.

Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"

It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with LoadOneNeutronDataFile


Ann
Posts: 8
Joined: 2018-01-07
Location: United States

sjr51 wrote:
I just looked again at the code and how it works. Forget my earlier answer. I see now how it is supposed to run (I think).

When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.

Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"

It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with LoadOneNeutronDataFile


Hi,

Thank you very much for your explanation.

I got your point. I tried to modify the code in a different way. For your attention, my target is to read only _1.dat files of series 340. Since Each series has almost 2500 files, so I switched from "Concatenate [/DL /KILL /NP[=dim ] /O] [typeFlags ] {wave1, wave2, wave3,...}, destWave" command to "Concatenate [ /DL /KILL /NP[=dim ] /O] [typeFlags ] waveListStr, destWave" command, since according to Igor manual, the first one is limited to concatenate 100 waves or less, but there is no limit when using waveListStr. Now the code is looping over all files but I am facing another problem, it is not giving the correct length of the concatenated wave. For example, If I concatenate wave1 with N points to wave2 with M points, then I should have a concatenated wave with points (N+M). To check it, I took one file in a different folder (file "s_0340_r_000000_1.dat") and then ran the code. It gave me the correct number of points (274) which is equal to the number of events in that file. Then I add another file (file "s_0340_r_000001_1.dat") to that folder and ran the code again over these two files. Before that, I ran the code for the second file "s_0340_r_000001_1.dat" separately and got the correct number of data points and values in the table which is 326, the number of events in that file. So after running the code over these two files, I must have (274+326)=600 points in my concatenated wave. But instead of that, I am getting 652 points. I guess something wrong is happening when it is overwriting the existing waves, or something else, though not sure. I faced the same problem before also and tried to figure out the issue but couldn't. I hope now you will understand what is going on with the code. Any help will save my so many evening hours working on it.

Again thanks for your help.
#pragma TextEncoding = "MacRoman"
#pragma rtGlobals=3 // Use modern global access method and strict wave access.

//At first read a single file and then concatenate the data from each file
//Use the following command in the commmand window to read all the files
//LoadAndConcateNateAllFiles("")

//**************************
//READING A SINGLE FILE//
//*************************

Menu "Load Waves"
"Load One File.....", LoadOneNeutronDatFile("","")
"Load and Concatenate All Files in Folder.....", LoadAndConcateNateAllFiles("")
"Make Histogram for Proton Energy and Proton Time...", MakeHist()
"Make Two Dim Hisotgram....", MakeTwoDimHist()
End

static strConstant FileNameExtension=".dat"

//LoadOneNeutronDatFile(pathName, fileName)
//Creates the following wave: TimeStampForSpecAmp, TimeStampForPreAmp, ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp

Function LoadOneNeutronDatFile(pathName, fileName)
String pathName //Name of an Igor symbolic path or ""
String fileName //Name of file or full path to file

//First we have to get a valid reference to a file
if (strlen(pathName) == 0 ||strlen(fileName) == 0)
//Display dialog looking for file
variable refNum
Open/D=2/R/F=FileNameExtension/P=$pathName refNum as fileName
filename=S_fileName //S_filename is set by Open/D
If (strlen(fileName)==0) //User Cancelled?
Print "LoadOneNeutronDatFile was Cancelled"
return -1
endif
endif

//String fullpath = S_filename

Variable i=0, j=0, k
Variable Vnam1, Vnam2, NumEvent, SpecMax, PreAmpMax, SpecTiMax, PreAmpTiMax, SpecAmpAvg, PreAmpAvg, Numerator, Denominator, ProtonEnergy

//String fullpath = S_filename
Open/P=$pathName/R refNum as fileName // fullpath
Printf "Reading from file \"%s\". Number of Event is: ", fileName//fullpath // full path is basically the s_filename

FStatus refNum // Used to read the file size
NumEvent = V_logEOF/8208 // Gives the total Number of Events where 8208 is the total byte of a single event
Print NumEvent

//Making Waves

Make/O/N=(NumEvent) TimeStampForSpecAmp
Make/O/N=(NumEvent) TimeStampForPreAmp

Make/O/N=(NumEvent) ProtonEnergyFor_SpecAmp
Make/O/N=(Numevent) ProtonTime_SpecAmp

Make/O/N=(NumEvent) ProtonEnergyFor_PreAmp
Make/O/N=(NumEvent) ProtonTime_PreAmp

for (k=0; k Specmax)
Specmax = Vnam2
SpecTiMax = j*10^-7
endif
Numerator = Numerator + Vnam2
endfor
//SpecAmpAvg = Numerator / Denominator
SpecAmpAvg = Numerator / 2048
ProtonEnergy = Specmax - SpecAmpAvg
ProtonEnergyFor_SpecAmp[k] = ProtonEnergy
ProtonTime_SpecAmp[k] = SpecTimax

//Reading 8 byte time stamp for PreAmp start
Fbinread/B=3/F=5 refNum, Vnam1
TimeStampForPreAmp[k] = Vnam1

//Reading 2 byte for PreAmp
for (j = 0; j < 2048; j+=1)
Fbinread/B=3/F=2 refNum, Vnam2
if (Vnam2 > PreAmpmax)
PreAmpmax = Vnam2
PreAmpTimax = j*10^-7
endif
Numerator = Numerator + Vnam2
endfor
PreAmpAvg = Numerator / 2048
ProtonEnergy = PreAmpmax - PreAmpAvg
ProtonenergyFor_PreAmp[k] = ProtonEnergy
ProtonTime_PreAmp[k] = PreAmpTiMax

//*********End OF ONE EVENT**********
endfor
Close refNum
return 0 //Signifies Success
End

//*************************************//
//READING ALL FILES IN SERIES 340//
//************************************//

Function LoadAndConCateNateAllFiles(pathName)
String pathName // Name of symbolic path or " " to get dialog
String fileName
Variable index=0

if (strlen(pathName) == 0) // If no path specified, create one
NewPath/O temporaryPath // This will put up a dialog. Also NewPath sets the variable V_flag to zero if the operation succeeded or to nonzero if it failed
If (V_flag != 0)
return -1 // User Cancelled
endif
pathName = "temporaryPath"
endif

String ProtonEnerFor_SpecAmpWaveList = ""
String ProtonTimeFor_SpecAmpWaveList = ""

Variable result
do // Loop through each file in the folder
fileName = IndexedFile($pathName, index, FileNameExtension) // If index is greater than or equal to zero, IndexedFile returns a STRING containing the name of the index th file in the folder specified by pathName which matches the file type or extension specified by fileTypeOrExtStr
If (strlen(fileName) == 0) // No more files?
break //Break out of the loop
endif

// Create data folder from file Name
String dfName = ParseFilePath(0, fileName, ":", 1, 0) // Provides the ability to manipulate file paths & to extract sections of file paths and gives the file name eg s_0340_r_000000_1.dat
//Print dfName // Prints the file name
dfName = RemoveEnding(dfName, ".dat")
//print dfName // Prints the file name without extension i.e. s_0340_r_000000_1

Variable LengthOfFileNameString = strlen(dfName) // Determines the length of the file name string

String FirstPart = dfName[3,5] // For the purpose of reading series 340 only
//Print FirstPart = dfName[3,5] i.e. 340
String LastPart = dfName[LengthOfFileNameString - 1, LengthOfFileNameString - 1] // To split the file name for the purpose of reading only _1.dat files
//Print LastPart = dfName[LengthOfFileNameString - 1, LengthOfFileNameString - 1] i.e. 1

if (stringmatch(FirstPart, "*340*") && stringmatch(LastPart, "1"))
result = LoadOneNeutronDatFile(pathName, fileName)

if (result == 0)

// Create wave reference for the wave named "ProtonEnergyFor_SpecAmp" & "ProtonTime_SpecAmp
Wave/Z TargetWavProEngFor_SpecAmp = ProtonEnergyFor_SpecAmp
if (WaveExists(TargetWavProEngFor_SpecAmp)) // The WaveExists function returns one if wave reference is valid or zero if the wave reference is null
ProtonEnerFor_SpecAmpWaveList = AddListItem(GetWavesDataFolder(TargetWavProEngFor_SpecAmp, 2), ProtonEnerFor_SpecAmpWaveList) // AddListItem function Returns listStr after adding itemStr. GetWavesDataFolder function is used when we want to create a wave in the data folder containing a wave passed as a parameter
else
Print "Couldn't find a wave named \"ProtonEnergyFor_SpecAmp\" in the current data folder."
endif

Wave/Z TargetWavProTimFor_SpecAmp = ProtonTime_SpecAmp
if (WaveExists(TargetWavProTimFor_SpecAmp))
ProtonTimeFor_SpecAmpWaveList = AddListItem(GetWavesDataFolder(TargetWavProTimFor_SpecAmp, 2), ProtonTimeFor_SpecAmpWaveList)
else
Print "Couldn't find a wave named \"ProtonTime_SpecAmp\" in the current data folder"
endif

if (ItemsInList(ProtonEnerFor_SpecAmpWaveList) >= 0)
Concatenate/O/NP ProtonEnerFor_SpecAmpWaveList, ProtonEnergyFor_SpecAmpConcat // Concatenates source waves to the destinated waves
endif

if (ItemsInList(ProtonTimeFor_SpecAmpWaveList) >= 0)
Concatenate/O/NP ProtonTimeFor_SpecAmpWaveList, ProtonTime_SpecAmpConcat
endif

endif
endif

// if (DataFolderExists(dfName))
// String prompt
// Sprintf prompt, "Data folder name '%s' exists. Click Yes to overwrite the existing data, No to cancel.", dfName
// DoAlert 1, prompt
// if (V_flag == 2)
// Print "Load cancelled. You can rename the existing data folder and try again."
// return -1 // User Cancelled
// endif
// SetDataFolder $dfName
// else
// NewDataFolder/O/S $dfName
// endif

//SetDataFolder :: //Back to the parent data folder

//KilldataFolder $dfName
Printf "Loaded file %d: \"%s\"\r", index, fileName
index += 1

while (1)

if (Exists("temporaryPath"))
Killpath temporaryPath
endif

Wave ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
// Sort ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp, ProtonEnergyFor_SpecAmp

return 0 // Signifies success

End

//***************************************************************//
//MAKING HISTOGRAM FOR PROTON ENERGY AND PROTON TIME FOR SPECAMP//
//***************************************************************//

Function MakeHistogram()

WAVE ProtonEnergyFor_SpecAmpConCat, ProtonTime_SpecAmpConcat

// Histogram for proton energy for SpecAmp
Make/N=100/O ProtEnrgyFor_SpecAmpConCat_His; DelayUpdate
Histogram/C/B={0,300,100} ProtonEnergyFor_SpecAmpConCat, ProtEnrgyFor_SpecAmpConcat_His; DelayUpdate // binstatrt, binwidth, num of bins
Display ProtEnrgyFor_SpecAmpConcat_His
Modifygraph mode=5
ModifyGraph standoff=0
Label bottom "proton energy (a.u.)"
Label left "frequency of proton energ"
TextBox/C/N=text0/A=MC "proton energy histogram"

// Histogram for proton time for SpecAmp
Make/N=100/O ProtTime_SpecAmpConcat_His; DelayUpdate
Histogram/C/B={0,1.2e-6,100} ProtonTime_SpecAmpConcat, ProtTime_SpecAmpConcat_His; DelayUpdate
Display ProtTime_SpecAmpConcat_His
Modifygraph mode=5
ModifyGraph standoff=0
Label left "frequency of proton time"
Label bottom "proton time (s)"
TextBox/C/N=text0/A=MC "histogram for proton time"

End

//********************************
//MAKING TWO DIM JOINT HISTOGRAM
//********************************

Function TwoDimJointHist()
Wave ProtonEnergyFor_SpecAmpConcat, ProtonTime_SpecAmpConcat
JointHistogram/DEST=M_JointHistogram/BINS={100,100 } ProtonEnergyFor_SpecAmpConcat, ProtonTime_SpecAmpConcat
NewImage M_JointHistogram

End


sjr51
Posts: 254
Joined: 2012-12-16
Location: United Kingdom

I had another look at this. The original code as you have it works, I think the reason it fails is to do with reading from the file. So, I think it works because if you take the dat files ending 1_1 or 3_1 and make copies giving them other names e.g. 2_1, 4_1, and just have those in a directory. The code works. The concatenated waves are the right length and all is good.

So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.

I don't know enough about the files or the data structure to help further. The error is because of this part:

TimeStampForSpecAmp[k] = Vnam1

k is out of range. This is because:
NumEvent = V_logEOF/8208

which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.


Ann
Posts: 8
Joined: 2018-01-07
Location: United States

sjr51 wrote:
I had another look at this. The original code as you have it works, I think the reason it fails is to do with reading from the file. So, I think it works because if you take the dat files ending 1_1 or 3_1 and make copies giving them other names e.g. 2_1, 4_1, and just have those in a directory. The code works. The concatenated waves are the right length and all is good.

So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.

I don't know enough about the files or the data structure to help further. The error is because of this part:

TimeStampForSpecAmp[k] = Vnam1

k is out of range. This is because:
NumEvent = V_logEOF/8208

which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.

[quote=Ann]

Hi,

Thank you very much for your explanations and suggestions. I have figured out the problem with the code. Now it is running well and giving the concatenated wave with the correct length.
Again thanks for your time.


[ last edited January 11, 2018 - 09:03 ]

Back to top