Tue, 08/20/2013 - 03:38 pm
I am new to IGOR and programming in general. I've gone through the tutorials, but this seems more advanced. I have multiple CSV files (comma delimited) that I am loading into an experiment. I'm using an example from another post (http://www.igorexchange.com/node/1406) and that works great, but my files are "special". I have a text file that has the wavenames listed I want for the wave labels. The data files are delimited, but the date and time are spit out as "13-aug-13","12:50", (the quotes are included in the output). So when I use the code in the ref above, it loads my files, but tries to use the first data points as labels. So wave0 is labeled as X13_aug_13__00_00__ but date and time are supposed to be two separate waves. So how would I program in the text file with the wavenames in order to set my S_waveNames = the text? Example: The first row of data for the waves is "13-aug-13","12:50",4.012,0.0226,1.314 and I want the waves labeled: Date Time Flow V-ref V-meas My text file has these names in list form, i.e:
Date
Time
Flow
V-ref
Vmeas
So to review:
How do I load the date as a date in the format it is given to me (and time)?
How would I incorporate the names listed in a text file as wave names in the loading portion of the program?
Where can I go to learn more advanced programming such as I need here (I'm assuming it comes with practice and examples such as seen here, but maybe there's an online class for more advanced programming?)?
I appreciate any help you all have!
Thank you
August 20, 2013 at 05:25 pm - Permalink
If this does not help, post a sample file or a representative excerpt along with the Igor command you are trying to use. Then I should be able to advise the best approach.
August 20, 2013 at 05:32 pm - Permalink
No. The csv files the instrument gives me start with
"13-aug-13","12:50",4.012,0.0226,1.314
and then repeats for every 5 minutes so next line is
"13-aug-13","12:55",4.012,0.0225,1.297
and so on for a few days worth of data.
I made a simple text file that lists the titles I would like to insert as the wave titles instead of wave0 wave1 ...
I will try the Loadwave command tonight (don't have my igor computer with me at the moment).
The following data is copied and pasted directly from the first 6 lines of the csv file. No titles or labels are included. I was turning it off and on a bunch so the time skips a few points in the data.
"13-aug-13","12:50", 4.0, 0.0221, 2.2011
"13-aug-13","12:55", 4.0, 0.0221, 2.1932
"13-aug-13","13:00", 4.0, 0.0221, 2.1854
"13-aug-13","13:20", 4.0, 0.0224, 2.1958
"13-aug-13","13:40", 4.0, 0.0225, 2.1961
"13-aug-13","13:45", 4.0, 0.0225, 2.1851
"13-aug-13","13:50", 4.0, 0.0225, 2.1744
"13-aug-13","13:55", 4.0, 0.0225, 2.1646
"13-aug-13","14:00", 4.0, 0.0225, 2.1552
Ideally, I'd like to combine date and time into one wave and then eventually graph time vs voltage over several weeks. But I think I can figure that out. Thanks for the reply
August 21, 2013 at 03:18 pm - Permalink
August 21, 2013 at 03:39 pm - Permalink
Both the date and the time are stored in seconds so you can just add the time wave to the date wave and then kill the time wave.
This probably has to do with your use of either the /W flag or the /L flag. Keep in mind that line numbers are zero-based (the first line is line 0).
Here is how I would load this data:
"Load ChemGoof File...", LoadChemGoofFile("", "") // Add menu item to Data->Load Waves submenu
End
// LoadChemGoofFile(pathName, fileName)
// Returns a semicolon-separated list of waves loaded or "" if cancel.
Function LoadChemGoofFile(pathName, fileName)
String pathName // Name of an Igor symbolic path or "".
String fileName // Name of file or full path to file.
// First get a valid reference to a file.
if ((strlen(pathName)==0) || (strlen(fileName)==0))
// Display dialog looking for file.
Variable refNum
Open/D/R/P=$pathName refNum as fileName
fileName = S_fileName // S_fileName is set by Open/D
if (strlen(fileName) == 0) // User cancelled?
return -1
endif
endif
// Now load the data. The /V flag specifies the accepted delimiters in the data file.
// Add the /A flag if you don't want the "Loading Delimited Text" dialog.
// Add the /O flag if you want to overwrite existing waves with the same names.
// "Date" and "Time" are not available as wave names because they are Igor function names
String columnInfoStr = "N=vDateTime;N=vTime;N=Flow;N=VRef;N=VMeas;"
String delimiters = "," // Comma-delimited
String skipChars = "\"" // Skip quotes around date and time
LoadWave /J /D /A /E=1 /K=0 /W /V={delimiters,skipChars,0,0} /B=columnInfoStr /R={English,1,3,1,2,"Year-Month-DayOfMonth",40} /P=$pathName fileName
Variable numWavesLoaded = V_flag // V_flag is set by LoadWave
if (numWavesLoaded != 5)
Print "Error loading file - wrong number of waves loaded"
return -1
endif
Wave vDateTime = vDateTime // Create reference to wave created by LoadWave
Wave vTime = vTime // Create reference to wave created by LoadWave
vDateTime += vTime // Add time to date to obtain date/time
KillWaves/Z vTime // This is no longer needed
// Tweak date/time table formatting
ModifyTable format(vDateTime)=8, width(vDateTime)=150
return 0 // Success
End
August 21, 2013 at 04:28 pm - Permalink
August 21, 2013 at 05:24 pm - Permalink
Variable refNum //Define refNum as number
String message = "Select one or more files" //Defines text names/labels to use
String outputPaths //Defines "outputPaths" as text string
String fileFilters = "Data Files (*.txt,*.dat,*.csv):.txt,.dat,.csv;" //Dictates which file extensions can be chosen
fileFilters += "All Files:.*;"
Open/D/R/MULT=1/F=fileFilters/M=message refNum //Show open window to choose file(s) from
outputPaths = S_fileName //Sets output path as string variable in carriage-return-delimited list of the ful path
if (strlen(outputPaths) == 0) //Says print cancelled if you click cancel
Print "Cancelled"
else
Variable numFilesSelected = ItemsInList(outputPaths, "\r") //Defines number of files chosen to open
Variable i //Sets i as a number
for(i=0; i<numFilesSelected; i+=1) //Sets limits on i so if i=0, i is less than # files or i is bigger than 1, then do the following
String columnInfoStr = "N=timewave;N=vTime;N=Flow;N=VRef;N=VMeas;"
//Above sets names of wave labels being loaded
String delimiters = "," //defines text name "delimiters" as comma
String skipChars = "\"" //defines "skipChars" as skipping the quote character
String path = StringFromList(i, outputPaths, "\r") //sets "path" as a string from the list of files loaded
Printf "%d: %s\r", i, path //Prints the Path and filename beingd loaded
LoadWave/A/D/E=1/J/K=0/R={English, 1, 3, 2, 1, "DayOfMonth-Month-Year", 40}/B=columnInfoStr/V={delimiters,skipChars,0,0} path //Loads the data from the delimited files
Wave timewave = timewave //Defines timewave as a wave
Wave vTime = vTime // " vTime " "
ModifyTable format(timewave)=8 //Format column to combined Date and Time
timewave += vTime //Sets timewave equal to itself plus the vTime data
RemovefromTable vTime //Take the wave off the table
Killwaves/Z vTime //Erase the full vTime wave
endfor
endif
return outputPaths // Will be empty if user cancelled; Stops procedure
End
Now the only problem is I want the files to load concurrently on the existing waves. So when I open the 8/13 file, it works great, but the 8/14 file I open next has different wave labels and I want them to just open on the same table from the 8/13. Is that possible? Also, any critique of the program is welcomed. Thank you
August 26, 2013 at 12:10 pm - Permalink
Cheers
August 26, 2013 at 10:29 am - Permalink
So K=0 causes the waves to be detected as text and K=1 kills all the data. /E=2 with /K=0 gives me exactly what I needed in terms of format of all the numbers going into one table, but that doesn't help me when I need to do a concurrent calculation using those waves and they come out as text... I'M SO CLOSE! THIS IS SO FRUSTRATING!
Thanks
August 26, 2013 at 10:44 am - Permalink
So that didn't work how I thought. /E=2 causes some issues too. It adds the data from the next day's file into the timewave wave, but it also recalculates the entire wave so that didn't work. So changed the /E=1. It now names the next file's data waves as timewave1, vTime1 etc... and puts them in their own columns.
Also, I think the text vs numeric problem in the /V I think. When I put the \" in the delimsStr of /V, it adds extra columns I have to erase, but keeps the numbers as numbers. When I skipCharsstr the \", all my numbers are text.
August 26, 2013 at 12:09 pm - Permalink
This is because of a mistake I made in constructing the /V flag to skip the quotation marks. I told LoadWave to skip the quotes but not to skip spaces. Your file has spaces before the comma delimiters so LoadWave decided the column contained text rather than numbers. I have corrected that below.
I think you want to concatenate the data loaded from one file onto the waves created by previously loading another file. If that is correct then you need to load the second file into separate waves and then use the Concatenate operation to concatenate the new data onto the old data.
One way to do this is to use different names for the new waves, then concatenate onto the old waves, then kill the new waves. Another way to do it is to load the new waves into a new data folder, concatenate onto the old waves, and kill the new data folder.
Here is code that does that using a temporary data folder. It includes a second function, ConcatenateChemGoofFile, which calls the original routine, LoadChemGoofFile, to load the new data which it then concatenates onto the original data:
"Load ChemGoof File...", LoadChemGoofFile("", "", 1)
"Concatenate ChemGoof File...", ConcatenateChemGoofFile("", "")
End
// LoadChemGoofFile(pathName, fileName, makeTable)
// Returns a semicolon-separated list of waves loaded or "" if cancel.
Function LoadChemGoofFile(pathName, fileName, makeTable)
String pathName // Name of an Igor symbolic path or "".
String fileName // Name of file or full path to file.
Variable makeTable // 1 to make table, 0 to not make table
// First get a valid reference to a file.
if ((strlen(pathName)==0) || (strlen(fileName)==0))
// Display dialog looking for file.
Variable refNum
Open/D/R/P=$pathName refNum as fileName
fileName = S_fileName // S_fileName is set by Open/D
if (strlen(fileName) == 0) // User cancelled?
return -1
endif
endif
// Now load the data. The /V flag specifies the accepted delimiters in the data file.
// Add the /A flag if you don't want the "Loading Delimited Text" dialog.
// Add the /O flag if you want to overwrite existing waves with the same names.
// "Date" and "Time" are not available as wave names because they are Igor function names
String columnInfoStr = "N=vDateTime;N=vTime;N=Flow;N=VRef;N=VMeas;"
String delimiters = "," // Comma-delimited
String skipChars = "\" " // Skip quotes around date and time and spaces before commas
LoadWave /J /D/A /K=0 /V={delimiters,skipChars,0,0} /B=columnInfoStr /R={English,1,3,1,2,"Year-Month-DayOfMonth",40} /P=$pathName fileName
Variable numWavesLoaded = V_flag // V_flag is set by LoadWave
if (numWavesLoaded != 5)
Print "Error loading file - wrong number of waves loaded"
return -1
endif
// Create reference to waves created by LoadWave
Wave vDateTime, vTime, Flow, VRef, VMeas
vDateTime += vTime // Add time to date to obtain date/time
KillWaves/Z vTime // This is no longer needed
if (makeTable)
// Tweak date/time table formatting
Edit vDateTime, Flow, VRef, VMeas
ModifyTable format(vDateTime)=8, width(vDateTime)=150
endif
return 0 // Success
End
Function ConcatenateChemGoofFile(pathName, fileName)
String pathName
String fileName
// Create a new data folder
NewDataFolder/O/S ChemGoofTemp
Variable result = LoadChemGoofFile(pathName, fileName, 0)
SetDataFolder :: // Back to original data folder
if (result != 0)
return result // Error loading new waves
endif
// Concatenate new waves onto old
Concatenate/NP {:ChemGoofTemp:vDateTime}, vDateTime
Concatenate/NP {:ChemGoofTemp:Flow}, Flow
Concatenate/NP {:ChemGoofTemp:VRef}, VRef
Concatenate/NP {:ChemGoofTemp:VMeas}, VMeas
// Kill temp data folder
KillDataFolder/Z :ChemGoofTemp
return 0 // Success
End
I made three changes to LoadChemGoofFile from the previous version.
First I fixed the /V flag by adding a space to it in addition to the quotation mark.
Second I removed /W which tells LoadWave to look for wave names as there are no wave names in the file and we are naming wave using /B.
Third I removed /E=1 from the LoadWave command because we don't want to create a new table when concatenating. Also using LoadWave/E=1 displayed the vTime wave in the table but the vTime wave is killed after the time data is added to the date data.
If this does not solve the problem for you then I recommend that you post a zip file containing three sample files along with any further instructions.
August 26, 2013 at 12:52 pm - Permalink
August 26, 2013 at 04:37 pm - Permalink
Here is a function that does that. It always appends so you must load one file first using Load rather than Concatenate or Append. Then you can load the rest of the files using this function.
String list = GetMultipleTextFileDialog()
if (strlen(list) == 0)
return -1 // User cancelled
endif
Variable numFiles = ItemsInList(list, "\r")
Variable index
for(index=0; index<numFiles; index+=1)
String fullPath = StringFromList(index, list, "\r")
if (strlen(fullPath) == 0)
break // No more files
endif
ConcatenateChemGoofFile("Igor", fullPath) // The path Igor will be ignored because we are using a full path
endfor
End
It would be possible to write this so that it automatically detected whether the waves exist and called Load or Concatenate automatically. I will leave that as an exercise for the reader. (Hint: Use Wave/Z and WaveExists.)
Also, it would be a good idea to sort the data since the files may not be loaded in the right order. After loading, execute:
// Create reference to waves in current data folder
Wave vDateTime, Flow, VRef, VMeas
Sort vDateTime, vDateTime, Flow, VRef, VMeas
End
August 26, 2013 at 05:15 pm - Permalink
September 5, 2013 at 12:43 pm - Permalink
So I discovered a possible different way to load the files. Its based off one of the help files in the help menu; LoadandGraphAll
I did a quick edit of the code there and just have:
//"Load ChemGoof File...", LoadChemGoofFile("", "", 1)
"Load ChemGoof File...", LoadAllFiles("")
End
// LoadChemGoofFile(pathName, fileName, makeTable)
// Returns a semicolon-separated list of waves loaded or "" if cancel.
Function LoadChemGoofFile(pathName, fileName, makeTable)
String pathName // Name of an Igor symbolic path or "".
String fileName // Name of file or full path to file.
Variable makeTable // 1 to make table, 0 to not make table
// First get a valid reference to a file.
if ((strlen(pathName)==0) || (strlen(fileName)==0))
// Display dialog looking for file.
Variable refNum
Open/D/R/P=$pathName refNum as fileName
fileName = S_fileName // S_fileName is set by Open/D
if (strlen(fileName) == 0) // User cancelled?
return -1
endif
endif
// Now load the data. The /V flag specifies the accepted delimiters in the data file.
// Add the /A flag if you don't want the "Loading Delimited Text" dialog.
// Add the /O flag if you want to overwrite existing waves with the same names.
// "Date" and "Time" are not available as wave names because they are Igor function names
String columnInfoStr = "N=vDateTime;N=vTime;N=Flow;N=VRef;N=VMeas;"
String delimiters = "," // Comma-delimited
String skipChars = "\" " // Skip quotes around date and time and spaces before commas
LoadWave /J /D/A /K=0 /V={delimiters,skipChars,0,0} /B=columnInfoStr /R={English,1,3,1,2,"Year-Month-DayOfMonth",40} /P=$pathName fileName
Variable numWavesLoaded = V_flag // V_flag is set by LoadWave
if (numWavesLoaded != 5)
Print "Error loading file - wrong number of waves loaded"
return -1
endif
// Create reference to waves created by LoadWave
Wave vDateTime, vTime, Flow, VRef, VMeas
vDateTime += vTime // Add time to date to obtain date/time
KillWaves/Z vTime // This is no longer needed
if (makeTable)
// Tweak date/time table formatting
Edit vDateTime, Flow, VRef, VMeas
ModifyTable format(vDateTime)=8, width(vDateTime)=150
endif
return 0 // Success
End
Function ConcatenateChemGoofFile(pathName, fileName)
String pathName
String fileName
// Create a new data folder
NewDataFolder/O/S ChemGoofTemp
Variable result = LoadChemGoofFile(pathName, fileName, 0)
SetDataFolder :: // Back to original data folder
if (result != 0)
return result // Error loading new waves
endif
// Concatenate new waves onto old
Concatenate/NP {:ChemGoofTemp:vDateTime}, vDateTime
Concatenate/NP {:ChemGoofTemp:Flow}, Flow
Concatenate/NP {:ChemGoofTemp:VRef}, VRef
Concatenate/NP {:ChemGoofTemp:VMeas}, VMeas
// Kill temp data folder
KillDataFolder/Z :ChemGoofTemp
return 0 // Success
End
Function LoadAllFiles(pathName)
String pathName // Name of symbolic path or "" to get dialog
String fileName
Variable index=0
if (strlen(pathName)==0) // If no path specified, create one
NewPath/O temporaryPath // This will put up a dialog
if (V_flag != 0)
return -1 // User cancelled
endif
pathName = "temporaryPath"
endif
Variable result
do // Loop through each file in folder
fileName = IndexedFile($pathName, index, ".csv")
if (strlen(fileName) == 0) // No more files?
break // Break out of loop
endif
result = ConcatenateChemGoofFile(pathName, fileName)
if (result == 0) // Did LoadAndGraph succeed?
// Print the graph.
fileName = WinName(0, 1) // Get the name of the top graph
//String cmd
//sprintf cmd, "PrintGraphs %s", fileName
//Execute cmd // Explained below.
//DoWindow/K $fileName // Kill the graph
//KillWaves/A/Z // Kill all unused waves
endif
index += 1
while (1)
if (Exists("temporaryPath")) // Kill temp path if it exists
KillPath temporaryPath
endif
return 0 // Signifies success.
End
There is a lot of excess code that can be trimmed out of here, but it seems to load all the files without my having to load one file first. Can you see a problem that might develop here?
It is nice because the Concatenate function uses the load function and the Loadall function uses the concatenate function. Everything shows up in order and the data looks right when compared with the previous code.
I will still do my homework though and try to get the SortWaves function you posted working.
Thanks
September 10, 2013 at 12:43 pm - Permalink
For the life of me, I can't figure out the
String list = GetMultipleTextFileDialog()It looks like its trying to call an internal function, but I don't have it and can't find it on the internet. Or is that a personal function? Sorry for the confusion. I just don't know what that function does really (though I have an idea from the name).
September 12, 2013 at 01:28 pm - Permalink
Variable refNum
String message = "Select one or more files"
String outputPaths
String fileFilters = "Data Files (*.txt):.txt;"
fileFilters += "All Files:.*;"
Open /D /R /MULT=1 /F=fileFilters /M=message refNum
outputPaths = S_fileName
return outputPaths // "" if user cancelled
End
September 12, 2013 at 02:08 pm - Permalink