2.18.3 Batch Processing

Batch-Processing

Batch ProcessingOne may often encounter the need to perform batch processing of multiple sets of data files or datasets in Origin, repeating the same analysis procedure on each set of data. This can be achieved in three different ways, and the following sections provide information and examples of each.

Processing Each Dataset in a Loop

One way to achieve batch processing is to loop over multiple files or datasets, and within the loop process each dataset by calling appropriate X-Functions and other script commands to perform the necessary data processing.

The following example shows how to import 10 files and perform a curve fit operation and print out the fitting results:

// Find all files using wild card
string path$ = system.path.program$ + "Samples\Batch Processing";  // Path to find files
// Find the files in the folder specified by path$ variable (default)
// The result file names are stored in the string variable fname$
// Separated by CRLF (default). Here wild card * is used, which means
// all files start with "T", and with the extension "csv"
findFiles ext:="T*.csv";

// Start a new book with no sheets
newbook sheet:=0;
// Loop over all files
for(int iFile = 1; iFile <= fname.GetNumTokens(CRLF); iFile++)
{
	// Get file name
	string file$ = fname.GetToken(iFile, CRLF)$;
	// Import file into a new sheet
	newsheet;
	impasc file$;
	// Perform gaussian fitting to col 2 of the current data
	nlbegin iy:=2 func:=gaussamp nltree:=myfitresult;
	// Just fit and end with no report
	nlfit;
	nlend;
	// Print out file name and results
	type "File Name: %(file$)";
	type "   Peak Center= $(myfitresult.xc)";	
	type "   Peak Height= $(myfitresult.A)";	
	type "   Peak Width=  $(myfitresult.w)";	
}

Using Analysis Template in a Loop

Custom templates for analysis can be created in Origin by performing the necessary data processing from the GUI on a representative dataset and then saving the workbook, or the entire project, as an Analysis Template. The following example shows how to make use of an existing analysis template to perform curve fitting on 10 files:

// Find all files using wild card
string fpath$ = "Samples\Batch Processing\";
string path$ = system.path.program$ + fpath$;  // Path to find files
// Find the files in the folder specified by path$ variable (default)
// The result file names are stored in the string variable fname$
// Separated by CRLF (default). Here wild card * is used, which means
// all files start with "T", and with the extension "csv"
findFiles ext:="T*.csv";
 
// Set path of Analysis Template
string templ$ = path$ + "Peak Analysis.OGW";
// Loop over all files
for(int iFile = 1; iFile <= fname.GetNumTokens(CRLF); iFile++)
{	
   // Open an instance of the analysis template
   doc -a %(templ$);
   // Import current file into first sheet
   page.active = 1;
   impasc fname.GetToken(iFile, CRLF)$
}

// Issue a command to update all pending operations
// in case the operations were set to manual recalculate in the template
run -p au;

Using Batch Processing X-Functions

Origin provides script-accessible X-Functions to perform batch processing, where there is no need to loop over files or datsets. One simply creates a list of desired data to be processed and calls the relevant X-Function. The X-Function then either uses a template or a theme to process all of the specified data. Some of these X-Functions can also create an optional summary report that contains results from each file/dataset that were marked for reporting by the user, in their custom analysis template.

The table below lists X-Functions available for batch analysis:

The following script shows how to use the batchProcess X-Function to perform curve fitting of data from 10 files using an analysis template, with a summary report created at the end of the process.

// Find all files using wild card
string path$ = system.path.program$ + "Samples\Batch Processing\";  // Path to find files
// Find the files in the folder specified by path$ variable (default)
// The result file names are stored in the string variable fname$
// Separated by CRLF (default). Here wild card * is used, which means
// all files start with "T", and with the extension "csv"
findFiles ext:="T*.csv";
 
// Set path of Analysis Template
string templ$ = path$ + "Peak Analysis.OGW";

// Call the Batch Processing X-Function
// Keep only the final summary sheet, delete intermediate books
batchProcess batch:=1 name:=templ$ data:=0 fill:="Raw Data" 
             append:="Summary" remove:=1 method:=impASC;

Batch processing using X-Functions can also be performed by calling Origin from an external console; for more see Running Scripts From Console.