Mellau Consulting

SpectrumFit program features

SpectrumFit is a data analysis program writen with the SyMath software platform.

The basic idea that I set up at the beginning was the definition of a data structure that I named "SpectrumDatabase DATA FORMAT". This data structure contains all information about the spectrum under analysis : input raw measured spectrum, corrected (as background corrected) spectrum, the predicted spectrum calculated using the lineshape parameters, the lineshape analytical functions, the error curve i.e the difference between measured and calculated spectrum etc.... The program maintains all this data sets "on-line". After any change in number of peaks, lineshape parameters, lineshape functions etc...  the calculated data sets are automatically recalculated. The concept of a generalised spectrum data format is also necessary to be abel to append and merge spectra measured with different methods and instruments. The SyMath programs Merger and Appender allows you to merge the physical information that is contented in the spectra measured even with different instruments. This concept is necessary for 2 reasons : first to speed up the analysis procedure - all but really all data regarded to a project is in one single file - and as second to make some new data analysis procedures possible.

In a generalised spectrum one can define a set of unlimited number of peaks where each peak is defined not only by the parameters frequency and intensity, each peak has its own lineshape function, a definition range, a set of unlimited number of lineshape parameters ( peak position, intensity, linewidth etc... ) defined for each peak and a set of parameters that are shared by all peaks with the same lineshape ( as for FT-IR spectra the aperture width, a parameter shared by all peaks in the spectrum ). This definition should allow one to use Symath for any type of peak fitting problem that one can meet in spectroscopy (in fact this is realised fully in version 2.0 only). The SpectrumFit basic procedures can be used for the analysis of any data which is "spectrum like" even if in this manual I will explain the spectrum analysis of spectra usually measured in high and low resolution molecular spectroscopy.

"SpectrumFit" was designed for a fully "automatic" creation of a peak list for a spectrum. For this SpectrumFit can identify and fit to any user given lineshape all peaks in the spectrum. The result of the peak fitting is the list of the lineshape parameters. Such an analysis is perfect if the difference between the measured spectrum and the spectrum predicted using the found lineshape parameters contains only the noise of the measurement. Sure, for this task it is first necessary to found the peaks in the spectrum. SpectrumFit can do all this for you AND - this is VERY important - it will do this task of "peak searching and fitting" completely automatically even for very dense spectra with a very large number of peaks ( i.e. than 10000 peaks).

Another very important aspect of the SpectrumDatabase idea is that this database allows also the integration of peak assignment information in the database. The peak assignment information is not only a crude text that one can assign to each peak but contains also all the physical information in a computer readable format i.e. each peak "knows" exactly for the corresponding transition its quantum numbers, corresponding sample molecule structure, references for the publications where the peak was already measured and assigned etc.... You can use SpectrumFit to display this assignment information on the spectrum but to do an assignment analysis you need the Symath Calibration, BandSearch and Molecule programs

The main features implemented in SpectrumFit version 1.1. and 1.2 are :

  1. Any type of spectrum, as FT-IR, microwave, laser etc. can be processed. The input spectrum can be a simple (x,y) ASCII file. Also the import of some binary data formats is supported as *.1 or *.dat type files. The data do not have to be equally spaced. All SpectrumFit analysis procedures work on lists of (x,y) data pairs.
  2. Any function can be used as lineshape function. You only have to type your lineshape function using the LineshapeEditor and start working.
  3. All parameters of the fitting procedure are stored in a data structure which includes the MODEL ( lineshape functions ) also. In this way the spectroscopic data carry their theoretical background too. If after 10 years somebody will load a spectrum in Symath SpectrumDatabase format he will be able to do any manipulations or changes on the peaklist with the same lineshape function even if Symath itself was in the meantime completely rewritten.
  4. You can define for the lineshape model two distinct types of parameters: "b" type parameters belong to each peak, while "a" type parameters are shared by all peaks having the same lineshape. There are no limitations on the number of these parameters.
  5. The lineshape function can have ANY number of parameters. All interactive tables are created in a dynamic way so that they "grow" as the number of parameters increases without limitations. You may ask why a theoretical lineshape will ever have more than 3-5 parameters. The answer is that you can fit not only single peaks using Symath, you can set up any theoretical model that models a set of peaks and fit them together with any number of parameters that belong to the set of the peaks.
  6. ANY number of peaks with any number of parameters can be fitted together. There are no limitations on the number of peaks that you can fit together other than the computer speed (for example, the fit of 100 peaks with Doppler lineshape takes several minutes).
  7. There is no limit on the spectral range to be fitted. The number of data points that can be fitted together will be limited only by the computer speed.
  8. Peaks to be fitted together  must not necessarily be in the neighborhood of each other. You can fit peaks with index 2,200 and 3000 together with any number of common parameters. Only those data points will be considered that are around the peaks 2,200 and 3000. This is very important if you are using the manual fit window and the spectrum is not very dense. In this case you can have for example 3 wavenumbers in the window with 20 lines. Fitting all x data points where really no line information exists would increase dramatically the fit time. Fitting these peaks "together" will not take longer than if 20 peaks would be in a range of only 0.01 wavenumbers. This is also very important if you set up "lineshapes" that model a set of the lines which are far away from each other in the spectrum (in this case the lineshape function models more than one peak).
  9. Each peak in the spectrum can have its own lineshape.You can fit peaks together where each peak has its own lineshape. This can be important if you have peaks from different molecules, and some of them can be best fitted with one lineshape and other can be best fitted with a second lineshape. Symath allows you to append any number of spectra measured under different experimental conditions/different spectrometers (and fitted with different lineshapes) so that you have to handle for the assignment work only a single spectrum. In the appended spectrum each peak knows its original lineshape and parameters.
  10. EACH parameter for EACH peak can be set to ADJUST or FIX for the fit.These settings are stored and used by the automatic processing system. This is very important for dense spectra where the lineshape width parameters must be fixed to an average value. One can fix for example manually for all group of peaks which overlap and fit for all non overlapping peaks the line width parameter, the automatic fitting procedure will use all these preselections. Fitting manually a set of peaks and then setting all their parameters to fix will have the effect that any later automatic procedure will leave these peaks unchanged. The a type parameters common for a set of peaks can also be set to adjust or fix.
  11. The initial generation of a peaklist is done by a complex peak search program coupled with an expert system for peak validation and parameter estimation.The program can search for different user selectable predefined type of positions in the spectrum as maxima of peaks. Using the found positions ( (x,y) position of the peak maxima ) peak parameters are estimated for ANY lineshape using the so called estimating functions. In this manner a very complete first guess can be dome for the position, intensity and the number of peaks present in the spectrum for ANY spectrum and for ANY lineshape. For sparse spectra the first guess looks already as if the program had already fitted the spectra..
  12. The FIT of the complete spectrum can be fully AUTOMATIC.The program goes through the spectrum, fitting one peak after another. The automatic fit routine can check for overlapping peaks and select for a fit sets of overlapping peaks.
  13. The INSERTION of additional peaks in the peaklist is AUTOMATIC.For very dense spectra it could happen that only 50-80 % of the peaks can be detected using the PeakSearch searching methods. Such spectra can be first fitted using the automatic fit and than insert the missing peaks using the SpectrumFit automatic insert and fit procedure. This procedure can analyze the spectrum and make decisions about where to insert new peaks. If new peak is inserted, the region around the new peaks is refitted.
  14. The SpectrumFit uses the SymathUserInterface, a custom designed, flexible user interface using the capabilities given by the Mathematica 4.1 Notebook system. All working sheets can be saved on disk and the tables or plots introduced directly with Cut/Paste into a document. Data plots are always Postscript plots which can be saved and introduced into any other word processor. Lineshape expressions are introduced in conventional mathematical notation. You will find the system very easy to use and very near to the world of the spectroscopy. The SymathUserInterface is described in a separate manual.
  15. Symath SpectrumFit has a very powerful BACKGROUND CORRECTION SYSTEM for any type of background fringing/shift. Fitting peaks to lineshape models can be done only if the background is a flat line at 0.0 for emission and at 1.0 for transmission files. Any distortions will degrade the fit. The program can correct any type of irregularities which one can "see" on the background.
  16. Symath SpectrumFit has a very powerful MANUAL FIT WINDOW. Here the user can select interactively a portion of the spectrum. The spectrum, the predicted spectrum,the error curve and the predicted spectrum of each peak using its own lineshape are drawn in a window. For all peaks shown in the window a table is set up with peak parameters. You can change any of the parameters or set them to be adjusted or fixed in the next fit. You can insert peaks or fit the peaks shown in the window using the parameter set up. For many users this fit mode is even more interesting than the automatic procedures, but if your spectrum has for example 12000 peaks a manually fit of the spectrum is completely unpractical. You can make a quick estimate of how long it would take to analyze manually your spectrum if you need 3-5 minutes for the visual inspection and correction of 15 currently selected peaks.
  17. Symath SpectrumFit's fitting routine is a very complex (over 150 pages written code) nonlinear fitting routine.The basic principle is prediction and the usual least-squares method using the lineshape functions analytical derivatives BUT these methods are adapted to the task of simultaneous fitting of many spectral lines.
  18. Symath SpectrumFit's fitting routine has a very complex FIT REPORT STATUS Window where you can examine how a fit is converging during the fit loop. The program can create a table with all parameters for each loop and even plot the spectra and the predicted spectra for each loop. So if you have problems with your model you can find out why the fit does not converge.
  19. As already mentioned, you can simply type your function into the Symath SpectrumFit and start fitting. It is important to mention that this function can have any complexity and you can use any of the mathematical special functions defined within Mathematica, really all special functions used in physics.The formula can even contain sum, product, integral symbols. Any mathematical expression for which Mathematica can derive the derivatives ( or you can type it in the user derivative input fields ) of your formula and can calculate the model value for a set of parameters will work with Symath. This is the most important feature of this program ( and works only using Mathematica as interpreting programming language ). You have inside of an usual program the possibility to use the tremendous mathematical know-how of Mathematica. You have to realize that any fitting program written in any other computer language can not give you the possibilities that you have here. Sure you can do the same tasks if the program has the desired fitting model implemented but if you want to adapt the model to your set of data you will never succeed. In a compiled C or Fortran program you can do only and only  the tasks that are implemented in the program, any change that you would do are impossible. This is the basic feature of this program. There are programs with thousands of predefined model functions which even can found a model function that fits perfectly your data - which is in the case of modelling physical phenomena a completely wrong strategy. One must predefine the model using physical considerations and than fit it to the experimental data.

To each module and to each program corresponds a so called “UserExtensions file“. These are notebooks containing program code or program constants.

These notebooks contain program code which can be freely edited and modified by users. After any modification Symath will notice this and load the current changed version of this program code. Moreover, any program extensions that the user has written can be placed here and will be automatically loaded at any further Symath start into the Kernel.

With this feature it is very easy and convenient to create user defined extensions to any module or program. The user has not to care about loading any of its extensions.

In similar notebooks are placed all text resources used by the Symath programs and users are free to match these text strings to the need of their application.

© 2007 Dr.Georg Ch. Mellau