Local Level Processing

This document provides a guide to help the users analyzing the AGATA data produced at the local level processing i.e. before any building of events. It includes energy calibrations, time alignments, cross talk corrections and any other corrections to improve the quality of the data. Criteria for bad events rejection are also highlighted.

People involved in this document:

D. Bazzacco, A. Boston, E. Clement, N. Dosmes, J. Dudouet, A. Gadea, F. Holloway, L. Hongjie, A. Korichi, N. Lalovic, E. Legay, J. Ljungvall, C. Michelagnoli, R. Perez, D. Ralet, M. Siciliano and O. Stezowski

The different actors processing the data flow#

The Producer actor#

This actor is in charge to start the NARVAL chain. In online mode, it gets the traces from the front end electronics and sends it to the data flow. In offline mode, it reads input files (adf or traces files) and sends them to the NARVAL chain.

The Preprocessing actor#

This actor is in charge of preparing the data for the PSA algorithm. In order to do that job, two files which are named PreprocessingFilter.conf and PreprocessingFilterPSA.conf should contain the required information.

As any detector, any AGATA capsule should be calibrated in energy. Since it is also highly electrically segmented, cross talks have a significant importance and must be corrected. For details, see for instance these publications for such effects: B.Bruyneel et al., NIMA 608 (2009)

Be aware, there might be time delays between the different segments and the core signal. Any PSA code requires having all the signals from one capsule perfectly aligned and thus time alignment are also performed.

Here are snapshots of the two required configuration files.

PreprocessingFilter.conf:

ActualClass          PreprocessingFilterPSA
SaveDataDir          /global/path/to/Replay/Folder/run_xxxx/Out/00A
EnergyGain           4
XtalkFile            xinv_1325-1340.cal
WriteTraces          100

PreprocessingFilterPSA.conf:

segm         0   4600   500   0.300652   15   17.220
segm         1   4600   500   0.313630   15   17.873
...                                             
core         0   4390   500   0.482203    0   20.
core         1   4390   500   1.735468    0   20.
tntf   2097152

Note

the content may be slightly different depending on the version of the PreprocessingFilter actor used.

For the second file, here is the meaning of the different columns:


Type	segm/core
Id	Segments(0-35) or Cores(0-1) Id
Tfall	Pole zero / decay time [timestamp units]
Trise	Shaping time / Risetime [timestamp units]
egain	Slope of energy calibration (no offset at the preprocessing level) [keV/channel]
emink	Energy threshold [keV]
tmove	Shift to align in time the different segments [ns]

Energy calibration#

Explanations and goals:#

In digitizers, signals are processed to extract the amplitude using a trapezoidal filter. This value is written from the beginning of the chain into the data flow. This is the value which is used to calibrate the detectors in energy. Concerning the 36 segments, because of the various cross talks, the calibration in energy is done using events in which only one and only one segment in a given crystal has fired.

The obtained calibration coefficients are to be set in the 5^th col of the PreprocessingFilterPSA.conf file.

Note

All the different steps of the following procedures needs to be done for each crystal. The environment variable $CryId will be used in the following to refer to the current detector (ex: CryId=00A). We suppose for the given shell command that you are working in the Conf folder of the current detector.**

Tools available:#

To use those tools, spectra directly produced by the actors at running time are required. In particular the ones contained in the file Data/$CryId/Prod__4-38-32768-UI__Ampli.spec¹:

From the file name, we can deduce that this spectra file contains 4 libraries of 38 spectra written in 32768 unsigned integer bins, containing the amplitude spectra of segments and cores. The first library contains the raw amplitude from the trapezoidal filter. The second library contains events of segment multiplicity=1, with amplitude larger than the threshold and scaled by the EnergyGain (see conf file) factor. The calibration needs to be done on spectra from the second library. The two other libraries are not used in the energy calibration but can be necessary for x-talk corrections.

From these spectra files, a C program, called RecalEnergy², is able to find peaks and thus calibrate the different channels (core and segments) for one crystal. Here is the way to use it, with some options³

cd Conf/$CryId  # If not already done, go in the working directory
RecalEnergy -spe ../../Data/$CryId/Prod__4-38-32768-UI__Ampli.spec -sub 38 -num 38 -gain 2 -60Co |tee recal.out

Options:


-spe filename	Name of spectrum to analyze (mandatory)
-sub nn	Analysis starts from spec nn (-sub 38 to analyze the second library)
-num nn	Number of spectra to analyze (-num 38 for 36 segments + 2 cores)
-lim min max	Limit the search to this range in channels
-dwa w h	Default fwhm and minimum amplitude for the peak search
-60Co	Define the source (default is 60Co, more than one source is allowed)
-gain val	Scaling factor for the slope

The gain factor needs to be coherent with the gain applied to the spectra, this information can be found in the file CrystalProducer.conf:

ProjeM1               10 2 (threshold and gain)

A way to apply this for several crystals $DetList is the following:

Note

You need to be in the global folder (where Out, Data, Conf are located)

for i in $DetList ; 
do RecalEnergy -spe Data/$i/Prod__4-38-32768-UI__Ampli.spec -sub 38 -num 38 -gain 2 -60Co |tee Conf/$i/recal.out;
done

In the following, the commands will be written for only one detector, but this loop can be use to obtain automatic procedures on a list of detectors.

Check carefully (rEnergy, FM05, Chi2...) the output of each file to be sure that the fits are good. If not, reprocess the concerned files using more restrictive parameters (-lim, -dwa options). Note than in case of dead segments (see followings), the automatic procedure might not work and need to be done manually.

Note that for very bad crystals, typically neutron damaged detectors, the default fwhm parameter needs to be increased (the default value is 10) to obtain a reasonable fit result.

Once the calibration seems good the results should be inserted in the configuration file (i.e. add the recal.out coefficients to the 5^th column of PreprocessingFilterPSA.conf) so that the processing of the data should be done properly. A python script colupdate.py is provided to help with this⁴:

cp PreprocessingFilterPSA.conf PreprocessingFilterPSA.save # Just in case...
python colupdate.py PreprocessingFilterPSA.conf recal.out -c 4 13 -o PreprocessingFilterPSA.conf

-c column you want to use from file1 and file2 starting from 0 (5^th column in PreprocessingFilterPSA.conf (4 starting from 0), 14^th column in recal.out (13 starting from 0)
-o file output

TkT Calibration check:#

Once all crystals have been calibrated, we need to process a local level replay of the Preprocessing actor to check the calibrations. For this, we need to create a dedicated folder containing a link to the raw data, the Conf folder (containing the new PreprocessingFilterPSA.conf files), the gen_conf.py script adapted to offline femul replay (see section Replay) and a Topology file similar to this:

LOOP CRY 00B 00C 01B 01C 04B 04C 05A 05B 05C 06A 06B 06C

Chain 3   CRY
Producer  CrystalProducerATCA
Filter    PreprocessingFilterPSA
Consumer  None
ENDLOOP

The "None" consumer do not write any output file (we only need here the spectra files).

For preparing the following steps, we can modify in the gen_conf.py file the WriteDataMask value to 8 to write the "bdat" files used for x-talk corrections. If you don't plan to perform x-talk corrections, set this value to 0.

The following lines allows to process the replay:

Note

You need to be in the global folder (where Out, Data, Conf are located)

python gen_conf.py # (will generate the replay Conf and Out folders)
femul Topology.conf

Open the Preprocessing spectrum file: Out/$CryId/Prep__2-40-16384-UI__Ener.spec with TkT to check the energy calibrations (apply the gain factor written in PreprocessingFilter.conf file to obtain the correct energy).

Crosstalk corrections#

Explanations and goals:#

Crosstalk is present in any segmented detectors. Through couplings, the collection of the signal in one segment modifies the signal collected in the neighboring ones. One consequence is a shift in energy (applying a calibration based on events with just one segment fired) which increases as a function of the number of segment fired in one crystal. There are different crosstalk in segmented Germanium crystals, in particular differential and proportional. To have more information on such effect in different segmented Germanium detectors, you can read the following papers (non-exhautive list):

Vetter NIMA 452 (2000), Swensson NIMA 540 (2005), B.Rossé NIMA 565 (2006), B. Bruyneel NIMA 608 (2009)

Concerning the AGATA capsule, it is possible use the crosstalk correction procedure to recover up to one broken or missing segment per crystal (by using the fact that sum of the energies of the segments should be equal to the core energy).

Tools available:#

In order to perform this correction, we will use the files with the extension *.bdat (event_energy.bdat) produced by the CrystalProducer if the 4th bin in the WriteDataMask instruction of CrystalProducer.conf is set ("WriteDataMask 8"). The program used to perform this is called SortCapsule. Xtalk corrections needs to be done using a 60Co source.

First of all, it is necessary to write the preprocessing calibration file in a new format, use the following command to prepare the file in the proper format for the next steps:

cd Conf/$CryId          # Go in the working directory if not already done
rm -f ecalF1.cal        # if file was already existing
for i in {1..38} ; do cat PreprocessingFilterPSA.conf | head -n $i |awk -v var=$(( $i - 1 )) '{print "0\t" var "\t2\t0\t" $5}' | tail -n 1 >> ecalF1.cal ; 
done ; cat ecalF1.cal;

The ecalF1.cal file should be similar to:

0   0   2   0   0.301769
0   1   2   0   0.313686
...
0   36  2   0   0.484332
0   37  2   0   1.740927

2. Verify the calibration coefficients:

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -egain 5

This will generate the following files for each crystal:


proj__3-38-32768-UI__raw.spec raw projections	[0]=Tot [1]=SG@F1 [2]=CC@F1
spec__3-38-16384-UI__cal.spec calibrated spectra	[0]=Tot [1]=SG@F1 [2]=CC@F1
ssum__2-12-16384-UI__cal.spec sumEnergy[segFold]	[0]=SG [1]=CC

with [N] the library number.

The two last files can be read using TkT to check the segments energies. Energies needs to be good for fold 1, but note that the position of the peaks in the first set of spectra in ssum__2-12-16384-UI__cal.spec depends on the number of fired segments (Fold). No such dependency for the second set (core).

3. Call xTalkSort to sort and analyze the AGATA events dumped into event_energy.bdat.0000 by producing the cross-talk spectra for full energy release of 1332.5 keV in one segment (F1)

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -egain 5 -specXT -trigewin 1325 1340

with:


ecalF1	file with energy calibration coefficients for singles extracted above
egain	gain factor for energy spectra
specXT	generate cross talk spectra from Mseg=1
trigewin	energy window on trigger channel

4. Once the different spectra have been created, the RecalEnergy program can be used to build the direct Xtalk matrix:

RecalEnergy -spe xspe__36-37-16384-UI__cal.spec -num 1332 -ener 1332.5 -gain 5 -offs 1000 -Xtalk 37 |tee 1325-1340.txt

with:


Xtalk 37	Use the 37^th spectrum for the core in the Xtalk calculation
offs 1000	channel offset to subtract to the position of the peaks

5. Transform this file to the proper format for direct cross talk coefficients:

grep -v "^#" 1325-1340.txt |grep -v "^ *36 " |cut -b15-102 --complement |tee xdir_1325-1340.cal

6. Invert the cross talk coefficients and rename it:

xTalkInvert -f xdir_1325-1340.cal ; mv xdir_1325-1340.cal.inv xinv_1325-1340.cal

The inverse matrix is given the name of xdir_1325-1340.cal.inv. The traditional name of this file was called xinv_1325-1350.cal. Up to you to rename the file or keep the given name.

7. Call again xTalkSort using the Xtalk inverted matrix to check the corrected spectra:

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -egain 5 -recalXT xinv_1325-1340.cal

This will produce the spectra files:


-proj__3-38-32768-UI__raw.spec raw projections	[0]=Tot [1]=SG@F1 [2]=CC@F1
-spec__3-38-16384-UI__cal.spec calibrated spectra	[0]=Tot [1]=SG@F1 [2]=CC@F1
-spec__3-38-16384-UI__adj.spec adjusted spectra	[0]=Tot [1]=SG@F1 [2]=CC@F1
-ssum__2-12-16384-UI__cal.spec sumEnergy[segFold]	[0]=SG [1]=CC
-ssum__2-12-16384-UI__adj.spec sumAdjusted[segFold]	[0]=SG [1]=CC

with [N] the library number.

Check with TkT the spectra in spec__3-38-16384-UI__adj.spec and ssum__2-12-16384-UI__adj.spec. In particular the shift in position of the sum energy of the segments as a function of the segment fold should have disappeared.

Specific case of "lost" or "broken" segments:#

The part extends the treatment to the case of one dead segment which can be "lost" or "broken".

A broken segment is the result of a problem at the FET level with the consequence that the charge of the segment is not collected but flows to the neighbors. The salient effect of a broken segment is the presence of ghost peaks in the neighbors and of a strong step-like tail in the spectrum of the core. The ghost peaks and the left step can be seen as enhanced cross-talk.

The case of a lost segment is when the detector works normally but the signal is not present in the data due, e.g., to a broken wire or a faulty digitizer channel. In this case there are no ghost peaks.

It is worth remarking that segments with unstable gain could be transformed into (and treated as) lost segments by setting their energy calibration to zero. Of course this is possible only if all other segments in the detector work correctly.

To treat dead segments, we have to find a way to quantify the amount of missing energy. For broken segments, we have also to generate a specific set of cross-talk correction coefficients capable of removing the ghost peaks from the affected neighbors. Two different ways of treating this are possible

1. In the calibration file adapted for Xtalk calculations (ecalF1.cal) and in the PreprocessignFilterPSA.conf, the calibration parameter of the dead segment must to be set to 0.

Note

For the command lines in the following, it is supposed that you are in the Conf folder of the concerned detector named in the following: "$CryId"

2. To determine the energy released in the missing segment:

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -matCCSG

Open with Mat.exe the file CC-SG__50-1500-1500-US__ma.matr. Matrix #36 shows the correlation between the energy seen by the core and the sum-energy of all segments.

For a broken segment it should look like the following:

The main diagonal is not at 45° (as usual when the sum-energy is over all segments) and there is a lower limit line for the lost in the sum of segments (because part of the energy released in the broken segment is "collected" in its close neighbors, generating the "ghost peaks") and also a small loss of core energy (the slightly tilted lines coming down from the peaks which would be perfectly vertical if the segment would be disconnected).

The coefficients to manage the broken segment (here and also in PreprocessingFilterPSA) are:

deadXsg = 1267.5/1332.5 = 0.95122
$\rightarrow$ slope of main diagonal
deadXcc = (1332.5-1312.9)/(1267.5-366.2) = 0.021746
$\rightarrow$ 1/slope of core loss

For a lost segment:

the main diagonal has practically the same slope but the lower line is not present and the lines out of the peaks are just vertical. The coefficient to manage the lost segment are:

deadXsg = 0.95122
$\rightarrow$ slope of main diagonal
deadXcc = 0
$\rightarrow$ no core loss

3. We can verify these values from the core-segment correlation matrices produced after giving the index of the dead segment and the two coefficients:

For a lost segment (case of dead segment id = 4 [A5]):

We will use in the following the environment variable $DeadSeg=4

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -deadSeg $DeadSeg 0.95122 0. -matCCSG

The off diagonal stuff in sCC-eSumSG has disappeared by construction because the missing energy has been assigned to the lost segment. Indeed A5 is now present with the correct energy and we are essentially back to the normal situation and we can produce the cross-talk matrix as:

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -deadSeg $DeadSeg 0.95122 0. -matx1

and the cross talk correction coefficients as in the normal case:

xTalkMake -f xSG__36-36-100-1536-US__ij.matr

Test the result (see step 7 of standard Xtalk corrections):

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -deadSeg $DeadSeg 0.95122 0. -recalXT xinv_1325-1340.cal

For a broken segment (case of dead segment id = 4[A5]):

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -deadSeg $DeadSeg 0.95122 0.021746 -matCCSG

Also for this case the off diagonal stuff in sCC-eSumSG has disappeared and the matrix of the broken segment is no more empty. However, the assigned energy is much too small due to the fact that part of it is still in the neighbors. The slope of the diagonal line in the two matrices is:

slopeDeadSG = 897.5/1332.5 = 0.6734
$\rightarrow$ (matrix #4) needed for xTalkMake

To correct the remaining effects we need to determine the proper xTalk coefficients. The procedure described in the first part of this document is not suited as it looks only for slightly negative-energy peaks, missing completely the positive-energy ghosts. The program manages this case from the fact that the third parameter of the command line switch "-deadSeg" is positive:

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -egain 5 -deadSeg $DeadSeg 0.95122 0.021746 -matx1

In the generated file : xSG__36-36-1000-100-US__ij.matr, like for the standard xTalk matrices the x axis (second-last index) is the spectrum of the "affected segment" while the y axis (last index) is the spectrum of the "affecting (net-charge) segment". In this case the 100 channels-long Y axis is a [1300-1350] keV window on the core energy (0.5 kev/chan), while the 1000 channels-long x-axis is the spectrum of the affected segment for the energy range [-15 185]-keV (at a gain of 5 channels/keV). The matrix is incremented only if one segment sees more than 60% ($>$ 800 keV) of the core energy and that segment is considered as the affecting one. Have a look to the matrices [4][seg] to see the location of the ghost peaks in close neighbours of A5. Then, calculate the cross-talk correction coefficients by modifying the diagonal element of the broken segment with:

xTalkMake -f xSG__36-36-1000-100-US__ij.matr -dxy 1000 100 -gate 55 75 -xcal -15 5 -egam 1332.5 -dval $DeadSeg 0.6734

To be consistent with the Xtalk file names, rename the matrices:

mv xdir_0055-0075.cal xdir_1325-1340.cal
mv xinv_0055-0075.cal xinv_1325-1340.cal

Finally, test the result with (see step 7 of standard Xtalk corrections):

xTalkSort -ifile ../../Out/$CryId/event_energy.bdat.0000 -ecalF1 ecalF1.cal -deadSeg $DeadSeg 0.95122 0.021746 -recalXT xinv_1325-1340.cal

4. In the gen_conf.py , define in the PreprocessingFilter and PSA actors the dead segments as in the example bellow with 00A as a broken segment, and 11A as a lost one:

#########################
PreprocessingFilter=(
"ActualClass           PreprocessingFilterPSA", # name of the used daughter class
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/01A...
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
"XtalkFile             xinv_1325-1340.cal",     # cross talk correction coeffs for the energies
"WriteTraces           100" ,                   # number of traces written
#### command lines to be produced only for the specified crystals
{
'00A' : ("DeadSegment    29 0.950 0.049"), # case of a broken segment
'11A' : ("DeadSegment    18 0.947 0."),    # case of a lost segment
}
)
#########################
PSAFilter=(
"ActualClass           PSAFilterGridSearch",    # name of the used daughter class
"BasisFile             $SIGNAL_BASIS",          # this is generated from the GeDataBase structure
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/Data(online)
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
"XtalkFile             xdir_1325-1340.cal",     # cross talk correction coeffs for traces
"Threads               5 300",                  # number of threads, events/thread 
"GridSearchType        Adaptive" ,              # SegCenter, Adaptive, CoarseOnly or Full;
#### command lines to be produced only for the specified crystals
{
'00A' : ("DeadSegment    29"),
'11A' : ("DeadSegment    18"),
}
)

Check also that the files: xinv_1325-1340.cal xdir_1325-1340.cal are written in the "ExtraFiles" list of the gen_conf.py :

ExtraFiles={  
'CRYSTAL'   : "CrystalProducerATCA.conf PreprocessingFilterPSA.conf xinv_1325-1340.cal xdir_1325-1340.cal",
}

Time alignment#

Explanations and goals:#

The Preprocessing has another very important "ingredient\", i.e. the possibility to aligned in time (based on Straight line fit or Digital CFD of the signal rise-time) the traces of the different segments to the core one.

This operation is very important to obtain good performances of the PSA algorithm. In other words, the PSA tends to 'process' all the waveforms as beginning at the same time and therefore, if there is any mis-match in time, the reconstructed interaction points are prone to be unreliable. A bad time alignment can result in clusterisation patterns at the output of the PSA.

Tools available:#

The spectra to be used are in the Prep__6-40-1000-UI__TT.spec⁵ (before was Prep__2-40-1000-UI__TT1.spec). Library 1 (starting from 0) corresponds to timing before applying any shift. It must be used for time calibration. Library 3, after time shift, can be used to check the quality of the time alignment.

1. Time alignment of segments:

The command lines to be used per crystal is the following:

cd Conf/$CryId  # If not already done, go in the working directory 
RecalEnergy -spe ../../Out/$CryId/Prep__6-40-1000-UI__TT.spec -sub 40 -num 36 -T 500 |tee shift_TT.out

Options:


-sub 40	Second library
-num 36	36 segments
-T 500	we want all the peaks at this position

Add this coefficients to the 7th column of PreprocessingFilterPSA.conf using the colupdate.py script:

cp PreprocessingFilterPSA.conf PreprocessingFilterPSA.save2 # Just in case...
colupdate.py PreprocessingFilterPSA.conf shift_TT.out -c 6 13 -o PreprocessingFilterPSA.conf

2. Time alignment of the core:

The time alignment of the core is done using the spectrum file produced by the PSA actor name "Psa__40-1000-UI__Tzero.spec" (Here we don't care about what the PSA actor is doing, we only use this spectrum file). A replay from traces to PSA is thus required, taking into account the new time calibration of the segments.

Note

Be careful to have in the gen_conf.py file, the option "NoMultiHist" commented in the PSA actor (otherwise the spectra files will not be produce).

This kind of topology can be used:

LOOP CRY 00B 00C 01B 01C 04B 04C 05A 05B 05C 06A 06B 06C

Chain 4   CRY
Producer  CrystalProducerATCA
Filter    PreprocessingFilterPSA
Filter    PSAFilter
Consumer  None
ENDLOOP

The statistic needed here is not very important, few minutes should be enough. To align the timing of the core, look at the histogram n° 36 of the file Psa__40-1000-UI__Tzero.spec as shown below:

A well align core must contain only one peak, centered at channel 500. In the example above, a broad peak is observed with a peak position at 478. A shift of 22 channels is thus needed. Apply this shift to the timing value of the two cores set in the PreprocessingFilterPSA.conf file:

Before core alignment:

...
core         0   4390   500   0.482203    0      5.
core         1   4390   500   1.735468    0      5.
tntf   2097152

After core alignment:

...
core         0   4390   500   0.482203    0      27.
core         1   4390   500   1.735468    0      27.
tntf   2097152

Once the core has been aligned, start a new short replay to check the result, it should be similar to:

Finally, reprocess a replay to check the quality of the time alignment, looking at the spectra file "Prep__6-40-1000-UI__TT.spec" (library 3)

The Pulse Shape Analysis (PSA) actor#

The PSA actor is used to extract from the signals shapes the position of each interaction point with a 5mm precision. No specific calibration are needed for this step. But the quality of the PSA is highly dependent on the good calibrations at the Preprocessing level.

The PostPSA actor#

The PostPSA filter actor allows to make all the final operations on the local level data (neutron damage correction, time-dependent core-energy calibration, final energy calibration with an offset and core time alignment).

neutron damage corrections#

The damages caused by interactions between neutrons and the Germanium detectors deteriorate the gamma spectra quality. The typical effect of this neutron damages is a left tail on the peaks. This detector deterioration is increasing along time and its correction is mandatory to obtain a satisfying energy resolution.

This corrections is done in three steps:

First, a recalibration of the segments and core can be performed before the neutron damage correction.
The program named "SortPsaHits" is then used to estimate the neutron damages and correct the energies.
After the neutron damage correction, a final recalibration is processed to correct from possible shifts induced by the neutron damage correction.

Neutron damage correction procedure:

First, it is necessary to do a replay of a ⁶⁰Co source run from the traces including the "WritePsaHits" option in the PSA actor of the gen_conf.py file:

Note

It is necessary to have configured the correct path to the ADL signal bases in the gen_conf.py (see the Replay section)

#########################
PSAFilter=(
"ActualClass           PSAFilterGridSearch",    # name of the used daughter class
"BasisFile             $SIGNAL_BASIS",          # this is generated from the GeDataBase structure
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/Data(online)
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
"XtalkFile             xdir_1325-1340.cal",     # cross talk correction coeffs for traces
"Threads               5 300",                  # number of threads, events/thread 
"GridSearchType        Adaptive" ,              # SegCenter, Adaptive, CoarseOnly or Full;
"WritePsaHits" ,                                # writes the hits in binary
)

This will produce the files: Psa__0-16-F__Hits.fdat, needed to determine the coefficients of the neutron damage corrections.

This kind of topology can be used:

LOOP CRY 00B 00C 01B 01C 04B 04C 05A 05B 05C 06A 06B 06C

Chain 4   CRY
Producer  CrystalProducerATCA
Filter    PreprocessingFilterPSA
Filter    PSAFilter
Consumer  BasicAFC
ENDLOOP

The use of the BasicAFC consumer here will write the psa*.adf files that will be used for faster replays in the following.

In your calibration folder, create a new symbolic link "Out" linked to the output of this new replay

2. Prepare a dummy trapping file:

cd Conf/$CryId          # Go in the working directory if not already done
rm -f Trapping.cal      # If already existing
for i in {0..35} ; do echo -e "$i\t1.\t1.\t999999.9\t999999.9\t1.\t1." >> Trapping.cal ; 
done ; cat Trapping.cal

This file should be like following:

0   1.  1.  999999.9    999999.9    1.  1.
1   1.  1.  999999.9    999999.9    1.  1.
...
...
34  1.  1.  999999.9    999999.9    1.  1.
35  1.  1.  999999.9    999999.9    1.  1.

It can be understood as follow:

column 0: segment id
column 1: extra gain on the segment before correction
column 2: extra gain on the core before correction
column 3: electron-trapping correction (lambdaE)
column 4: hole-trapping correction (lambdaH)
column 5: extra gain on the segment after correction
column 6: extra gain on the core after correction

3. The program SortPSAHits is used both to sort PSA hits and generate the corresponding spectra files as well as to determine neutron damage correction parameters.

First use the program to generate the spectra file Pso__2-4-40-2048-UI__Ener.spec for the recalibration of the segment before neutron damage correction:

SortPsaHits -f ../../Out/$CryId/Psa__0-16-F__Hits.fdat -gain 5 -offs 5000 -fcal Trapping.cal

This Pso__2-4-40-2048-UI__Ener.spec file contains two spectra libraries of 4 sub libraries of 40 spectra:

Family 0: Segments
- Library 0: original spectra
- Library 1: original spectra + recal before neutron damage correction
- Library 2: spectra after neutron damage correction
- Library 3: neutron correction + final recalibration
Family 1: Core
- Library 0: ...

4. If necessary, recalibration of the segments and cores before neutron correction (the search peaks parameters "-dwa" probably needs to be adapted):

segments recalibration (Family 0, Library 0):

RecalEnergy -spe Pso__2-4-40-2048-UI__Ener.spec -num 36 -sub 0 -gain 5 -offs -5000 -noTR -dwa 30 5 | tail -n 36 |tee log_sg_pre.cal

cores recalibration for each segment (Family 1, Library 0):

RecalEnergy -spe Pso__2-4-40-2048-UI__Ener.spec -num 36 -sub 160 -gain 5 -offs -5000 -noTR -dwa 30 5 | tail -n 36 |tee log_cc_pre.cal

5. Update the dummy trapping file:

segments:

awk 'FNR==NR{a[NR]=$14;next}{$2=a[FNR]}1' log_sg_pre.cal Trapping.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' \
|tee Trapping_tmp.cal

cores:

awk 'FNR==NR{a[NR]=$14;next}{$3=a[FNR]}1' log_cc_pre.cal Trapping_tmp.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' \
|tee Trapping.cal

Note

Check carefully that the good peaks have been fitted, the coefficients should be all around 1. with a $\sim$ precision

The columns corresponding to pre-calibration should now be updated in the Trapping.cal files like following:

 0   0.999311   1.000675   999999.9   999999.9         1.         1. 
 1   1.000594   1.000559   999999.9   999999.9         1.         1. 
...
...
34   0.999558   1.000857   999999.9   999999.9         1.         1. 
35   0.999976   1.000779   999999.9   999999.9         1.         1.

6. Use now the program SortPSAHits to sort PSA hits and determine neutron damage correction parameters.

This step offers a significant degree of customization. Below, a typical command used to estimate the neutron damage parameters is provided. A more detailed explanation of the available customization options will follow.

SortPsaHits -f ../../Out/$CryId/Psa__0-16-F__Hits.fdat -best 1300 1350 -bpar 1 10000 0 -size 50 30 -algo 3 2 |tee sort_hit.log

The neutron damage correction parameters are estimated using a Grid-Search algorithm, which must be activated using the -best option. The numbers following the keyword specify the region of interest within the energy spectrum.

The algorithm processes only the events with a multiplicity of 1 and hits whose initial energy lies within the defined region of interest, highlighting the importance of performing an initial calibration beforehand.

The optimal set of parameters is selected based on a figure of merit (FOM). Two FOM calculation methods are available:

Default Method: the algorithm fits each spectrum with a double-tailed Gaussian and uses the Gaussian amplitude as the FOM. While this approach is computationally intensive, it performs best for detectors with moderate levels of damage.
Alternative Method: by including the -std option, an alternative FOM calculation can be enabled. In this method, the FOM is defined as the ratio between the height of the tallest bin and the standard deviation of the histogram (within the region of interest).

During the Grid-Search, the FOM for each parameter set is recorded separately for both segments and the core-contact, creating two grids of FOM values. These grids are normalized so that the best FOM in each grid is equals 1. A third grid is then calculated as the average of the two normalized grids. At the end of the process, the best parameter set for each selection is displayed.

The option -algo followed by depth and type allows to enable an Adaptive Grid-Search. This runs depth number of iterations, each time zooming around the best parameter set, according to type criteria (0 = SG, 1 = CC, 2 = avrg(SG,CC)).

The option -size, followed by two integers, allows to customize size of the Grid-Search.

6. Update the trapping file.

Use the following commands to select the parameters that optimize the SG spectra:

tail -n 39 sort_hit.log |head -n 36 > sort_hit_nohead.log;
awk 'FNR==NR{a[NR]=$4;next}{$4=a[FNR]}1'  sort_hit_nohead.log Trapping.cal \
 |awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' > Trapping_tmp.cal;
awk 'FNR==NR{a[NR]=$5;next}{$5=a[FNR]}1'  sort_hit_nohead.log Trapping_tmp.cal \
 |awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' |tee Trapping.cal;

Use the following commands to select the parameters that optimize the CC spectra:

tail -n 39 sort_hit.log |head -n 36 > sort_hit_nohead.log;
awk 'FNR==NR{a[NR]=$7;next}{$4=a[FNR]}1'  sort_hit_nohead.log Trapping.cal \
 |awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' > Trapping_tmp.cal;
awk 'FNR==NR{a[NR]=$8;next}{$5=a[FNR]}1'  sort_hit_nohead.log Trapping_tmp.cal \
 |awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' |tee Trapping.cal;

Use the following commands to select the parameters that optimize the avrg(SG,CC):

tail -n 39 sort_hit.log |head -n 36 > sort_hit_nohead.log;
awk 'FNR==NR{a[NR]=$10;next}{$4=a[FNR]}1'  sort_hit_nohead.log Trapping.cal \
 |awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' > Trapping_tmp.cal;
awk 'FNR==NR{a[NR]=$11;next}{$5=a[FNR]}1'  sort_hit_nohead.log Trapping_tmp.cal \
 |awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' |tee Trapping.cal;

The columns corresponding to neutron damage correction should now be updated in the Trapping.cal files like following:

 0   0.999311   1.000675       51.6       13.4         1.         1. 
 1   1.000594   1.000559     1482.1       11.1         1.         1.
...
...
34   0.999558   1.000857      326.8       11.1         1.         1. 
35   0.999976   1.000779      385.5       11.1         1.         1.

8. Apply the correction:

SortPsaHits -f ../../Out/$CryId/Psa__0-16-F__Hits.fdat -gain 5 -offs 5000 -fcal Trapping.cal

This will re-generate the file Pso__2-4-40-2048-UI__Ener.spec for the next step

9. Post trapping recalibration of the segments and cores after neutron correction (the search peaks parameters "-dwa" probably needs to be adapted):

segments recalibration (Family 0, Library 3):

RecalEnergy -spe Pso__2-4-40-2048-UI__Ener.spec -num 36 -sub 80 -gain 5 -offs -5000 -dwa 30 5 | tail -n 36 |tee log_sg_post.cal

cores recalibration for each segment (Family 1, Library 3):

RecalEnergy -spe Pso__2-4-40-2048-UI__Ener.spec -num 36 -sub 240 -gain 5 -offs -5000 -dwa 30 5 | tail -n 36 |tee log_cc_post.cal

Note

As we are here working on the recalibration after trapping, the spectra number offsets has been adapted: 80(Family 0, library 2) for segments, and 240(Family 1, library 2) for cores.

10. Insert the post-trapping recalibration parameters in the trapping file:

segments:

awk 'FNR==NR{a[NR]=$14;next}{$6=a[FNR]}1'  log_sg_post.cal Trapping.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' \
|tee Trapping_tmp.cal

cores:

awk 'FNR==NR{a[NR]=$14;next}{$7=a[FNR]}1'  log_cc_post.cal Trapping_tmp.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' \
|tee Trapping.cal

The columns corresponding to post-calibration should now be updated in the Trapping.cal files like following:

 0   0.999311   1.000675       51.6       13.4   0.998467   0.998754 
 1   1.000594   1.000559     1482.1       11.1   0.999063   0.999600 
...
...
34   0.999558   1.000857      326.8       11.1   0.999425   0.999294 
35   0.999976   1.000779      385.5       11.1   0.999054   0.999287

11. Re-generation of the file Pso__2-4-40-2048-UI__Ener.spec for final checks (be careful, a gain of 5 and an offset of 5000 is applied, the 1333 peak will thus be located at channel 1665):

SortPsaHits -f ../../Out/$CryId/Psa__0-16-F__Hits.fdat -gain 5 -offs 5000 -fcal Trapping.cal

The generated Trapping.cal file has to be copied to the corresponding folder Conf/CryId folder.

Finally, the gen_conf.py needs to be adapted in the PostPSA actor part, as follow (example for crystal 00A):

Note

If this line is already present, with other keywords, remove them, they will be added lated and must not be present for the moment.

#########################
PostPSAFilter=(
"ActualClass           PostPSAFilter",          # name of the used daughter class
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/Data(online)
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
#### command lines to be produced only for the specified crystals
{
'00A' : ("TrappingFile Trapping.cal"),
}
)

Check also that the file Trapping.cal is written in the "ExtraFiles" list of the gen_conf.py :

ExtraFiles={  
'CRYSTAL'   : "CrystalProducerATCA.conf PreprocessingFilterPSA.conf xinv_1325-1340.cal xdir_1325-1340.cal Trapping.cal",
}

Time-dependent core-energy (optional)#

Energy oscillations caused by the LN2 filling have been observed in multiple detectors (especially low-gain core contact) causing deterioration of resolution in order of ~1/1000. Example of this behavior is shown below:

To improve the resolution, an additional non-mandatory configuration file with time-dependent gain-correcting parameters TimeEvoCC.conf need to be created. File must have following structure

# TS_start        TS_end          gain
  540000000000    600000000000    0.999986788128527

To apply the correction parameters, the gen_conf.py needs to be modified in the PostPSA actor as follow (example for crystal 00A):

#########################
PostPSAFilter=(
"ActualClass           PostPSAFilter",          # name of the used daughter class
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/Data(online)
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
#### command lines to be produced only for the specified crystals
{
'00A' : ("TrappingFile Trapping.cal","TimeEvoCCFile TimeEvoCC.conf"
),
}
)

Check also that the file Trapping.cal is written in the "ExtraFiles" list of the gen_conf.py :

ExtraFiles={  
'CRYSTAL'   : "CrystalProducerATCA.conf PreprocessingFilterPSA.conf xinv_1325-1340.cal xdir_1325-1340.cal Trapping.cal TimeEvoCC.conf",
}

Getting the coefficients#

Esencially, the data has to be sliced in time and correction coefficients has to be found for each slice individually, with one of the serving as a reference. Several methods has been developed to automate this process, description of one of them called CCM (with a brief review of others) can be found in this NIM A paper. Code of CCM is available on github. It offers multiple settings which can be fine-tuned for the provided data using the agata_optimizer.cpp in order to achieve the best performance. Please follow project's README.md for detailed instruction.

Final energy re-calibrations (with offsets)#

In all the previous steps, energy calibrations were always linear. But the PostPSA actor allows to apply a final segment and energy recalibration, including an offset and if necessary higher order of calibrations. This step needs thus to be done on data taken with an ¹⁵²Eu source to have a good estimation of the non-linearity of the detectors. A very large statistics is thus needed for the backward segments. If you don't have such a dataset, use a ⁶⁰Co source,, by replacing in the following the "-152Eu" option by "-60Co".

The first step is to perform a replay of an ¹⁵²Eu run, from the traces, including the PostPSAFilter actor with neutron damage correction applied, and without recalibration (RecalEnergy2) and ForceSegmentToCore options.

For an easier and faster work in the following, this replay can be done in two steps:
1. Replay from traces to PSA (without the post PSA) and store the psa*.adf files using the BasicAFC consumer:
  
  You need to update the symbolic link "Data" in order to link to the ¹⁵²Eu data folder.
```
mv Out Out_save   # If you want to save the last replay from 60Co
rm Data
ln -s /path/to/152Eu/run/Data Data
python gen_conf.py
```
  This kind if topology can be used:
```
LOOP CRY 00B 00C 01B 01C 04B 04C 05A 05B 05C 06A 06B 06C

Chain 4   CRY
Producer  CrystalProducerATCA
Filter    PreprocessingFilterPSA
Filter    PSAFilter
Consumer  BasicAFC
ENDLOOP
```
  A large statistics is better, this replay can be long, but it will allow to no more perform the PSA later. The next replays will then be very fast.
2. Replay of the PostPSA only, using as input the psa*.adf files that have been produced at the previous step. For this, you need to move the Out folder in which the psa*.adf have been written in a new folder ("Data_Local" for example), and update the symbolic link "Data" to this folder, in such a way that the input files of the replay will be the psa*.adf that have been just produced:
```
mv Out Data_Local
rm Data
ln -s Data_Local Data
python gen_conf.py
```
  This kind if topology can be used:
```
LOOP CRY 00B 00C 01B 01C 04B 04C 05A 05B 05C 06A 06B 06C

Chain 3   CRY
Producer  BasicAFP
Filter    PostPSAFilter
Consumer  None
ENDLOOP
```
  This will produce the Post__5-40-16384-UI__Ener.spec spectra filename
Segments recalibration:
- case 1 : for detectors with Trapping.cal files present (should be the standard procedure):
```
RecalEnergy -spe ../../Out/$CryId/Post__5-40-16384-UI__Ener.spec -sub 40 -num 36 -gain 4 -poly1 -152Eu |tee recal.log
```
- case 2 : if for any reason the Trapping.cal file is not present:
```
RecalEnergy -spe ../../Out/$CryId/Post__5-40-16384-UI__Ener.spec -sub 0 -num 36 -gain 4 -poly1 -152Eu |tee recal.log
```
  This difference is due to the fact that the segment energy corrected from the trapping file is filled in the spectra of the second library of the file ("-sub 40"). But if there is no neutron damage correction (new detectors), these spectra are empty. Spectra of the first library ("-sub 0") need thus to be taken for the re-calibration.

Extract the re-calibration parameters:

tail -n 36 recal.log |tee recal_nohead.log ; 
awk -F' '  '{printf "segm %2.2s  %6.3f  %.6f \n",$1,$16,$17}' recal_nohead.log |tee RecalEnergy2.cal

Adapt the gen_conf.py as follow:

#########################
PostPSAFilter=(
"ActualClass           PostPSAFilter",          # name of the used daughter class
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/Data(online)
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
#### command lines to be produced only for the specified crystals
{
'00A' : ("TrappingFile Trapping.cal", "RecalEnergy2 RecalEnergy2.cal"),
}
)

Check also that the file: RecalEnergy2.cal is written in the "ExtraFiles" list of the gen_conf.py :

ExtraFiles={  
'CRYSTAL'   : "CrystalProducerATCA.conf PreprocessingFilterPSA.conf xinv_1325-1340.cal xdir_1325-1340.cal Trapping.cal RecalEnergy2.cal",
}

Core recalibration:

case 1 : for detectors with Trapping.cal files present (same as for segments):

RecalEnergy -spe ../../Out/$CryId/Post__5-40-16384-UI__Ener.spec -sub 79 -num 1 -gain 4 -poly1 -152Eu| tee recalCore.log

case 2 : for detectors without Trapping.cal files:

RecalEnergy -spe ../../Out/$CryId/Post__5-40-16384-UI__Ener.spec -sub 39 -num 1 -gain 4 -poly1 -152Eu| tee recalCore.log

Extract the re-calibration parameters:

tail -n 1 recalCore.log |tee recalCore_nohead.log ; 
awk -F' '  '{printf "\"RecalCC  %6.3f  %.6f\"\n",$16,$17}' recalCore_nohead.log

The output of the last command should be equivalent to: "RecalCC -0.115 1.000401" and needs to be copied in the gen_conf.py as follow:

#########################
PostPSAFilter=(
"ActualClass           PostPSAFilter",          # name of the used daughter class
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/Data(online)
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
#### command lines to be produced only for the specified crystals
{
'00A' : ("RecalCC -0.094  1.000651","TrappingFile Trapping.cal", "RecalEnergy2 RecalEnergy2.cal"),
}
)

Check the results:

To check that the recalibration has been well applied. A new replay is necessary with the modified gen_conf.py file. The file Post__5-40-16384-UI__Ener.spec can be analyzed with TkT. The different libraries are:
1. segment energies without correction
2. segment energies after trapping corrections
3. fraction of the core energy without correction
4. fraction of the core energy after trapping corrections
5. segment energies after trapping corrections and re-calibration.

Force Segments to Core#

The final step at the post PSA level is to decide of the use, or not, of the ForceSegmentsToCore option. If the core energy resolution is good and the counting rates not too important, it is better to use it. This option is defined for the desired detectors in the gen_conf.py , as follow:

#########################
PostPSAFilter=(
"ActualClass           PostPSAFilter",          # name of the used daughter class
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/Data(online)
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
#### command lines to be produced only for the specified crystals
{
'00A' : ("RecalCC -0.094  1.000651","TrappingFile Trapping.cal", "RecalEnergy2 RecalEnergy2.cal","ForceSegmentsToCore"),
}
)

Global time alignments#

The final time alignment of the cores is done using the output of the tracking. For this, this king of topology should be used:

LOOP CRY 00B 00C 01B 01C 04B 04C 05A 05B 05C 06A 06B 06C

Chain 3     CRY
Producer    BasicAFP 
Filter      PostPSAFilter
Dispatcher  EventBuilder
ENDLOOP

Chain 3     Builder/
Builder     EventBuilder
Filter      TrackingFilterOFT
Consumer    BasicAFC

Then, it is necessary to define in the gen_conf.py file the mapping of the detectors in the TrackingFilter actor as follow:

#########################
TrackingFilter=(
"ActualClass           PTrackingFilterOFT",     # name of the used daugther class 
"SaveDataDir           $SAVEDIR/$MERGER",       # Out/Merger
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
"OftParams             0.05 0.02 0.8",          # minprobtrack minprobsing sigma_thet (0==default)
"SourcePosition        0 0 0",                  # source pos with respect to the center of AGATA
"NumGeDets  24"                                 # for printing of TkT spectra
"SpecMap  0  0",                                # detID --> specID
"SpecMap  1  1",
"SpecMap  2  2",
"SpecMap  6  3",
...
...
"SpecMap 39 21",
"SpecMap 40 22",
"SpecMap 41 23",
)

Note

1- You need to be in the global folder (where Out, Data, Conf are located)

2- Be careful, the ordering of the SpecMap is very important !

Fit the time spectra:

The tacking actor produces a spectra file named: Track__N-N-1000-UI__TT.spec, with N the number of detectors defined in the TrackingFilter part of the gen_conf.py file (in the following example, N=24). This file is used for the global time alignment of the cores. The RecalEnergy code is here again used:
```
N=24; NN=$(( $N*$N )); 
RecalEnergy -spe Out/Global/Track__${N}-${N}-1000-UI__TT.spec -T 500 -num ${NN} |tee recalT.dat; 
tail -n ${NN} recalT.dat |tee recalT_nohead.dat;
```
Note

If the EventMerger actor has been used in the replay, the file will be located in "Out/Merger/" instead of "Out/Global/"

Apply the SolveTT.py script (this script is located in the agapro/zUseful folder):

solveTT.py -f recalT_nohead.dat -n ${N} -c 13 -p 500 |tee solveTT_tmp.dat

The end of the output of this script should be similar to:

Shifts that minimize Chi2
  -0.220
   3.565
   3.712
   1.187
   3.751
...
...
   3.505
   1.677
  -3.947
   1.248
  -0.931

Initial:    Average of 506 nonzero values is -499.98900    Chi2 = 692.93223
Corrected:  Average of 506 nonzero values is -499.98900    Chi2 = 17.45832

Extract the time shift values:

tail -n $(( $N+3 )) solveTT_tmp.dat |tee solveTT.dat |head -n ${N} |tee solveTT.dat; 
awk -F' '  '{printf "\"TimeShiftCC  %7.3f \"\n",$1}' solveTT.dat

The output of this last command give the time shifts to apply in the PostPSA actor for each crystal in the gen_conf.py file as follow:

#########################
PostPSAFilter=(
"ActualClass           PostPSAFilter",          # name of the used daughter class
"SaveDataDir           $SAVEDIR/$CRYSTAL",      # normally Out/Data(online)
"EnergyGain            4",                      # channels/keV of the calibrated energy spectra
#### command lines to be produced only for the specified crystals
{
'00A' : ("TimeShiftCC    -10.859" ,"RecalCC -0.094  1.000651","TrappingFile Trapping.cal", "RecalEnergy2 RecalEnergy2.cal","ForceSegmentsToCore"),
}
)

After a final global replay, check that the time alignment is correct. For this, open the spectra file Out/Global/Track__$N-$N-1000-UI__TT.spec (in the grid mode). The first set of N spectra will represent the timing between det 0 and the N-1 other... etc. All should be align at 500.

List of commands#

You can find bellow a list of commands for applying various steps of calibration for a list of detectors:

PreprocessingFilter#

Energy calibration#

#Define the crystal list
DET="00A 00B 00C 01A 01B 01C 02A 02B 02C"

### Commands to be applied in the replay folder:
### ==> where Conf, Data and Out dir, and gen_conf.py file are located

## Define Data dir (Out or Data as a function of spectra produced online or from replay)
DataDir=Data

## Define the source (-60Co or -152Eu) ==> 60Co is better for preprocessing level
Source=-60Co

for i in $DET; do cd Conf/$i;
RecalEnergy -spe ../../$DataDir/$i/Prod__4-38-32768-UI__Ampli.spec -sub 38 -num 38 -gain 2 -dwa 15 5 $Source |tee recal.out;
echo "ECalib for DET $i: press a key to save and continue"; read; 
cp PreprocessingFilterPSA.conf PreprocessingFilterPSA.save_energy # Just in case...
colupdate.py PreprocessingFilterPSA.conf recal.out -c 4 13 -o PreprocessingFilterPSA.conf
cd ../..; done;

Time alignment#

#Define the crystal list
DET="00A 00B 00C 01A 01B 01C 02A 02B 02C"

### Commands to be applied in the replay folder:
### ==> where Conf, Data and Out dir, and gen_conf.py file are located

## Define Data dir (Out or Data as a function of spectra produced online or from replay)
DataDir=Data

## Time alignment of the segments
for i in $DET;do cd Conf/$i;
RecalEnergy -spe ../../$DataDir/$i/Prep__6-40-1000-UI__TT.spec -sub 40 -num 36 -T 500 |tee shift_TT.out;
echo "ECalib for DET $i: press a key to save and continue"; read; 
cp PreprocessingFilterPSA.conf PreprocessingFilterPSA.save_time # Just in case...
colupdate.py PreprocessingFilterPSA.conf shift_TT.out -c 6 13 -o PreprocessingFilterPSA.conf;
cd ../../; done;

# For the time alignment of the cores, see the graphical explanations from the documentation

PostPSAFilter#

Neutron damage correction:#

#Define the crystal list
DET="00A 00B 00C 01A 01B 01C 02A 02B 02C"

### Commands to be applied in the replay folder:
### ==> where Conf, Data and Out dir, and gen_conf.py file are located

## Define Data dir (Out or Data as a function of spectra produced online or from replay)
DataDir=Out

## Create dummy empty file
for i in $DET; do cd Conf/$i;
rm -f Trapping.cal
for i in {0..35} ; do echo -e "$i\t1.\t1.\t999999.9\t999999.9\t1.\t1." >> Trapping.cal; done
cat Trapping.cal; cd ../.. ; done;

## Determine neutron damage parameters
for i in $DET; do cd Conf/$i;
SortPsaHits -f ../../$DataDir/$i/Psa__0-16-F__Hits.fdat -best 1300 1350 -bpar 1 10000 0 |tee sort_hit.log
tail -n 39 sort_hit.log |head -n 36 > sort_hit_nohead.log;
awk 'FNR==NR{a[NR]=$4;next}{$4=a[FNR]}1' sort_hit_nohead.log Trapping.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' > Trapping_tmp.cal;
awk 'FNR==NR{a[NR]=$5;next}{$5=a[FNR]}1' sort_hit_nohead.log Trapping_tmp.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' |tee Trapping.cal;
cd ../.. ; done;

## Generate output spectra
for i in $DET;do cd Conf/$i;
SortPsaHits -f ../../$DataDir/$i/Psa__0-16-F__Hits.fdat -gain 5 -offs 5000 -fcal Trapping.cal
cd ../..;done;

## Recalibration BEFORE neutron damage correction 
## ==> Not necessary if no REALLY bad calibration (more than 15keV shifted)
for i in $DET;do cd Conf/$i;
#segments
RecalEnergy -spe Pso__2-4-40-2048-UI__Ener.spec -num 36 -sub 0 -gain 5 -offs -5000 -noTR -dwa 30 2 |tail -n 36 |tee log_sg_pre.cal
#cores
RecalEnergy -spe Pso__2-4-40-2048-UI__Ener.spec -num 36 -sub 160 -gain 5 -offs -5000 -noTR -dwa 30 2 |tail -n 36 |tee log_cc_pre.cal
#update results
awk 'FNR==NR{a[NR]=$14;next}{$2=a[FNR]}1' log_sg_post.cal Trapping.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' |tee Trapping_tmp.cal
awk 'FNR==NR{a[NR]=$14;next}{$3=a[FNR]}1' log_cc_post.cal Trapping_tmp.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' |tee Trapping.cal
cd ../..;done;

# re-generate output spectra
for i in $DET;do cd Conf/$i;
SortPsaHits -f ../../Data/$i/Psa__0-16-F__Hits.fdat -gain 5 -offs 5000 -fcal Trapping.cal
cd ../..;done;

## Recalibration AFTER neutron damage correction 
## ==> Necessary for damaged detectors
for i in $DET;do cd Conf/$i;
#segments
RecalEnergy -spe Pso__2-4-40-2048-UI__Ener.spec -num 36 -sub 80 -gain 5 -offs -5000 -dwa 20 2 |tail -n 36 |tee log_sg_post.cal
#cores
RecalEnergy -spe Pso__2-4-40-2048-UI__Ener.spec -num 36 -sub 240 -gain 5 -offs -5000 -dwa 20 2 | tail -n 36 |tee log_cc_post.cal
read
#update results
awk 'FNR==NR{a[NR]=$14;next}{$6=a[FNR]}1' log_sg_post.cal Trapping.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' |tee Trapping_tmp.cal
awk 'FNR==NR{a[NR]=$14;next}{$7=a[FNR]}1' log_cc_post.cal Trapping_tmp.cal \
|awk '{printf "%2s %10s %10s %10s %10s %10s %10s \n", $1,$2,$3,$4,$5,$6,$7}' |tee Trapping.cal
cd ../..;done;

# re-generate output spectra
for i in $DET;do cd Conf/$i;
SortPsaHits -f ../../Data/$i/Psa__0-16-F__Hits.fdat -gain 5 -offs 5000 -fcal Trapping.cal
cd ../..;done;

### Check calibs
### This script allows to check the full Trapping files, 
### to help to spot potential errors in the calibration procedure 

for i in $DET;do cd Conf/$i;
echo -e "\n\033[36mDet: $i\n\033[0m"
for line in {1..36} ; do 
toto=`head -$line Trapping.cal |tail -1`;
id=`echo $toto |awk -F " " '{print $1}'`;
a=`echo $toto |awk -F " " '{print $2}'`;
diff=`printf "%.6f\n" $(echo "sqrt(($a - 1.)^2)" | bc -l)`
if ((`bc <<< "$diff<0.001"`)); then color="\033[32m";
elif ((`bc <<< "$diff<0.01"`)); then color="\033[33m";
else color="\033[31m";fi;a="$color$a\033[0m";
b=`echo $toto |awk -F " " '{print $3}'`;
diff=`printf "%.6f\n" $(echo "sqrt(($b - 1.)^2)" | bc -l)`
if ((`bc <<< "$diff<0.001"`)); then color="\033[32m";
elif ((`bc <<< "$diff<0.01"`)); then color="\033[33m";
else color="\033[31m";fi;b="$color$b\033[0m";
n1=`printf "%9.1f\n" $(echo $toto |awk -F " " '{print $4}')`;
n2=`printf "%9.1f\n" $(echo $toto |awk -F " " '{print $5}')`;
c=`echo $toto |awk -F " " '{print $6}'`;
diff=`printf "%.6f\n" $(echo "sqrt(($c - 1.)^2)" | bc -l)`
if ((`bc <<< "$diff<0.001"`)); then color="\033[32m";
elif ((`bc <<< "$diff<0.01"`)); then color="\033[33m";
else color="\033[31m";fi;c="$color$c\033[0m";
d=`echo $toto |awk -F " " '{print $7}'`;
diff=`printf "%.6f\n" $(echo "sqrt(($d - 1.)^2)" | bc -l)`
if ((`bc <<< "$diff<0.001"`)); then color="\033[32m";
elif ((`bc <<< "$diff<0.01"`)); then color="\033[33m";
else color="\033[31m";fi;d="$color$d\033[0m";
echo -e "$id\t$a\t$b\t$n1\t$n2\t$c\t$d"
done ;
cd ../..;done;

Final energy recalibration (RecalEnergy2)#

#Define the crystal list
DET="00A 00B 00C 01A 01B 01C 02A 02B 02C"

### Commands to be applied in the replay folder:
### ==> where Conf, Data and Out dir, and gen_conf.py file are located

## Define Data dir (Out or Data as a function of spectra produced online or from replay)
DataDir=Out

## Define the source (-60Co or -152Eu) ==> 152Eu is better for final recalibrations
Source=-152Eu

## Segments recalibration (if no trapping file, see the documentation):
for i in $DET;do cd Conf/$i;
RecalEnergy -spe ../../$DataDir/$i/Post__5-40-16384-UI__Ener.spec -sub 40 -num 36 -gain 4 -poly1 -dwa 30 2 $Source |tee recal.log
echo "ECalib for DET $i: press a key to continue"; read;
tail -n 36 recal.log |tee recal_nohead.log ; 
awk -F' '  '{printf "segm %2.2s %6.3f %.6f \n",$1,$16,$17}' recal_nohead.log |tee RecalEnergy2.cal
cd ../..;done;

## core recalibration (if no trapping file, see the documentation):
rm -f recalCore.log
for i in $DET;do cd Conf/$i;
RecalEnergy -spe ../../$DataDir/$i/Post__5-40-16384-UI__Ener.spec -sub 79 -num 1 -gain 4 -poly1 -dwa 40 5 $Source |tee recalCore.log;
tail -n 1 recalCore.log > recalCore_nohead.log ; 
cd ../..;
awk -v var=$i -F' '  '{printf "%s: \"RecalCC %6.3f %.6f\"\n",var,$16,$17}' Conf/$i/recalCore_nohead.log >> recalCore.log ;
done; cat recalCore.log ;

## Quick Check of the FWHM:
for i in $DET;do cd Conf/$i;
tail -n 1 recalCore.log > recalCore_nohead.log ; 
awk -v var=$i -F' '  '{printf "%s: \"FWHM %6.3f\"\n",var,$6}' recalCore_nohead.log
cd ../..;done;

### Check calibs
### This script allows to check the full Trapping files, 
### to help to spot potential errors in the calibration procedure 

for i in $DET;do cd Conf/$i;
echo -e "\n\033[36mDet: $i\n\033[0m"
for line in {1..36} ; do 
toto=`head -$line RecalEnergy2.cal |tail -1`;
id=`echo $toto |awk -F " " '{print $2}'`;
offset=`echo $toto |awk -F " " '{print $3}'`;
offsetabs=$(echo "sqrt(($offset)^2)" | bc -l)
if ((`bc <<< "$offsetabs<1"`)); then color="\033[32m";
elif ((`bc <<< "$offsetabs<5"`)); then color="\033[33m";
else color="\033[31m";fi;offset="$color$offset\033[0m";
gain=`echo $toto |awk -F " " '{print $4}'`;
diff=`printf "%.6f\n" $(echo "sqrt(($gain - 1.)^2)" | bc -l)`
if ((`bc <<< "$diff<0.001"`)); then color="\033[32m";
elif ((`bc <<< "$diff<0.01"`)); then color="\033[33m";
else color="\033[31m";fi;gain="$color$gain\033[0m";
echo -e "segm $id\t$offset\t$gain"
done ;
cd ../..;done

Global time alignment#

## N define the number of detectors:
## ==> must be identical as the value defined in the Tracking actor of the gen_conf.py
N=38;

## Define Data dir (Out/Merger or Out/Global as a function of the topology used)
DataDir=Out/Merger

NN=$(( $N*$N )); \
RecalEnergy -spe $DataDir/Track__${N}-${N}-1000-UI__TT.spec -T 500 -num ${NN} |tee recalT.dat;
tail -n ${NN} recalT.dat |tee recalT_nohead.dat;
solveTT.py -f recalT_nohead.dat -n ${N} -c 13 -p 500 |tee solveTT_tmp.dat;
tail -n $(( $N+3 )) solveTT_tmp.dat |tee solveTT.dat |head -n ${N} |tee solveTT.dat;
awk -F' ' '{printf "\"TimeShiftCC %7.3f \"\n",$1}' solveTT.dat;

These spectra are produced by the CrystalProducer actor ↩
See zPrograms directory in the agapro package ↩
For a fill list of options, RecalEnergy -h ↩
See zUseful directory in the agapro package ↩
This set of spectra is produced by the Preprocessing filter. ↩